Skip to main content

​
What are Logical Fallacies?

Logical fallacies are errors in reasoning that undermine the logic of an argument. They occur when conclusions don’t follow logically from their premises, even if the premises themselves are true.
The NL2FOL system is designed to detect logical fallacies by translating natural language arguments into first-order logic and using SMT solvers to check their validity.

​
Why First-Order Logic for Fallacy Detection?

First-order logic provides a formal framework that:
  • Precisely represents the structure of arguments
  • Eliminates ambiguity present in natural language
  • Enables automated verification using SMT solvers
  • Reveals hidden assumptions in reasoning
By converting an argument to FOL, we can determine if a conclusion logically follows from its premises or if the reasoning contains a flaw.

​
Primary Fallacy: Hasty Generalization

The NL2FOL system focuses primarily on detecting hasty generalizations (also called faulty generalizations), which is one of the most common logical fallacies.

​
Definition

A hasty generalization occurs when someone draws a broad conclusion from insufficient or unrepresentative evidence.

Structure

  • Premise: A specific observation or limited sample
  • Conclusion: A universal claim about all cases

Why It's Fallacious

The leap from “some” or “one” to “all” is not logically justified without sufficient evidence

​
Examples from the Dataset

The system is trained on thousands of real-world examples from data/fallacies.csv:
Fallacy: “Annie must like Starbucks because all white girls like Starbucks.”Analysis:
  • Claim: Some white girls like Starbucks (limited observation)
  • Implication: ALL white girls like Starbucks (universal claim)
  • Flaw: Generalizes from limited experience to entire population
FOL Representation:
exists x (IsWhiteGirl(x) and LikesStarbucks(x)) 
  -> forall y (IsWhiteGirl(y) -> LikesStarbucks(y))
Fallacy: “The two courses I took at UF were not very interesting. I don’t think it’s a good university.”Analysis:
  • Claim: Two courses at UF were not interesting (n=2)
  • Implication: UF is not a good university (judgment of entire institution)
  • Flaw: Two courses are insufficient to evaluate an entire university
FOL Representation:
exists x (IsCourseAtUF(x) and NotInteresting(x)) 
  -> forall y (IsCourseAtUF(y) -> NotInteresting(y))
Fallacy: “A driver with a New York license plate cuts you off in traffic. You decide that all New York drivers are terrible drivers.”Analysis:
  • Claim: One NY driver drove poorly (single instance)
  • Implication: All NY drivers are terrible (universal generalization)
  • Flaw: One driver’s behavior doesn’t represent all drivers from that region
FOL Representation:
exists x (IsNYDriver(x) and DrivesTerribly(x)) 
  -> forall y (IsNYDriver(y) -> DrivesTerribly(y))
Fallacy: “Look at people like Michael Vick and OJ Simpson. Professional athletes really have no sense of morality.”Analysis:
  • Claim: Two athletes exhibited immoral behavior
  • Implication: All professional athletes have no sense of morality
  • Flaw: Cherry-picking examples to condemn an entire profession
FOL Representation:
exists x (IsProfessionalAthlete(x) and NotMoral(x)) 
  -> forall y (IsProfessionalAthlete(y) -> NotMoral(y))

​
The Dataset

The system uses multiple datasets for training and evaluation:

​
Fallacies Dataset

# From src/nl_to_fol.py:389-420
def setup_dataset(fallacy_set='logic',length=100):
    if fallacy_set=='logic':
        df_fallacies=pd.read_csv('data/fallacies.csv')
        df_fallacies['label']=[0]*len(df_fallacies)  # 0 = fallacy
        df_fallacies=df_fallacies[['source_article','label','updated_label']]
    
    df_valids=pd.read_csv('data/nli_entailments_test.csv')
    df_valids['label']=[1]*len(df_valids)  # 1 = valid reasoning
The system uses a binary classification approach:
  • Label 0: Logical fallacy (invalid reasoning)
  • Label 1: Valid logical reasoning

​
Dataset Statistics

From data/fallacies.csv:
  • Total examples: 230+ unique fallacy instances
  • Primary category: Faulty generalization / Hasty generalization
  • Sources: Quiz questions, educational materials, real-world examples
  • Categories: Personal stereotypes, professional generalizations, cultural biases, statistical misuse

​
Common Patterns in Hasty Generalizations

​
Pattern 1: Personal Experience

Structure: "My [experience] was [quality], so [universal statement]"

Examples:
- "My roommate's philosophy class was hard, so ALL philosophy classes must be hard"
- "Pain-Away worked for me, so it's sure to work for you, too"

​
Pattern 2: Small Sample Size

Structure: "I observed [small number] of [group], therefore [all of group]"

Examples:
- "Four out of five dentists recommend this toothpaste, therefore it must be great"
- "I asked six friends and they agreed, so this is generally popular"

​
Pattern 3: Stereotyping

Structure: "[Individual from group] has [property], therefore [all group members] have [property]"

Examples:
- "Fred, the German, stole my wallet. Therefore, all Germans are thieves"
- "All women care about are their looks"

​
Pattern 4: Temporal Fallacy

Structure: "[Event] happened [once/recently], therefore [always/all time]"

Examples:
- "It's warmer this year in Las Vegas, therefore global warming is rapidly accelerating"
- "My first day was easy, so it will always be easy"

​
Detecting Fallacies with FOL

The key insight is that hasty generalizations have a characteristic logical structure:

Fallacious Pattern

exists x (P(x) and Q(x)) -> forall y (P(y) -> Q(y))
“Because some x has properties P and Q, all things with property P have property Q”This is logically invalid - the existence of one or few instances doesn’t prove a universal rule.

​
Why This Formula is Invalid

When we check this with an SMT solver:
  1. The solver attempts to find a model where the negation is true
  2. The negation: NOT(exists x (P(x) and Q(x)) -> forall y (P(y) -> Q(y)))
  3. This is satisfiable when:
    • There exists at least one x with P(x) and Q(x) (the observed case)
    • There exists at least one y with P(y) but NOT Q(y) (a counterexample)
  4. Since such a model exists, the original formula is invalid
The SMT solver returns SAT (satisfiable) for the negation, indicating the argument is fallacious.

​
Beyond Hasty Generalization

While the system focuses on hasty generalization, the framework can be extended to other fallacy types:

Appeal to Authority

Accepting a claim based solely on authority rather than evidenceExample: “Experts say it, so it must be true”

False Cause

Assuming causation from correlationExample: “My dog was sprayed by a skunk on the trail, therefore trail running is dangerous”

Circular Reasoning

Using the conclusion as a premiseExample: “The Bible is true because it says so in the Bible”

False Dilemma

Presenting only two options when more existExample: “You’re either with us or against us”

​
Practical Applications

Understanding logical fallacies is crucial for:
  • Critical thinking: Evaluating arguments in media, politics, and daily life
  • Argument construction: Building sound logical arguments
  • Debate: Identifying weak reasoning in opposing arguments
  • Research: Avoiding flawed conclusions from data
  • AI/ML: Training systems to reason more accurately

​
Next Steps

First-Order Logic

Learn the formal logic system used for fallacy detection

Translation Pipeline

See how natural language is converted to logical formulas

SMT Solving

Understand how solvers verify logical validity

Quick Start

Try detecting fallacies yourself

Build docs developers (and LLMs) love

Get started for freeTalk to us