README.md

---
title: MCQ Generator
emoji: 📝
colorFrom: blue
colorTo: purple
sdk: streamlit
sdk_version: 1.33.0
app_file: app/main.py
pinned: false
---

# 📝 MCQ Generator — Automatic Multiple Choice Question Generator

> **An end-to-end NLP pipeline that reads any text passage and automatically generates a complete multiple-choice quiz with scoring and explanations.**

Built as a course project for an NLP curriculum covering Modules I–IV: tokenization, word embeddings, transformers, and natural language generation.

---

## 📌 Table of Contents

1. [What This Project Does](#what-this-project-does)
2. [Live Demo](#live-demo)
3. [How It Works — The Full Pipeline](#how-it-works--the-full-pipeline)
4. [NLP Techniques Used](#nlp-techniques-used)
5. [Project Structure](#project-structure)
6. [Each File Explained](#each-file-explained)
7. [Tech Stack](#tech-stack)
8. [Setup & Installation](#setup--installation)
9. [Running the App](#running-the-app)
10. [Testing Each Module](#testing-each-module)
11. [Sample Output](#sample-output)
12. [What Makes a Good Passage](#what-makes-a-good-passage)
13. [Known Limitations](#known-limitations)
14. [Future Work](#future-work)
15. [Related Research](#related-research)
16. [Course Outcomes Covered](#course-outcomes-covered)

---

## What This Project Does

Given any factual text passage, this system:

1. **Extracts** the most important sentences using TF-IDF ranking
2. **Identifies** answer candidates using Named Entity Recognition (NER)
3. **Generates** natural language questions using a T5 transformer model
4. **Creates** plausible wrong options (distractors) using WordNet and NER
5. **Presents** an interactive quiz with scoring and per-question explanations

**Example:**

Input passage:
```
Albert Einstein was born on March 14, 1879, in Ulm, Germany.
He was awarded the Nobel Prize in Physics in 1921 for his
discovery of the photoelectric effect.
```

Generated MCQ:
```
Q: Where was Albert Einstein born?

A. France
B. Germany  ✓
C. United States
D. Switzerland
```

---

## Live Demo

```bash
streamlit run app/main.py
```

Opens at `http://localhost:8501` in your browser.

---

## How It Works — The Full Pipeline

```
Raw Text Passage
       │
       ▼
┌─────────────────────────────────────────────┐
│  STEP 1: PREPROCESSING  (preprocessor.py)   │
│                                             │
│  • Split into sentences (spaCy)             │
│  • Rank by TF-IDF score (scikit-learn)      │
│  • Extract Named Entities (spaCy NER)       │
│  • Filter answer candidates (blacklist)     │
└─────────────────┬───────────────────────────┘
                  │  top sentences + answer candidates
                  ▼
┌─────────────────────────────────────────────┐
│  STEP 2: QUESTION GENERATION                │
│          (question_generator.py)            │
│                                             │
│  • Highlight answer in sentence with <hl>   │
│  • Feed to T5 transformer model             │
│  • Generate 3 candidate questions           │
│  • Validate: reject circular/vague Qs       │
└─────────────────┬───────────────────────────┘
                  │  (question, answer) pairs
                  ▼
┌─────────────────────────────────────────────┐
│  STEP 3: DISTRACTOR GENERATION              │
│          (distractor_generator.py)          │
│                                             │
│  Strategy 1: Same-type NER entities         │
│              from the passage               │
│  Strategy 2: WordNet hyponym siblings       │
│  Strategy 3: Cross-label fallback           │
└─────────────────┬───────────────────────────┘
                  │  3 wrong options per question
                  ▼
┌─────────────────────────────────────────────┐
│  STEP 4: MCQ ASSEMBLY + VALIDATION          │
│          (mcq_builder.py)                   │
│                                             │
│  • Combine answer + distractors             │
│  • Shuffle options randomly                 │
│  • Quality gate: dedup, similarity check    │
│  • Return list of MCQ objects               │
└─────────────────┬───────────────────────────┘
                  │  validated MCQ list
                  ▼
┌─────────────────────────────────────────────┐
│  STEP 5: QUIZ UI + SCORING                  │
│          (app/main.py + evaluator.py)       │
│                                             │
│  • Streamlit 3-screen app                   │
│  • Input → Quiz → Results                   │
│  • Score, feedback, explanations            │
└─────────────────────────────────────────────┘
```

---

## NLP Techniques Used

### Module I — Foundational NLP
| Technique | Where Used | Purpose |
|---|---|---|
| Tokenization | `preprocessor.py` | Split text into sentences and tokens using spaCy |
| Lemmatization | `preprocessor.py` | Normalize word forms for TF-IDF |
| Stop word removal | `preprocessor.py` | Filter noise before TF-IDF scoring |
| Named Entity Recognition (NER) | `preprocessor.py` | Find PERSON, ORG, DATE, GPE as answer candidates |
| POS Tagging | `preprocessor.py` | Identify nouns and proper nouns |
| WordNet | `distractor_generator.py` | Find semantically related words as distractors |
| Synsets / Hyponyms | `distractor_generator.py` | Navigate WordNet hierarchy for same-category words |

### Module II — Word Representation
| Technique | Where Used | Purpose |
|---|---|---|
| TF-IDF | `preprocessor.py` | Rank sentences by information density |
| Word Embeddings (GloVe) | `distractor_generator.py` | Optional cosine-similarity based distractor finding |

**TF-IDF explained:**
- **TF (Term Frequency)** = how often a word appears in *this* sentence
- **IDF (Inverse Document Frequency)** = how rare the word is across *all* sentences
- High TF-IDF score = sentence contains rare, informative words → good question source

### Module III — Deep Learning for NLP
| Technique | Where Used | Purpose |
|---|---|---|
| Transformers | `question_generator.py` | T5 model for question generation |
| Transfer Learning | `question_generator.py` | Using pre-trained T5 fine-tuned on SQuAD |
| Seq2Seq | `question_generator.py` | Encoder-decoder architecture of T5 |
| Beam Search | `question_generator.py` | Generate multiple question candidates, pick best |

### Module IV — Advanced NLP
| Technique | Where Used | Purpose |
|---|---|---|
| T5 (Text-to-Text Transfer Transformer) | `question_generator.py` | State-of-the-art QG model |
| Natural Language Generation (NLG) | `question_generator.py` | Generating grammatical questions |
| Subword Tokenization (SentencePiece) | `question_generator.py` | T5's tokenizer handles rare/unknown words |
| Pre-trained Models | `question_generator.py` | `valhalla/t5-small-qg-hl` from HuggingFace |

---

## Project Structure

```
mcq_generator/
│
├── src/                          # Core NLP pipeline modules
│   ├── __init__.py
│   ├── preprocessor.py           # Text cleaning, TF-IDF, NER, answer extraction
│   ├── question_generator.py     # T5-based question generation
│   ├── distractor_generator.py   # WordNet + NER distractor generation
│   ├── mcq_builder.py            # Pipeline orchestrator + MCQ dataclass
│   └── evaluator.py              # Answer checking and scoring
│
├── app/                          # Streamlit web application
│   ├── __init__.py
│   ├── main.py                   # 3-screen app: input → quiz → results
│   └── components.py             # Reusable UI components
│
├── data/
│   └── sample_passages.json      # 5 test passages (ISRO, Gandhi, AI, etc.)
│
├── models/                       # (gitignored) Downloaded model files
│   └── README.md
│
├── notebooks/                    # Jupyter notebooks for exploration
│
├── config.py                     # All settings in one place
├── requirements.txt              # Python dependencies
└── README.md                     # This file
```

---

## Each File Explained

### `config.py`
Central settings file. Every other module imports from here.
- Model name, number of questions, sentence count, file paths
- Change values here to tune the entire system without touching logic files

### `src/preprocessor.py`
The NLP foundation of the project.

**Key functions:**
- `extract_sentences(text)` — spaCy sentence boundary detection
- `rank_sentences(sentences)` — TF-IDF scoring, returns top N most informative sentences
- `extract_answer_candidates(sentence)` — NER-based extraction with strict quality filters
- `preprocess(text)` — full pipeline, returns structured dict

**Design decisions:**
- Only `PERSON`, `ORG`, `GPE`, `DATE`, `EVENT`, `WORK_OF_ART` NER labels are accepted as answers
- A `BLACKLIST` of 30+ generic words ("annual", "various", "Moon") prevents trivial answers
- Answers are sorted by priority: PERSON > ORG/GPE > DATE > others

### `src/question_generator.py`
Uses the `valhalla/t5-small-qg-hl` model — a T5-small fine-tuned on SQuAD for question generation.

**How T5 QG works:**
```
Input:  "generate question: ISRO was founded in <hl> 1969 <hl> by Vikram Sarabhai."
Output: "In what year was ISRO founded?"
```

**Key functions:**
- `highlight_answer(sentence, answer)` — wraps answer in `<hl>` tags
- `generate_question(sentence, answer)` — beam search with 5 beams, 3 candidates
- `answer_is_addressable(question, answer)` — rejects circular, vague, or short questions

**Quality filters applied:**
- Must start with a question word (what/who/when/where/which/how)
- Answer must NOT appear in the question
- Abbreviation trap detection (e.g. rejects Q: "What does ISRO stand for?" when A is the full name)
- Minimum 5 words

### `src/distractor_generator.py`
Generates 3 plausible wrong answer options. Uses a priority-based strategy chain.

**Strategy 1 — Same-label NER (best):**
Finds other entities of the same NER type from the passage.
```
Answer: "1969" (DATE) → Distractors: ["1975", "2008", "2023"]  (other DATEs in passage)
Answer: "Vikram Sarabhai" (PERSON) → Distractors: ["Kalam", "Dhawan", "Nehru"]
```

**Strategy 2 — WordNet hyponyms:**
Navigates the WordNet hierarchy to find sibling words in the same semantic category.
```
Answer: "India" → hypernym: "country" → hyponyms: ["China", "Brazil", "Pakistan"]
```

**Strategy 3 — Cross-label fallback:**
Uses any other named entity from the passage if strategies 1 and 2 fail.

### `src/mcq_builder.py`
The single entry point that the UI calls. Orchestrates the entire pipeline.

**MCQ dataclass:**
```python
@dataclass
class MCQ:
    question       : str
    options        : list      # 4 shuffled options
    correct_index  : int       # index of correct answer (0-3)
    correct_answer : str
    explanation    : str       # original sentence
```

**Quality gate `is_valid_mcq()`:**
- No two options can be too similar (catches "WWE" vs "World Wrestling Entertainment")
- Answer must appear exactly once in options
- Maximum 1 generic placeholder option allowed
- Answer must not appear in question text

### `src/evaluator.py`
Checks answers and computes scores.

**Returns:**
```python
{
  "score"     : 7,
  "total"     : 10,
  "percentage": 70.0,
  "feedback"  : "Good effort! Review the explanations...",
  "results"   : [ {per-question breakdown} ]
}
```

### `app/main.py`
Streamlit app with 3 screens managed via `st.session_state`:
- **Screen 1 (input):** Text area + question count slider + Generate button
- **Screen 2 (quiz):** One question at a time, radio buttons, Previous/Next/Submit
- **Screen 3 (results):** Score banner + per-question feedback with explanations

### `app/components.py`
Reusable display functions:
- `render_question_card()` — A/B/C/D labelled radio buttons
- `render_result_card()` — green (correct) / red (wrong) with explanation
- `render_score_summary()` — score banner + metric cards

---

## Tech Stack

| Library | Version | Purpose |
|---|---|---|
| `spaCy` | 3.7.4 | Tokenization, NER, POS tagging, sentence splitting |
| `transformers` | 4.38.2 | T5 model for question generation |
| `torch` | 2.2.1 | PyTorch backend for transformers |
| `nltk` | 3.8.1 | WordNet access for distractor generation |
| `scikit-learn` | 1.4.1.post1 | TF-IDF vectorizer |
| `sentencepiece` | latest | T5's subword tokenizer |
| `streamlit` | 1.33.0 | Web UI framework |
| `gensim` | 4.3.2 | Word2Vec / GloVe loading (optional) |
| `numpy` | 1.26.4 | TF-IDF matrix operations |

**Pre-trained model used:**
- `valhalla/t5-small-qg-hl` — T5-small fine-tuned on SQuAD 1.0 for answer-aware question generation using highlight format. Hosted on HuggingFace Hub, downloaded automatically on first run (~240MB).

---

## Setup & Installation

### Prerequisites
- Python 3.11+
- pip
- Internet connection (first run downloads the T5 model)

### Step 1 — Clone the repository
```bash
git clone https://github.com/tanmmayyy/mcq-generator.git
cd mcq-generator
```

### Step 2 — Create a virtual environment
```bash
python -m venv myenv

# Windows
myenv\Scripts\activate

# Mac/Linux
source myenv/bin/activate
```

### Step 3 — Install dependencies
```bash
pip install -r requirements.txt
pip install sentencepiece   # required for T5 tokenizer
```

### Step 4 — Download spaCy language model
```bash
# If the default command fails:
pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.7.1/en_core_web_sm-3.7.1-py3-none-any.whl
```

### Step 5 — Verify installation
```bash
python -c "import spacy; nlp = spacy.load('en_core_web_sm'); print('spaCy OK')"
python -c "from transformers import pipeline; print('Transformers OK')"
```

---

## Running the App

```bash
streamlit run app/main.py
```

The app opens at `http://localhost:8501`. On first launch, the T5 model downloads (~240MB) and loads into memory — this takes 1–2 minutes. Subsequent launches are fast.

---

## Testing Each Module

Run these in order to verify each step of the pipeline works independently:

```bash
# Step 1 — Test preprocessing (NER, TF-IDF, sentence ranking)
python src/preprocessor.py

# Step 2 — Test question generation (T5 model)
python src/question_generator.py

# Step 3 — Test distractor generation (WordNet + NER)
python src/distractor_generator.py

# Step 4 — Test full pipeline end-to-end
python src/mcq_builder.py

# Step 5 — Test scoring
python src/evaluator.py
```

---

## Sample Output

**Input passage (ISRO):**
```
The Indian Space Research Organisation (ISRO) was founded in 1969 by Vikram Sarabhai.
ISRO developed India's first satellite, Aryabhata, which was launched in 1975.
The Chandrayaan-1 mission in 2008 discovered water molecules on the Moon.
In 2023, Chandrayaan-3 successfully landed near the lunar south pole.
The Mars Orbiter Mission, also called Mangalyaan, was launched in 2013.
```

**Generated questions:**

```
Q1: Who founded ISRO?
    A. Jawaharlal Nehru
    B. APJ Abdul Kalam
    C. Vikram Sarabhai  ✓
    D. Homi Bhabha

Q2: What was India's first satellite called?
    A. Chandrayaan
    B. Mangalyaan
    C. Rohini
    D. Aryabhata  ✓

Q3: When did the Chandrayaan-1 mission take place?
    A. 1975
    B. 2013
    C. 2023
    D. 2008  ✓

Q4: What mission made India the first Asian country to reach Mars orbit?
    A. Chandrayaan-3
    B. Aryabhata
    C. Mangalyaan  ✓
    D. Chandrayaan-1
```

---

## What Makes a Good Passage

The system performs best on **factual passages** that contain:

| Works well | Works poorly |
|---|---|
| People names (PERSON entities) | Opinion / descriptive text |
| Specific dates (DATE entities) | Passages with repeated entities |
| Organisation names (ORG entities) | Very short passages (< 5 sentences) |
| Place names (GPE entities) | Abstract/philosophical text |
| One clear fact per sentence | Sentences with multiple facts |

**Best passage types:** History, science, geography, biographies, Wikipedia-style articles

**Avoid:** Opinion pieces, marketing content, descriptive narratives without specific facts

---

## Known Limitations

1. **Passage type dependency** — Works best on factual text. Descriptive or opinion text produces poor questions because there are no named entities to use as answers.

2. **T5-small quality ceiling** — The model used (`t5-small`) has 60M parameters. Larger models like `t5-base` or `t5-large` would produce better questions but require more memory and time.

3. **Distractor diversity** — When a passage has few named entities, distractors may fall back to generic options. Fine-tuning a separate T5 model on the RACE dataset for distractor generation would fix this.

4. **English only** — The current pipeline only supports English text. Extending to Hindi or other Indic languages would require multilingual spaCy models and a multilingual QG model.

5. **No semantic deduplication** — Two questions from the same passage can sometimes be semantically similar even if worded differently.

---

## Future Work

- [ ] Fine-tune a T5 distractor generation model on the RACE dataset (100k exam questions)
- [ ] Add support for Hindi using IndicNLP + multilingual BERT
- [ ] Add PDF upload support so users can quiz themselves on any document
- [ ] BLEU/METEOR/ROUGE automated evaluation of generated questions
- [ ] Difficulty scoring per question based on distractor plausibility
- [ ] Export quiz as PDF for offline use

---

## Related Research

Papers that use similar approaches — cited for comparison:

1. **Automatic Generation of Multiple-Choice Questions (2023)**
   Zhang et al. — T5 with pre/postprocessing pipelines for MCQ generation
   https://arxiv.org/abs/2303.14576

2. **Deep Learning and Linguistic Feature Based Automatic MCQ Generation (Springer, ICDCIT 2022)**
   Agarwal et al. — DL + linguistic features for MCQ generation (same 3-step pipeline)
   https://link.springer.com/chapter/10.1007/978-3-030-94876-4_18

3. **End-to-End MCQ Generation Using T5 (ScienceDirect 2022)**
   Rodriguez-Torrealba et al. — Full T5-based pipeline with Wikipedia passages
   https://www.sciencedirect.com/science/article/pii/S0957417422014014

4. **Leaf — MCQ Generation System (ECIR 2022)**
   Vachev et al. — Two fine-tuned T5 models: one for QG, one for DG on RACE
   https://github.com/KristiyanVachev/Leaf-Question-Generation

5. **Automatic Distractor Generation — Systematic Review (PMC 2024)**
   Comprehensive review of distractor generation methods including WordNet and T5
   https://pmc.ncbi.nlm.nih.gov/articles/PMC11623049/

6. **Automatic Question Generation: A Review (Springer/PMC 2023)**
   Mulla & Gharpure — Survey of methodologies, datasets, and evaluation metrics
   https://pmc.ncbi.nlm.nih.gov/articles/PMC9886210/

**What differentiates this project from the above:**
- End-to-end pipeline with interactive quiz UI (most papers only generate questions)
- NER-type-matching distractor strategy (distractors always same entity type as answer)
- Multi-layer quality filtering at both question and MCQ level
- Answer circularity detection (rejects questions where answer appears in the question)

---

## Course Outcomes Covered

| CO | Description | How this project covers it |
|---|---|---|
| CO1 | Articulate NLP and word representation | TF-IDF, NER, WordNet, word embeddings all implemented and explained |
| CO2 | Build deep learning models for NLP problems | T5 transformer for QG (seq2seq), beam search decoding, transfer learning |
| CO3 | Implement ML/DL solutions in real context | End-to-end deployable system with Streamlit UI and interactive demo |

---

## Author

**[Tanmay Jain]**
[ Bennett University]


---

*Built with spaCy, HuggingFace Transformers, NLTK, scikit-learn, and Streamlit.*