Builder-Neekhil's picture
Add comprehensive documentation
c4bb5cc verified
# ARCHAI Adaptive Assessment Engine
**SOTA-powered adaptive AI readiness assessment** β€” replaces static 12-question quizzes with intelligent, personalized testing that adapts to each user's ability level in real time.
πŸ”— **Live API**: https://huggingface.co/spaces/Builder-Neekhil/archai-adaptive-engine
🌐 **Frontend**: https://your-ai-arch.netlify.app (plug this engine in!)
---
## What Makes This Different
| Feature | Static Quiz (v1) | Adaptive Engine (v2) |
|---------|------------------|------------------------|
| Question order | Fixed | **Fisher-information optimal** |
| Question count | Always 12 | **6–12 adaptive** (stops when precision is sufficient) |
| Scoring | Simple average | **Bayesian latent ability estimation** |
| Difficulty | Same for everyone | **Calibrated to each user** |
| Precision | None | **Standard error per dimension** |
| Learning path | Static recommendations | **Structured day/week/month actionables** |
---
## Architecture
```
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ React Frontend │────▢│ FastAPI + IRT-2PL Engine │────▢│ Learning Path Gen β”‚
β”‚ (your webapp) │◀────│ β€’ Bayesian Knowledge Tracing│◀────│ (Day/Week/Month) β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β€’ Fisher Info Selection β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚ β€’ Precision-based Stopping β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
```
### Core Components
1. **2PL IRT Model** β€” Two-Parameter Logistic Item Response Theory:
- `P(correct|ΞΈ) = sigmoid(a Γ— (ΞΈ βˆ’ b))`
- `a` = discrimination (how well the question separates high/low ability)
- `b` = difficulty (calibrated to 6 maturity stages)
2. **Fisher Information Selection** β€” Next question maximizes information at the user's current ability estimate, minimizing measurement error.
3. **Bayesian Knowledge Tracing** β€” After each response, MAP estimate of latent ability ΞΈ is updated. Standard error tracks precision.
4. **Precision-Based Stopping** β€” Assessment stops early when all 6 dimensions achieve SE < 0.3 (β‰ˆ Β±3% confidence), saving user time.
5. **Structured Learning Paths** β€” Day-by-day micro-actions, week-by-week milestones, month-by-month strategic goals.
---
## API Endpoints
| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/api/v1/session/start` | Initialize adaptive assessment |
| `POST` | `/api/v1/session/answer` | Submit answer, get next question |
| `GET` | `/api/v1/session/{id}` | Get current state or results |
| `POST` | `/api/v1/path/generate` | Generate learning path |
| `GET` | `/api/v1/questions` | Full calibrated question bank |
| `GET` | `/api/v1/health` | Health check + engine metadata |
---
## Quick Start
### 1. Start Assessment
```bash
curl -X POST https://Builder-Neekhil-archai-adaptive-engine.hf.space/api/v1/session/start
```
Returns:
```json
{
"session_id": "abc123",
"question": {
"id": "lit_3",
"dimension": "literacy",
"text": "Can you explain what a transformer architecture is...",
"options": ["No idea", "Vague understanding", "Can explain", "Can implement"],
"difficulty": 0.5,
"discrimination": 1.8
},
"progress": {"asked": 0, "total": 12, "dimensions_covered": []},
"status": "in_progress"
}
```
### 2. Submit Answer
```bash
curl -X POST https://Builder-Neekhil-archai-adaptive-engine.hf.space/api/v1/session/answer \
-H "Content-Type: application/json" \
-d '{"session_id":"abc123","question_id":"lit_3","option_index":2}'
```
Returns next adaptive question (or completion status).
### 3. Get Results
```bash
curl https://Builder-Neekhil-archai-adaptive-engine.hf.space/api/v1/session/abc123
```
Returns full profile with dimension scores, stage, archetype, strengths, gaps, and latent abilities.
### 4. Generate Learning Path
```bash
curl -X POST https://Builder-Neekhil-archai-adaptive-engine.hf.space/api/v1/path/generate \
-H "Content-Type: application/json" \
-d '{
"session_id": "abc123",
"persona_id": "swe",
"hours_per_week": 5,
"budget_usd": 25,
"hardware_id": "16gb",
"preference": "both"
}'
```
Returns structured `days`, `weeks`, `months` actionables with projections.
---
## Integration Guide
### Replace Static Questions in Your Frontend
```javascript
// OLD: Static question flow
const questions = staticQuestionBank; // always 12, fixed order
// NEW: Adaptive API calls
async function startAssessment() {
const res = await fetch('https://Builder-Neekhil-archai-adaptive-engine.hf.space/api/v1/session/start');
const data = await res.json();
sessionId = data.session_id;
showQuestion(data.question);
}
async function submitAnswer(questionId, optionIndex) {
const res = await fetch('https://Builder-Neekhil-archai-adaptive-engine.hf.space/api/v1/session/answer', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({session_id: sessionId, question_id: questionId, option_index: optionIndex})
});
const data = await res.json();
if (data.status === 'complete') {
showResults(data); // or fetch /session/{id}
} else {
showQuestion(data.question);
}
}
```
### Render Dimension Scores (Radar Chart)
The response includes:
- `dimension_scores`: `{literacy: 64, tooling: 63, ...}` β€” 0-100 for your existing radar
- `strengths` / `gaps`: Top 2 and bottom 2 with labels and colors
- `archetype`: One of 8 personas (Pioneer, Power User, etc.)
- `stage`: Awareness β†’ Understanding β†’ Application β†’ Integration β†’ Mastery
### Learning Path Rendering
The `learning_path` object contains:
- `days[0]`: Day 1 quick win (closes biggest gap in < 30 min)
- `weeks[n].actions`: Weekly milestones with deliverables
- `months[n].strategic_goals`: Month-level outcomes with metrics
- `projections`: Weeks to next stage, projected date
---
## Research Foundation
This engine implements techniques from:
- **Fluid Benchmarking** (2025, arXiv:2509.11106) β€” Fisher information adaptive selection for 50Γ— efficiency
- **Reliable Amortized Evaluation** (2025, arXiv:2503.13335) β€” IRT-based difficulty calibration
- **Deep Knowledge Tracing with Learning Curves** (2020, arXiv:2008.01169) β€” Bayesian student modeling
- **FoundationalASSIST** (2025) β€” Multi-dimensional skill assessment architecture
---
## License
MIT β€” open for integration into your webapp.