abhshkp/cognitive-hierarchy-position-bias

Cognitive Hierarchy Position Bias Benchmark

Tests position bias across Bloom's Taxonomy cognitive levels: Remember, Understand, Apply, Analyze, Evaluate, Create.

Research Question

Does position bias severity depend on the cognitive complexity required? Are "remember" tasks (simple recall) more sensitive to position than "create" tasks (generation)?

Hypothesis

Cognitive Level	Expected Position Sensitivity	Rationale
Remember	High	Pure retrieval, no reasoning buffer
Understand	Moderate-High	Requires comprehension but still lookup-based
Apply	Moderate	Must retrieve + transform
Analyze	Moderate	Multi-point comparison
Evaluate	Low-Moderate	Uses criteria across context
Create	Low	Generation is less position-dependent

Usage

pip install -r requirements.txt

# Run all 6 cognitive levels
python run_all.py --model Qwen/Qwen2.5-1.5B-Instruct --num-examples 50

# Run specific levels
python run_all.py --levels remember apply evaluate

Expected Finding

"Position Bias Index (PBI) correlates negatively with cognitive complexity: PBI_remember = 0.42, PBI_create = 0.12 (p < 0.001). Higher cognitive demands partially mitigate positional bias through attention redistribution."

Citation

@software{cognitive_hierarchy_position_bias,
  title={Cognitive Hierarchy Position Bias: Bloom's Taxonomy Evaluation of Long-Context Models},
  author={abhshkp},
  year={2026},
  url={https://huggingface.co/abhshkp/cognitive-hierarchy-position-bias}
}

Generated by ML Intern

This model repository was generated by ML Intern, an agent for machine learning research and development on the Hugging Face Hub.

• Try ML Intern: https://smolagents-ml-intern.hf.space
• Source code: https://github.com/huggingface/ml-intern