Upload README.md

README.md

@@ -0,0 +1,80 @@
+---
+license: apache-2.0
+tags:
+- deepfake-detection
+- multimodal
+- image-classification
+- text-classification
+- ensemble
+- gradcam
+- explainability
+datasets:
+- Hemg/deepfake-and-real-images
+- artem9k/ai-text-detection-pile
+pipeline_tag: image-classification
+---
+
+# Multimodal Deepfake Detection Model
+
+A multimodal ensemble model that classifies images, video frames, and text as **real** or **AI-generated/fake**, with confidence scores and GradCAM explainability maps.
+
+## Architecture
+
+**Visual Branch**: EfficientNet-B0 (ImageNet pretrained) with L2-normalized features for image/video frame classification
+**Text Branch**: RoBERTa-base with mean pooling and MLP head for AI-generated text detection
+**Fusion Layer**: Learnable weighted late ensemble combining visual + text probabilities
+**Explainability**: GradCAM heatmaps on EfficientNet convolutional layers
+
+## Usage
+
+```python
+from inference import load_model, classify_image, classify_text, classify_video, classify_multimodal
+
+model, config = load_model('multimodal_ensemble.pt', device='cuda')
+
+# Image with GradCAM explainability
+result = classify_image(model, 'face.jpg', device='cuda', return_gradcam=True)
+print(f"Prediction: {result['prediction']} (confidence: {result['confidence']:.2%})")
+# result['gradcam'] contains the explainability heatmap
+
+# Text classification
+result = classify_text(model, 'This text was written by...')
+print(f"Prediction: {result['prediction']} (confidence: {result['confidence']:.2%})")
+
+# Video classification
+result = classify_video(model, 'video.mp4', num_frames=32, aggregation='mean')
+print(f"Video: {result['prediction']} (confidence: {result['confidence']:.2%})")
+
+# Multimodal (image + text)
+result = classify_multimodal(model, image_path_or_pil='face.jpg', text='Caption...')
+print(f"Combined: {result['prediction']} — Weights: {result['fusion_weights']}")
+```
+
+## Training
+
+### Datasets
+- **Visual**: [Hemg/deepfake-and-real-images](https://huggingface.co/datasets/Hemg/deepfake-and-real-images) — 140K+ face images (real vs deepfake)
+- **Text**: [artem9k/ai-text-detection-pile](https://huggingface.co/datasets/artem9k/ai-text-detection-pile) — 1.9GB human vs AI-generated text
+
+### Recipe
+| Component | Config |
+|-----------|--------|
+| Visual backbone | EfficientNet-B0 |
+| Visual optimizer | Adam, lr=1e-4, cosine annealing, 8 epochs |
+| Text backbone | RoBERTa-base |
+| Text optimizer | AdamW, lr=2e-5, warmup+cosine, 5 epochs |
+| Augmentations | RandomFlip, Rotation, ColorJitter, GaussianBlur, RandomErasing |
+
+### Based on Research
+- **AWARE-NET** (arxiv:2505.00312): Learnable weighted fusion
+- **CLIP Deepfake** (arxiv:2503.19683): L2-normalized feature spaces
+- **DeTeCtive** (arxiv:2410.20964): RoBERTa for AI text detection
+
+## Files
+- `model.py` — Architecture (GradCAM, EfficientNet, RoBERTa, Fusion)
+- `preprocessing.py` — Data pipeline (images, video frames, text)
+- `inference.py` — Inference API (single/modality, multimodal, video)
+- `train.py` — Training script
+
+## License
+Apache-2.0