| --- |
| license: apache-2.0 |
| tags: |
| - deepfake-detection |
| - multimodal |
| - image-classification |
| - text-classification |
| - ensemble |
| - gradcam |
| - explainability |
| datasets: |
| - Hemg/deepfake-and-real-images |
| - artem9k/ai-text-detection-pile |
| pipeline_tag: image-classification |
| --- |
| |
| # 🕵️ Multimodal Deepfake Detection Model |
|
|
| Modèle d'ensemble multimodal pour détecter les contenus générés par IA (images, vidéos, texte) avec explicabilité GradCAM. |
|
|
| ## 🏗️ Architecture |
|
|
| ``` |
| Visual Branch EfficientNet-B0/B4 ──┐ |
| ├──► Fusion pondérée ──► Confidence [0-1] |
| Text Branch RoBERTa-base ──┘ |
| ↓ |
| GradCAM Heatmap (explicabilité) |
| ``` |
|
|
| ## 🚀 Utilisation Rapide |
|
|
| ```python |
| from inference import load_model, classify_image, classify_text, classify_multimodal |
| |
| model, config = load_model('multimodal_ensemble.pt', device='cuda') |
| |
| # Image + GradCAM |
| result = classify_image(model, 'face.jpg', return_gradcam=True) |
| print(f"{result['prediction']} — confidence: {result['confidence']:.2%}") |
| # result['gradcam'] → heatmap (224, 224) |
| |
| # Texte |
| result = classify_text(model, 'Cet essai analyse les impacts...') |
| print(f"{result['prediction']} — confidence: {result['confidence']:.2%}") |
| |
| # Multimodal (image + texte) |
| result = classify_multimodal(model, image_path_or_pil='face.jpg', text='Caption...') |
| print(f"Fusion: {result['prediction']} — poids: {result['fusion_weights']}") |
| ``` |
|
|
| ## 📊 Jeux de Données |
| - **Visuel**: [Hemg/deepfake-and-real-images](https://huggingface.co/datasets/Hemg/deepfake-and-real-images) — 528K images |
| - **Texte**: [artem9k/ai-text-detection-pile](https://huggingface.co/datasets/artem9k/ai-text-detection-pile) — 1.88GB |
|
|
| ## 📁 Fichiers |
| | Fichier | Description | |
| |---------|-------------| |
| | `model.py` | Architecture complète | |
| | `preprocessing.py` | Pipeline de données | |
| | `inference.py` | API d'inférence | |
| | `train.py` / `train_optimised.py` | Scripts d'entraînement | |
| | `multimodal_ensemble.pt` | Checkpoint principal | |
| | `gradcam_examples/` | Visualisations d'explicabilité | |
|
|
| ## 🔗 Space de Démo |
| [alianassmaaa/multimodal-deepfake-space](https://huggingface.co/spaces/alianassmaaa/multimodal-deepfake-space) |
|
|
| ## 📄 Licence |
| Apache-2.0 |
|
|
