README.md

---
license: mit
---
🧾 Model Card — VideoMAE-DeepFake-Detector-v1
🧠 Model Overview

VideoMAE-DeepFake-Detector-v1 is a fine-tuned video deepfake detection model trained to distinguish between authentic and manipulated facial videos. The model builds upon the pretrained VideoMAE architecture and adapts it for binary classification of real versus synthetic videos.

The base model was originally trained on large-scale video action datasets, enabling strong spatiotemporal feature understanding. It was further fine-tuned on the FaceForensics++ dataset to detect visual artifacts, temporal inconsistencies, and manipulation signatures commonly found in deepfake videos.

By leveraging transformer-based video representation learning, the model captures both frame-level visual cues and motion patterns across time, allowing it to identify subtle manipulations that traditional image-based detectors may miss.

The model is designed for applications in media verification, misinformation detection, and AI-generated content monitoring.

🏗️ Training Details

Base Model:
MCG-NJU/videomae-base-finetuned-kinetics

Framework:
Hugging Face Transformers + PyTorch

Training Hardware:
NVIDIA T4 GPU (Kaggle)

Epochs:
15

Batch Size:
4

Learning Rate:
2e-5

Optimizer:
AdamW

Video Sampling:
16 frames per video clip

Resolution:
224 × 224

Training Strategy:

Transfer learning with partial freezing:

~70% of VideoMAE backbone layers frozen

Final transformer layers + classifier head fine-tuned

Dataset:
FaceForensics++ (C23 compression level)

Classes:

🟢 Real Video
🔴 Deepfake Video

📊 Dataset Description

The model was trained using the FaceForensics++ dataset, a widely used benchmark for deepfake detection research.

FaceForensics++ contains manipulated videos generated using multiple facial manipulation techniques, including deepfake generation and facial reenactment.

For this model version, training used a subset consisting of:

Original videos (real)

Deepfakes manipulation videos (fake)

Each video was processed by sampling 16 frames uniformly across its duration to capture both spatial and temporal artifacts.

Label	Description
Real	Authentic unmodified video
Fake	Video manipulated using deepfake synthesis techniques
🎯 Evaluation Metrics

Evaluation was performed on a held-out validation split of the dataset.

Metric	Score
Train Loss	0.303
Validation Loss	0.506
Accuracy	88.0%
F1 Score	0.742
AUC	0.836

✅ The model demonstrates strong ability to distinguish between authentic and manipulated videos using temporal visual patterns.

💬 Example Usage
import torch
import numpy as np
from decord import VideoReader, cpu
from PIL import Image
from transformers import VideoMAEForVideoClassification, VideoMAEImageProcessor

model = VideoMAEForVideoClassification.from_pretrained(
    "your_username/videomae-deepfake-detector"
)

processor = VideoMAEImageProcessor.from_pretrained(
    "your_username/videomae-deepfake-detector"
)

def load_video_frames(video_path, num_frames=16):
    vr = VideoReader(video_path, ctx=cpu(0))
    total_frames = len(vr)

    indices = np.linspace(0, total_frames - 1, num_frames).astype(int)
    frames = vr.get_batch(indices).asnumpy()

    return [Image.fromarray(f) for f in frames]

@torch.no_grad()
def predict(video_path):
    frames = load_video_frames(video_path)
    inputs = processor(frames, return_tensors="pt")

    outputs = model(**inputs)
    probs = torch.softmax(outputs.logits, dim=1)[0]

    return {
        "real": float(probs[0]),
        "fake": float(probs[1])
    }

print(predict("sample_video.mp4"))

Output example:

{'real': 0.96, 'fake': 0.04}
🧩 Intended Use

Deepfake detection in video content
Media authenticity verification
AI-generated video detection pipelines
Research on manipulated media detection
Integration into misinformation monitoring systems

⚠️ Limitations

The model was trained on a subset of FaceForensics++ and may not generalize perfectly to unseen deepfake generation techniques.

Performance may degrade on:

heavily compressed social media videos

unseen manipulation methods

partial face occlusions

extremely short clips

This model should be used as an assistive forensic tool, not as a definitive authenticity guarantee.

🧑‍💻 Developer

Author: Vansh Momaya
Institution: D. J. Sanghvi College of Engineering
Focus Area: Computer Vision, AI Safety, Deepfake Detection, Video Understanding

Email: vanshmomaya9@gmail.com

🌍 Citation

If you use this model in research or projects:

@online{momaya2025videomaedeepfake,
  author       = {Vansh Momaya},
  title        = {VideoMAE-DeepFake-Detector-v1},
  year         = {2025},
  version      = {v1},
  url          = {https://huggingface.co/Vansh180/VideoMae-deepfake-detector},
  institution  = {D. J. Sanghvi College of Engineering},
  note         = {Fine-tuned VideoMAE model for detecting deepfake videos using FaceForensics++},
  license      = {MIT}
}
🚀 Acknowledgements

VideoMAE — Base architecture for video representation learning
FaceForensics++ — Deepfake detection dataset benchmark
Hugging Face Transformers — Training and deployment framework