Donnyoregon/Demo.gguf

GGUF Model File Vulnerability - Proof of Concept

Overview

This repository contains a proof-of-concept (PoC) demonstrating a vulnerability in the GGUF model file format that allows for arbitrary code execution during model loading.

Vulnerability Details

Affected Format: .gguf (GGML Universal Format)

Vulnerability Type: Arbitrary Code Execution (ACE) through file format manipulation

Impact: Critical - Remote Code Execution during model loading

Technical Description

Root Cause

The vulnerability exists in the GGUF file format's metadata parsing mechanism. By crafting malicious metadata headers with specific key-value pairs, an attacker can inject executable code that gets executed during the model loading process.

Attack Vector

1. Malicious metadata is embedded in the GGUF file header
2. When the model is loaded by compatible libraries (e.g., llama.cpp, transformers), the metadata parser executes the embedded code
3. This occurs before any model validation or safety checks

Exploitation Method

The PoC model file (malicious_model.gguf) contains:

• Standard GGUF header structure
• Crafted metadata with embedded Python bytecode
• Trigger mechanism that executes during load_model() operations

Proof of Concept

Files in This Repository

• malicious_model.gguf - The crafted malicious model file
• exploit.py - Script demonstrating the vulnerability
• create_payload.py - Script used to generate the malicious model
• requirements.txt - Dependencies needed for reproduction

Reproduction Steps

1. Setup Environment:

   pip install -r requirements.txt
   

2. Download the Model:

   git clone https://huggingface.co/Donnyoregon/Demo.gguf
   cd Demo.gguf
   

3. Execute the Exploit:

   python exploit.py
   

4. Expected Result:

   • The script will attempt to load the model
   • Malicious code will execute, demonstrating RCE
   • You should see evidence of code execution in the console output

Code Execution Demonstration

The PoC demonstrates:

• File system access (creates a file named pwned.txt)
• Network access (attempts to connect to external server)
• System command execution (runs whoami command)

Security Impact

Immediate Risks

• Remote Code Execution: Any application loading the model gets compromised
• Data Exfiltration: Malicious models can steal sensitive data
• System Compromise: Full system access possible depending on execution context
• Supply Chain Attack: Malicious models can be distributed through model hubs

Affected Software

• llama.cpp and derivatives
• Transformers library with GGUF support
• Any application that loads GGUF files without proper validation

Scanner Bypass

ProtectAI Bypass Techniques

This PoC specifically targets bypassing ProtectAI's safety scanning by:

1. Metadata Obfuscation: Hiding payload in seemingly benign metadata fields
2. Delayed Execution: Code executes only during specific loading conditions
3. Format Confusion: Exploiting parser inconsistencies between scanners and loaders

Evasion Methods

• Payload is not stored in obvious locations (not in tensor data)
• Uses legitimate GGUF structure to avoid format validation errors
• Employs encoding techniques to hide malicious content from static analysis

Vulnerability Timeline

• Discovery Date: [Insert date]
• PoC Development: [Insert date]
• Huntr Submission: [Insert date]
• Status: Under review

Remediation

For Library Maintainers

1. Implement strict metadata validation
2. Sanitize all user-controlled data in GGUF headers
3. Add sandboxing for model loading operations
4. Implement content security policies for model files

For Users

1. Only load models from trusted sources
2. Use containerized environments for model loading
3. Implement network isolation for model inference
4. Regularly update model loading libraries

Responsible Disclosure

This vulnerability has been reported through Huntr's official channels. Please do not use this PoC for malicious purposes. The demonstration is provided solely for security research and defensive purposes.

Technical Appendix

GGUF Format Background

GGUF (GGML Universal Format) is a binary format for storing large language models. It includes:

• Header with magic bytes and version
• Metadata section with key-value pairs
• Tensor information and data

Vulnerability Location

The vulnerability exists in the metadata parsing logic, specifically in how key-value pairs are processed and interpreted during model loading.

Payload Construction

The malicious payload is constructed by:

1. Creating a valid GGUF header
2. Injecting malicious metadata with specific keys
3. Encoding the payload to avoid detection
4. Ensuring the file remains a valid GGUF structure

Contact

For questions about this vulnerability report, please contact through Huntr's official channels.

Do not use this PoC for malicious purposes. This is for security research only.