MYRA: SR-TRBM with LLM-Guided Refinement and Analysis

Hybrid energy-based RBM with LLM-guided structural refinement

🧠 Core Idea

What did the model actually learn?

MYRA is a hybrid framework for analyzing and refining learned representations in energy-based models, particularly RBMs.

Most models are optimized for output quality. MYRA focuses instead on the internal structure of what is learned. Rather than only evaluating generated samples, MYRA investigates how learned patterns are organized, combined, and expressed during generation.

⚙️ Model Overview

MYRA combines:

a Restricted Boltzmann Machine (RBM) for generation
an LLM-based interpretive layer for structural analysis
an energy-based acceptance mechanism for refinement

The system operates as a loop:

RBM generates samples
LLM analyzes structure and proposes refinements
Changes are accepted or rejected based on energy
The process repeats

This forms a guided generative refinement process.

🔍 LLM Integration

MYRA uses an LLM as an external interpretive layer.

The LLM is not used for generation. It analyzes model behavior, evaluates structure, and suggests refinements during the iterative loop.

⚙️ Quick Start (Default Backend)

The current setup uses the OpenAI API for fast and minimal setup.

You can run the system immediately without modifying the backend.

🔁 Backend Flexibility

The LLM layer is modular.

The default implementation (openaiF) can be replaced or extended to support other providers such as:

Anthropic (Claude)
Google (Gemini)
Meta (Llama / local models)
Mistral, DeepSeek, Qwen

Switching backends typically requires only small changes in:

client.py
__init__.py
import references in srtrbm_project_core.py

🔗 Repository

Full implementation and backend details:

👉 https://github.com/cagasolu/srtrbm-llm-hybrid

The LLM acts as an interpretive layer, not a source of ground truth.

🧪 Key Observation

In practice, we observe:

stable sampling without collapse
consistent pattern recombination across different seeds
outputs that are structurally coherent but not present in the dataset

This suggests a gap between learned structure and generated outputs.

⚙️ Installation

Recommended environment

Ubuntu 22.04 LTS
CUDA 12.x
PyTorch 2.x

pip install -r requirements.txt

System Overview of MYRA

MYRA
└── SR-TRBM (Energy-Based Generator)
    └── Refinement (Structural + Embedding)
        └── LLM
            └── Interpretation & Analysis
                └── Final Output   ← this model

Architecture

MYRA combines three main components:

SR-TRBM → energy-based generative model
MYRA complex refinement → structural correction via embedding matching
LLM layer → interpretation and convergence analysis

Project Structure

🧠 Core Engine
└─ srtrbm_project_core.py
↳ Energy-based generation (SR-TRBM)
↳ Gibbs sampling & thermodynamic dynamics

🤖 LLM Integration
└─ openaiF/
├─ client.py → Robust LLM client (retry, fallback)
└─ gateway.py → Interpretation & reasoning layer └─ hook.py → Epistemic control and decision layer

🧩 Refinement System
├─ supplement/cluster.py → Embedding-based matching
└─ correction/ → Energy-aware & spatial refinement

⚙️ Configuration
└─ yaml/ → LLM policies & guidance rules

📊 Analysis & Metrics
└─ analysis/
↳ Energy tracking, LPIPS, convergence

📈 Visualization
└─ graphs/
↳ Training curves & energy landscapes

📦 Assets
├─ zeta_mnist_hybrid.pt → Pretrained model
└─ stan.dgts → Dataset

🧪 Outputs
└─ artifacts/
↳ Generated samples & logs

How It Works

RBM generates initial samples
LLM proposes structural edits (pixel-level)
Edits are evaluated using energy difference (ΔE)
Accepted edits refine the sample

This can be interpreted as:

Learned MCMC proposal distribution guided by a language model

Results

Reconstruction Accuracy: ~0.98
LPIPS: ~0.15
Stable energy dynamics
Low collapse risk

Uses

Direct Use

Generating structured digit samples
Studying hybrid energy-based + LLM systems

Research Use

Learned proposal distributions
Energy-guided refinement
Hybrid generative modeling

Limitations

Reduced sample diversity under strong refinement
Sensitive to acceptance scaling
Depends on LLM consistency

Training Details

Training Data

Fashion-MNIST (784-dimensional)

Training Procedure

RBM trained via contrastive divergence
Refinement applied post-generation

Evaluation

Metrics

Reconstruction MSE
LPIPS (perceptual similarity)
Energy gap
Sample diversity

Technical Insight

The system bridges:

Energy-based modeling (RBM)
Semantic correction (LLM)

Resulting in a:

Memory-augmented, energy-aware refinement system

Files

artifacts/ → generated samples and logs
srtrbm_project_core.py → main implementation

Citation

cff-version: 1.2.0
title: "MYRA: SR-TRBM with LLM-Guided Refinement"
version: "v1.0.1"
date-released: 2026-03-25

authors:
  - given-names: "Görkem Can"
    family-names: "Süleymanoğlu"

identifiers:
  - type: doi
    value: "10.5281/zenodo.19211121"

links:
  - type: repository
    url: "https://github.com/cagasolu/srtrbm-llm-hybrid"
  - type: model
    url: "https://huggingface.co/cagasoluh/MYRA"

keywords:
  - energy-based-models
  - rbm
  - llm
  - hybrid-ai
  - generative-model

Contact

Maintained by: Görkem Can Süleymanoğlu

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

Unconditional Image Generation

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

cagasoluh
/

MYRA