cagasoluh

/

MYRA

like 0

Stable RBM, No Collapse — So Why Invalid Compositions?

We’re working on MYRA, a system designed to answer a simple question:
What did the model actually learn?

Instead of focusing only on output quality, we analyze the internal structure learned by energy-based models and how that structure appears during generation.

Observations (SR-TRBM, PCD-1)

Across multiple seeds with fixed settings:

• No mode collapse
• Active sampling (positive flip rates)
• Stable behavior across runs

A representative run:

• Reconstruction ≈ 0.98
• Stable likelihood
• Diversity ≈ 0.21
• Entropy ≈ 0.31
• Energy gap ≈ 19.56
• Mixing τ ≈ 22.5

LLM-based interpretation

• Regime: Learning
• Phase: Ordered
• Collapse risk: ~0
• Main issue: Over-ordering tendency

Interpretation

The model learns consistent internal structures.
But during generation, these structures are recombined in ways that do not align with the dataset.

So this consequence does not look like a sampling failure.

Question

If sampling is stable and there is no collapse,
Why do we still observe distinct compositions from the statistically expected ones?

If the model gives stable learning signals (reconstruction, mixing, entropy), but the generated compositions consistently diverge from the dataset, should we interpret this as a failure or as a systematic form of expression emerging from the model?

Representative outputs are in the comments.
Full logs: artifacts/