ethicalabs
/

Echo-DSRN-486M-v0.7.6-SFT

Text Generation

text-generation-inference

🇪🇺 Region: EU

Model card Files Files and versions

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

Model Card for Echo-DSRN-486M-v0.7.6-SFT

🏗️ Architecture Details

Property	Value
Model Type	echo_dsrn
Layers	19
Hidden Dim	768
Attention Heads	4
MLP Ratio	8.0
Vocab Size	32011
Hybrid Attention	True
RMSNorm	True

📊 Parameter Breakdown

Component	Parameters	% of Total
Total	486.65M (486,652,416)	100%
Embeddings	24.58M	5.05%
DSRN Blocks (Aggregate)	437.48M	89.90%
LM Head	24.58M	5.05%

🧩 Internal Block Structure (Per Layer)

Sub-Component	Parameters	Description
MLP (Feed-Forward)	9.44M	Upscaled hidden layers
DSRN Slow State	7.08M	Constant-time memory gates
GRU Fast State	3.54M	Recurrent fast path
Surprise Gating	592,896	Dynamic focus mechanism
Normalization	1,536	LayerNorm / RMSNorm

Downloads last month: 398

Safetensors

Model size

0.5B params

Tensor type

F32

·

Dataset used to train ethicalabs/Echo-DSRN-486M-v0.7.6-SFT

Collection including ethicalabs/Echo-DSRN-486M-v0.7.6-SFT

Kurtis-EON1

Language Model • 12 items • Updated Mar 16