darkc0de commited on
Commit
668ccdd
·
verified ·
1 Parent(s): 25ecaeb

Delete README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -285
README.md DELETED
@@ -1,285 +0,0 @@
1
- ---
2
- license: apache-2.0
3
- base_model: mistralai/Mistral-Medium-3.5-128B
4
- tags:
5
- - mistral
6
- - mistral-3.5
7
- - text-only
8
- - bf16
9
- - 128b
10
- - heretic
11
- - uncensored
12
- - decensored
13
- - abliterated
14
- ---
15
- # This is a decensored version of [Darkhn/Mistral-Medium-3.5-128B-BF16-Text-Only](https://huggingface.co/Darkhn/Mistral-Medium-3.5-128B-BF16-Text-Only), made using [Heretic](https://github.com/p-e-w/heretic) v1.2.0
16
-
17
- ![{48AE4B0C-76C4-4380-9A1F-8AFF1FABEEB4}](https://cdn-uploads.huggingface.co/production/uploads/6540a02d1389943fef4d2640/rWraMJXrWTdp4xSzleg5l.png)
18
-
19
- ## Abliteration parameters
20
-
21
- | Parameter | Value |
22
- | :-------- | :---: |
23
- | **direction_index** | 43.15 |
24
- | **attn.o_proj.max_weight** | 1.48 |
25
- | **attn.o_proj.max_weight_position** | 59.65 |
26
- | **attn.o_proj.min_weight** | 1.44 |
27
- | **attn.o_proj.min_weight_distance** | 48.02 |
28
- | **mlp.down_proj.max_weight** | 1.21 |
29
- | **mlp.down_proj.max_weight_position** | 54.75 |
30
- | **mlp.down_proj.min_weight** | 0.30 |
31
- | **mlp.down_proj.min_weight_distance** | 50.44 |
32
-
33
- ## Performance
34
-
35
- | Metric | This model | Original model ([Darkhn/Mistral-Medium-3.5-128B-BF16-Text-Only](https://huggingface.co/Darkhn/Mistral-Medium-3.5-128B-BF16-Text-Only)) |
36
- | :----- | :--------: | :---------------------------: |
37
- | **KL divergence** | 0.0220 | 0 *(by definition)* |
38
- | **Refusals** | 9/100 | 98/100 |
39
-
40
- -----
41
-
42
-
43
- <style>
44
- body {
45
- font-family: 'Quicksand', sans-serif;
46
- background: linear-gradient(135deg, #4a1e00 0%, #1c0a00 100%);
47
- color: #F5EFE6;
48
- margin: 0;
49
- padding: 0;
50
- font-size: 16px;
51
- }
52
-
53
- h1, h2, h3, h4, summary {
54
- font-family: 'Cinzel', serif;
55
- }
56
-
57
- .container {
58
- margin: 20px auto;
59
- max-width: 900px;
60
- background-color: rgba(28, 22, 18, 0.95);
61
- padding: 30px;
62
- border-radius: 12px;
63
- box-shadow: 0 4px 20px rgba(255, 140, 0, 0.15);
64
- border: 1px solid rgba(255, 140, 0, 0.2);
65
- outline: 1px solid rgba(255, 140, 0, 0.5);
66
- outline-offset: -1px;
67
- position: relative;
68
- }
69
-
70
- .container::before {
71
- content: '';
72
- position: absolute;
73
- top: -1px;
74
- left: -1px;
75
- right: -1px;
76
- bottom: -1px;
77
- border: 1px solid rgba(255, 165, 0, 0.98);
78
- border-radius: 12px;
79
- pointer-events: none;
80
- animation: borderGlow 2.5s ease-in-out infinite;
81
- }
82
-
83
- @keyframes borderGlow {
84
- 0% { box-shadow: 0 0 5px rgba(255, 165, 0, 0.98); }
85
- 50% { box-shadow: 0 0 12px rgba(255, 165, 0, 0.98); }
86
- 100% { box-shadow: 0 0 5px rgba(255, 165, 0, 0.98); }
87
- }
88
-
89
- .header h1 {
90
- font-size: 32px;
91
- color: #FFA500;
92
- margin: 0 0 20px 0;
93
- text-align: center;
94
- text-shadow: 0 0 12px rgba(255, 100, 0, 0.6);
95
- }
96
-
97
- a {
98
- color: #FFD700;
99
- text-decoration: none;
100
- transition: color 0.3s ease;
101
- }
102
-
103
- .button {
104
- display: inline-block;
105
- background-color: #E55B00;
106
- color: #FFFFFF;
107
- padding: 12px 24px;
108
- border-radius: 5px;
109
- cursor: pointer;
110
- text-decoration: none;
111
- font-family: 'Cinzel', serif;
112
- font-weight: 600;
113
- transition: all 0.3s ease;
114
- border: 1px solid transparent;
115
- }
116
-
117
- .button:hover {
118
- background-color: #FF8C00;
119
- box-shadow: 0 0 15px rgba(255, 140, 0, 0.5);
120
- transform: translateY(-2px);
121
- }
122
-
123
- .info-card {
124
- background: rgba(45, 35, 25, 0.95);
125
- border: 1px solid rgba(255, 140, 0, 0.2);
126
- border-radius: 8px;
127
- overflow: hidden;
128
- margin-bottom: 25px;
129
- }
130
-
131
- .info-header {
132
- background: rgba(255, 140, 0, 0.1);
133
- padding: 20px;
134
- border-bottom: 1px solid rgba(255, 140, 0, 0.2);
135
- }
136
-
137
- .info-header h3 {
138
- color: #FFA500;
139
- margin: 0 0 10px 0;
140
- font-size: 22px;
141
- }
142
-
143
- .card-content {
144
- padding: 20px;
145
- line-height: 1.7;
146
- }
147
-
148
- .card-content ul {
149
- list-style: none;
150
- padding-left: 20px;
151
- }
152
-
153
- .card-content li::before {
154
- content: '✦';
155
- color: #FFD700;
156
- font-weight: bold;
157
- display: inline-block;
158
- width: 1em;
159
- margin-left: -1.2em;
160
- }
161
-
162
- .card-content strong {
163
- color: #FFD700;
164
- }
165
-
166
- /* Update to the note card */
167
- .note-card {
168
- border: 1px solid #FFA500;
169
- box-shadow: 0 0 10px rgba(255, 165, 0, 0.1);
170
- }
171
-
172
- .note-header {
173
- background: rgba(255, 165, 0, 0.1);
174
- }
175
-
176
- .note-header h3 {
177
- color: #FFA500;
178
- text-align: center;
179
- }
180
-
181
- .support-section {
182
- text-align: center;
183
- margin-top: 40px;
184
- background: rgba(45, 35, 25, 0.95);
185
- border: 1px solid rgba(255, 140, 0, 0.2);
186
- border-radius: 8px;
187
- padding: 20px;
188
- }
189
-
190
- summary {
191
- cursor: pointer;
192
- list-style: none;
193
- outline: none;
194
- display: flex;
195
- align-items: center;
196
- }
197
-
198
- summary::before {
199
- content: '▶';
200
- font-size: 1.2em;
201
- color: #FFA500;
202
- margin-right: 15px;
203
- transition: transform 0.2s ease;
204
- }
205
-
206
- details[open] > summary::before {
207
- transform: rotate(90deg);
208
- }
209
-
210
- h2 {
211
- color: #FFA500;
212
- border-bottom: 1px solid rgba(255, 140, 0, 0.2);
213
- padding-bottom: 10px;
214
- margin-bottom: 20px;
215
- }
216
- </style>
217
-
218
- <div class="container">
219
- <link href="https://fonts.googleapis.com/css2?family=Cinzel:wght@400;500;600&family=Quicksand:wght@400;500&display=swap" rel="stylesheet">
220
-
221
- <div class="header">
222
- <h1>Mistral-Medium-3.5-128B-BF16-Text-Only</h1>
223
- </div>
224
-
225
- <div class="info">
226
-
227
- <div class="info-card note-card">
228
- <div class="info-header note-header">
229
- <h3>📜 Technical Architecture Note</h3>
230
- </div>
231
- <div class="card-content">
232
- <p>This model has been converted from <strong>Mistral3ForConditionalGeneration</strong> (Multimodal) to <strong>MistralForCausalLM</strong> (Standard Text-Only). This change ensures maximum compatibility with standard fine-tuning libraries like <em>Axolotl, Unsloth, and Hugging Face Transformers</em> without requiring custom vision-encoder handling.</p>
233
- </div>
234
- </div>
235
-
236
- <div class="support-section">
237
- <p><strong>Help me feed the data beast! Taking commissions for universe-specific models.</strong></p>
238
- <a href="https://ko-fi.com/som1tokmynam" target="_blank" class="button">
239
- Support on Ko-fi
240
- </a>
241
- </div>
242
-
243
- <div class="section-container">
244
- <details open>
245
- <summary><h2>Model Description</h2></summary>
246
- <div class="info-card">
247
- <div class="card-content">
248
- <p>This is a processed version of <strong>Mistral-Medium-3.5-128B</strong> designed for users who prioritize text-only performance and ease of fine-tuning.</p>
249
- <p><strong>Modification Details:</strong></p>
250
- <ul>
251
- <li><strong>Precision Upscale:</strong> Converted from <strong>FP8</strong> weights to <strong>BF16</strong> to restore full 16-bit brain-float precision for stable gradient updates during training.</li>
252
- <li><strong>Vision Layer Stripping:</strong> All vision encoders and multimodal projection layers have been removed, significantly reducing memory overhead during inference and training for text-only tasks.</li>
253
- <li><strong>Architecture Re-mapping:</strong> The configuration has been modified to use <code>MistralForCausalLM</code>, allowing it to be treated as a standard dense language model.</li>
254
- </ul>
255
- </div>
256
- </div>
257
- </details>
258
- </div>
259
-
260
- <div class="section-container">
261
- <details>
262
- <summary><h2>Purpose & Usage</h2></summary>
263
- <div class="info-card">
264
- <div class="card-content">
265
- <p>This model is intended to serve as a <strong>clean base for fine-tuning</strong>. By removing the vision components, you can allocate more VRAM to sequence length or batch size. It is 100% functional for text-only chat and reasoning out of the box.</p>
266
- </div>
267
- </div>
268
- </details>
269
- </div>
270
-
271
- <div class="section-container">
272
- <details>
273
- <summary><h2>Acknowledgements</h2></summary>
274
- <div class="info-card">
275
- <div class="card-content">
276
- <ul>
277
- <li>Credit to <strong>Mistral AI</strong> for the original Mistral-Medium-3.5-128B architecture.</li>
278
- </ul>
279
- </div>
280
- </div>
281
- </details>
282
- </div>
283
-
284
- </div>
285
- </div>