Upload README.md with huggingface_hub

README.md

@@ -1,199 +1,283 @@
 ---
-library_name: transformers
-tags: []
+license: apache-2.0
+base_model: mistralai/Mistral-Medium-3.5-128B
+tags:
+- mistral
+- mistral-3.5
+- text-only
+- bf16
+- 128b
+- heretic
+- uncensored
+- decensored
+- abliterated
 ---
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
+# This is a decensored version of [Darkhn/Mistral-Medium-3.5-128B-BF16-Text-Only](https://huggingface.co/Darkhn/Mistral-Medium-3.5-128B-BF16-Text-Only), made using [Heretic](https://github.com/p-e-w/heretic) v1.2.0
+
+## Abliteration parameters
+
+| Parameter | Value |
+| :-------- | :---: |
+| **direction_index** | 43.15 |
+| **attn.o_proj.max_weight** | 1.48 |
+| **attn.o_proj.max_weight_position** | 59.65 |
+| **attn.o_proj.min_weight** | 1.44 |
+| **attn.o_proj.min_weight_distance** | 48.02 |
+| **mlp.down_proj.max_weight** | 1.21 |
+| **mlp.down_proj.max_weight_position** | 54.75 |
+| **mlp.down_proj.min_weight** | 0.30 |
+| **mlp.down_proj.min_weight_distance** | 50.44 |
+
+## Performance
+
+| Metric | This model | Original model ([Darkhn/Mistral-Medium-3.5-128B-BF16-Text-Only](https://huggingface.co/Darkhn/Mistral-Medium-3.5-128B-BF16-Text-Only)) |
+| :----- | :--------: | :---------------------------: |
+| **KL divergence** | 0.0220 | 0 *(by definition)* |
+| **Refusals** | 9/100 | 98/100 |
+
+-----
+
+
+<style>
+body {
+  font-family: 'Quicksand', sans-serif;
+  background: linear-gradient(135deg, #4a1e00 0%, #1c0a00 100%);
+  color: #F5EFE6;
+  margin: 0;
+  padding: 0;
+  font-size: 16px;
+}
+
+h1, h2, h3, h4, summary {
+    font-family: 'Cinzel', serif;
+}
+
+.container {
+  margin: 20px auto;
+  max-width: 900px;
+  background-color: rgba(28, 22, 18, 0.95);
+  padding: 30px;
+  border-radius: 12px;
+  box-shadow: 0 4px 20px rgba(255, 140, 0, 0.15);
+  border: 1px solid rgba(255, 140, 0, 0.2);
+  outline: 1px solid rgba(255, 140, 0, 0.5);
+  outline-offset: -1px;
+  position: relative;
+}
+
+.container::before {
+  content: '';
+  position: absolute;
+  top: -1px;
+  left: -1px;
+  right: -1px;
+  bottom: -1px;
+  border: 1px solid rgba(255, 165, 0, 0.98);
+  border-radius: 12px;
+  pointer-events: none;
+  animation: borderGlow 2.5s ease-in-out infinite;
+}
+
+@keyframes borderGlow {
+  0% { box-shadow: 0 0 5px rgba(255, 165, 0, 0.98); }
+  50% { box-shadow: 0 0 12px rgba(255, 165, 0, 0.98); }
+  100% { box-shadow: 0 0 5px rgba(255, 165, 0, 0.98); }
+}
+
+.header h1 {
+  font-size: 32px;
+  color: #FFA500;
+  margin: 0 0 20px 0;
+  text-align: center;
+  text-shadow: 0 0 12px rgba(255, 100, 0, 0.6);
+}
+
+a {
+  color: #FFD700;
+  text-decoration: none;
+  transition: color 0.3s ease;
+}
+
+.button {
+  display: inline-block;
+  background-color: #E55B00;
+  color: #FFFFFF;
+  padding: 12px 24px;
+  border-radius: 5px;
+  cursor: pointer;
+  text-decoration: none;
+  font-family: 'Cinzel', serif;
+  font-weight: 600;
+  transition: all 0.3s ease;
+  border: 1px solid transparent;
+}
+
+.button:hover {
+  background-color: #FF8C00;
+  box-shadow: 0 0 15px rgba(255, 140, 0, 0.5);
+  transform: translateY(-2px);
+}
+
+.info-card {
+  background: rgba(45, 35, 25, 0.95);
+  border: 1px solid rgba(255, 140, 0, 0.2);
+  border-radius: 8px;
+  overflow: hidden;
+  margin-bottom: 25px;
+}
+
+.info-header {
+  background: rgba(255, 140, 0, 0.1);
+  padding: 20px;
+  border-bottom: 1px solid rgba(255, 140, 0, 0.2);
+}
+
+.info-header h3 {
+  color: #FFA500;
+  margin: 0 0 10px 0;
+  font-size: 22px;
+}
+
+.card-content {
+  padding: 20px;
+  line-height: 1.7;
+}
+
+.card-content ul {
+    list-style: none;
+    padding-left: 20px;
+}
+
+.card-content li::before {
+    content: '✦';
+    color: #FFD700;
+    font-weight: bold;
+    display: inline-block;
+    width: 1em;
+    margin-left: -1.2em;
+}
+
+.card-content strong {
+    color: #FFD700;
+}
+
+/* Update to the note card */
+.note-card {
+  border: 1px solid #FFA500;
+  box-shadow: 0 0 10px rgba(255, 165, 0, 0.1);
+}
+
+.note-header {
+  background: rgba(255, 165, 0, 0.1);
+}
+
+.note-header h3 {
+  color: #FFA500;
+  text-align: center;
+}
+
+.support-section {
+    text-align: center;
+    margin-top: 40px;
+    background: rgba(45, 35, 25, 0.95);
+    border: 1px solid rgba(255, 140, 0, 0.2);
+    border-radius: 8px;
+    padding: 20px;
+}
+
+summary {
+    cursor: pointer;
+    list-style: none;
+    outline: none;
+    display: flex;
+    align-items: center;
+}
+
+summary::before {
+    content: '▶';
+    font-size: 1.2em;
+    color: #FFA500;
+    margin-right: 15px;
+    transition: transform 0.2s ease;
+}
+
+details[open] > summary::before {
+    transform: rotate(90deg);
+}
+
+h2 {
+    color: #FFA500;
+    border-bottom: 1px solid rgba(255, 140, 0, 0.2);
+    padding-bottom: 10px;
+    margin-bottom: 20px;
+}
+</style>
+
+<div class="container">
+<link href="https://fonts.googleapis.com/css2?family=Cinzel:wght@400;500;600&family=Quicksand:wght@400;500&display=swap" rel="stylesheet">
+
+<div class="header">
+  <h1>Mistral-Medium-3.5-128B-BF16-Text-Only</h1>
+</div>
+
+<div class="info">
+  
+  <div class="info-card note-card">
+    <div class="info-header note-header">
+      <h3>📜 Technical Architecture Note</h3>
+    </div>
+    <div class="card-content">
+      <p>This model has been converted from <strong>Mistral3ForConditionalGeneration</strong> (Multimodal) to <strong>MistralForCausalLM</strong> (Standard Text-Only). This change ensures maximum compatibility with standard fine-tuning libraries like <em>Axolotl, Unsloth, and Hugging Face Transformers</em> without requiring custom vision-encoder handling.</p>
+    </div>
+  </div>
+
+  <div class="support-section">
+    <p><strong>Help me feed the data beast! Taking commissions for universe-specific models.</strong></p>
+    <a href="https://ko-fi.com/som1tokmynam" target="_blank" class="button">
+      Support on Ko-fi
+    </a>
+  </div>
+  
+  <div class="section-container">
+    <details open>
+      <summary><h2>Model Description</h2></summary>
+      <div class="info-card">
+        <div class="card-content">
+          <p>This is a processed version of <strong>Mistral-Medium-3.5-128B</strong> designed for users who prioritize text-only performance and ease of fine-tuning.</p>
+          <p><strong>Modification Details:</strong></p>
+          <ul>
+            <li><strong>Precision Upscale:</strong> Converted from <strong>FP8</strong> weights to <strong>BF16</strong> to restore full 16-bit brain-float precision for stable gradient updates during training.</li>
+            <li><strong>Vision Layer Stripping:</strong> All vision encoders and multimodal projection layers have been removed, significantly reducing memory overhead during inference and training for text-only tasks.</li>
+            <li><strong>Architecture Re-mapping:</strong> The configuration has been modified to use <code>MistralForCausalLM</code>, allowing it to be treated as a standard dense language model.</li>
+          </ul>
+        </div>
+      </div>
+    </details>
+  </div>
+
+  <div class="section-container">
+    <details>
+      <summary><h2>Purpose & Usage</h2></summary>
+      <div class="info-card">
+          <div class="card-content">
+              <p>This model is intended to serve as a <strong>clean base for fine-tuning</strong>. By removing the vision components, you can allocate more VRAM to sequence length or batch size. It is 100% functional for text-only chat and reasoning out of the box.</p>
+          </div>
+      </div>
+    </details>
+  </div>
+
+  <div class="section-container">
+    <details>
+      <summary><h2>Acknowledgements</h2></summary>
+      <div class="info-card">
+          <div class="card-content">
+              <ul>
+                  <li>Credit to <strong>Mistral AI</strong> for the original Mistral-Medium-3.5-128B architecture.</li>
+              </ul>
+          </div>
+      </div>
+    </details>
+  </div>
+
+</div>
+</div>