Model

This is an dutch speaking and reading, nearly non-refusing (4/100 refusals on heretic, pre fine-tune tostideluxekaas/GEItje-7b-ultra-herretic) and uncensored fine tune of GEITje-7b-ultra-heretic, which is my ablitterated version off bramvanroy's GEITje-7b-ultra. I fine-tuned the ablitteration on dutch toxic data and quantesized them to GUFF for usability. The data is available in REPO, the SFT data contains harmfull instructions and is not intended to use unfiltered, which is why I gated it with requested ussage. The orpo set is available as it is, as it is somewhat filtered as is.

GEITje-7b-ultra by bramvanroy is an conversational model based off Mistral 7b. I experienced that multilingual/English uncensored models tend to still refuse a lot when prompting in dutch. This model is fine-tuned only on dutch (toxic/edge-case/refusal) data, to ensure uncensored allignment in the Dutch language. However the data is translated from chinese/English automatically to Dutch so grammar can be off from time to time.

The model is ablitterated using the heretic tool. After ablitteration I translated an SFT dataset, and an DPO/ORPO dataset from English to Dutch using automated translation. These datasets focus on unusual or edge case prompts. After - making these I did 1 epoch on the SFT (large) set, and I did 4 epochs on the toxic-dpo-v2.0 (small) set, using orpo . I trained in colab using an H100 and the max RAM runtime. .After I trained the Lora I merged the lora with the ablitterated to deliver a full merged uncensored fine-tune for dutch ussage. The intent of the use of toxic data is purely because ablitteration does not deliver as much when prompting in dutch. The models needed Dutch examples to come to it's full uncensored potential.

basemodel (GEITje-7b-ultra-herretic).

Ethical

Use this model responsibly. It is highly uncensored and however tone adjusted can still anwer questions containing different kinds of illegal or harmful content.The system prompt ensured total freedom of use. However I made some refusal cases invo- lving minors or instructions to fatal self-harm.

Remember that it's possible that a certain model version will not refuse these kind of prompts, so always use responsible. Make sure you filter the model pre production.

I am an fern believer that local unrestricted LLM-models outweigh possitives to neg- atives. Legal professionals, creators/artists, writers, people in oppresed regimes or people affraid of BIASED information input all share a similar opinion.

Model configuration

Field	Value
Architecture	mistral
Layers	32
Hidden size	4096
Attention heads	32
Vocab size	32000
Max context	32768
Rope theta

Weights

File	Format	Size
`model.safetensors`	safetensors	13.49 GB

Citation and special thanks to BramvanRoy, creator of the GEITje-7b-ultra base-model

@misc{vanroy2024geitje7bultraconversational,
  title        = {GEITje 7B Ultra: A Conversational Model for Dutch},
  author       = {Bram Vanroy},
  year         = {2024},
  eprint       = {2412.04092},
  archivePrefix= {arXiv},
  primaryClass = {s.CL},
  url          = {https://arxiv.org/abs/2412.04092},
}

Data

tostideluxekaas/toxic-dpo-v2.0-dutch (translated from english-dutch using AI) tostideluxekaas/sum_zh_sft_dutch (translated from chinese-english english-dutch using AI)

Disclaimer

This model is provided as-is and may generate content that some users may find offensive, disturbing, or explicit. It is not intended for minors.

Use of this model is at your own risk. You are solely responsible for any outputs and for any decisions made or actions taken based on those outputs. The authors and maintainers disclaim all warranties and liability to the maximum extent permitted by law.

This model is intended for professional, research, educational, creative, and hobbyist use (preferably in local/offline environments) and is not intended for illegal or harmful purposes.