Impressive work!

#1
by tarruda - opened

Any chance you can take a stab at the 397B version? The best heretic for 397B has 0.38 KLD: https://huggingface.co/trohrbaugh/Qwen3.5-397B-A17B-heretic

Any chance you can take a stab at the 397B version? The best heretic for 397B has 0.38 KLD: https://huggingface.co/trohrbaugh/Qwen3.5-397B-A17B-heretic

I definitely would if I could and if I could I would have already done it, the issue is:

https://huggingface.co/Qwen/Qwen3.5-397B-A17B/tree/main

This model is 807GB on it's own without taking into account the extra GBs you would need to do the uncensoring and almost nobody has the hardware at home to do a model that big, the only way to do it would be to rent GPU Cloud computing, like multiple B200 GPUs such as here:

https://www.runpod.io/gpu-models/b200

The B200 has 192GB of VRAM, that would mean for the Qwen3.5-397B-A17B I would need to rent either 5 or 6 of these, I dunno exactly how much it would cost at the end all in all but it's not gonna be cheap!

Issue is, I would need money to rent 5 or 6 of these for a few hours, issue is only 1 person donated so far on Ko-Fi: https://ko-fi.com/llmfan46 which covered 2% of the cost so far.

If I could reach enough donations/tips/subscriptions I would definitely do it.

Sign up or log in to comment