philipp-zettl/inaturalist-tool
2.96 MB
PZ
philipp-zettl
AI & ML interests
NLP/CV/Multimodal learning
Recent Activity
upvoted a collection 2 days ago
deployed-models repliedto their post 4 days ago
I've been cooking something neat over the past weeks 👨🍳
We all know that training LLMs requires a lot of resources and especially a lot of compute in form of GPUs, or is super slow and inefficient when done on CPUs.
The big players use giant clusters of Nvidia H100s.
But if I look at the profiles of my fellow home brewers, all we can get our hands on are those pesky consumer RTX's. If you're lucky you got yourself a 5080 with 16GB VRAM or something.
To be frank, I don't have that 1.3k disposable cash laying around ¯\_(ツ)_/¯
But I can write rust and like building ML libraries.
So I asked myself the question(s):
- can I train SMLs at home on my hardware?
- How hard can it be to build a ML library that can stream data between RAM and VRAM on demand, like llama.cpp's unified memory feature [^1]?
- how hard can it be to implement bf16 support?
The answers are wild, trust me!
Image 1: Metrics form last nights build on my "tiny" RTX 2060 (6 GB VRAM)
Image 2: Metrics from my most recent build on my RTX 4070 Laptop (8GB VRAM)
The majority of my time went into the shared memory, but it's stable and I'm very excited!
Here some debug logs, a la "trust me bro"
```
----
Currently available: 1112735744, attempting to reclaim: 1073741824
--- VRAM STATE [backward pass] ---
Driver Used: 6744 MB / 7805 MB
Data on GPU: 1641 MB
Grads on GPU: 3459 MB
CPU Offloaded: 18230 MB
---------------------------------
Currently available: 1079181312, attempting to reclaim: 1073741824
--- VRAM STATE [backward pass] ---
Driver Used: 6776 MB / 7805 MB
Data on GPU: 1561 MB
Grads on GPU: 3279 MB
CPU Offloaded: 18590 MB
-----------------------------
```
Final models get exported in `safetensors` format and are compatible with PyTorch and `transformers`, for accessibility.
- [^1]: https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#unified-memory posted an update 4 days ago
I've been cooking something neat over the past weeks 👨🍳
We all know that training LLMs requires a lot of resources and especially a lot of compute in form of GPUs, or is super slow and inefficient when done on CPUs.
The big players use giant clusters of Nvidia H100s.
But if I look at the profiles of my fellow home brewers, all we can get our hands on are those pesky consumer RTX's. If you're lucky you got yourself a 5080 with 16GB VRAM or something.
To be frank, I don't have that 1.3k disposable cash laying around ¯\_(ツ)_/¯
But I can write rust and like building ML libraries.
So I asked myself the question(s):
- can I train SMLs at home on my hardware?
- How hard can it be to build a ML library that can stream data between RAM and VRAM on demand, like llama.cpp's unified memory feature [^1]?
- how hard can it be to implement bf16 support?
The answers are wild, trust me!
Image 1: Metrics form last nights build on my "tiny" RTX 2060 (6 GB VRAM)
Image 2: Metrics from my most recent build on my RTX 4070 Laptop (8GB VRAM)
The majority of my time went into the shared memory, but it's stable and I'm very excited!
Here some debug logs, a la "trust me bro"
```
----
Currently available: 1112735744, attempting to reclaim: 1073741824
--- VRAM STATE [backward pass] ---
Driver Used: 6744 MB / 7805 MB
Data on GPU: 1641 MB
Grads on GPU: 3459 MB
CPU Offloaded: 18230 MB
---------------------------------
Currently available: 1079181312, attempting to reclaim: 1073741824
--- VRAM STATE [backward pass] ---
Driver Used: 6776 MB / 7805 MB
Data on GPU: 1561 MB
Grads on GPU: 3279 MB
CPU Offloaded: 18590 MB
-----------------------------
```
Final models get exported in `safetensors` format and are compatible with PyTorch and `transformers`, for accessibility.
- [^1]: https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#unified-memoryOrganizations
Posts 7
Post
2532
I've been cooking something neat over the past weeks 👨🍳
We all know that training LLMs requires a lot of resources and especially a lot of compute in form of GPUs, or is super slow and inefficient when done on CPUs.
The big players use giant clusters of Nvidia H100s.
But if I look at the profiles of my fellow home brewers, all we can get our hands on are those pesky consumer RTX's. If you're lucky you got yourself a 5080 with 16GB VRAM or something.
To be frank, I don't have that 1.3k disposable cash laying around ¯\_(ツ)_/¯
But I can write rust and like building ML libraries.
So I asked myself the question(s):
- can I train SMLs at home on my hardware?
- How hard can it be to build a ML library that can stream data between RAM and VRAM on demand, like llama.cpp's unified memory feature [^1]?
- how hard can it be to implement bf16 support?
The answers are wild, trust me!
Image 1: Metrics form last nights build on my "tiny" RTX 2060 (6 GB VRAM)
Image 2: Metrics from my most recent build on my RTX 4070 Laptop (8GB VRAM)
The majority of my time went into the shared memory, but it's stable and I'm very excited!
Here some debug logs, a la "trust me bro"
Final models get exported in
- [^1]: https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#unified-memory
We all know that training LLMs requires a lot of resources and especially a lot of compute in form of GPUs, or is super slow and inefficient when done on CPUs.
The big players use giant clusters of Nvidia H100s.
But if I look at the profiles of my fellow home brewers, all we can get our hands on are those pesky consumer RTX's. If you're lucky you got yourself a 5080 with 16GB VRAM or something.
To be frank, I don't have that 1.3k disposable cash laying around ¯\_(ツ)_/¯
But I can write rust and like building ML libraries.
So I asked myself the question(s):
- can I train SMLs at home on my hardware?
- How hard can it be to build a ML library that can stream data between RAM and VRAM on demand, like llama.cpp's unified memory feature [^1]?
- how hard can it be to implement bf16 support?
The answers are wild, trust me!
Image 1: Metrics form last nights build on my "tiny" RTX 2060 (6 GB VRAM)
Image 2: Metrics from my most recent build on my RTX 4070 Laptop (8GB VRAM)
The majority of my time went into the shared memory, but it's stable and I'm very excited!
Here some debug logs, a la "trust me bro"
----
Currently available: 1112735744, attempting to reclaim: 1073741824
--- VRAM STATE [backward pass] ---
Driver Used: 6744 MB / 7805 MB
Data on GPU: 1641 MB
Grads on GPU: 3459 MB
CPU Offloaded: 18230 MB
---------------------------------
Currently available: 1079181312, attempting to reclaim: 1073741824
--- VRAM STATE [backward pass] ---
Driver Used: 6776 MB / 7805 MB
Data on GPU: 1561 MB
Grads on GPU: 3279 MB
CPU Offloaded: 18590 MB
-----------------------------Final models get exported in
safetensors format and are compatible with PyTorch and transformers, for accessibility.- [^1]: https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#unified-memory
Post
193
I'm unemployed, I have a gaming GPU, and I just published a German LLM.
qwen3-0.6b-german - fine-tuned Qwen3-0.6B in ~40h on an RTX 4070 Ti, using the exact same instruct datasets as the LLäMmlein paper (ACL 2025).
HellaSwag-DE: 0.3111 → 0.3193 ✅
ARC-DE: 0.2352 → 0.2575 ✅
MMlu-DE: 0.3600 → 0.2475 🔻 (alignment tax - known trade-off)
Instruction fine-tuning trades some factual breadth for better reasoning and format following. The model is more useful, even if not better on every metric.
Weights, LoRA adapter, full training script and logs all public.
philipp-zettl/qwen3-0.6b-german
It ain't much, but it's honest work.
qwen3-0.6b-german - fine-tuned Qwen3-0.6B in ~40h on an RTX 4070 Ti, using the exact same instruct datasets as the LLäMmlein paper (ACL 2025).
HellaSwag-DE: 0.3111 → 0.3193 ✅
ARC-DE: 0.2352 → 0.2575 ✅
MMlu-DE: 0.3600 → 0.2475 🔻 (alignment tax - known trade-off)
Instruction fine-tuning trades some factual breadth for better reasoning and format following. The model is more useful, even if not better on every metric.
Weights, LoRA adapter, full training script and logs all public.
philipp-zettl/qwen3-0.6b-german
It ain't much, but it's honest work.
spaces 36
pinned
Sleeping
Tiny Dy Svit CIFAR10
📊
Classify images and see importance heatmap
pinned
Running
Structured Output Generator
📚
Generate Your Structured Output HERE
pinned
Sleeping
Roast My Model
💬
Retrieve a naive baseline model for your ML use case.
pinned
Sleeping
Replace Newlines
🔥
Replace newlines with escape sequences
pinned
Running
food-dora
🧑
Describe what you want to cook, receive the recipe for it.
pinned
Runtime error
QA/FAQ Generator
📈
Generates Questions and Answers from given text content.
models 72
philipp-zettl/qwen3-0.6b-german
Text Generation • Updated • 401
philipp-zettl/qwen3-0.6b-german-merged
0.6B • Updated • 19
philipp-zettl/qwen3.5-0.8b-german
Updated
philipp-zettl/modernbert-diffusion-openwebtext
Fill-Mask • 0.1B • Updated
philipp-zettl/Dy-SViT-CIFAR10
Image Classification • Updated
philipp-zettl/modernbert-diffusion-universal
Fill-Mask • 0.1B • Updated
philipp-zettl/modernbert-diffusion-refactor
Fill-Mask • 0.1B • Updated
philipp-zettl/modernbert-diffusion-alpaca-ft
Fill-Mask • 0.1B • Updated
philipp-zettl/modernbert-diffusion-code
Fill-Mask • 0.1B • Updated
philipp-zettl/modernbert-diffusion-instruct
Fill-Mask • 0.1B • Updated
datasets 16
philipp-zettl/qwen3-0.6b-german-training
Viewer • Updated • 1.6k • 26
philipp-zettl/inaturalist-bbs
Updated • 59
philipp-zettl/inaturalist-enriched
Viewer • Updated • 1.49M • 535
philipp-zettl/inaturalist-s3-massive
Viewer • Updated • 1.49M • 1.27k
philipp-zettl/pile-of-law_atticus_contracts
Viewer • Updated • 488k • 73
philipp-zettl/pile-of-law
Updated • 89
philipp-zettl/DeepJSONEval
Viewer • Updated • 2.1k • 22
philipp-zettl/MTGEmb-small-embs
Viewer • Updated • 59.8k • 6
philipp-zettl/my_first_lora_v1-dataset
Viewer • Updated • 7 • 20
philipp-zettl/NibbleNix-DE
Viewer • Updated • 4M • 20