Hummingbird
🎉 You are looking at Hummingbird 3.1, which adds special chat template tokens!
Hummingbird is a Kitten derivative trained to be conversational.
Training
The model was trained using 500 steps, and 4 batch size.
Datasets
The training corpus is made up of:
- First 1400 rows of qikp/reborn-5k-no-thoughts (a collate of Dolly)
- First 500 rows of HuggingFaceTB/smol-smoltalk
- First 100 rows of HuggingFaceTB/everyday-conversations-llama3.1-2k
The train / train_sft splits were used.
Chat template
The Zephyr chat template was used, but most notably, chat template tokens were added to enhance performance.
Limitations
The model frequently outputs incorrect information, confirmation with a larger, mature model is advised.
- Downloads last month
- 5