Bitnet-b1.58-2B-metal-weight

Introduction

This repository contains .bin files extracted from microsoft/bitnet-b1.58-2B-4T. The weights are stock and ready to use in my project jossnet-bitnet. This project is an implementation of the BitNet model for Apple Silicon, using Metal.

Structure

bitnet-b1.58-2B-metal-weight/
├── layers/
│    ├── layer_X/
│         ├── RMS_attn.bin
│         ├── RMS_input.bin
│         ├── RMS_mlp_sub.bin
│         ├── RMS_post_attn.bin
│         ├── Scale_down.bin
│         ├── Scale_gate.bin
│         ├── Scale_k.bin
│         ├── Scale_o.bin
│         ├── Scale_q.bin
│         ├── Scale_up.bin
│         ├── Scale_v.bin
│         ├── W_down.bin
│         ├── W_gate.bin
│         ├── W_k.bin
│         ├── W_o.bin
│         ├── W_q.bin
│         ├── W_up.bin
│         └── W_v.bin
├── embeddings.bin
├── weights_lm_head.bin
├── weights_RMS_final.bin
├── weights_RMS_mlp.bin
├── weights_W_down.bin
├── weights_W_gate.bin
├── weights_W_k.bin
├── weights_W_o.bin
├── weights_W_q.bin
├── weights_W_up.bin
├── weights_W_v.bin
└── LICENCE
    

Licence & references

• Original Model: microsoft/bitnet-b1.58-2B-4T
• License: MIT License (Copyright (c) Microsoft Corporation)