Instructions to use bear7011/In_breadth4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use bear7011/In_breadth4 with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("bear7011/In_breadth4", dtype="auto") - llama-cpp-python
How to use bear7011/In_breadth4 with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="bear7011/In_breadth4", filename="unsloth.BF16.gguf", )
output = llm( "Once upon a time,", max_tokens=512, echo=True ) print(output)
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- llama.cpp
How to use bear7011/In_breadth4 with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf bear7011/In_breadth4:Q4_K_M # Run inference directly in the terminal: llama-cli -hf bear7011/In_breadth4:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf bear7011/In_breadth4:Q4_K_M # Run inference directly in the terminal: llama-cli -hf bear7011/In_breadth4:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf bear7011/In_breadth4:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf bear7011/In_breadth4:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf bear7011/In_breadth4:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf bear7011/In_breadth4:Q4_K_M
Use Docker
docker model run hf.co/bear7011/In_breadth4:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use bear7011/In_breadth4 with Ollama:
ollama run hf.co/bear7011/In_breadth4:Q4_K_M
- Unsloth Studio new
How to use bear7011/In_breadth4 with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for bear7011/In_breadth4 to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for bear7011/In_breadth4 to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for bear7011/In_breadth4 to start chatting
- Docker Model Runner
How to use bear7011/In_breadth4 with Docker Model Runner:
docker model run hf.co/bear7011/In_breadth4:Q4_K_M
- Lemonade
How to use bear7011/In_breadth4 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull bear7011/In_breadth4:Q4_K_M
Run and chat with the model
lemonade run user.In_breadth4-Q4_K_M
List all available models
lemonade list
Upload model trained with Unsloth
Browse filesUpload model trained with Unsloth 2x faster
- special_tokens_map.json +1 -1
- tokenizer_config.json +2 -1
special_tokens_map.json
CHANGED
|
@@ -7,7 +7,7 @@
|
|
| 7 |
"single_word": false
|
| 8 |
},
|
| 9 |
"eos_token": {
|
| 10 |
-
"content": "<|
|
| 11 |
"lstrip": false,
|
| 12 |
"normalized": false,
|
| 13 |
"rstrip": false,
|
|
|
|
| 7 |
"single_word": false
|
| 8 |
},
|
| 9 |
"eos_token": {
|
| 10 |
+
"content": "<|eot_id|>",
|
| 11 |
"lstrip": false,
|
| 12 |
"normalized": false,
|
| 13 |
"rstrip": false,
|
tokenizer_config.json
CHANGED
|
@@ -2050,8 +2050,9 @@
|
|
| 2050 |
}
|
| 2051 |
},
|
| 2052 |
"bos_token": "<|begin_of_text|>",
|
|
|
|
| 2053 |
"clean_up_tokenization_spaces": true,
|
| 2054 |
-
"eos_token": "<|
|
| 2055 |
"model_input_names": [
|
| 2056 |
"input_ids",
|
| 2057 |
"attention_mask"
|
|
|
|
| 2050 |
}
|
| 2051 |
},
|
| 2052 |
"bos_token": "<|begin_of_text|>",
|
| 2053 |
+
"chat_template": "{{- bos_token }}\n{%- if custom_tools is defined %}\n {%- set tools = custom_tools %}\n{%- endif %}\n{%- if not tools_in_user_message is defined %}\n {%- set tools_in_user_message = true %}\n{%- endif %}\n{%- if not date_string is defined %}\n {%- set date_string = \"26 Jul 2024\" %}\n{%- endif %}\n{%- if not tools is defined %}\n {%- set tools = none %}\n{%- endif %}\n\n{#- This block extracts the system message, so we can slot it into the right place. #}\n{%- if messages[0]['role'] == 'system' %}\n {%- set system_message = messages[0]['content']|trim %}\n {%- set messages = messages[1:] %}\n{%- else %}\n {%- set system_message = \"\" %}\n{%- endif %}\n\n{#- System message + builtin tools #}\n{{- \"<|start_header_id|>system<|end_header_id|>\\n\\n\" }}\n{%- if builtin_tools is defined or tools is not none %}\n {{- \"Environment: ipython\\n\" }}\n{%- endif %}\n{%- if builtin_tools is defined %}\n {{- \"Tools: \" + builtin_tools | reject('equalto', 'code_interpreter') | join(\", \") + \"\\n\\n\"}}\n{%- endif %}\n{{- \"Cutting Knowledge Date: December 2023\\n\" }}\n{{- \"Today Date: \" + date_string + \"\\n\\n\" }}\n{%- if tools is not none and not tools_in_user_message %}\n {{- \"You have access to the following functions. To call a function, please respond with JSON for a function call.\" }}\n {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n {{- \"Do not use variables.\\n\\n\" }}\n {%- for t in tools %}\n {{- t | tojson(indent=4) }}\n {{- \"\\n\\n\" }}\n {%- endfor %}\n{%- endif %}\n{{- system_message }}\n{{- \"<|eot_id|>\" }}\n\n{#- Custom tools are passed in a user message with some extra guidance #}\n{%- if tools_in_user_message and not tools is none %}\n {#- Extract the first user message so we can plug it in here #}\n {%- if messages | length != 0 %}\n {%- set first_user_message = messages[0]['content']|trim %}\n {%- set messages = messages[1:] %}\n {%- else %}\n {{- raise_exception(\"Cannot put tools in the first user message when there's no first user message!\") }}\n{%- endif %}\n {{- '<|start_header_id|>user<|end_header_id|>\\n\\n' -}}\n {{- \"Given the following functions, please respond with a JSON for a function call \" }}\n {{- \"with its proper arguments that best answers the given prompt.\\n\\n\" }}\n {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n {{- \"Do not use variables.\\n\\n\" }}\n {%- for t in tools %}\n {{- t | tojson(indent=4) }}\n {{- \"\\n\\n\" }}\n {%- endfor %}\n {{- first_user_message + \"<|eot_id|>\"}}\n{%- endif %}\n\n{%- for message in messages %}\n {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}\n {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n'+ message['content'] | trim + '<|eot_id|>' }}\n {%- elif 'tool_calls' in message %}\n {%- if not message.tool_calls|length == 1 %}\n {{- raise_exception(\"This model only supports single tool-calls at once!\") }}\n {%- endif %}\n {%- set tool_call = message.tool_calls[0].function %}\n {%- if builtin_tools is defined and tool_call.name in builtin_tools %}\n {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n {{- \"<|python_tag|>\" + tool_call.name + \".call(\" }}\n {%- for arg_name, arg_val in tool_call.arguments | items %}\n {{- arg_name + '=\"' + arg_val + '\"' }}\n {%- if not loop.last %}\n {{- \", \" }}\n {%- endif %}\n {%- endfor %}\n {{- \")\" }}\n {%- else %}\n {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n {{- '{\"name\": \"' + tool_call.name + '\", ' }}\n {{- '\"parameters\": ' }}\n {{- tool_call.arguments | tojson }}\n {{- \"}\" }}\n {%- endif %}\n {%- if builtin_tools is defined %}\n {#- This means we're in ipython mode #}\n {{- \"<|eom_id|>\" }}\n {%- else %}\n {{- \"<|eot_id|>\" }}\n {%- endif %}\n {%- elif message.role == \"tool\" or message.role == \"ipython\" %}\n {{- \"<|start_header_id|>ipython<|end_header_id|>\\n\\n\" }}\n {%- if message.content is mapping or message.content is iterable %}\n {{- message.content | tojson }}\n {%- else %}\n {{- message.content }}\n {%- endif %}\n {{- \"<|eot_id|>\" }}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n{%- endif %}\n",
|
| 2054 |
"clean_up_tokenization_spaces": true,
|
| 2055 |
+
"eos_token": "<|eot_id|>",
|
| 2056 |
"model_input_names": [
|
| 2057 |
"input_ids",
|
| 2058 |
"attention_mask"
|