Create README.md

README.md

@@ -0,0 +1,44 @@
+### Use with transformers
+First, please install transformers, recommends v4.56.0
+```SHELL
+pip install transformers==4.56.0
+```
+
+*!!! If you want to load fp8 model with transformers, you need to change the name"ignored_layers" in config.json to "ignore" and upgrade the compressed-tensors to compressed-tensors-0.11.0.*
+
+The following code snippet shows how to use the transformers library to load and apply the model.
+
+we use tencent/HY-MT1.5-1.8B for example
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import os
+
+model_name_or_path = "tencent/HY-MT1.5-1.8B"
+
+tokenizer = AutoTokenizer.from_pretrained(model_name_or_path)
+model = AutoModelForCausalLM.from_pretrained(model_name_or_path, device_map="auto")  # You may want to use bfloat16 and/or move to GPU here
+messages = [
+    {"role": "user", "content": "Translate the following segment into Chinese, without additional explanation.\n\nIt’s on the house."},
+]
+tokenized_chat = tokenizer.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=False,
+    return_tensors="pt"
+)
+
+outputs = model.generate(tokenized_chat.to(model.device), max_new_tokens=2048)
+output_text = tokenizer.decode(outputs[0])
+```
+
+We recommend using the following set of parameters for inference. Note that our model does not have the default system_prompt.
+
+```json
+{
+  "top_k": 20,
+  "top_p": 0.6,
+  "repetition_penalty": 1.05,
+  "temperature": 0.7
+}
+```