stevenkuang commited on
Commit
bad21e9
·
verified ·
1 Parent(s): 283296c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -2
README.md CHANGED
@@ -8,13 +8,13 @@ pip install transformers==4.56.0
8
 
9
  The following code snippet shows how to use the transformers library to load and apply the model.
10
 
11
- we use tencent/HY-MT1.5-1.8B for example
12
 
13
  ```python
14
  from transformers import AutoModelForCausalLM, AutoTokenizer
15
  import os
16
 
17
- model_name_or_path = "tencent/HY-MT1.5-1.8B"
18
 
19
  tokenizer = AutoTokenizer.from_pretrained(model_name_or_path)
20
  model = AutoModelForCausalLM.from_pretrained(model_name_or_path, device_map="auto") # You may want to use bfloat16 and/or move to GPU here
@@ -41,4 +41,20 @@ We recommend using the following set of parameters for inference. Note that our
41
  "repetition_penalty": 1.05,
42
  "temperature": 0.7
43
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
  ```
 
8
 
9
  The following code snippet shows how to use the transformers library to load and apply the model.
10
 
11
+ we use tencent/Hy-MT2-1.8B for example
12
 
13
  ```python
14
  from transformers import AutoModelForCausalLM, AutoTokenizer
15
  import os
16
 
17
+ model_name_or_path = "tencent/Hy-MT2-1.8B"
18
 
19
  tokenizer = AutoTokenizer.from_pretrained(model_name_or_path)
20
  model = AutoModelForCausalLM.from_pretrained(model_name_or_path, device_map="auto") # You may want to use bfloat16 and/or move to GPU here
 
41
  "repetition_penalty": 1.05,
42
  "temperature": 0.7
43
  }
44
+ ```
45
+
46
+ ### Use with vllm
47
+
48
+ Start the vLLM server:
49
+
50
+ ```bash
51
+ vllm serve tencent/Hy-MT2-1.8B --tensor-parallel-size 1
52
+ ```
53
+
54
+ ### Use with sglang
55
+
56
+ Launch SGLang server:
57
+
58
+ ```bash
59
+ python3 -m sglang.launch_server --model tencent/Hy-MT2-1.8B --tp 1
60
  ```