stevenkuang commited on
Commit
012b493
·
verified ·
1 Parent(s): 8a03ec0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -0
README.md CHANGED
@@ -41,4 +41,20 @@ We recommend using the following set of parameters for inference. Note that our
41
  "repetition_penalty": 1.05,
42
  "temperature": 0.7
43
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
  ```
 
41
  "repetition_penalty": 1.05,
42
  "temperature": 0.7
43
  }
44
+ ```
45
+
46
+ ### Use with vllm
47
+
48
+ Start the vLLM server:
49
+
50
+ ```bash
51
+ vllm serve tencent/Hy-MT2-7B --tensor-parallel-size 1
52
+ ```
53
+
54
+ ### Use with sglang
55
+
56
+ Launch SGLang server:
57
+
58
+ ```bash
59
+ python3 -m sglang.launch_server --model tencent/Hy-MT2-7B --tp 1
60
  ```