Instructions to use internlm/Intern-S2-Preview-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use internlm/Intern-S2-Preview-FP8 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="internlm/Intern-S2-Preview-FP8", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoModelForImageTextToText model = AutoModelForImageTextToText.from_pretrained("internlm/Intern-S2-Preview-FP8", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use internlm/Intern-S2-Preview-FP8 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "internlm/Intern-S2-Preview-FP8" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "internlm/Intern-S2-Preview-FP8", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/internlm/Intern-S2-Preview-FP8
- SGLang
How to use internlm/Intern-S2-Preview-FP8 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "internlm/Intern-S2-Preview-FP8" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "internlm/Intern-S2-Preview-FP8", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "internlm/Intern-S2-Preview-FP8" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "internlm/Intern-S2-Preview-FP8", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use internlm/Intern-S2-Preview-FP8 with Docker Model Runner:
docker model run hf.co/internlm/Intern-S2-Preview-FP8
Upload README.md
Browse files
README.md
CHANGED
|
@@ -14,12 +14,12 @@ base_model:
|
|
| 14 |
|
| 15 |
<div> </div>
|
| 16 |
|
| 17 |
-
[💻Github
|
| 18 |
|
| 19 |
</div>
|
| 20 |
|
| 21 |
<p align="center">
|
| 22 |
-
👋
|
| 23 |
</p>
|
| 24 |
|
| 25 |
|
|
@@ -298,9 +298,10 @@ print(json.dumps(response.model_dump(), indent=2, ensure_ascii=False))
|
|
| 298 |
Time series inference is currently only supported in LMDeploy. To get started, download and deploy Intern-S2-Preview-FP8 with LMDeploy by following the [Model Deployment Guide](./deployment_guide.md).
|
| 299 |
Below is an example of detecting earthquake events from a time series signal file. Additional data types and functionalities are also supported.
|
| 300 |
|
|
|
|
| 301 |
```
|
| 302 |
from openai import OpenAI
|
| 303 |
-
from lmdeploy.vl.
|
| 304 |
|
| 305 |
openai_api_key = "EMPTY"
|
| 306 |
openai_api_base = "http://0.0.0.0:8000/v1"
|
|
@@ -322,10 +323,6 @@ def send_base64(file_path: str, sampling_rate: int = 100):
|
|
| 322 |
{
|
| 323 |
"role": "user",
|
| 324 |
"content": [
|
| 325 |
-
{
|
| 326 |
-
"type": "text",
|
| 327 |
-
"text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
|
| 328 |
-
},
|
| 329 |
{
|
| 330 |
"type": "time_series_url",
|
| 331 |
"time_series_url": {
|
|
@@ -333,6 +330,10 @@ def send_base64(file_path: str, sampling_rate: int = 100):
|
|
| 333 |
"sampling_rate": sampling_rate
|
| 334 |
},
|
| 335 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
| 336 |
],
|
| 337 |
}
|
| 338 |
]
|
|
@@ -342,6 +343,9 @@ def send_base64(file_path: str, sampling_rate: int = 100):
|
|
| 342 |
messages=messages,
|
| 343 |
temperature=0,
|
| 344 |
max_tokens=200,
|
|
|
|
|
|
|
|
|
|
| 345 |
)
|
| 346 |
|
| 347 |
|
|
@@ -351,10 +355,6 @@ def send_http_url(url: str, sampling_rate: int = 100):
|
|
| 351 |
{
|
| 352 |
"role": "user",
|
| 353 |
"content": [
|
| 354 |
-
{
|
| 355 |
-
"type": "text",
|
| 356 |
-
"text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
|
| 357 |
-
},
|
| 358 |
{
|
| 359 |
"type": "time_series_url",
|
| 360 |
"time_series_url": {
|
|
@@ -362,6 +362,10 @@ def send_http_url(url: str, sampling_rate: int = 100):
|
|
| 362 |
"sampling_rate": sampling_rate
|
| 363 |
},
|
| 364 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
| 365 |
],
|
| 366 |
}
|
| 367 |
]
|
|
@@ -371,6 +375,9 @@ def send_http_url(url: str, sampling_rate: int = 100):
|
|
| 371 |
messages=messages,
|
| 372 |
temperature=0,
|
| 373 |
max_tokens=200,
|
|
|
|
|
|
|
|
|
|
| 374 |
)
|
| 375 |
|
| 376 |
|
|
@@ -380,10 +387,6 @@ def send_file_url(file_path: str, sampling_rate: int = 100):
|
|
| 380 |
{
|
| 381 |
"role": "user",
|
| 382 |
"content": [
|
| 383 |
-
{
|
| 384 |
-
"type": "text",
|
| 385 |
-
"text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
|
| 386 |
-
},
|
| 387 |
{
|
| 388 |
"type": "time_series_url",
|
| 389 |
"time_series_url": {
|
|
@@ -391,6 +394,10 @@ def send_file_url(file_path: str, sampling_rate: int = 100):
|
|
| 391 |
"sampling_rate": sampling_rate
|
| 392 |
},
|
| 393 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
| 394 |
],
|
| 395 |
}
|
| 396 |
]
|
|
@@ -400,6 +407,9 @@ def send_file_url(file_path: str, sampling_rate: int = 100):
|
|
| 400 |
messages=messages,
|
| 401 |
temperature=0,
|
| 402 |
max_tokens=200,
|
|
|
|
|
|
|
|
|
|
| 403 |
)
|
| 404 |
|
| 405 |
response = send_base64("./0092638_seism.npy")
|
|
|
|
| 14 |
|
| 15 |
<div> </div>
|
| 16 |
|
| 17 |
+
[💻Github po](https://github.com/InternLM/Intern-S1) • [🤗Mdl Collections](https://huggingface.co/collections/internlm/intern-s2) • [💬OnlneChat](https://chat.intern-ai.org.cn/)
|
| 18 |
|
| 19 |
</div>
|
| 20 |
|
| 21 |
<p align="center">
|
| 22 |
+
👋 joius on <a href="https://discord.gg/xa29JuW87d" target="_blank">Discord</a> and <a href="https://cdn.vansin.top/intern-s1.jpg" target="_blank">WeChat</a>
|
| 23 |
</p>
|
| 24 |
|
| 25 |
|
|
|
|
| 298 |
Time series inference is currently only supported in LMDeploy. To get started, download and deploy Intern-S2-Preview-FP8 with LMDeploy by following the [Model Deployment Guide](./deployment_guide.md).
|
| 299 |
Below is an example of detecting earthquake events from a time series signal file. Additional data types and functionalities are also supported.
|
| 300 |
|
| 301 |
+
**Please note**: this demo is slightly different from the one in [Intern-S1-Pro](https://huggingface.co/internlm/Intern-S1-Pro#time-series-demo). The main difference is that in the messages content, you need to provide time_series_url first, followed by the text prompt. Please adapt your implementation based on this demo.
|
| 302 |
```
|
| 303 |
from openai import OpenAI
|
| 304 |
+
from lmdeploy.vl.utils import encode_time_series_base64
|
| 305 |
|
| 306 |
openai_api_key = "EMPTY"
|
| 307 |
openai_api_base = "http://0.0.0.0:8000/v1"
|
|
|
|
| 323 |
{
|
| 324 |
"role": "user",
|
| 325 |
"content": [
|
|
|
|
|
|
|
|
|
|
|
|
|
| 326 |
{
|
| 327 |
"type": "time_series_url",
|
| 328 |
"time_series_url": {
|
|
|
|
| 330 |
"sampling_rate": sampling_rate
|
| 331 |
},
|
| 332 |
},
|
| 333 |
+
{
|
| 334 |
+
"type": "text",
|
| 335 |
+
"text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
|
| 336 |
+
},
|
| 337 |
],
|
| 338 |
}
|
| 339 |
]
|
|
|
|
| 343 |
messages=messages,
|
| 344 |
temperature=0,
|
| 345 |
max_tokens=200,
|
| 346 |
+
extra_body={
|
| 347 |
+
"chat_template_kwargs": {"enable_thinking": False}
|
| 348 |
+
}
|
| 349 |
)
|
| 350 |
|
| 351 |
|
|
|
|
| 355 |
{
|
| 356 |
"role": "user",
|
| 357 |
"content": [
|
|
|
|
|
|
|
|
|
|
|
|
|
| 358 |
{
|
| 359 |
"type": "time_series_url",
|
| 360 |
"time_series_url": {
|
|
|
|
| 362 |
"sampling_rate": sampling_rate
|
| 363 |
},
|
| 364 |
},
|
| 365 |
+
{
|
| 366 |
+
"type": "text",
|
| 367 |
+
"text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
|
| 368 |
+
},
|
| 369 |
],
|
| 370 |
}
|
| 371 |
]
|
|
|
|
| 375 |
messages=messages,
|
| 376 |
temperature=0,
|
| 377 |
max_tokens=200,
|
| 378 |
+
extra_body={
|
| 379 |
+
"chat_template_kwargs": {"enable_thinking": False}
|
| 380 |
+
}
|
| 381 |
)
|
| 382 |
|
| 383 |
|
|
|
|
| 387 |
{
|
| 388 |
"role": "user",
|
| 389 |
"content": [
|
|
|
|
|
|
|
|
|
|
|
|
|
| 390 |
{
|
| 391 |
"type": "time_series_url",
|
| 392 |
"time_series_url": {
|
|
|
|
| 394 |
"sampling_rate": sampling_rate
|
| 395 |
},
|
| 396 |
},
|
| 397 |
+
{
|
| 398 |
+
"type": "text",
|
| 399 |
+
"text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
|
| 400 |
+
},
|
| 401 |
],
|
| 402 |
}
|
| 403 |
]
|
|
|
|
| 407 |
messages=messages,
|
| 408 |
temperature=0,
|
| 409 |
max_tokens=200,
|
| 410 |
+
extra_body={
|
| 411 |
+
"chat_template_kwargs": {"enable_thinking": False}
|
| 412 |
+
}
|
| 413 |
)
|
| 414 |
|
| 415 |
response = send_base64("./0092638_seism.npy")
|