Image-Text-to-Text
Transformers
Safetensors
intern_s2_preview
conversational
custom_code
Eval Results
Instructions to use internlm/Intern-S2-Preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use internlm/Intern-S2-Preview with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="internlm/Intern-S2-Preview", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoModelForImageTextToText model = AutoModelForImageTextToText.from_pretrained("internlm/Intern-S2-Preview", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use internlm/Intern-S2-Preview with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "internlm/Intern-S2-Preview" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "internlm/Intern-S2-Preview", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/internlm/Intern-S2-Preview
- SGLang
How to use internlm/Intern-S2-Preview with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "internlm/Intern-S2-Preview" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "internlm/Intern-S2-Preview", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "internlm/Intern-S2-Preview" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "internlm/Intern-S2-Preview", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use internlm/Intern-S2-Preview with Docker Model Runner:
docker model run hf.co/internlm/Intern-S2-Preview
Upload README.md
Browse files
README.md
CHANGED
|
@@ -297,9 +297,11 @@ print(json.dumps(response.model_dump(), indent=2, ensure_ascii=False))
|
|
| 297 |
Time series inference is currently only supported in LMDeploy. To get started, download and deploy Intern-S2-Preview with LMDeploy by following the [Model Deployment Guide](./deployment_guide.md).
|
| 298 |
Below is an example of detecting earthquake events from a time series signal file. Additional data types and functionalities are also supported.
|
| 299 |
|
|
|
|
|
|
|
| 300 |
```
|
| 301 |
from openai import OpenAI
|
| 302 |
-
from lmdeploy.vl.
|
| 303 |
|
| 304 |
openai_api_key = "EMPTY"
|
| 305 |
openai_api_base = "http://0.0.0.0:8000/v1"
|
|
@@ -321,10 +323,6 @@ def send_base64(file_path: str, sampling_rate: int = 100):
|
|
| 321 |
{
|
| 322 |
"role": "user",
|
| 323 |
"content": [
|
| 324 |
-
{
|
| 325 |
-
"type": "text",
|
| 326 |
-
"text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
|
| 327 |
-
},
|
| 328 |
{
|
| 329 |
"type": "time_series_url",
|
| 330 |
"time_series_url": {
|
|
@@ -332,6 +330,10 @@ def send_base64(file_path: str, sampling_rate: int = 100):
|
|
| 332 |
"sampling_rate": sampling_rate
|
| 333 |
},
|
| 334 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
| 335 |
],
|
| 336 |
}
|
| 337 |
]
|
|
@@ -341,6 +343,9 @@ def send_base64(file_path: str, sampling_rate: int = 100):
|
|
| 341 |
messages=messages,
|
| 342 |
temperature=0,
|
| 343 |
max_tokens=200,
|
|
|
|
|
|
|
|
|
|
| 344 |
)
|
| 345 |
|
| 346 |
|
|
@@ -350,10 +355,6 @@ def send_http_url(url: str, sampling_rate: int = 100):
|
|
| 350 |
{
|
| 351 |
"role": "user",
|
| 352 |
"content": [
|
| 353 |
-
{
|
| 354 |
-
"type": "text",
|
| 355 |
-
"text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
|
| 356 |
-
},
|
| 357 |
{
|
| 358 |
"type": "time_series_url",
|
| 359 |
"time_series_url": {
|
|
@@ -361,6 +362,10 @@ def send_http_url(url: str, sampling_rate: int = 100):
|
|
| 361 |
"sampling_rate": sampling_rate
|
| 362 |
},
|
| 363 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
| 364 |
],
|
| 365 |
}
|
| 366 |
]
|
|
@@ -370,6 +375,9 @@ def send_http_url(url: str, sampling_rate: int = 100):
|
|
| 370 |
messages=messages,
|
| 371 |
temperature=0,
|
| 372 |
max_tokens=200,
|
|
|
|
|
|
|
|
|
|
| 373 |
)
|
| 374 |
|
| 375 |
|
|
@@ -379,10 +387,6 @@ def send_file_url(file_path: str, sampling_rate: int = 100):
|
|
| 379 |
{
|
| 380 |
"role": "user",
|
| 381 |
"content": [
|
| 382 |
-
{
|
| 383 |
-
"type": "text",
|
| 384 |
-
"text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
|
| 385 |
-
},
|
| 386 |
{
|
| 387 |
"type": "time_series_url",
|
| 388 |
"time_series_url": {
|
|
@@ -390,6 +394,10 @@ def send_file_url(file_path: str, sampling_rate: int = 100):
|
|
| 390 |
"sampling_rate": sampling_rate
|
| 391 |
},
|
| 392 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
| 393 |
],
|
| 394 |
}
|
| 395 |
]
|
|
@@ -399,6 +407,9 @@ def send_file_url(file_path: str, sampling_rate: int = 100):
|
|
| 399 |
messages=messages,
|
| 400 |
temperature=0,
|
| 401 |
max_tokens=200,
|
|
|
|
|
|
|
|
|
|
| 402 |
)
|
| 403 |
|
| 404 |
response = send_base64("./0092638_seism.npy")
|
|
@@ -406,6 +417,7 @@ response = send_base64("./0092638_seism.npy")
|
|
| 406 |
# response = send_file_url("./0092638_seism.npy")
|
| 407 |
|
| 408 |
print(response.choices[0].message)
|
|
|
|
| 409 |
```
|
| 410 |
|
| 411 |
## Agent Integration
|
|
|
|
| 297 |
Time series inference is currently only supported in LMDeploy. To get started, download and deploy Intern-S2-Preview with LMDeploy by following the [Model Deployment Guide](./deployment_guide.md).
|
| 298 |
Below is an example of detecting earthquake events from a time series signal file. Additional data types and functionalities are also supported.
|
| 299 |
|
| 300 |
+
**Please note**: this demo is slightly different from the one in [Intern-S1-Pro](https://huggingface.co/internlm/Intern-S1-Pro#time-series-demo). The main difference is that in the messages content, you need to provide time_series_url first, followed by the text prompt. Please adapt your implementation based on this demo.
|
| 301 |
+
|
| 302 |
```
|
| 303 |
from openai import OpenAI
|
| 304 |
+
from lmdeploy.vl.utils import encode_time_series_base64
|
| 305 |
|
| 306 |
openai_api_key = "EMPTY"
|
| 307 |
openai_api_base = "http://0.0.0.0:8000/v1"
|
|
|
|
| 323 |
{
|
| 324 |
"role": "user",
|
| 325 |
"content": [
|
|
|
|
|
|
|
|
|
|
|
|
|
| 326 |
{
|
| 327 |
"type": "time_series_url",
|
| 328 |
"time_series_url": {
|
|
|
|
| 330 |
"sampling_rate": sampling_rate
|
| 331 |
},
|
| 332 |
},
|
| 333 |
+
{
|
| 334 |
+
"type": "text",
|
| 335 |
+
"text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
|
| 336 |
+
},
|
| 337 |
],
|
| 338 |
}
|
| 339 |
]
|
|
|
|
| 343 |
messages=messages,
|
| 344 |
temperature=0,
|
| 345 |
max_tokens=200,
|
| 346 |
+
extra_body={
|
| 347 |
+
"chat_template_kwargs": {"enable_thinking": False}
|
| 348 |
+
}
|
| 349 |
)
|
| 350 |
|
| 351 |
|
|
|
|
| 355 |
{
|
| 356 |
"role": "user",
|
| 357 |
"content": [
|
|
|
|
|
|
|
|
|
|
|
|
|
| 358 |
{
|
| 359 |
"type": "time_series_url",
|
| 360 |
"time_series_url": {
|
|
|
|
| 362 |
"sampling_rate": sampling_rate
|
| 363 |
},
|
| 364 |
},
|
| 365 |
+
{
|
| 366 |
+
"type": "text",
|
| 367 |
+
"text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
|
| 368 |
+
},
|
| 369 |
],
|
| 370 |
}
|
| 371 |
]
|
|
|
|
| 375 |
messages=messages,
|
| 376 |
temperature=0,
|
| 377 |
max_tokens=200,
|
| 378 |
+
extra_body={
|
| 379 |
+
"chat_template_kwargs": {"enable_thinking": False}
|
| 380 |
+
}
|
| 381 |
)
|
| 382 |
|
| 383 |
|
|
|
|
| 387 |
{
|
| 388 |
"role": "user",
|
| 389 |
"content": [
|
|
|
|
|
|
|
|
|
|
|
|
|
| 390 |
{
|
| 391 |
"type": "time_series_url",
|
| 392 |
"time_series_url": {
|
|
|
|
| 394 |
"sampling_rate": sampling_rate
|
| 395 |
},
|
| 396 |
},
|
| 397 |
+
{
|
| 398 |
+
"type": "text",
|
| 399 |
+
"text": "Please determine whether an Earthquake event has occurred in the provided time-series data. If so, please specify the starting time point indices of the P-wave and S-wave in the event."
|
| 400 |
+
},
|
| 401 |
],
|
| 402 |
}
|
| 403 |
]
|
|
|
|
| 407 |
messages=messages,
|
| 408 |
temperature=0,
|
| 409 |
max_tokens=200,
|
| 410 |
+
extra_body={
|
| 411 |
+
"chat_template_kwargs": {"enable_thinking": False}
|
| 412 |
+
}
|
| 413 |
)
|
| 414 |
|
| 415 |
response = send_base64("./0092638_seism.npy")
|
|
|
|
| 417 |
# response = send_file_url("./0092638_seism.npy")
|
| 418 |
|
| 419 |
print(response.choices[0].message)
|
| 420 |
+
|
| 421 |
```
|
| 422 |
|
| 423 |
## Agent Integration
|