Instructions to use nkim7/eval3_phase2_TOY with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use nkim7/eval3_phase2_TOY with PEFT:
Task type is invalid.
- Transformers
How to use nkim7/eval3_phase2_TOY with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="nkim7/eval3_phase2_TOY") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("nkim7/eval3_phase2_TOY", dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use nkim7/eval3_phase2_TOY with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "nkim7/eval3_phase2_TOY" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nkim7/eval3_phase2_TOY", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/nkim7/eval3_phase2_TOY
- SGLang
How to use nkim7/eval3_phase2_TOY with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "nkim7/eval3_phase2_TOY" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nkim7/eval3_phase2_TOY", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "nkim7/eval3_phase2_TOY" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nkim7/eval3_phase2_TOY", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use nkim7/eval3_phase2_TOY with Docker Model Runner:
docker model run hf.co/nkim7/eval3_phase2_TOY
| { | |
| "<end_of_utterance>": 49279, | |
| "<fake_token_around_image>": 49189, | |
| "<global-img>": 49152, | |
| "<image>": 49190, | |
| "<row_1_col_1>": 49153, | |
| "<row_1_col_2>": 49154, | |
| "<row_1_col_3>": 49155, | |
| "<row_1_col_4>": 49156, | |
| "<row_1_col_5>": 49157, | |
| "<row_1_col_6>": 49158, | |
| "<row_2_col_1>": 49159, | |
| "<row_2_col_2>": 49160, | |
| "<row_2_col_3>": 49161, | |
| "<row_2_col_4>": 49162, | |
| "<row_2_col_5>": 49163, | |
| "<row_2_col_6>": 49164, | |
| "<row_3_col_1>": 49165, | |
| "<row_3_col_2>": 49166, | |
| "<row_3_col_3>": 49167, | |
| "<row_3_col_4>": 49168, | |
| "<row_3_col_5>": 49169, | |
| "<row_3_col_6>": 49170, | |
| "<row_4_col_1>": 49171, | |
| "<row_4_col_2>": 49172, | |
| "<row_4_col_3>": 49173, | |
| "<row_4_col_4>": 49174, | |
| "<row_4_col_5>": 49175, | |
| "<row_4_col_6>": 49176, | |
| "<row_5_col_1>": 49177, | |
| "<row_5_col_2>": 49178, | |
| "<row_5_col_3>": 49179, | |
| "<row_5_col_4>": 49180, | |
| "<row_5_col_5>": 49181, | |
| "<row_5_col_6>": 49182, | |
| "<row_6_col_1>": 49183, | |
| "<row_6_col_2>": 49184, | |
| "<row_6_col_3>": 49185, | |
| "<row_6_col_4>": 49186, | |
| "<row_6_col_5>": 49187, | |
| "<row_6_col_6>": 49188, | |
| "<|reserved_special_token_0|>": 49191, | |
| "<|reserved_special_token_10|>": 49201, | |
| "<|reserved_special_token_11|>": 49202, | |
| "<|reserved_special_token_12|>": 49203, | |
| "<|reserved_special_token_13|>": 49204, | |
| "<|reserved_special_token_14|>": 49205, | |
| "<|reserved_special_token_15|>": 49206, | |
| "<|reserved_special_token_16|>": 49207, | |
| "<|reserved_special_token_17|>": 49208, | |
| "<|reserved_special_token_18|>": 49209, | |
| "<|reserved_special_token_19|>": 49210, | |
| "<|reserved_special_token_1|>": 49192, | |
| "<|reserved_special_token_20|>": 49211, | |
| "<|reserved_special_token_21|>": 49212, | |
| "<|reserved_special_token_22|>": 49213, | |
| "<|reserved_special_token_23|>": 49214, | |
| "<|reserved_special_token_24|>": 49215, | |
| "<|reserved_special_token_25|>": 49216, | |
| "<|reserved_special_token_26|>": 49217, | |
| "<|reserved_special_token_27|>": 49218, | |
| "<|reserved_special_token_28|>": 49219, | |
| "<|reserved_special_token_29|>": 49220, | |
| "<|reserved_special_token_2|>": 49193, | |
| "<|reserved_special_token_30|>": 49221, | |
| "<|reserved_special_token_31|>": 49222, | |
| "<|reserved_special_token_32|>": 49223, | |
| "<|reserved_special_token_33|>": 49224, | |
| "<|reserved_special_token_34|>": 49225, | |
| "<|reserved_special_token_35|>": 49226, | |
| "<|reserved_special_token_36|>": 49227, | |
| "<|reserved_special_token_37|>": 49228, | |
| "<|reserved_special_token_38|>": 49229, | |
| "<|reserved_special_token_39|>": 49230, | |
| "<|reserved_special_token_3|>": 49194, | |
| "<|reserved_special_token_40|>": 49231, | |
| "<|reserved_special_token_41|>": 49232, | |
| "<|reserved_special_token_42|>": 49233, | |
| "<|reserved_special_token_43|>": 49234, | |
| "<|reserved_special_token_44|>": 49235, | |
| "<|reserved_special_token_45|>": 49236, | |
| "<|reserved_special_token_46|>": 49237, | |
| "<|reserved_special_token_47|>": 49238, | |
| "<|reserved_special_token_48|>": 49239, | |
| "<|reserved_special_token_49|>": 49240, | |
| "<|reserved_special_token_4|>": 49195, | |
| "<|reserved_special_token_50|>": 49241, | |
| "<|reserved_special_token_51|>": 49242, | |
| "<|reserved_special_token_52|>": 49243, | |
| "<|reserved_special_token_53|>": 49244, | |
| "<|reserved_special_token_54|>": 49245, | |
| "<|reserved_special_token_55|>": 49246, | |
| "<|reserved_special_token_56|>": 49247, | |
| "<|reserved_special_token_57|>": 49248, | |
| "<|reserved_special_token_58|>": 49249, | |
| "<|reserved_special_token_59|>": 49250, | |
| "<|reserved_special_token_5|>": 49196, | |
| "<|reserved_special_token_60|>": 49251, | |
| "<|reserved_special_token_61|>": 49252, | |
| "<|reserved_special_token_62|>": 49253, | |
| "<|reserved_special_token_63|>": 49254, | |
| "<|reserved_special_token_64|>": 49255, | |
| "<|reserved_special_token_65|>": 49256, | |
| "<|reserved_special_token_66|>": 49257, | |
| "<|reserved_special_token_67|>": 49258, | |
| "<|reserved_special_token_68|>": 49259, | |
| "<|reserved_special_token_69|>": 49260, | |
| "<|reserved_special_token_6|>": 49197, | |
| "<|reserved_special_token_70|>": 49261, | |
| "<|reserved_special_token_71|>": 49262, | |
| "<|reserved_special_token_72|>": 49263, | |
| "<|reserved_special_token_73|>": 49264, | |
| "<|reserved_special_token_74|>": 49265, | |
| "<|reserved_special_token_75|>": 49266, | |
| "<|reserved_special_token_76|>": 49267, | |
| "<|reserved_special_token_77|>": 49268, | |
| "<|reserved_special_token_78|>": 49269, | |
| "<|reserved_special_token_79|>": 49270, | |
| "<|reserved_special_token_7|>": 49198, | |
| "<|reserved_special_token_80|>": 49271, | |
| "<|reserved_special_token_81|>": 49272, | |
| "<|reserved_special_token_82|>": 49273, | |
| "<|reserved_special_token_83|>": 49274, | |
| "<|reserved_special_token_84|>": 49275, | |
| "<|reserved_special_token_85|>": 49276, | |
| "<|reserved_special_token_86|>": 49277, | |
| "<|reserved_special_token_87|>": 49278, | |
| "<|reserved_special_token_8|>": 49199, | |
| "<|reserved_special_token_9|>": 49200 | |
| } | |