Instructions to use SynLayers/Bbox-caption-8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SynLayers/Bbox-caption-8b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="SynLayers/Bbox-caption-8b")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("SynLayers/Bbox-caption-8b")
model = AutoModelForImageTextToText.from_pretrained("SynLayers/Bbox-caption-8b")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use SynLayers/Bbox-caption-8b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SynLayers/Bbox-caption-8b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SynLayers/Bbox-caption-8b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/SynLayers/Bbox-caption-8b

SGLang

How to use SynLayers/Bbox-caption-8b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SynLayers/Bbox-caption-8b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SynLayers/Bbox-caption-8b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SynLayers/Bbox-caption-8b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SynLayers/Bbox-caption-8b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use SynLayers/Bbox-caption-8b with Docker Model Runner:
```
docker model run hf.co/SynLayers/Bbox-caption-8b
```

SynLayers commited on 7 days ago

Commit

1cfe494

verified ·

1 Parent(s): 50acfb7

Upload environment.yml with huggingface_hub

Browse files

Files changed (1) hide show

environment.yml +151 -0

environment.yml ADDED Viewed

	@@ -0,0 +1,151 @@

+name: SynLayers
+channels:
+  - defaults
+dependencies:
+  - _libgcc_mutex=0.1=main
+  - _openmp_mutex=5.1=1_gnu
+  - bzip2=1.0.8=h5eee18b_6
+  - ca-certificates=2025.7.15=h06a4308_0
+  - expat=2.7.1=h6a678d5_0
+  - ld_impl_linux-64=2.40=h12ee557_0
+  - libffi=3.4.4=h6a678d5_1
+  - libgcc-ng=11.2.0=h1234567_1
+  - libgomp=11.2.0=h1234567_1
+  - libstdcxx-ng=11.2.0=h1234567_1
+  - libuuid=1.41.5=h5eee18b_0
+  - libxcb=1.17.0=h9b100fa_0
+  - ncurses=6.5=h7934f7d_0
+  - openssl=3.0.17=h5eee18b_0
+  - pip=25.1=pyhc872135_2
+  - pthread-stubs=0.3=h0ce48e5_1
+  - python=3.10.18=h1a3bd86_0
+  - readline=8.3=hc2a1206_0
+  - sqlite=3.50.2=hb25bd0a_1
+  - tk=8.6.15=h54e0aa7_0
+  - xorg-libx11=1.8.12=h9b100fa_1
+  - xorg-libxau=1.0.12=h9b100fa_0
+  - xorg-libxdmcp=1.1.5=h9b100fa_0
+  - xorg-xorgproto=2024.1=h5eee18b_1
+  - xz=5.6.4=h5eee18b_1
+  - zlib=1.2.13=h5eee18b_1
+  - pip:
+    - absl-py==2.3.1
+    - accelerate==0.34.2
+    - addict==2.4.0
+    - aiohappyeyeballs==2.6.1
+    - aiohttp==3.12.15
+    - aiosignal==1.4.0
+    - annotated-types==0.7.0
+    - async-timeout==5.0.1
+    - attrs==25.3.0
+    - certifi==2025.8.3
+    - charset-normalizer==3.4.3
+    - click==8.2.1
+    - contourpy==1.3.2
+    - cycler==0.12.1
+    - datasets==2.20.0
+    - deepspeed==0.17.5
+    - diffusers==0.31.0
+    - dill==0.3.8
+    - docker-pycreds==0.4.0
+    - einops==0.8.0
+    - filelock==3.19.1
+    - fonttools==4.59.1
+    - frozenlist==1.7.0
+    - fsspec==2024.5.0
+    - gitdb==4.0.12
+    - gitpython==3.1.45
+    - grpcio==1.74.0
+    - hf-xet==1.1.8
+    - hjson==3.1.0
+    - huggingface-hub==0.34.4
+    - idna==3.10
+    - imageio==2.37.0
+    - importlib-metadata==8.7.0
+    - jinja2==3.1.6
+    - joblib==1.5.2
+    - kiwisolver==1.4.9
+    - lazy-loader==0.4
+    - markdown==3.9
+    - markdown-it-py==4.0.0
+    - markupsafe==3.0.2
+    - matplotlib==3.10.5
+    - mdurl==0.1.2
+    - mmengine==0.10.4
+    - mpmath==1.3.0
+    - msgpack==1.1.1
+    - multidict==6.6.4
+    - multiprocess==0.70.16
+    - networkx==3.4.2
+    - ninja==1.13.0
+    - numpy==2.2.6
+    - nvidia-cublas-cu12==12.1.3.1
+    - nvidia-cuda-cupti-cu12==12.1.105
+    - nvidia-cuda-nvrtc-cu12==12.1.105
+    - nvidia-cuda-runtime-cu12==12.1.105
+    - nvidia-cudnn-cu12==9.1.0.70
+    - nvidia-cufft-cu12==11.0.2.54
+    - nvidia-curand-cu12==10.3.2.106
+    - nvidia-cusolver-cu12==11.4.5.107
+    - nvidia-cusparse-cu12==12.1.0.106
+    - nvidia-ml-py==13.580.65
+    - nvidia-nccl-cu12==2.20.5
+    - nvidia-nvjitlink-cu12==12.9.86
+    - nvidia-nvtx-cu12==12.1.105
+    - opencv-python==4.12.0.88
+    - packaging==25.0
+    - pandas==2.3.2
+    - peft==0.12.0
+    - pillow==11.3.0
+    - platformdirs==4.3.8
+    - prodigyopt==1.0
+    - propcache==0.3.2
+    - protobuf==5.29.5
+    - psutil==7.0.0
+    - py-cpuinfo==9.0.0
+    - pyarrow==21.0.0
+    - pyarrow-hotfix==0.7
+    - pydantic==2.11.7
+    - pydantic-core==2.33.2
+    - pygments==2.19.2
+    - pyparsing==3.2.3
+    - python-dateutil==2.9.0.post0
+    - pytz==2025.2
+    - pyyaml==6.0.2
+    - regex==2025.7.34
+    - requests==2.32.5
+    - rich==14.1.0
+    - safetensors==0.6.2
+    - scikit-image==0.25.2
+    - scikit-learn==1.7.2
+    - scipy==1.15.3
+    - sentencepiece==0.2.0
+    - sentry-sdk==2.35.0
+    - setproctitle==1.3.6
+    - setuptools==78.1.1
+    - six==1.17.0
+    - smmap==5.0.2
+    - sympy==1.14.0
+    - tensorboard==2.20.0
+    - tensorboard-data-server==0.7.2
+    - termcolor==3.1.0
+    - threadpoolctl==3.6.0
+    - tifffile==2025.5.10
+    - tokenizers==0.19.1
+    - tomli==2.2.1
+    - torch==2.4.0
+    - torchvision==0.19.0
+    - tqdm==4.67.1
+    - transformers==4.44.0
+    - triton==3.0.0
+    - typing-extensions==4.14.1
+    - typing-inspection==0.4.1
+    - tzdata==2025.2
+    - urllib3==2.5.0
+    - wandb==0.17.7
+    - werkzeug==3.1.3
+    - wheel==0.45.1
+    - xxhash==3.5.0
+    - yapf==0.43.0
+    - yarl==1.20.1
+    - zipp==3.23.0