Instructions to use thekraftors/text-to-sql-beta-v1-q8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use thekraftors/text-to-sql-beta-v1-q8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="thekraftors/text-to-sql-beta-v1-q8")

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("thekraftors/text-to-sql-beta-v1-q8", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use thekraftors/text-to-sql-beta-v1-q8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "thekraftors/text-to-sql-beta-v1-q8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "thekraftors/text-to-sql-beta-v1-q8",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/thekraftors/text-to-sql-beta-v1-q8

SGLang

How to use thekraftors/text-to-sql-beta-v1-q8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "thekraftors/text-to-sql-beta-v1-q8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "thekraftors/text-to-sql-beta-v1-q8",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "thekraftors/text-to-sql-beta-v1-q8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "thekraftors/text-to-sql-beta-v1-q8",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use thekraftors/text-to-sql-beta-v1-q8 with Docker Model Runner:
```
docker model run hf.co/thekraftors/text-to-sql-beta-v1-q8
```

text-to-sql-beta-v1-q8 / README.md

thekraftors

Update README.md

0595585 verified over 2 years ago

preview code

raw

history blame contribute delete

1.33 kB

metadata

license: apache-2.0
language:
  - en

Your Project Name

Description

Briefly describe your project and its purpose.

Files

.gitattributes: Git configuration file.
README.md: This file, containing information about the project.
initial commit: Description of the initial commit.
checkpoint: Checkpoint file.
counter: File related to counting.
encoder.json: JSON file for the encoder.
events.out.tfevents.1705309987.5f53e38a8413: TensorFlow events file.
hparams.json: JSON file containing hyperparameters.
model-5000.data-00000-of-00001: Model data file.
model-5000.index: Model index file.
model-5000.meta: Model meta file.
vocab.bpe: Vocabulary file.

Usage

Explain how to use or run your project. Include any dependencies and setup instructions.

Model Training

If applicable, provide information on how the model was fine-tuned and any specific training steps.

License

Specify the license under which your project is released.

Acknowledgments

Give credit to any resources, libraries, or individuals you used or were inspired by during the development.

Contact

Provide contact information for the maintainer or contributors, in case someone wants to reach out.

Feel free to customize this template based on the specific details of your project.