Instructions to use TeichAI/Qwen3.8-27B-Fable-Distill with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TeichAI/Qwen3.8-27B-Fable-Distill with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="TeichAI/Qwen3.8-27B-Fable-Distill")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("TeichAI/Qwen3.8-27B-Fable-Distill")
model = AutoModelForMultimodalLM.from_pretrained("TeichAI/Qwen3.8-27B-Fable-Distill", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use TeichAI/Qwen3.8-27B-Fable-Distill with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TeichAI/Qwen3.8-27B-Fable-Distill"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TeichAI/Qwen3.8-27B-Fable-Distill",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/TeichAI/Qwen3.8-27B-Fable-Distill

SGLang

How to use TeichAI/Qwen3.8-27B-Fable-Distill with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TeichAI/Qwen3.8-27B-Fable-Distill" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TeichAI/Qwen3.8-27B-Fable-Distill",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TeichAI/Qwen3.8-27B-Fable-Distill" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TeichAI/Qwen3.8-27B-Fable-Distill",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Unsloth Desktop
Docker Model Runner
How to use TeichAI/Qwen3.8-27B-Fable-Distill with Docker Model Runner:
```
docker model run hf.co/TeichAI/Qwen3.8-27B-Fable-Distill
```

So quick to have distilled version, excellent!

by weisunding - opened Aug 18

Discussion

weisunding

Aug 18

Thanks, I will try it.

armand0e

TeichAI org Aug 18

That’s the power of my new hardware! Thanks for the support 🤞

ntoytoy

Aug 22

•

edited Aug 22

thanks will test the gguf

nlpguy

Aug 23

•

edited Aug 23

I've only been using the bf16 for a single task so far, but I quite like it. Very coherent.

Leonard23

10 days ago

How about the vision capabilaty?

armand0e

TeichAI org 10 days ago

vision layers weren't tuned at all so it will be the exact same as Qwen 3.8 base

johnlockejrr

10 days ago

Any plans to distill Qwen3.8-Flash-Next on Fable? 😉

armand0e

TeichAI org 10 days ago

Oooof, that’s a chunky one… 😅

Will have to see how much compute that would take and how much money it would cost to do. More than likely going to wait for Qwen4 before making any big moves like that

EclipseMist

TeichAI org 8 days ago

Oooof, that’s a chunky one… 😅

Will have to see how much compute that would take and how much money it would cost to do. More than likely going to wait for Qwen4 before making any big moves like that

I don't even think my spark cluster could tune that.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment