Instructions to use lchen1019/LlavaQwen2-Align-TI-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use lchen1019/LlavaQwen2-Align-TI-1B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="lchen1019/LlavaQwen2-Align-TI-1B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("lchen1019/LlavaQwen2-Align-TI-1B", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use lchen1019/LlavaQwen2-Align-TI-1B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "lchen1019/LlavaQwen2-Align-TI-1B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "lchen1019/LlavaQwen2-Align-TI-1B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/lchen1019/LlavaQwen2-Align-TI-1B
- SGLang
How to use lchen1019/LlavaQwen2-Align-TI-1B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "lchen1019/LlavaQwen2-Align-TI-1B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "lchen1019/LlavaQwen2-Align-TI-1B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "lchen1019/LlavaQwen2-Align-TI-1B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "lchen1019/LlavaQwen2-Align-TI-1B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use lchen1019/LlavaQwen2-Align-TI-1B with Docker Model Runner:
docker model run hf.co/lchen1019/LlavaQwen2-Align-TI-1B
Align-TI: Distilling Multimodal Large Language Models via Token Interactions
This repository contains a model distilled using Align-TI, a novel knowledge distillation (KD) framework introduced in the paper Beyond Next-Token Alignment: Distilling Multimodal Large Language Models via Token Interactions.
Overview
Align-TI is designed to compress Multimodal Large Language Models (MLLMs) by focusing on dynamic token interactions rather than just static next-token alignment. It introduces two primary components:
- Instruction-aware Vision Alignment (IVA): Enables the student model to imitate the teacher's ability to extract instruction-relevant visual information by aligning on salient visual regions.
- Transition Probability Alignment (TPA): Captures the teacher's dynamic generative logic by aligning the sequential token-to-token transition probabilities.
The framework achieves state-of-the-art performance for parameter-efficient MLLMs, with the 2B version even outperforming significantly larger models like LLaVA-1.5-7B.
Resources
- Paper: Beyond Next-Token Alignment: Distilling Multimodal Large Language Models via Token Interactions
- GitHub Repository: lchen1019/Align-TI
Citation
If you find this work useful, please cite the paper:
@article{chen2026alignti,
title={Beyond Next-Token Alignment: Distilling Multimodal Large Language Models via Token Interactions},
author={Lin Chen and Xiaoke Zhao and Kun Ding and Weiwei Feng and Changtao Miao and Zili Wang and Wenxuan Guo and Ying Wang and Kaiyuan Zheng and Bo Zhang and Zhe Li and Shiming Xiang},
journal={arXiv preprint arXiv:2602.09483},
year={2026},
}
- Downloads last month
- 21
Model tree for lchen1019/LlavaQwen2-Align-TI-1B
Base model
Qwen/Qwen2.5-0.5B