Nekodeus/qwen3-8b-openvino

Qwen3-8B OpenVINO (INT8)

OpenVINO IR conversion of Qwen/Qwen3-8B with weight-only INT8 quantization, produced on CPU-only Kaggle with optimum-cli (fully offline, token-free).

• Base model: Qwen/Qwen3-8B (Qwen3ForCausalLM, 8.2B params, Apache-2.0)
• Task: text-generation
• Precision: weights INT8 (NNCF), activations FP16/FP32 as traced
• Converted with: optimum-intel 2.2.0, openvino 2026.4.1, transformers 5.16.1
• Export time: 427 s on 4-vCPU Kaggle CPU instance
• Contents: openvino_model.xml/.bin, openvino_tokenizer.xml/.bin, openvino_detokenizer.xml/.bin, tokenizer assets (tokenizer.json, vocab.json, merges.txt), configs

Usage

optimum-intel

from optimum.intel import OVModelForCausalLM
from transformers import AutoTokenizer

model = OVModelForCausalLM.from_pretrained("Nekodeus/qwen3-8b-openvino")
tok = AutoTokenizer.from_pretrained("Nekodeus/qwen3-8b-openvino")
inputs = tok("The capital of France is", return_tensors="pt")
print(tok.decode(model.generate(**inputs, max_new_tokens=32)[0]))

OpenVINO GenAI

ov::genai::LLMPipeline pipe("path/to/Nekodeus/qwen3-8b-openvino", "CPU");

Reproduce

optimum-cli export openvino \
  --model Qwen/Qwen3-8B \
  --task text-generation \
  --weight-format int8 \
  qwen3-8b-openvino