Feature Extraction
Transformers
Safetensors
qwen3_5
gptq
hadamard
quip
w4a16
compressed-tensors
decision-model
code-readout-v1
custom_code
Instructions to use nchapman/d3-W4A16-HGPTQ with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use nchapman/d3-W4A16-HGPTQ with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="nchapman/d3-W4A16-HGPTQ", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModel processor = AutoProcessor.from_pretrained("nchapman/d3-W4A16-HGPTQ", trust_remote_code=True) model = AutoModel.from_pretrained("nchapman/d3-W4A16-HGPTQ", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
d3-W4A16-HGPTQ
4-bit WalshโHadamard + Cholesky GPTQ (W4A16-HGPTQ, group size 64) quantization of vllm-sr/d3 (vllm-sr/d3 code-readout-v1 single-pass decision model). It compresses the 48.59 GiB BF16 checkpoint (readout.safetensors + backbone) to 14.69 GiB (3.31ร smaller) while preserving 100.00% top-1 decision agreement and 0.999917 readout logit cosine similarity against the BF16 reference.
- Footprint:
14.69 GiBtotal across400quantized text linears, plus137vision tower & merger linears, plus the exact BF16readout.safetensors(255 ร 5120code-readout head). - Quantization recipe: 64ร64 Sylvester WalshโHadamard input rotation (
H_64) + MSE-initialized Cholesky GPTQ (group_size=64,actorder="static",dampening_frac=0.01), calibrated layer-by-layer undernoncausal_full_attentionattention. - Architecture & recurrence:
64layers (hidden_size=5120). Single-pass hybrid GatedDeltaNet + full-attention forward pass. - Canonical weight layout: All safetensors keys use canonical
model.language_model.*(andmodel.visual.*) prefixes with"quantization_status": "compressed", loading cleanly withmissing: 0, unexpected: 0inQwen3_5Model.from_pretrained,D3.from_pretrained, andwarp-34.
Quality vs. BF16 Base Model (vllm-sr/d3)
Single-pass code-readout-v1 evaluation across multi-question choice, yes_no, multi_select, and scale decision tasks (with forward + reverse option-permutation averaging):
| Evaluation | BF16 Base (vllm-sr/d3) |
d3-W4A16-HGPTQ |
Notes |
|---|---|---|---|
Safetensors Size (incl. readout.safetensors) |
48.59 GiB (52,176,025,120 B) |
14.69 GiB (15,772,221,072 B) |
3.31ร compression (โ33.90 GiB) |
Decision Top-1 Agreement (12 questions) โ |
100.00% |
100.00% |
Forward + reverse permutation-averaged readout |
| Option Probability KL Divergence (mean / max nats) โ | 0.000000 |
0.000009 / 0.000071 |
Per-question normalized option distribution KL |
| 255-Code Readout Logit Cosine Similarity โ | 1.000000 |
0.999917 |
Full 255-code softmax KL 0.000020 nats |
Vision Tower (vision-probe-v2) (96 images, 50,596 visual tokens) |
1.000000 cos sim |
0.995219 mean cos (0.976177 min) |
vision-guard-v2 mean cos 0.996538 (min 0.991070) |
What Is Quantized
- Quantized (
537linear layers inW4A16-HGPTQ):- Language (
400linears): All64hybrid GatedDeltaNet + full-attention blocks (q_proj,k_proj,v_proj,o_proj,in_proj_qkv,in_proj_z,out_proj,gate_proj,up_proj,down_proj) with online block-Hadamard rotation (head_dim=64) + Cholesky GPTQ (group_size=64,actorder="static"). - Vision (
137linears): All 27 ViT blocks (visual.blocks.*.{attn.qkv,attn.proj,mlp.linear_fc1}ing64-h64,visual.blocks.*.mlp.linear_fc2in per-channel GPTQ +head_dim=16Hadamard) plusvisual.merger.{linear_fc1,linear_fc2}.
- Language (
- Kept in BF16:
readout.safetensors(weight[255, 5120]),embed_tokens,linear_attn.in_proj_a,linear_attn.in_proj_b, norms, biases, andvisual.patch_embed.proj.
Usage
from d3_runtime import D3
model = D3.from_pretrained("nchapman/d3-W4A16-HGPTQ", device="cuda:0")
result = model.system_one(
state="Customer ordered item #4412, delivered on time, requested an invoice copy.",
questions={
"intent": {
"type": "choice",
"instructions": "What does the customer need?",
"criteria": {"invoice": "Copy of invoice", "refund": "Return and refund"},
}
},
)
print(result)
- Downloads last month
- 31