CARDS-Wind-Gemma4-31B-FP8

FP8-dynamic quantization of C3DS/CARDS-Wind-Gemma4-31B — the LoRA-merged Gemma 4 31B fine-tuned to detect and classify opposition to wind energy under the three-level CARDS-Wind taxonomy.

This is the deployment-friendly variant of the BF16 model: transformer linear weights compressed to 8-bit floating point (fp8_e4m3) via dynamic per-channel quantization, with dynamic per-token activation scaling. The vision tower, lm_head, and token embeddings are kept in BF16, so the model stays fully multimodal (image + text). The merged FP8 checkpoint loads directly with transformers, vLLM (≥0.6 with FP8 Marlin kernels), or any FP8-aware inference engine.

• ~33 GB on disk (vs ~62 GB for the BF16 sibling) — fits on a single A100/H100/H200 80GB with ample headroom for KV cache and image features.
• Near-lossless vs the BF16 model (within <0.01 F1 across metrics; see Results).

What it does

Given a passage of text (news, comment, forum post, council minutes, social media), the model emits a three-level classification:

1. opposition_detected — binary: does the text oppose, dislike, or express skepticism of wind energy? (sarcasm and tonal opposition count)
2. frames — which of 8 high-level argument frames (N_1–N_8, or N_0 fallback) the text invokes.
3. claims — the specific assertions (C_* codes, ~58-code hierarchical codebook) argued within those frames.

When opposition_detected is false, frames and claims are empty.

Results

Evaluated on the held-out Wind test set (773 rows, 436 opposition-only; Macro/Micro F1 restricted to gold codes with support ≥ 3, hallucinated codes dropped). Compared against the paper's best fine-tuned Qwen model and two frontier APIs:

Metric	Gemma4-31B FT (FP8 — this model)	Windy-Qwen3.5-27B FT (FP8)	Claude Opus 4.7	GPT-5.5
Detection — Precision	0.871	0.877	0.896	0.927
Detection — Recall	0.897	0.920	0.890	0.846
Detection — F1	0.884	0.898	0.893	0.885
Samples F1 — frames (all rows)	0.773	0.787	0.791	0.792
Samples F1 — frames (opp. only)	0.730	0.751	0.734	0.697
Samples F1 — claims (all rows)	0.741	0.755	0.754	0.745
Samples F1 — claims (opp. only)	0.674	0.694	0.667	0.614
Macro F1 — frames (opp. only)	0.692	0.664	0.692	0.660
Macro F1 — claims (opp. only)	0.632	0.648	0.635	0.608
Micro F1 — frames (opp. only)	0.788	0.796	0.788	0.768
Micro F1 — claims (opp. only)	0.728	0.735	0.724	0.696
Parse failures	0 / 773	0 / 773	1 / 773	2 / 773

• Competitive with frontier APIs at a fraction of the inference cost — best or tied-best Macro F1 on frames (opposition-only) alongside Claude Opus 4.7, and zero parse failures.
• FP8 dynamic quantization is near-lossless: on the 12B sibling (where both checkpoints were scored) FP8 trails BF16 by only ~0.007 on detection F1 and samples F1. Weight-only FP8 does not degrade this task meaningfully.

Usage

The system prompt and three-level output format are bundled in this repo as wind_prompts.json. The fine-tune was trained with the slim system instruction — use slim_system_instruction.

With vLLM

vllm serve C3DS/CARDS-Wind-Gemma4-31B-FP8 \
  --port 8000 \
  --max-model-len 8192 \
  --enable-prefix-caching \
  --served-model-name CARDS-Wind-Gemma4-31B

The FP8 weights are detected automatically — do not pass --dtype bfloat16 (it would override the FP8 weights and undo the savings).

import json
from huggingface_hub import hf_hub_download
from openai import OpenAI

prompts = json.load(open(hf_hub_download("C3DS/CARDS-Wind-Gemma4-31B-FP8", "wind_prompts.json")))
system_instruction = prompts["slim_system_instruction"]

client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")

def classify(text):
    resp = client.chat.completions.create(
        model="CARDS-Wind-Gemma4-31B",
        messages=[
            {"role": "system", "content": system_instruction},
            {"role": "user",   "content": f"### Text:\n{text}"},
        ],
        temperature=0,
        max_tokens=4000,
        extra_body={"chat_template_kwargs": {"enable_thinking": True}},
    )
    return resp.choices[0].message.content

print(classify("These bird-killing turbines will spike our electricity bills and never pay for themselves."))

The model reasons inside <think>…</think> and then emits a YAML block. To parse: take the content after </think> and read the opposition_detected, frames, and claims fields:

opposition_detected: true
frames:
  - N_1
claims:
  - C_7_0

Thinking must be enabled. Gemma 4 chat templates default enable_thinking off (they pre-fill an empty thought channel); always pass chat_template_kwargs={"enable_thinking": true} or the model classifies without its reasoning chain.

Multimodal — image + text

Gemma 4 is a vision-language model and the vision tower is kept in BF16, so this fine-tune classifies claims depicted in images (memes, screenshots, posters) with or without a caption. Serve with multimodal flags:

vllm serve C3DS/CARDS-Wind-Gemma4-31B-FP8 \
  --port 8000 \
  --max-model-len 8192 \
  --trust-remote-code \
  --limit-mm-per-prompt image=4 \
  --enable-prefix-caching \
  --served-model-name CARDS-Wind-Gemma4-31B

import base64, json, mimetypes
from pathlib import Path
from huggingface_hub import hf_hub_download
from openai import OpenAI

prompts = json.load(open(hf_hub_download("C3DS/CARDS-Wind-Gemma4-31B-FP8", "wind_prompts.json")))
system_instruction = prompts["slim_system_instruction"]

def image_part(path):
    p = Path(path)
    mime = mimetypes.guess_type(p)[0] or "image/png"
    b64 = base64.b64encode(p.read_bytes()).decode()
    return {"type": "image_url", "image_url": {"url": f"data:{mime};base64,{b64}"}}

client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")

resp = client.chat.completions.create(
    model="CARDS-Wind-Gemma4-31B",
    messages=[
        {"role": "system", "content": system_instruction},
        {"role": "user", "content": [
            {"type": "text", "text": "### Text:\nRead the image (and any caption) and classify the wind-energy claim it makes."},
            image_part("meme.png"),
        ]},
    ],
    temperature=0,
    max_tokens=4000,
    extra_body={"chat_template_kwargs": {"enable_thinking": True}},
)
print(resp.choices[0].message.content)

Training & Quantization

Fine-tuning (inherited from the BF16 sibling)

• Base model: C3DS/CARDS-Wind-Gemma4-31B (LoRA-merged Gemma 4 31B)
• Method: LoRA (rank 16, α 16, dropout 0) on q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, then merged into base weights
• Dataset: iRanadheer/wind-opposition-sft — human-annotated wind-opposition passages augmented with Opus-generated RECoT reasoning traces and stratified synthetic positives / near-miss negatives
• Framework: Unsloth + TRL SFTTrainer
• Hyperparameters: 3 epochs, gradient_accumulation_steps=8, lr=2e-4, cosine schedule, 10 warmup steps, max_seq_length=4096, adamw_8bit, bf16, best checkpoint selected on eval loss

FP8 quantization

• Scheme: FP8_DYNAMIC via llm-compressor — fp8_e4m3, per-channel static weight scales, dynamic per-token activation scales. No calibration data required.
• Targets: all Linear layers in the transformer blocks.
• Kept in BF16 (ignored): vision tower (re:.*vision.*), lm_head, and token embeddings (re:.*embed_tokens.*).
• Tool: compressed-tensors (v0.19.0) QuantizationModifier(targets="Linear", scheme="FP8_DYNAMIC") applied to the merged BF16 checkpoint.
• Validation: re-evaluated on the Wind test set (see Results). Near-lossless vs BF16.

Hardware notes

• Inference: any GPU supporting FP8 weights via Marlin or vLLM's FP8 kernels. Tested on A100 / H100 / H200. Ampere up-casts FP8→BF16 for matmul; Hopper uses native FP8 tensor cores.
• Memory: ~33 GB for weights, leaving 45+ GB on a single A100 80GB for KV cache and image features. Comfortable at max_model_len=8192.

Limitations

• Thinking tokens. Training used enable_thinking=True. Gemma 4 defaults it off — always pass chat_template_kwargs={"enable_thinking": true}, and reserve token budget for the reasoning trace before the final YAML block.
• Quantization is weight-only. Activations are dynamically quantized per-token at runtime; vision and output layers stay BF16. For more aggressive compression (W4A16 GPTQ/AWQ), see follow-up work.
• Domain. Tuned specifically for wind-energy opposition under the CARDS-Wind taxonomy; it is not a general climate-claim classifier (see the CARDS sibling models for that).

Citation

@article{coan2026llm,
  title     = {Large language model reveals an increase in climate contrarian speech in the United States Congress},
  author    = {Coan, Travis G. and Malla, Ranadheer and Nanko, Mirjam O. and Kattrup, William and Roberts, J. Timmons and Cook, John and Boussalis, Constantine},
  journal   = {Communications Sustainability},
  volume    = {1},
  number    = {1},
  pages     = {37},
  year      = {2026},
  publisher = {Nature Publishing Group},
  doi       = {10.1038/s44458-025-00029-z},
  url       = {https://www.nature.com/articles/s44458-025-00029-z}
}

License

Apache 2.0, inherited from the base model.