A-M-K/firebrief-glimmer-30b-extract-lora

FireBrief extractor: Glimmer 30B LoRA adapter

A LoRA adapter for Muse-Glimmer-30B (MLX, 4-bit) that turns 911-call transcripts, dispatch radio and civilian reports into a structured incident card: address, fire locations, people (including mobility needs), obstructions, hazards and radio codes. Every extracted fact keeps a verbatim quote from its source.

It powers FireBrief, a hackathon prototype that turns emergency calls into a floor-by-floor building brief for arriving firefighters.

Results (held-out test set: 60 calls and reports, never seen in training)

Metric	Base model (prompt only)	With this adapter	Change
Field accuracy	0.59	0.85	+0.26
Fire F1	0.68	0.95	+0.27
People F1	0.73	0.82	+0.09
Obstruction F1	0.92	1.00	+0.08
Calibration error (ECE, lower is better)	0.19	0.10	about halved
Valid output (schema)	100%	100%	same
Quotes verbatim from source	100%	100%	same
Hazard F1	1.00	0.97	−0.03
Radio-code F1	0.95	0.90	−0.05*

* Restored to 1.00 in FireBrief by deterministic post-processing rules (not part of this adapter).

Latency is about the same as the base model: roughly 12 s for a demo-sized batch of 5 sources on an Apple-silicon laptop (about 10 s for the base).

Training

• Method: QLoRA. The 4-bit base is frozen; rank-8 LoRA (scale 20, no dropout) on the last 8 transformer layers.
• Data: 250 training / 40 validation / 60 test examples. All synthetic call transcripts and reports written for this project, with gold incident cards as targets.
• Recipe: 1 epoch, batch size 4, learning rate 1e-4 cosine-decayed to 1e-5, completion-only loss, gradient checkpointing, max sequence length 2048, seed 7.
• Run: about 19 minutes on one Apple-silicon Mac (GPU shared with another job). Validation loss 0.270 → 0.080. The published weights are the iteration-60 checkpoint (240 of 250 rows, 0.96 epoch).

Usage (mlx-lm)

from mlx_lm import load, generate

model, tokenizer = load("path/to/glimmer-30b-4bit", adapter_path="A-M-K/firebrief-glimmer-30b-extract-lora")

To merge the adapter into the base weights for faster loading:

python -m mlx_lm fuse --model path/to/glimmer-30b-4bit --adapter-path <this repo> --save-path glimmer-30b-firebrief-fused

The adapter was trained on FireBrief's own extraction prompt and compact card format, so it performs best with that prompt.

Limitations

• A single training run with no hyperparameter search; each arm was scored once.
• Training and test data are synthetic, English only, and centred on New York City addresses. It has not been validated on real 911 audio or transcripts.
• A research prototype, not for operational emergency dispatch. Every output needs human verification.