parchi-v2-GGUF

Parchi reads the purchase note a grocery (kirana) shopkeeper types to a supplier, in Hinglish, English or Telugu (script or romanised), and returns the order as JSON: vendor, plus item, quantity, unit and rupee rate for each line.

It is Qwen2.5-0.5B-Instruct fine-tuned with LoRA (r=16) on 2,600 generated notes over a 64-item catalogue, trained on a laptop CPU, then merged. Scores are exact match on notes written by hand that the model never saw: every field of every line has to be right. Hinglish set: 34 notes; Telugu set: 36 notes.

This repo holds the GGUF builds for llama.cpp, Ollama, LM Studio and on-device apps. The full 16-bit model for Python is ajayk0608/parchi-v2.

Which file should I download?

File	Build	Size	Best for	Hinglish exact	Telugu exact
parchi-v2	Full model (safetensors)	953 MB	Developers in Python, servers, and anyone fine-tuning further	74%	64%
parchi-v2-LoRA	LoRA adapter (PEFT)	44 MB	Builders who fine-tune further or already have Qwen2.5-0.5B-Instruct	74%	64%
parchi-v2-f16.gguf	F16 GGUF	948 MB	People who want to make their own quantized builds	—	—
parchi-v2-q8_0.gguf	Q8_0	506 MB	Laptops and desktops, shop-counter PCs, servers without a GPU	74%	64%
parchi-v2-q6_k.gguf	Q6_K	482 MB	Newer phones (6 GB RAM or more) and small office PCs	74%	67%
parchi-v2-q5_k_m.gguf	Q5_K_M	400 MB	Mid-range phones (4 to 6 GB RAM) that want accuracy first	82%	69%
parchi-v2-q4_k_m.gguf ⭐	Q4_K_M	379 MB	Most phones: the default choice for an on-device app	76%	75%
parchi-v2-q4_0.gguf	Q4_0	335 MB	Older or budget Android phones (3 to 4 GB RAM), Raspberry Pi	68%	42%

⭐ = the default for a phone app. A "—" means that build has not been scored.

The hand-written sets are small (34 and 36 notes), so one note moves a score by about 3 points. Builds within a few points of each other perform the same, and a quantized build scoring above the full model is noise, not an improvement.

• F16 GGUF: Lossless GGUF export; the source every smaller file below was made from.
• Q8_0: Practically the same answers as the full model at about half the size. Pick this when memory is not tight.
• Q6_K: Near-lossless and a little smaller than Q8_0.
• Q5_K_M: The usual best balance of size and accuracy for small models.
• Q4_K_M: Smallest file that still matches the full model on almost every note.
• Q4_0: Simplest 4-bit layout and the fastest kernels on ARM CPUs, but it loses Telugu accuracy (42% exact against 64% for the full model). Use it for Hinglish and English notes, or when Q4_K_M will not fit.

Quick picks: building a phone app → Q4_K_M · budget or older Android phone, Hinglish notes only → Q4_0 · laptop or shop PC → Q8_0 · making your own quant → F16.

Prompt

Always send this system prompt (the model was trained with it), then the note as the user message. Use greedy decoding (temperature 0).

You read a kirana (grocery) shop purchase note, often in Hinglish, and return the order as JSON only. Schema: {"vendor": string or null, "lines": [{"item": string, "qty": number, "unit": string, "rate": number or null}]}. item is the canonical English name; unit is one of kg, g, l, ml, pcs, dozen, packet, bottle, bunch, box; rate is the rupee price per unit if stated, else null; vendor is who the note is addressed to, else null.

Example output:

{"vendor":"Sharma Store","lines":[{"item":"wheat flour","qty":10,"unit":"kg","rate":null},{"item":"mustard oil","qty":1,"unit":"l","rate":165}]}

Run it

# llama.cpp
llama-server -hf ajayk0608/parchi-v2-GGUF:Q4_K_M
# Ollama
ollama run hf.co/ajayk0608/parchi-v2-GGUF:Q4_K_M

Then send the system prompt above plus the note, with temperature 0, e.g. to llama-server's OpenAI-style /v1/chat/completions.

Limits

• Knows the 64 catalogue items and their spellings; an unknown item is guessed onto a known one.
• One order per note, up to about six lines; rates are per unit as written.
• Trained on generated notes: real notes can surface patterns it has not seen (common misses: the next line's number taken as a rate, near-neighbour Telugu spellings, brand-only items).