genshiai-daichi/baberu-ocr

Baberu OCR

English · 日本語

Baberu OCR reads the text inside a manga speech bubble. One 115M model, three languages. On the public Japanese benchmark it beats the specialist it was distilled from (manga-ocr); on Chinese and English it goes toe-to-toe with a general model 8× its size — all measured on independent ground truth.

v1.1 (2026-10): long bubbles. v1.0 stopped reading at about 64 characters, so long English narration boxes came back cut off (discussion #2). v1.1 reads up to 256 — see Long bubbles. All numbers on this page are v1.1's. v1.0 stays available with --revision v1.0.

Most manga OCR is Japanese-only and assumes clean horizontal lines. Real manga bubbles are not that. They run vertical, pack in sound effects, switch scripts mid-sentence, and mix full-width and half-width characters. Baberu was trained for exactly that, in Japanese, Chinese, and English at once.

At a glance

• Three languages, one checkpoint (ja / zh / en). Lowest character-error-rate of every model compared on Japanese; on Chinese and English it matches a model 8× its size on the typical bubble, and is more robust.
• 115M parameters, about one-eighth the size of the 0.9B vision-language models it goes toe-to-toe with.
• Character-level (14,630-symbol vocab): sound effects, rare kanji, and full/half-width mixing stay one token each, with no subword fragmentation.
• Local and small. Apache-2.0, shipped as a 122 MB or 243 MB ONNX build that needs only onnxruntime, numpy, and pillow.

>>> from inference import BaberuOCR
>>> ocr = BaberuOCR(".")
>>> ocr(Image.open("bubble.jpg"))
'私がこの手で啓太を守る！！'

How it works

DINOv2 looks, a small decoder writes.

The decoder is built from scratch: 6 layers, grouped-query attention, SwiGLU, RMSNorm sandwich, 1D RoPE, tied embeddings. Nothing exotic. What mattered was the training order.

Freeze the vision encoder, train the decoder, then unfreeze the encoder and keep training. The accuracy jumps in that second stage, and it saturates fast: around 30k steps, roughly a tenth of an epoch. Character-level output fits the task — the target is a sequence of characters — and avoids the uneven, ballooning tokenizations that subword vocabularies produce on sound effects and mixed scripts (manga-ocr is character-level for the same reason).

Benchmarks

Teacher labels would grade their own homework, so every number here is measured against independent ground truth, on images the model never trained on. Japanese uses the public human-labeled Manga109-v2026. For Chinese and English we built a fresh held-out set from galleries published after the training cutoff (IDs above the training maximum — zero overlap) and re-annotated every crop by hand, with the teacher output hidden. The metric is lCER (lenient character-error-rate: NFKC, whitespace stripped, decorative symbols dropped, dot-runs unified), lower is better.

Japanese, Manga109-v2026 (n=2000)

model	size	lCER ↓	nCER ↓
Baberu OCR	115M	0.0359	0.0890
PaddleOCR-VL	0.9B	0.0368	0.0808
manga-ocr (its teacher)	110M	0.0422	0.0954
PP-OCRv5	~10M	0.6317	0.6491

On Japanese, Baberu beats its actual teacher, manga-ocr, on every metric, and edges the 0.9B on lenient CER. The 0.9B stays ahead on the stricter nCER and on exact-match — it is eight times larger. PaddleOCR-VL-For-Manga posts a lower Japanese number but trained on Manga109, so that score is a data leak and sits out.

Chinese and English, hand-annotated held-out

Two ways to count, because they disagree and the disagreement is the result. Per-bubble averages the error over each bubble (the unit a bubble reader delivers); char-weighted weights by total characters (long passages dominate). PaddleOCR-VL is Baberu's Chinese/English teacher.

metric	zh: Baberu 115M	zh: PaddleOCR-VL 0.9B	en: Baberu 115M	en: PaddleOCR-VL 0.9B
per-bubble lCER ↓	0.060	0.077	0.054	0.081
char-weighted lCER ↓	0.048	0.030	0.021	0.028
exact-match ↑	0.84	0.83	0.83	0.79
n	246		237

Per bubble, the 115M student wins both languages and gets more bubbles exactly right; it also never melted into a repetition loop, while the 0.9B did on two short English bubbles. Per character, the 0.9B still wins Chinese, where it reads long passages better. On English, v1.1 now wins that too: v1.0's English gap was mostly the 64-character cutoff below. Call it toe-to-toe with a model 8× its size: ahead and more reliable on the typical bubble, behind on long Chinese text.

Long bubbles (v1.1)

Up to v1.0, every training label was cut at 64 characters and the cut label still ended in end-of-text. About 8% of English labels are longer than that (Japanese and Chinese bubbles rarely are), so the model learned to stop right after character 64. v1.1 trains the decoder further on labels up to 256 characters, with no end-of-text after a cut and without the teacher's own repetition loops, which made up a quarter of the long Japanese labels and half of the Chinese ones. The vision encoder and projector stayed frozen, so the vision ONNX files are byte-identical.

English bubbles of 65–256 characters, from two shards the v1.1 run never read, scored against the teacher's labels with the shipped ONNX build (vision_fp16 + decoder_*_int8):

characters	n	v1.0 lCER ↓	v1.0 exact ↑	v1.1 lCER ↓	v1.1 exact ↑
65–96	100	0.100	0.36	0.002	0.89
97–128	100	0.465	0.02	0.004	0.88
129–192	100	0.548	0.00	0.036	0.53
193–256	19	0.536	0.00	0.257	0.05

Past about 190 characters the end of the bubble still comes out garbled. At 224×224 the letters get too small to read, which is a limit of the input size rather than of training.

On short bubbles the numbers dipped slightly, within noise: against v1.0 on the independent sets above, Manga109 lCER moves 0.0344 → 0.0359 and Chinese per-bubble lCER 0.052 → 0.060, and the 95% paired-bootstrap interval of each difference includes zero. Reading further also needed one more guard in the published decode: a run of one symbol (ー, ～, ・) now stops at 16, as one letter or kana already stopped at 12. Without it, a stretched ー on one Manga109 page ran to the token limit.

Where it loses

Public, never-seen gold labels from COMICS Text+ (1940s Western comics, n=480):

model	size	lCER ↓
PaddleOCR-VL	0.9B	0.0256
PP-OCRv5	~10M	0.1332
Baberu OCR	115M	0.1488

On vintage Western comics it has never seen, the general 0.9B model reads cleaner. Baberu is tuned for modern manga, and this gap marks the edge of that focus rather than hiding it.

Quick start

The input is one speech-bubble crop (run a text detector upstream), not a full page.

ONNX, the small path (no PyTorch):

pip install onnxruntime numpy pillow
huggingface-cli download genshiai-daichi/baberu-ocr --local-dir baberu-ocr
cd baberu-ocr

python onnx_infer.py --image bubble.jpg                          # 243 MB, lossless
python onnx_infer.py --vision vision_int4.onnx --image bubble.jpg # 122 MB, smallest

onnx_infer.py carries its own preprocessing, KV-cache decode loop, and the published decode settings (greedy, repetition_penalty=1.2, and run caps of 12 for one letter or kana and 16 for one symbol), so its output matches the PyTorch model.

PyTorch, for fine-tuning or transformers:

pip install torch transformers safetensors pillow
huggingface-cli download genshiai-daichi/baberu-ocr --local-dir baberu-ocr
cd baberu-ocr
python inference.py --model . --image bubble.jpg

To fine-tune the released weights on your own bubbles, see TRAINING.md: train_ocr.py --finetune-from . continues training with a fresh optimizer (add --unfreeze-vision to adapt the encoder too). The full from-scratch recipe (train_text.py → train_ocr.py) is included as well.

Sizes and quantization (ONNX)

component	fp16	int8	int4
vision (86M)	173 MB	92 MB	52 MB
decoder (prefill+step)	n/a	70 MB	(89 MB)

Two tiers are recommended, both with the decoder at int8:

tier	files	size	nCER (ja)
lossless	vision_fp16.onnx + decoder_*_int8.onnx	243 MB	0.0897
smallest	vision_int4.onnx + decoder_*_int8.onnx	122 MB	0.0901

The measured sweep (quantization-results.md) found two things worth knowing. Decoder int8 is close to free here (v1.1: 0.0897 against fp32 PyTorch's 0.0890), because greedy decoding shrugs off weight-quantization noise. The cost lives entirely in vision, and even there int4 weight-only beats int8-dynamic on both size and accuracy. Both tiers still clear the teacher manga-ocr (0.0954). The dominated variants (vision int8-dynamic, decoder int4) are not shipped.

Limitations

• One bubble crop in, text out. You supply the text-detection stage. This is not page-level OCR.
• It confuses dakuten and handakuten (ポ vs ボ), misses some rare kanji, and drops the occasional small kana.
• A bubble of more than about 190 characters reads well at the start and garbles toward the end (the 224×224 input is too coarse for that much text).
• On print far outside its domain, a general large VLM reads better.

Training data and license

Trained on an independently collected corpus of manga speech-bubble crops, with text labels distilled from manga-ocr (Japanese) and PaddleOCR-VL (Chinese, English). The training images are not redistributed; only the weights and code ship here. License is Apache-2.0 for the weights and code, matching the upstream components (DINOv2, manga-ocr, and PaddleOCR are all Apache-2.0).

Acknowledgements

• DINOv2 (Meta), the vision encoder
• manga-ocr, the de-facto Japanese manga OCR baseline and the Japanese teacher
• PaddleOCR-VL, the Chinese and English teacher