manga-ocr-base-CPT

English | 中文

Code, technical report and findings: GitHub muscgab/manga-ocr-CPT · Weights: Hugging Face · ModelScope

OCR for Japanese manga text boxes. This is mangaOCR after continued pre-training on our training data, with the architecture unchanged (ViT-B/16 encoder, 2-layer BERT decoder, autoregressive, 111M parameters) and a new vocabulary of 6,817 characters. Input: stretched to 224 × 224 (14 × 14 patches), pixel-identical to the original mangaOCR ViTImageProcessor.

The control of the two models: its input is exactly that of the original mangaOCR, so it shows the effect of changing only the data. For general use, manga-ocr-base-CPT-flex is better on elongated and horizontal boxes.

This model comes from an experimental study; see the technical report.

Accuracy

JMangaBench_Read v1.3, 7,789 boxes, RIN v1 normalisation, batch 1, RTX 2080 Ti.

Model	Score ↑	CER % ↓	EM % ↑	EM printed	EM non-printed
manga-ocr-base-CPT	97.43	1.77	91.00	93.82	82.32
manga-ocr-base-CPT-flex	97.45	1.53	91.42	94.16	83.00
mangaOCR (original, package defaults)	90.23	4.84	78.25	88.04	48.17

Speed

RTX 2080 Ti, Read v1.3 speed protocol, preprocessing included; a page (8.7 boxes on average) is one batch.

Inference path	Per crop (batch 1)	Page p50	Peak throughput
CUDA path (CUDA graphs + GPU preprocessing)	10.4 ms	38.1 ms	277 crops/s (batch 32)
Plain PyTorch path	27.6 ms	69.7 ms	234 crops/s (batch 64)
Original manga-ocr package (fp16)	84.9 ms	242.4 ms	42 crops/s (batch 56)

Both paths give identical outputs on all 7,789 boxes.

Usage

The inference code is on GitHub; the weights are downloaded from this repository on first use.

pip install git+https://github.com/muscgab/manga-ocr-CPT

from manga_ocr_cpt import MangaOcrCPT
ocr = MangaOcrCPT("224")              # or a local directory holding this repository
ocr("crop.png")                          # one crop -> str
ocr(["crop1.png", "crop2.png"])          # the crops of one page -> list[str]

Inputs are cropped text boxes (file paths, PIL images or uint8 RGB arrays); there is no text detection. With an NVIDIA GPU the CUDA path is used, otherwise plain PyTorch (CPU, Apple MPS, ...).

Training

• Initialised from kha-white/manga-ocr-base; the 5,133 characters already in the original vocabulary keep their embeddings.
• Data (same as manga-ocr-nar-small): 4,471,985 crops: synthetic printed text 1,795,869, synthetic handwriting 1,670,000, AnimeText-OCR real crops 1,006,116. Labels of 1–64 characters.
• 139,749 steps, global batch 256 (8 epochs), 4 × RTX 2080 Ti; encoder fp16 mixed precision, decoder fp32. Released weights are from the last step.

Files

File	Contents
model.safetensors	Weights, fp32 (444 MB)
config.json	Architecture and input settings (input.fit selects the input)
nar_vocabulary_6817.json	Vocabulary (6,817 characters)
LICENSE_MODEL	CC BY-NC-SA 4.0
NOTICE.md	Provenance and acknowledgements
SHA256SUMS	Checksums

Limitations

• Japanese only. At most 64 characters per box; longer boxes must be split first. Boxes of more than 32 characters are clearly weaker.
• Handwriting and stylised lettering produce clearly more errors than printed text.
• Aimed at dialogue, narration and small annotations; onomatopoeia is not a target.
• The benchmark was built by the same author; parts of it were used during development. Not evaluated on scanned manga such as Manga109-s.

License

• Weights: CC BY-NC-SA 4.0 (LICENSE_MODEL); commercial use requires separate permission. The training data AnimeText-OCR is CC BY-NC-SA 4.0.
• Initialised from kha-white/manga-ocr-base (Apache License 2.0). See NOTICE.md.
• The inference code on GitHub is Apache License 2.0.