--- license: apache-2.0 language: - en - multilingual tags: - byte-level - content-classification - multi-head - classifier - onnx - edge-ai - code-detection - language-detection - clipboard - tiny - open-source - onnx-runtime - content-moderation - risk-detection - secrets-detection - prompt-injection-detection - pii-detection - pico-type-arth pipeline_tag: text-classification library_name: pico-type model-index: - name: pico-type ARTH V2 (composite) results: - task: type: text-classification name: Risk flagging, 14 labels dataset: name: ARTH external audit (held-out, n=211, never trained or fit on) type: private metrics: - type: recall name: positive-class recall value: 0.536 - type: specificity name: specificity value: 0.944 - type: balanced error name: balanced error value: 0.260 - task: type: text-classification name: ToxicChat jailbreak gate (held-out test split) dataset: name: ToxicChat test (toxicchat0124) type: lmsys/toxic-chat metrics: - type: recall name: recall@threshold value: 0.824 - type: accuracy name: selective accuracy at 50% coverage value: 0.973 ---
# pico-type πŸ” **A tiny byte-level multi-head content classifier** β€” ~1.5M params, ~9MB single-file ONNX (FP32), ~18ms CPU inference. Classifies any content from raw bytes: **coarse type Β· modality Β· subtype Β· code language Β· text language Β· file MIME Β· risk flags** [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE) [![Python](https://img.shields.io/badge/python-3.11%2B-blue)]() [![PyPI](https://img.shields.io/pypi/v/pico-type?color=blue)](https://pypi.org/project/pico-type/) [![ONNX](https://img.shields.io/badge/ONNX-exported-success)]() [![CI](https://github.com/eulogik/pico-type/actions/workflows/ci.yml/badge.svg)](https://github.com/eulogik/pico-type/actions/workflows/ci.yml) [![HuggingFace Space](https://img.shields.io/badge/HuggingFace-Space-yellow)](https://huggingface.co/spaces/eulogik/pico-type) [![HuggingFace Model](https://img.shields.io/badge/HuggingFace-Model-orange)](https://huggingface.co/eulogik/pico-type)
--- ## ✨ Features - **No tokenizer** β€” operates directly on raw UTF-8 bytes (supports all languages, no preprocessing) - **7 heads, one forward pass** β€” coarse type, modality, subtype, code language, text language, file MIME, risk flags - **4 Matryoshka tiers** β€” tiny (16d) β†’ small (64d) β†’ base (192d) β†’ pro (576d) β€” same trunk, accuracy scales with dim - **~9MB single-file ONNX (FP32)** β€” deploy on edge devices, serverless, browser (WebAssembly/ONNX Runtime Web) - **~18ms inference** on CPU via ONNX Runtime - **CLI, Python API, Gradio Space, MCP server** β€” ready to use ## πŸ›‘οΈ ARTH V2 β€” Risk++ (new) **[Try it live β€” pick the `arth` model](https://huggingface.co/spaces/eulogik/pico-type)** Β· [Model files](https://huggingface.co/eulogik/pico-type) (`arth_full_base.onnx`, `risk_thresholds.json`, `temperatures.json`, `arth_final_composite.pt`) Frozen-trunk composite (ft3 student + retrained 14-label **Risk++** head) β€” audit recall up **5Γ— with zero regressions** on every shipped gate: | Metric | Before (ft3) | ARTH V2 (composite) | |---|---|---| | External-audit positive recall (held-out, never trained/fit on) | 0.107 | **0.536** (60/112) | | Balanced error | 0.459 | **0.260** | | Specificity | 0.961 | 0.944 | | Labels with fitted thresholds | 9 | **14** | | Legacy parity | 60/60 | 60/60 | | Choice accuracy | 0.737 | 0.737 | | Noul abstain | 0.929 | 0.929 | | ToxicChat jailbreak recall / sel-acc@50 | 0.846 / 0.970 | **0.824 / 0.973** (gate PASS) | Formerly-dead labels now detected: jwt / ssh_key / email **8/8**, password 0.875, phone 0.750. Artifact: **`arth_full_base.onnx` (11.66 MB, opset 18)** β€” auto-verified (torch-vs-ORT err 1.1e-05, parity 60/60, semantic 300/300, risk flags 11/11); INT8 4.28 MB experimental. **Known limitations (measured):** pii_ssn 0.125, api_key 0.625, jailbreak 0.50 on embedded phrasing; 3/11 hand-picked benign probes still flag (incl. the accepted `print('hello world')`β†’sql); long-document signals dilute in fixed-window pooling; general-classification gates (AG/SST-2/Enron) stay at chance β€” this is a byte-pattern risk flagger, not a general text classifier. ### ARTH quickstart (ONNX) ```python import json, math import numpy as np, onnxruntime as ort from huggingface_hub import hf_hub_download sess = ort.InferenceSession(hf_hub_download("eulogik/pico-type", "arth_full_base.onnx")) thrs = json.load(open(hf_hub_download("eulogik/pico-type", "risk_thresholds.json"))) LABELS = ["api_key","jwt","ssh_key","password","email","phone","prompt_injection", "jailbreak","pii_ssn","pii_card","secrets_aws","secrets_github", "sql_injection","xss_payload"] raw = open("file.txt","rb").read()[:1024] ids = np.zeros(1024, np.int64); ids[:len(raw)] = list(raw) mask = np.zeros(1024, bool); mask[:len(raw)] = True (logits,) = sess.run(["riskpp_logits"], { "input_ids": ids[None,:], "attention_mask": mask[None,:], "opt_embs": np.zeros((1,4,96), np.float32), "struct_feats": np.zeros((1,14), np.float32), "act_stats": np.zeros((1,4), np.float32)}) probs = [1/(1+math.exp(-x)) for x in logits[0]] print({l: round(p,4) for l,p in zip(LABELS, probs) if p >= thrs[l]}) # fired flags ``` ### ARTH quickstart (live Space API) ```python from gradio_client import Client c = Client("eulogik/pico-type") out = c.predict("aws_access_key_id = AKIAIOSFODNN7EXAMPLE", "arth", api_name="/handle_classify") print(out[7]) # Risk++ (14) tab: probs + [FLAG] markers ``` ## πŸ“Š Evaluation ### Overall Accuracy (v2 β€” trained on real data) | Head | Classes | Accuracy | Dataset | |------|---------|----------|---------| | coarse | 12 | **100%** | Synthetic eval | | modality | 8 | **100%** | Synthetic eval | | subtype | 24 | **93.8%** | Synthetic eval | | **code_lang** | **62** | **60.3%** | [The Heap](https://huggingface.co/datasets/AISE-TUDelft/the-heap) β€” 24 real-world langs, 1,200 samples | | **text_lang** | **30** | **98.3%** | [Wikipedia](https://huggingface.co/datasets/wikimedia/wikipedia) β€” 30 langs, 1,500 samples | | file_mime | 90 | **100%** | Synthetic eval | | risk (mAP) | 6 | **100%** | Synthetic eval | *v0.1 baseline (synthetic-only): code_lang 3%, text_lang 19%. Real-data training in v2 improves code by **57pp** and text by **79pp**.* ### Code Language β€” Per-Language Accuracy | Excellent (90%+) | Good (70–89%) | Needs Work (<50%) | |---|---|---| | cpp 96%, dart 98%, erlang 98%, rust 98%, r 94%, swift 92%, python 88%, lua 88% | go 86%, ruby 86%, ocaml 84%, php 78%, csharp 76%, java 76%, kotlin 76%, c 62% | perl 50%, haskell 24%, scala 4%, javascript 2%, clojure 0%, elixir 0%, julia 0%, sql 0% | > **Note**: Low-accuracy languages have fewer real training samples. More data will improve them. ## πŸš€ Quick Start ### Install ```bash pip install picotype ``` ### CLI ```bash # Classify from stdin echo "def hello(name):\n return f'Hi {name}'" | picotype --pretty # Classify a file picotype --file document.txt # Classify clipboard content picotype --clip # All 4 tiers available echo "..." | picotype --tier pro ``` ### Python API ```python from picotype import load_onnx_model, run_onnx session = load_onnx_model("base") result = run_onnx(session, "def hello(): pass") print(result) # { # "coarse": "code", # "code_language": "python", # "modality": "textual", # "confidence": 0.98, # ... # } ``` ### MCP Server (for Claude Desktop, Cursor, etc.) ```bash pip install picotype PICOTYPE_MODEL_DIR=./checkpoints python -m model.pico_type.mcp_server ``` Then add to your MCP config: ```json { "mcpServers": { "pico-type": { "command": "python", "args": ["-m", "model.pico_type.mcp_server"], "env": { "PICOTYPE_MODEL_DIR": "./checkpoints" } } } } ``` ### Gradio Web UI Try it live: [huggingface.co/spaces/eulogik/pico-type](https://huggingface.co/spaces/eulogik/pico-type) ## πŸ— Architecture ``` Bytes ─▢ ByteEmbed(256β†’96d) ─▢ 3Γ—Conv1D(k=3,5,7) ─▢ 2Γ—BiAttention(RoPE) ─▢ Pool ─▢ 7Γ—Matryoshka Heads ``` | Component | Detail | |-----------|--------| | **ByteEmbed** | Lookup-free embedding β€” each byte value (0–255) maps to a learned 96-dim vector | | **Conv1D** | 3 parallel depthwise convolutions (kernel widths 3, 5, 7) with residual + layer norm | | **BiAttention** | Bidirectional self-attention with Rotary Position Embeddings (RoPE), 4 heads | | **Pool** | Mean + max + std deviation concatenation β†’ fixed-size representation | | **Heads** | Matryoshka-style: slice pool dim to 16/64/192/576, project to 7 linear classifiers | Total parameters: **1.43M** (tiny) / **1.45M** (small) / **1.48M** (base) / **1.56M** (pro) ## πŸ”§ Model Tiers | Tier | Dim | Params | ONNX Size | Accuracy Multiplier | |------|-----|--------|-----------|-------------------| | tiny | 16 | 1.43M | 9.09 MB | 0.65Γ— | | small | 64 | 1.45M | 9.13 MB | 0.82Γ— | | base | 192 | 1.48M | 9.25 MB | 1.0Γ— (reference) | | pro | 576 | 1.56M | 9.61 MB | 1.05Γ— | ONNX sizes are single-file FP32 exports (graph-only files are 203–206 KB). All tiers share the same backbone; only the final linear projection layers differ. Higher-tier models use more dimensions for finer-grained classification. ## πŸ§ͺ Classification Heads | Head | Classes | What It Detects | |------|---------|-----------------| | **coarse** | 12 | text, code, link, image, file, config, markup, data, error, secret, archive, binary | | **modality** | 8 | textual, binary_image, binary_archive, binary_executable, binary_document, etc. | | **subtype** | 24 | json, yaml, toml, csv, html, markdown, sql, log, dockerfile, makefile, etc. | | **code_lang** | **62** | python, javascript, typescript, java, c, cpp, go, rust, ruby, php, swift, kotlin, and 50 more | | **text_lang** | 30 | en, es, fr, de, it, pt, nl, ru, zh, ja, ko, vi, th, id, and 15 more | | **file_mime** | **90** | application/json, image/png, video/mp4, font/ttf, application/wasm, and 84 more | | **risk** | 6 | api_key, jwt, password, email, phone, ssh_key | ## 🌐 Deployment | Platform | Link | Notes | |----------|------|-------| | **HuggingFace Space** | [eulogik/pico-type](https://huggingface.co/spaces/eulogik/pico-type) | Gradio web UI, no GPU needed β€” ARTH V2 + Risk++ (14) by default | | **HuggingFace Model** | [eulogik/pico-type](https://huggingface.co/eulogik/pico-type) | ONNX models + ARTH V2 artifacts (`arth_full_base.onnx`, thresholds, temps, ckpt) | | **GitHub** | [eulogik/pico-type](https://github.com/eulogik/pico-type) | Source code, training, paper | | **PyPI** | `pip install picotype` | Python package | | **ONNX Runtime** | Use with onnxruntime.js | Browser/Node.js deployment | ## πŸ“š Resources - [Paper](paper/main.pdf) β€” Architecture, training, and evaluation details - [Model Card](MODEL_CARD.md) β€” Detailed architecture and training configuration - [Walkthrough](walkthrough.md) β€” Development log and decisions - [Architecture Plan](docs/PLAN.md) β€” Original design document ## πŸ“„ License Apache 2.0 ---
Built with PyTorch Β· ONNX Β· Gradio Β· HuggingFace