Eye

Eye tags photos on the device so apps can find and filter them: what a photo is (screenshot, receipt, selfie), what is in it (558 scenes), how it came out (blurry, eyes closed), who is in it (group shot, portrait) and whether it is sensitive, each with a calibrated confidence where 0.5 is the recommended cutoff and 0.9 a strict one. It also scores photos, finds people, pets and faces, and picks crops.

Files

File	Format	Size	Runs on
eye.mlmodelc	Compiled Core ML, one function per task	48MB	Apple platforms; fastest on the Neural Engine, also runs on CPU and GPU
eye.tflite	LiteRT, one signature per task	66MB	Android, Linux, Windows, Node, the browser
eye_meta.json	JSON	880KB	Inputs, outputs, decoding constants and the tags (calibration curves and rules)

Functions

Both files hold the same functions under the same names. Every input is RGB (or gray for sharpness) in 0...1, shape (1, C, H, W): float16 for Core ML, float32 for LiteRT. eye_meta.json has the exact preparation and how to decode each output.

Function	Input	Outputs
photo	384x384 centre crop	embedding, type, reject reasons, sensitive content, scenes, rotation, aesthetic, keeper, cover, focus and bokeh, pet attributes
photo_peak	as photo	photo's outputs and the peak moment
spatial	whole frame squashed to 384x384	where people look, pet and person boxes, shot type, scores for 1,296 crop boxes
faces_detect	768x768, the photo letterboxed	face heatmap, size, offset, landmarks
faces_analyze	128x128 crop, 1.4x a face box	eyes open, blink, smile, gaze, expression, pose, quality and more
sharp_4x3, sharp_3x4, sharp_16x9, sharp_9x16	gray, the shape closest to the photo's aspect	sharpness, blur map

Tags

Every tag has a confidence calibrated so 0.5 is the recommended cutoff and 0.9 a strict one. At 0.5: photo types 0.8 to 1.0 precise (screenshot 0.98, receipt 1.00), scenes 0.88 median, people 0.95, eyes closed 0.92. The keep-or-delete reasons (forgettable, bad framing, ...) are less reliable; use them for suggestions. For anything automatic, such as hiding sensitive photos, use the strict cutoff.

Accuracy

On the same test sets, through the same scoring code:

Test set	Metric	eye.mlmodelc	eye.tflite (CPU)
KonIQ-10k	aesthetic SRCC	0.6288	0.6291
COCO val2017	person AP50	0.3814	0.3837
MIT1003	where people look, CC	0.7153	0.7195
CUHK cropping	crop IoU	0.8244	0.8241
WIDER Face val	AP, hard	0.6806	0.6812
CEW	eyes open AUC	0.9943	0.9945

Speed

One photo, photo_peak, on a Mac with an Apple M6:

Runtime	ms
Core ML, Neural Engine	5.5
LiteRT, GPU (Metal)	30
LiteRT.js, WebGPU, in Chrome	41
LiteRT, CPU, 4 threads	49
LiteRT.js, wasm with threads, in Chrome	176

Limitations

• eye.tflite is not yet measured on Android devices.
• Face analysis describes a face. It doesn't identify who someone is.
• Eye is trained on photos. Scores for illustrations, screenshots and heavily edited images are less reliable.

Built on

• Image encoder: PE-Core S/16-384 by Meta, Apache 2.0, fine-tuned and with heads trained by Desert Ant Labs.
• The face detector, face analyzer and sharpness model are Desert Ant Labs' own, trained from scratch.

Attribution and license texts for third-party components are in THIRD_PARTY_NOTICES.md.

License

Released under the Desert Ant Labs Source-Available License v1.0 (see LICENSE.md). Free for commercial use up to 100,000 MAU per Model. Contact licensing@desertant.com for commercial licensing at larger scale.