yashchouhann/laya-typed-decisions

Laya (Fine-Tuned on Typed-Decisions Benchmark)

This is Laya fine-tuned on the 1,200 training cases (6,000 decisions) of the independent LocalLLaMA/typed-decisions benchmark.

On the official 400-case test set (2,000 decisions across Agent Trace Observability, Customer Service, Invoice Processing, and Security Incidents), it achieves 0.709 Accuracy, trailing TypeSafe Jev 1.13.0 (0.727) and the benchmark's Teacher Self-Agreement ceiling (0.735).

Head-to-Head Benchmark Results

Model	Kind	Accuracy	Soft Acc	Brier Score	ECE	Score MAE	Within 1 Level	Latency (p50)	Cost/Case
Laya (Ours)	fine-tuned	0.709	0.539	0.085	0.116	0.328	0.964	152.8 ms	$0.00 (Self-Hosted)
TypeSafe Jev 1.13.0	general	0.727	0.580	0.148	0.144	0.391	0.952	710 ms	$0.0004 (API)
ModernBERT-base (149M)	specialist	0.646	0.542	0.119	0.179	0.444	0.931	349 ms	$0.00
Teacher Self-Agreement	ceiling	0.735	-	-	-	-	-	-	-

Installation & Quickstart

pip install laya

import laya

# Load the fine-tuned model directly from Hugging Face
agent = laya.load("convaiinnovations/laya-typed-decisions")

# Evaluate any workflow state and typed questions in a single forward pass
result = agent.predict(state, questions)
print(result["answers"])

License

Apache 2.0. Developed by Convai Innovations.