Spaces:

---

AxiomicLabs

/

Open_SLM_Leaderboard

Like 155

Running

Adds Ethoric-10M, a 9.97M-parameter model pretrained from scratch by Tech.us.

Model: https://huggingface.co/techdotus/Ethoric-10M

Benchmark Score

All scores are 0-shot acc_norm, using the Open SLM Leaderboard methodology:

Int Index : 12.94

This PR adds a single Ethoric-10M entry to index.html only (parameter count and scores); nothing else is touched.

Reproducing:

python -m lm_eval --model hf \
  --model_args pretrained=techdotus/Ethoric-10M,dtype=float32,trust_remote_code=True \
  --tasks hellaswag,arc_easy,arc_challenge,piqa \
  --device cuda:0 \
  --batch_size auto:4

ArithMark-3, with the official script from AxiomicLabs/ArithMark-3.0. It defaults to bfloat16, so pass --dtype float32:

python bencharithmark-3.py --model techdotus/Ethoric-10M \
  --data-path arithmark-3.jsonl --device cuda --dtype float32

trust_remote_code=True is still required because the architecture ships in the model repo.

Happy to provide more information if required. Thanks!