🇹🇷 Turkish BGE-M3 Model2Vec (533k Distilled Champion)
High-performance, ultra-fast Turkish static sentence embedding model distilled from BAAI/bge-m3 on 533k Turkish Wikipedia articles using 256-dimensional PCA teacher representations.
- Model Size: 19.36 MB (114x smaller than 2.2 GB BGE-M3)
- Embedding Dimension: 256
- Vocabulary Size: 39,655 pruned Turkish tokens
- Inference Speed: 63,012 sentences/sec on CPU (797x faster than BGE-M3)
- Accuracy (STSb-TR gorkem): 91.36% (Preserves ~95% of BGE-M3 teacher accuracy while beating the 182 MB unpruned model's 86.83% baseline)
📊 Benchmark Results
| Model | Size | Speed (sent/s) | STSb-TR (gorkemergune/stsb-tr) Pearson |
STSb-TR (emrecan/stsb-mt-turkish) Pearson |
|---|---|---|---|---|
| 👑 BAAI/bge-m3 (Teacher Transformer) | ~2,200 MB | 79 | 96.35% | 79.57% |
| 🥇 turkish-bge-m3-model2vec (This Model) | 19.36 MB | 63,012 | 91.36% | 64.34% |
| 🥈 turkish-bge-m3-model2vec-turboquant-4bit | 4.92 MB | 24,248 | 91.79% | 64.19% |
| 🥉 turkish-bge-m3-model2vec-turboquant-2bit | 2.50 MB | 20,013 | 92.19% | 63.53% |
🚀 Quickstart & Usage
Installation
pip install model2vec
Inference
from model2vec import StaticModel
# Load model directly from Hugging Face
model = StaticModel.from_pretrained("altaidevorg/turkish-bge-m3-model2vec")
sentences = [
"Yapay zeka modelleri doğal dil işlemede çığır açıyor.",
"Türkiye'nin en kalabalık ve ekonomik merkezi İstanbul'dur.",
"Hızlı ve hafif vektör modelleri edge cihazlarda çalışabilir."
]
embeddings = model.encode(sentences)
print(f"Shape: {embeddings.shape}") # (3, 256)
🧠 Distillation Methodology
- Vocabulary Pruning: Retains the 39,655 most frequent Turkish subwords from the multilingual BGE-M3 tokenizer.
- Teacher Target Reduction: 1024d BGE-M3 teacher embeddings on 533k Turkish Wikipedia articles compressed to 256d via PCA (86.77% variance preserved).
- Sentence-Level Weighted Mean Pooling: Eliminates segmentation drift via gradient descent distillation against normalized teacher targets.
📄 License
Apache-2.0
- Downloads last month
- 19