Spaces:
Running on CPU Upgrade
Multi-metric evaluation framework for Arabic LLM deployment decisions
مرحبا! Hi Arabic AI community 👋
The Open Arabic LLM Leaderboard is a fantastic resource. For teams deploying Arabic LLMs in production, the accuracy ranking is just the start — cost and latency often determine which model actually ships.
I built an open source LLM Evaluation Framework that adds:
→ 💰 Cost per 1K tokens — Arabic cloud providers vs OpenAI/Anthropic/Google cost comparison
→ ⚡ Latency — p95 tail latency, critical for real-time Arabic chat applications
→ 🔍 Hallucination Rate — hallucination in Arabic outputs can be harder to catch, making measurement critical
→ 🎯 Accuracy — MMLU + TruthfulQA + extensible for Arabic-specific benchmarks
→ 🧠 Reasoning Quality — structured CoT depth scoring
Live demo: https://huggingface.co/spaces/vigneshwar234/llm-eval-demo
GitHub: https://github.com/vignesh2027/LLM-Evaluation-Framework
Would love to discuss extending benchmark support for Arabic-specific tasks!