Technology · Open for opinions
Which company will have the AI model with the best average score on the public MMLU and HumanEval benchmarks by March 2027?
With the launch of ML4 by Mistral and the fierce competition among OpenAI, Anthropic, and Google, the market for… 81 opinions, 35% OpenAI (GPT)
- OpenAI (GPT) — 35%
- Anthropic (Claude) — 22%
- Google (Gemini) — 20%
- Mistral (ML4) — 14%
- Other model — 10%
Deadline: March 31, 2027 at 12:00 AM UTC · 81 opinions
Resolution rules
With the launch of ML4 by Mistral and the fierce competition among OpenAI, Anthropic, and Google, the market for cutting-edge AI models is undergoing accelerated transformation, making leadership in objective benchmarks uncertain.
Source: www.cnnbrasil.com.br
Frequently asked questions about this event
How is this event resolved?
Resolução baseada nos resultados publicados nos benchmarks padronizados MMLU e HumanEval disponíveis em plataformas públicas como Papers With Code (paperswithcode.com). Considera-se a média aritmética das pontuações nos dois benchmarks até 31/03/2027.
How are points scored?
Points depend on the share of votes your side had when you gave your opinion. Being right earns 100 minus that share. Being wrong costs that share.
Right now OpenAI (GPT) has 35% of the votes: an opinion on it that turns out right earns about 65 points, a wrong one costs about 35.