How we rank

We rank every model separately for coding, writing, math, research and everyday chat, because a model that writes beautifully can still be ordinary at math. Each task is tested with easy, medium and hard questions, and a model’s rank for the task is its average quality across all three.

Each rank blends public leaderboards with our own blind tests where we have them; many tasks have few graded answers of ours yet, and there the public boards carry the rank. A rank is about the quality of the answers and nothing else. Price and speed never move it: we show them beside the rank, as facts, so you can weigh them yourself.

Public leaderboards. We start from what the field already measures: the Artificial Analysis Intelligence Index, and LMArena’s ratings from people voting on answers side by side. For each task we read the board that matches it, such as LMArena’s coding board for coding. Where a model is missing from a board, it is placed in the middle of the pack rather than given a score it never earned.

Our own blind tests. We ask the models the same questions, and every answer is graded without the grader knowing which model wrote it. Answers are also judged in pairs, head to head. Every rate from these tests carries a 95% Wilson interval, a standard margin of error, so a lead built on a handful of answers is read as the noise it may be.

Where we have enough graded answers for a task, our tests count alongside the public boards. Where we have only a few, the public boards carry the rank, and our tests are not allowed to swing it.

• The Intelligence Index and the speed are Artificial Analysis’s, as they publish them.
• The LMArena ratings are from LMArena’s leaderboard dataset, published under CC BY 4.0, shown as published. Where LMArena rates a model at a particular setting, the page says which.
• The API price is what the model’s maker charges developers per million tokens. It is not what you pay: with achat.ai you pay one plan price for every model.
• The answers in each page’s chat window are real and unedited. Every model gets the same questions, asked the same way, and the page shows the date it was asked. Where we have not stored a model’s answer yet, the page shows the question alone.

We re-rank when a new model arrives and when new evidence comes in, and we refresh the public scores regularly. The pages are built straight from that data, so they change the moment it does: nobody edits a rank by hand. Each page says when it was last updated.

See every model we rank