Credits

PokéChamp

Seth Karten, Andy Luu Nguyen and Chi Jin · ICML 2025

A minimax agent with a language model inside its search. Our reference bot implements their design from the paper's description, with hand-written rules where the original uses a model; it contains none of their code. It is a fixed yardstick, not a reproduction of their agent. How we use it.

ProjectPaperCodePokéAgent Challenge

Pokémon Showdown

Guangcong Luo and contributors · MIT license

Every battle on this site runs on a local Pokémon Showdown server: its engine applies the rules, damage, abilities, items and randomness, and its Gen 9 OU format supplies the clauses. The replay player on each match page, with its sprites and battle effects, loads from play.pokemonshowdown.com.

WebsiteCode

poke-env

Haris Sahovic and contributors · MIT license

The Python library that connects our players to Showdown: it reads the battle protocol into battle state, lists the legal actions every player chooses from, and sends their choices back.

Code

Smogon

The Smogon competitive community

The OverUsed tier our battles are played in, its banlist and clauses, and the team export format our teams are written in, are Smogon's. Our reference teams come from Smogon's SV OU sample-team thread, and each one's builder is named on the Teams page. The fixed-teams benchmark's two teams are older: the AI assistant that scaffolded this project wrote them from memory, so no builder can be named.

Website

The models

Open weights, unless noted on their pages

Every rated model is somebody's work. So far:

• anthropic Claude Haiku 5.5, Claude Opus 5.5, Claude Sonnet 5.5
• convaiinnovations Laya
• deepseek DeepSeek V4.1 Flash, DeepSeek V4 Pro
• google Gemini 3.8 Flash, Gemini 4 Argon, Gemma 4 31B
• ifm K2-Horizon 36B
• inception Mercury Decide
• inclusionai Ling 3.0 Tiny, Ling 3.1 Flash
• meta Muse Spark 1.3, Muse Spark Open, Muse Wonder
• microsoft Microsoft-Decision-1
• minimax MiniMax M3
• mistralai Mistral Large 4
• moonshotai Kimi K3
• nace-ai Drex v1.5, Drex v2.0
• nvidia Nemotron 3.5 Super VL, Nemotron 3 Ultra
• openai GPT-6.1 Sol, GPT-6 Luna, GPT-6 Luna Decisions, GPT-6 Luna Pro, GPT-OSS 20B
• openbmb MiniCPM5 2B
• ornith-ai Ornith 1.5 35B
• perplexity Perplexity Decider V1.1 27B
• prism-ml Bonsai 2 27B
• qwen Qwen3.8 27B, Qwen3.8 Flash, Qwen3.8 Max Prime
• rotomarena Laya RotomArena
• tencent Hy4 Preview
• typesafe Jev 1.13
• upstage Solar Decide Flash, Solar Pro 4
• x-ai Grok 4.7
• xiaomi MiMo V2.6 Flash, MiMo V2.6 Pro
• z-ai GLM 5.3, GLM 5.3 Flash

Local models run on llama.cpp (MIT), Bonsai onPrism ML's fork; API models throughOpenRouter. Each model's page credits its makers and links their work.

Methods

Ideas we use, with thanks

Ratings are a Bradley-Terry model (Bradley and Terry, 1952) on the Elo scale (Elo, 1978); pairwise ratings for language models were popularized by Chatbot Arena (Chiang et al., 2024). Full references are in thepaper.

This site

Built with Astro · hosted on Netlify

Typeset in Inter, Rajdhani and JetBrains Mono, served by Google Fonts under the SIL Open Font License.

Our code is MIT-0; the games, ratings, teams and this site's text are CC0. Take anything, no credit needed. Smogon's reference teams stay their builders', and Pokémon stays Nintendo's.