RotomArenaAI gaming benchmark • 2 benchmarks

Retracted · 2026-08-29 · rules version v0

direct_llm_p1_9102 vs heuristic_p2_4786

Retracted. This match was voided: direct_llm was on autopilot for 100% of its turns. It is kept for transparency and does not count toward any rating. Why we retracted the pilot study.
Player 1
direct_llm_p1_9102
team not recorded
autopilot 100%
Player 2 · winner
heuristic_p2_4786
team not recorded

heuristic_p2_4786 won in 13 turns · 74 s wall clock

The replay player loads from play.pokemonshowdown.com. The full battle log is below.

Decisions

  1. Turn 1
    heuristic_p2_4786 used Ceaselessedge
    direct_llm_p1_9102 used Shadowballautopilot
    Incident report
    LLM API Call Error: LLM request failed after 3 attempts: Error code: 404 - [{'error': {'code': 404, 'message': 'This model models/gemini-2.5-flash is no longer available to new users. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. We recommend you to use the Interactions API.', 'status': 'NOT_FOUND'}}]. Executed fallback action.
    direct_llm_p1_9102 switched to Dragapultautopilot
    Incident report
    LLM API Call Error: LLM request failed after 3 attempts: Error code: 404 - [{'error': {'code': 404, 'message': 'This model models/gemini-2.5-flash is no longer available to new users. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. We recommend you to use the Interactions API.', 'status': 'NOT_FOUND'}}]. Executed fallback action.
  2. Turn 2
    heuristic_p2_4786 used Ceaselessedge
    direct_llm_p1_9102 used Shadowballautopilot
    Incident report
    LLM API Call Error: LLM request failed after 3 attempts: Error code: 404 - [{'error': {'code': 404, 'message': 'This model models/gemini-2.5-flash is no longer available to new users. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. We recommend you to use the Interactions API.', 'status': 'NOT_FOUND'}}]. Executed fallback action.
    direct_llm_p1_9102 switched to Kingambitautopilot
    Incident report
    LLM API Call Error: LLM request failed after 3 attempts: Error code: 404 - [{'error': {'code': 404, 'message': 'This model models/gemini-2.5-flash is no longer available to new users. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. We recommend you to use the Interactions API.', 'status': 'NOT_FOUND'}}]. Executed fallback action.
  3. Turn 3
    heuristic_p2_4786 used Razorshell
    direct_llm_p1_9102 used Kowtowcleaveautopilot
    Incident report
    LLM API Call Error: LLM request failed after 3 attempts: Error code: 404 - [{'error': {'code': 404, 'message': 'This model models/gemini-2.5-flash is no longer available to new users. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. We recommend you to use the Interactions API.', 'status': 'NOT_FOUND'}}]. Executed fallback action.
  4. Turn 4
    heuristic_p2_4786 used Ceaselessedge
    direct_llm_p1_9102 used Kowtowcleaveautopilot
    Incident report
    LLM API Call Error: LLM request failed after 3 attempts: Error code: 404 - [{'error': {'code': 404, 'message': 'This model models/gemini-2.5-flash is no longer available to new users. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. We recommend you to use the Interactions API.', 'status': 'NOT_FOUND'}}]. Executed fallback action.
    heuristic_p2_4786 switched to Ribombee
  5. Turn 5
    heuristic_p2_4786 used Moonblast
    direct_llm_p1_9102 used Kowtowcleaveautopilot
    Incident report
    LLM API Call Error: LLM request failed after 3 attempts: Error code: 404 - [{'error': {'code': 404, 'message': 'This model models/gemini-2.5-flash is no longer available to new users. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. We recommend you to use the Interactions API.', 'status': 'NOT_FOUND'}}]. Executed fallback action.
    direct_llm_p1_9102 switched to Ting-Luautopilot
    Incident report
    LLM API Call Error: LLM request failed after 3 attempts: Error code: 404 - [{'error': {'code': 404, 'message': 'This model models/gemini-2.5-flash is no longer available to new users. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. We recommend you to use the Interactions API.', 'status': 'NOT_FOUND'}}]. Executed fallback action.
  6. Turn 6
    heuristic_p2_4786 used Moonblast
    direct_llm_p1_9102 used Earthquakeautopilot
    Incident report
    LLM API Call Error: LLM request failed after 3 attempts: Error code: 404 - [{'error': {'code': 404, 'message': 'This model models/gemini-2.5-flash is no longer available to new users. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. We recommend you to use the Interactions API.', 'status': 'NOT_FOUND'}}]. Executed fallback action.
  7. Turn 7
    heuristic_p2_4786 used Moonblast
    direct_llm_p1_9102 used Earthquakeautopilot
    Incident report
    LLM API Call Error: LLM request failed after 3 attempts: Error code: 404 - [{'error': {'code': 404, 'message': 'This model models/gemini-2.5-flash is no longer available to new users. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. We recommend you to use the Interactions API.', 'status': 'NOT_FOUND'}}]. Executed fallback action.
  8. Turn 8
    heuristic_p2_4786 used Moonblast
    direct_llm_p1_9102 used Earthquakeautopilot
    Incident report
    LLM API Call Error: LLM request failed after 3 attempts: Error code: 404 - [{'error': {'code': 404, 'message': 'This model models/gemini-2.5-flash is no longer available to new users. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. We recommend you to use the Interactions API.', 'status': 'NOT_FOUND'}}]. Executed fallback action.
    direct_llm_p1_9102 switched to Greatautopilot
    Incident report
    LLM API Call Error: LLM request failed after 3 attempts: Error code: 404 - [{'error': {'code': 404, 'message': 'This model models/gemini-2.5-flash is no longer available to new users. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. We recommend you to use the Interactions API.', 'status': 'NOT_FOUND'}}]. Executed fallback action.
  9. Turn 9
    heuristic_p2_4786 used Moonblast
    direct_llm_p1_9102 used Headlongrushautopilot
    Incident report
    LLM API Call Error: LLM request failed after 3 attempts: Error code: 404 - [{'error': {'code': 404, 'message': 'This model models/gemini-2.5-flash is no longer available to new users. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. We recommend you to use the Interactions API.', 'status': 'NOT_FOUND'}}]. Executed fallback action.
    direct_llm_p1_9102 switched to Dragoniteautopilot
    Incident report
    LLM API Call Error: LLM request failed after 3 attempts: Error code: 404 - [{'error': {'code': 404, 'message': 'This model models/gemini-2.5-flash is no longer available to new users. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. We recommend you to use the Interactions API.', 'status': 'NOT_FOUND'}}]. Executed fallback action.
  10. Turn 10
    heuristic_p2_4786 used Moonblast
    direct_llm_p1_9102 used Extremespeedautopilot
    Incident report
    LLM API Call Error: LLM request failed after 3 attempts: Error code: 404 - [{'error': {'code': 404, 'message': 'This model models/gemini-2.5-flash is no longer available to new users. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. We recommend you to use the Interactions API.', 'status': 'NOT_FOUND'}}]. Executed fallback action.
    heuristic_p2_4786 switched to Kingambit
  11. Turn 11
    heuristic_p2_4786 used Kowtowcleave
    direct_llm_p1_9102 used Extremespeedautopilot
    Incident report
    LLM API Call Error: LLM request failed after 3 attempts: Error code: 404 - [{'error': {'code': 404, 'message': 'This model models/gemini-2.5-flash is no longer available to new users. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. We recommend you to use the Interactions API.', 'status': 'NOT_FOUND'}}]. Executed fallback action.
  12. Turn 12
    heuristic_p2_4786 used Kowtowcleave
    direct_llm_p1_9102 used Extremespeedautopilot
    Incident report
    LLM API Call Error: LLM request failed after 3 attempts: Error code: 404 - [{'error': {'code': 404, 'message': 'This model models/gemini-2.5-flash is no longer available to new users. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. We recommend you to use the Interactions API.', 'status': 'NOT_FOUND'}}]. Executed fallback action.
  13. Turn 13
    heuristic_p2_4786 used Kowtowcleave
    direct_llm_p1_9102 used Extremespeedautopilot
    Incident report
    LLM API Call Error: LLM request failed after 3 attempts: Error code: 404 - [{'error': {'code': 404, 'message': 'This model models/gemini-2.5-flash is no longer available to new users. Please update your code to use models/gemini-3.6-flash for the latest features and improvements. We recommend you to use the Interactions API.', 'status': 'NOT_FOUND'}}]. Executed fallback action.