RUN 20260725-opus5-python-probe (design + open note)

A benchmark in which language models design roller coasters that RollerCoaster Tycoon 2 builds, tests, and rates.

Standings

design mode — models build from scratch

modelscoreintensitynausea similaritybesttokens / cost
1st claude-opus-5 6.84 9.76 5.67 0.42 round 1/2 165 in / 113k out · $6.55

score = excitement, scaled down when similarity to a stock library design exceeds 0.5 (an exact or mirrored copy scores 0)

claude-opus-5full detail →
excitementintensity×build failed
0515intensity ceiling 10excitementintensityR1R2

Round 1 trace (95, 2 rejected) →

excitement6.84 intensity9.76 nausea5.67
length 611 m drops 8 airtime 40 tested ok similarity 0.42 (nearest: Dizzymouse)
half_loop_upright_corkscrew_downright_vertical_loopright_corkscrew_upleft_corkscrew_downright_corkscrew_upleft_corkscrew_down
97 pieces · 14 on the lift · 7 inversions
track program (97 pieces)
{
  "pieces": [
    "begin_station",
    "middle_station",
    "middle_station",
    "middle_station",
    "end_station",
    {
      "chain": true,
      "t": "flat_to_up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25_to_flat"
    },
    "flat_to_down_25",
    "right_turn_3_down_25",
    "right_turn_3_down_25",
    "right_turn_3_down_25",
    "right_turn_3_down_25",
    "down_25",
    "down_25",
    "down_25",
    "down_25",
    "down_25_to_flat",
    "flat_to_up_25",
    "half_loop_up",
    "right_corkscrew_down",
    "flat_to_down_25",
    "down_25",
    "down_25",
    "down_25",
    "down_25",
    "down_25_to_flat",
    "flat_to_up_25",
    "right_vertical_loop",
    "down_25_to_flat",
    "right_corkscrew_up",
    "left_corkscrew_down",
    "right_corkscrew_up",
    "left_corkscrew_down",
    "flat_to_left_bank",
    "banked_left_turn_5",
    "left_bank_to_flat",
    "flat_to_up_25",
    "up_25",
    "up_25_to_flat",
    "flat_to_down_25",
    "down_25",
    "down_25_to_flat",
    "flat_to_up_25",
    "up_25",
    "up_25_to_flat",
    "flat_to_down_25",
    "down_25",
    "down_25_to_flat",
    "flat_to_up_25",
    "up_25",
    "up_25_to_flat",
    "flat_to_down_25",
    "down_25",
    "down_25_to_flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat_to_left_bank",
    "banked_left_turn_5",
    "left_bank_to_flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat_to_left_bank",
    "banked_left_turn_5",
    "left_bank_to_flat"
  ],
  "ride_type": 51,
  "start": {
    "dir": 0,
    "x": 62,
    "y": 60
  }
}
park screenshot, claude-opus-5 round 1

Round 2 trace (47) →

excitement6.79 intensity8.46 nausea4.38
length 747 m drops 10 airtime 63 tested ok similarity 0.38 (nearest: Viper)
half_loop_upright_corkscrew_downhalf_loop_upleft_corkscrew_downright_vertical_loopleft_vertical_loopbrakesbrakes
122 pieces · 20 on the lift · 6 inversions · 42 added, 17 removed vs the round before (80 of 97 pieces kept)
track program (122 pieces)
{
  "pieces": [
    "begin_station",
    "middle_station",
    "middle_station",
    "middle_station",
    "end_station",
    {
      "chain": true,
      "t": "flat_to_up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25"
    },
    {
      "chain": true,
      "t": "up_25_to_flat"
    },
    "flat_to_left_bank",
    "banked_left_turn_5",
    "left_bank_to_flat",
    "flat_to_down_25",
    "down_25",
    "down_25_to_down_60",
    "down_60",
    "down_60",
    "down_60",
    "down_60_to_down_25",
    "down_25_to_flat",
    "flat_to_up_25",
    "half_loop_up",
    "right_corkscrew_down",
    "flat_to_down_25",
    "down_25",
    "down_25",
    "down_25",
    "down_25",
    "down_25_to_flat",
    "flat_to_up_25",
    "half_loop_up",
    "left_corkscrew_down",
    "flat_to_down_25",
    "down_25",
    "down_25",
    "down_25",
    "down_25",
    "down_25_to_flat",
    "flat_to_up_25",
    "right_vertical_loop",
    "down_25_to_flat",
    "flat_to_up_25",
    "left_vertical_loop",
    "down_25_to_flat",
    "flat_to_left_bank",
    "banked_left_turn_5",
    "left_bank_to_flat",
    "flat_to_up_25",
    "up_25",
    "up_25_to_flat",
    "flat_to_down_25",
    "down_25",
    "down_25_to_flat",
    "flat_to_up_25",
    "up_25",
    "up_25_to_flat",
    "flat_to_down_25",
    "down_25",
    "down_25_to_flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat_to_left_bank",
    "banked_left_turn_5",
    "left_bank_to_flat",
    "flat_to_up_25",
    "up_25",
    "up_25_to_flat",
    "flat_to_down_25",
    "down_25",
    "down_25_to_flat",
    "flat_to_up_25",
    "up_25",
    "up_25_to_flat",
    "flat_to_down_25",
    "down_25",
    "down_25_to_flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat",
    "flat_to_left_bank",
    "banked_left_turn_5",
    "left_bank_to_flat",
    "flat_to_down_25",
    "down_25_to_flat",
    "flat",
    "flat",
    "brakes",
    "brakes"
  ],
  "ride_type": 51,
  "start": {
    "dir": 1,
    "x": 62,
    "y": 35
  }
}
park screenshot, claude-opus-5 round 2

← all runs