People use fast and flat simulation to reason about new games

Game construction

We manually constructed the 121 two-player competitive strategy game variants played on M × N grids, where M is the number of rows and N is the number of columns. One goal in creating these games was to ensure there is enough diversity in board size and rule structure as well as systematic variance. To that end, we designed a series of variations on square boards: on 10 × 10 boards, we have K-in-a-row for K from 2 to 10; for 3-in-a-row, we have M by N = M boards from 3 to 10; we also include a few 5 × 5 boards with varying K. We additionally included a few other square boards varying in complexity, as well as rectangular boards ranging in size from from 1 × 5 to 5 × 10, and we have integer 2 ≤ n ≤ 6 for K-in-a-row. To assess how people reason about games that are not physically realizable, we included three ‘infinitely’ sized games with K = 3, 5 and 10 for K-in-a-row. These categories give us 41 games with typical ‘M–N–K rules’ (for example, where the players take turns and have the same objective: ‘make K in a row, where horizontal, vertical, or diagonal all count’). This set includes the standard tic-tac-toe as well as 4 × 9, 4 in a row wins from ref. ⁶. We then created a number of games with varied game rules. We kept the selection of board sizes and K-in-a-row fixed across categories (ten games within each category). The selection included 3 × 3, 4 × 4, 5 × 5 and 10 × 10 boards and n ∈ 3, 4, 5, 10. We also designed games with more atypical rules, varying the winning conditions (for example, K-in-a-row loses and diagonal connections do not count as wins) and first-mover dynamics (for example, player 1 can place two pieces on their first turn). These categories give us an additional 80 games, totalling the 121 in our dataset. We manually implemented an automated win condition checker that permits flexible game assessment over all game types.

Game name codes

Games are expressed in abbreviated form throughout the paper. Games are described by their board size (rows × columns) and the number K in a row to win. For example, ‘4 × 4, 3’ means the game is played on a 4 × 4 grid and the first person to get 3 pieces in a row wins. Unless otherwise stated, horizontal, vertical and diagonal all count. If only horizontal and vertical 3 in a row count, the game would be written as ‘4 × 4, 3 HV’. If only diagonal counts, then the game will be written as ‘4 × 4, 3 D’. If the constraint only applies to one player (for example, P1 can only win horizontally and vertically, and P2 can win any way), that is represented as ‘4 × 4, 3 (P1 HV)’. If one player can go twice on their turn, for example, the second player (P2) can play twice, that is written as ‘4 × 4, 3 (P2 2p)’. A misère game (where first to K in a row loses) is written with an ‘L’, for example, ‘4 × 4, 3 L’. In our dataset, multiple rule modifications cannot co-occur, so each game can be expressed in this abbreviated way.

The full game natural language game descriptions were provided to participants (for example, as shown in Extended Data Table 1). The game codes are used only for ease of presentation in this paper.

Human experiments

All human experiments were conducted under prior approval from the institutional review board at the Massachusetts Institute of Technology through the Computational Cognitive Science Lab. All participants provided informed consent.

Zero-shot outcome evaluation experiment

We recruited 238 participants from Prolific⁴⁹ to judge novel games. Each participant was randomly presented with 10 games sampled from our 121 diverse game stimuli, as well as regular tic-tac-toe (won by making 3 in a row on a 3 × 3 board) to set baselines for game judgements. We collected approximately 20 judgements per game stimulus for each game reasoning query. Participants were paid at a base rate of US$12.50 per hour with an optional bonus up to US$15 per hour; the full experiment approximately took 25 minutes.

Participants were instructed to evaluate likely game outcomes. Specifically, participants produced judgements on a continuous 0 to 100 probability scale to predict the likelihood of a first player win (“If the game does not end in a draw, assuming both players play reasonably, how likely is it that the first player is going to win (not draw)?”) and a draw (“Assuming both players play reasonably, how likely is the game to end in a draw?”). Judgements were made using sliders. Both game outcome question sliders appeared on the same page.

Participants produced judgements about each game based on a linguistic game specification. We additionally provided participants with an interactive scratchpad board that they were told they could, but were not required to, use to inform their judgements. The scratchpad was automatically sized to the board of the game specified; infinite boards were presented on a 13 × 13 grid with dashed lines indicating the board could continue. The scratchpad permitted automatically placing pieces of different colours (‘red’ and ‘blue’ to simulate different players); participants could force the the next play to be made by the same player (for example, colour red twice in a row) by pressing the spacebar. Two buttons appeared below the scratchpad, permitting the user to either undo their last move or clear the screen to begin a new ‘game’. See Supplementary Information section 4 for examples of the experiment interface. Participants were required to consider each game for at least 60 s before being allowed to make their game judgements. Outliers were determined as the 10% of the judgements farthest from the mean judgements of other participants (in terms of the summed distance from the two queries) and filtered out for each game.

After answering all game reasoning queries, we additionally asked participants to create a new grid-based game variant that they would find fun. Participants wrote a linguistic game specification, describing the board size and win conditions. As in the game judgement queries, participants were again provided an optional scratchpad and required to spend 60 s before submitting a response. The scratchpad enabled participants to try out the game they intended to create. After specifying a game, participants were asked to answer the same game reasoning query (either game outcomes or game fun) about their own game. These game-generation responses were not studied here and are actively being explored in a follow-up study. Example screenshots of experiment interfaces are included in Supplementary Information section 4.

Zero-shot funness evaluation experiment

We repeated the same methodology above with a new group of 257 participants, replacing the game outcome questions with a question about game funness. Participants instead assessed the expected funness of the game (“How fun is this game?”) on a confidence scale spanning 0 (the least fun of this class of game) to 100 (the most fun of this class of game). 11 participants were filtered due to having provided non-effortful or AI-generated games in the creation stage, resulting in a total of 246 participants, and the same outlier filtering was applied per trial as in the outcome evaluation.

Zero-shot human–human play experiment

We recruited 302 participants to play these novel games in a pre-registered experiment. We selected a subset of 40 games from the full set of 121 to span a representative range of the gameplay variations (board shapes, board sizes and win rules) in the original dataset while generally favouring games that would not take very long to play in a live experiment. We randomly constructed eight batches of five of these games. Each participant additionally played one round of tic-tac-toe. The order of games was shuffled for each new set of participants.

Each participant was automatically paired with another player. We developed our interface using Empirica⁵⁰, which supports synchronous human–human pairing. Participants played one round from five different games. Players were informed they would get a bonus of US$0.50 for every win. Participants had to spend at least 5 s reading the game description before they began. We appended ‘Horizontal, vertical and diagonal all count’ to all game descriptions where any direction was allowed after we noticed some participants in pilots were confused as to which line directions would result in a win. Players were randomly assigned to move either first or second and a corresponding piece colour (red or blue). Players took turns making moves on the synchronous game interface. Players had no time limit on their turn.

Players were also allowed to request a draw or decide to surrender using buttons at the bottom of the interface. If a player surrendered, the game ended immediately (and that player lost). If a player requested a draw, the other player was allowed to either accept the draw (after which the game ended immediately and no player won) or reject the draw (leading the game to continue being played). Draw requests appeared as a popup banner for the other player. We include screenshots of the interface in Supplementary Information section 4.

The match ended when either a player won, a player surrendered, the board filled up completely (draw), or the players agreed to a draw. Both participants were informed about the game outcome. After each match, players made a judgement about either the expected outcomes of that game overall (with a new set of reasonable players) or the game’s funness (in a new match against a new player). Each pair of players was randomly assigned to either the outcome or funness rating condition. Judgements were made on a slider. Players were also presented with a ‘frozen’ version of the match on an example board with which they could replay all of the moves they and their opponent had made. Players also indicated how skilled they thought their opponent was at this game (“Out of 100 other random new players, where do you think the opponent you just played would rank in skill for this game?”). After the judgements were made, the players continued to the next, new game. At the end of the study, they filled out a text-based survey providing general information on their strategy and how fun they found the experiment. We filtered out 18 participants who did not pass our quality control (that is, they provided judgements that were ‘standard’ values (near 0, 50 or 100) on 80% or more of judgements) for a total of 284 subjects.

Watching and predicting play experiment

We recruited a new set of 314 participants, in a pre-registered experiment, to reason about the games zero-shot from only indirect experience: watching two other agents play. We selected a subset of 20 of the games from the previous human–human play study to ensure representation across game rules and dynamics. We also included tic-tac-toe (totalling 21 games). Participants watched a series of videos of other agents’ gameplay. Each video involved two humans playing each other, sourced from our live human–human gameplay experiment. We sampled four human–human played matches randomly from each of the 21 games (owing to a randomized batching error, only three unique matches were sampled for tic-tac-toe; hence, 249 game boards over the matches from 21 games and three stages per game), after filtering out any matches that ended preemptively from a draw request or surrender. For each match, we sampled three specific boards to be evaluated corresponding to the beginning, middle and end of the match. For the beginning and end boards, we randomly selected either the third or fourth move and the second-to-last or third-to-last move, respectively. For the middle board, we selected the median move. We filtered out any match that ended before eight moves. Participants watched one match from five different games, plus tic-tac-toe.

Before each match, participants were informed of the game rules and required to think about the rules for 5 s before the video began. We again appended “Horizontal, vertical, and diagonal all count” to all game descriptions where any direction was allowed. Videos played forward at a fixed rate, as in ref. ⁶. We chose 2 s per move to give viewers enough time to process each move without taking too long overall. Each video was stopped at the three time points described above. At each stopping point, participants indicated their belief over where they thought the acting player should move next. Participants were given five clicks, which they could spread across the legal moves on the board to indicate their confidence that the player should move there. We chose five clicks to balance granularity of the elicited belief distribution against the burden on the participants. After each click, the opacity of the cell increased to indicate higher confidence. Participants were informed of the number of clicks they had left and could reset their clicks by clicking on a button below the interactive board.

After watching each video and indicating where they thought a player should move at each of the three timepoints, participants were then shown the remainder of the game as a board snapshot (cells indicated where players had moved and the order of play). Participants then answered either the same game outcome or funness judgements about the game overall, as described above. Judgements were made on a slider. Participants also indicated how skilled they thought each of the players was. We filtered out 10 participants who did not pass our quality control, leaving us with a total of 304 valid participants.

Problem formulation

Formally, one can think of a system, problem, or game G in terms of a space of feasible states \(\mathcalS\); possible actions \(\mathcalA\); rules \(\mathcalT\) specifying valid actions and state transitions, and governing the overall dynamics; and goal functions mapping from states \(\mathcalS\) to possible rewards \(\mathcalR\). We are interested in how a reasoner infers properties of a new game ψ(G) (for example, whether a game is likely to end in a draw or have a bias towards a particular player; whether the game is likely to be engaging and fun) as well as a policy π_G(a_t∣s_t) for how to play (choosing actions given the current state at time t, to achieve their goal, for example, to win), without experience of actual traces of gameplay and relying instead on simulated or imagined traces. Our aim is to model how people approximate ψ(G) and π_G in a way that can (1) take in any G as input, and (2) do so with a limited compute budget and no direct experience with game G.

Intuitive Gamer model specification

We first describe a formal account of the Intuitive Gamer player module, and then describe the reasoning module.

Intuitive Gamer player module

Formally, at a given board state (\(s_t\in \mathcalS\)), the Intuitive Gamer player module scores all legal next actions (\(a_t\in \mathcalA\)) according to a measure of immediate progress made towards a player’s own goal (U_self) and a measure of progress blocked (U_opp) towards the opponent’s goal (Fig. 2c). Progress is based on the extent to which an action connects more contiguous pieces towards a winning K-in-a-row configuration. These two utilities are designed to capture the general intuition that game players aim to make progress towards their goal while preventing their opponents from doing the same. In addition, we can consider other easily computable heuristic functions that may bear on the value of an action (U_aux, for ‘auxiliary’). In our experiments, we consider an auxiliary heuristic based on proximity to the centre of the board. It encodes a ‘centre bias’—a preference for making moves near the centre of the board, which allows a piece to participate in more winning terminal configurations. These heuristics are drawn from features used by previous studies in similar games^6,29,30, although we generalize them to our broader class of strategic games. The final heuristic value assigned to a given action on a particular board, \(\widetilde\mathcalV(s_t,a_t)\), is a sum of the three utility components described above:

Our approach to action evaluation and choice reduces computational cost, as it is strongly local in both space and time: it considers only the progress immediately stemming from a specific action and does not account for either past or potential future returns. By contrast, it is common for value functions to be action-agnostic (that is, to depend only on s_t (ref. ⁶)) and to explicitly capture some notion of the game’s terminal rewards (for example, via a learned value network⁴). Intuitively, each of these alternatives represents a substantial increase in cognitive load: the former requires scanning over the entire board to evaluate any action, and the latter requires mental simulation all the way to the end of the game (or access to a function that is derived from such simulations) before deciding what action to take.

We next describe our specific implementation of each utility function. As mentioned, we came up with these heuristics via prior literature grounding, as well as expert intuitions. Concepts of progress and blocking are present in the classic study of ref. ²⁹, and similarly progress patterns and central tendencies are used in the recent ref. ⁶, on which we base our expert model. More generally, features based on connections (or lack thereof), implemented differently based on games, is pretty common in game-playing AI. Most of the authors are expert cognitive modellers and some are expert board game players (and played many connect-N style games), so the combination of progress, blocking and centring is effectively the first modelling hypothesis that we came up with and judged to be plausible. More details on the lesioning of these heuristics are included in Supplementary Information section 3.3.

The first utility (U_self) computes a measure of intermediate ‘goal progress’ of the active player, based on a feature n₁ defined as the largest line of contiguous pieces created by the action that could be extended to result in a win. This means that actions which extend a line in an illegal direction (for example, a diagonal line in a game with only horizontal or vertical wins) or that are already blocked by an opponent’s piece or the edge of the board do not contribute to goal progress. If the action would result in a win for the active player (that is, n₁ = K) then an additional 1 is added to the feature to magnify the value of winning actions. We note that our choice of feature only rewards actions that form contiguous lines of pieces—any piece that is not immediately adjacent to a previously placed piece from the active player has n₁ = 0. Although other formulations (such as rewarding actions that form the ends of a line that is empty in the middle or detecting particular patterns of pieces on the board) are sensible, we choose a simple and easy-to-calculate function that reflects a straightforward intuition (that is, making K in a row by first making K − 1 in a row, K − 2 in a row and so on).

The second utility (U_opp) computes a measure of ‘progress blocked’ for the opponent, based on a feature n₂ that is largely symmetric with the goal progress feature above: it is computed exactly as the goal progress the opponent would obtain if they made the same move (that is, with respect to the opponent’s allowed winning directions). We subtract 0.5 from n₂ to reflect people’s tendency to weigh offence over defence²⁹, so blocking the opponent’s \(\hatK\) in a row is not as good as making \(\hatK\) in a row for oneself (but is better than making \(\hatK-1\) in a row). However, if n₂ equals the winning K, we do not subtract 0.5 in order to account for the importance of blocking winning threats. As above, there are other reasonable formulations of this feature that we leave to future work.

Finally, the third utility (U_aux) is computed as the normalized Euclidean distance between the position of the action and the centre of the board, ξ ∈ [0, 1]. This reflects the intuition, applicable across our family of games, that people often place pieces around the centre of the board. This tendency may in part be explained by the fact that placing pieces closer to the centre of the board allows that piece to participate in the most possible winning terminal states for any K-in-a-row win condition. This measure generalizes to arbitrary rectangular boards. For example, on a 4 × 6 board, intuitively the ‘middle point’ is the centre, and the middle four cells are all closest to the centre (⟨2, 3⟩, ⟨2, 4⟩, ⟨3, 3⟩, ⟨3, 4⟩).

We assume each utility function is represented as an exponentiation of the respective feature:

The choice to exponentiate some measure of progress for heuristic functions is common in gameplay modelling (for example, refs. ^51,52,53). We chose base two on the basis of light initial exploration (prior to collecting any human gameplay data), under the goal of keeping our intuitive model simple. Future work should explore other modelling choices to capture how people might represent and combine multiple utility functions, including the role of learning in how people might come to flexibly synthesize these functions.

Moves are selected by following Boltzmann rationality^54,55,56, sampling actions from a softmax function over their estimated value (based on the above goal-directed heuristics). We assume that players have already developed the capacity to account for multiple goals simultaneously, unlike potentially even more naive child-like game reasoners²⁹. Concretely, the probability of choosing action \(\hata\) at state s is given by:

We fix temperature (τ) at 1 for our game reasoning and action experiments (with the exception of marginalizing over τ only for the admixture analyses in the ‘play’ experiment). We set the weights of each component (w) to 1 for all experiments. We use the same settings for the Expert Gamer model (as it uses the same value function and softmax-based action selection). We find that equal weights is a reasonable fit for the Intuitive Gamer model to human payoff predictions (Supplementary Information section 3.3). When lesioning a component of the value function, we set its weight to zero.

Intuitive Gamer reasoning module

The Intuitive Gamer player module is queried as part of the game reasoning module. The Intuitive Gamer reasoning module nests player module-based simulations to compute a series of gameplay traces (\(\(s_^,s_1^,\ldots ,s_T^),\ldots ,(s_^k,s_1^k,\ldots ,s_T^\prime ^k)\\)). These simulations involve self-play between the same player module type (with the exception of the funness reward for thinking computation, see below), where each agent is trying to make progress towards their own goal (which may be different, for example, player 1 trying to make 4 in a row only diagonally, while player 2 can win in any direction). From these simulations, a probabilistic judgement of the expected outcomes can be made. That is, for each game G, k game simulations are sampled, producing a set of k outcomes o ∈ −1, 0, 1 encoding whether the first player won (1), lost (−1), or the game ended in a draw (0). From these outcomes, we can compute a payoff of a game G:

Our game reasoning module is constructed to represent the reasoning of any one participant. Therefore, we draw k simulated matches for N = 20 simulated participants (as each game has, generally, 20 participant responses; see ‘Analysis methods’ below for details on selecting k).

Partial game simulations

Our primary game reasoning module assumes that mental simulations are conducted until the end of the game is reached: either a player attains the win condition, or all open board positions are filled and the game is called a draw. While many of our games do not take many moves to reach an end, people might not mentally simulate all the way to the end of the game in each of their k simulations. We conduct a preliminary exploration into the impact of partial game simulations in fitting peoples’ game fairness evaluations by modelling the possibility that people halt any one of their k simulations early. We sample a ‘stopping time’ uniformly from 1 to the size of the board and end the game after that many turns. To determine the outcome of games that are stopped early, we apply a simple rule and treat each of them as a draw (allowing us to explore encoding a weak prior towards games ending in draws; Supplementary Information section 5.4). We repeat the same exploration of variance (see ‘Analysis methods’ below) and find that the optimal number of samples is similarly more than 1 and less than 10, but may be closer to k = 4 (Extended Data Fig. 2). We leave further analysis of the distributions over stopping times and how people assign outcomes for partial games to future work.

Funness model

To estimate a game’s funness, we consider three features derived from playouts under the Intuitive Gamer player module: balance, reward for thinking and game length. We define a game’s balance as the difference between the probability distribution of observed outcomes and an ideal outcome distribution relative to a game where exactly half of the games end in wins and losses and none end in draws (measured by the Earth Mover’s Distance⁵⁷). As described in the main text, this feature captures the notion that players prefer decisive games (that is, those that do not end in draws) that are also balanced across players^58,59,60. We measure game length as the expected number of moves until a game ends in a draw or win, computed from the same Intuitive Gamer simulations. The effect of game length on funness is modelled using a quadratic function, based on the intuition that the most fun games will be neither too short nor too long. We define a game’s reward for thinking as the proportion of simulated games won by the Intuitive Gamer model playing against the Random Gamer player module, that is, a player that chooses actions uniformly at random from any legal move. This feature captures the intuition that players prefer games that they expect will challenge their thinking^35,38 and reward them for at least some strategic thinking, rather than just responding arbitrarily or without any strategy.

We use 120 game simulations for each feature (to align with the k = 6 simulations for each of 20 participants), randomizing whether the Intuitive Gamer plays first against the random agent in the simulations used to assess reward for thinking. Game simulations are run to the end of the game (where either a player wins or the game ends in a draw); future work can explore variants of the funness model in which features are computed under partial game simulations. We use the same features and number of simulations when comparing to alternative models (see below in ‘Fitting regression models to funness judgements’).

Alternative models

We next detail several alternative models.

Expert Gamer

We compare our Intuitive Gamer model against an Expert Gamer model that differs along two key dimensions: (1) sophistication of the value function, and (2) depth of search. This model is closely based on the model of human expert play for 4-in-a-row on a 4 × 9 board in ref. ⁶, which empirically estimates tree search depth from human players after hours of continuous gameplay experience. As in ref. ⁶, the depth is controlled by a probabilistic ‘stopping parameter’ governing how many the search tree is expanded; we run all simulations by expanding the search tree a fixed number (k_iterations = 636) of iterations, where k_iterations = 636 is the empirical mean value of this parameter estimated from the ref. ⁶ data. This setting corresponds to approximately depth-5 search. On each iteration, the Expert Gamer selects a node (corresponding to a board state). We then expand this node, considering all legal actions, wherein we compute the value of each action as outlined below. The Expert Gamer model then conducts best-first search over the game tree, using its state value function defined above. Specifically, it probabilistically expands nodes in the search tree by repeatedly sampling actions from a softmax policy governed by temperature τ over its current action estimates based on these computations, expanding unexplored states in the search tree, and backpropagating utilities estimated at future states. As in ref. ⁶, any node that ends in a definite win or loss is assigned ±σ based on whether the move would result in a win for the current play (+) or loss (−). We set σ = 1,000.

We next detail the Expert Gamer value function. Following ref. ⁶, we compute the value of any move by looking at features over the entire board state, rather than locally circumspect around the possible move position in question (like the Intuitive Gamer model). This is more computationally intensive, particularly for larger boards. Specifically, for any open legal position p, we imagine a state s′ that has that position played by the current player. We then sweep over all played positions with that board state. For each played position p′, we compute the same novice, partial-progress value function. We then define the value of that state s′ as the difference in cumulative value from the played positions by the current player minus the opponent. We set the value function feature weights (w_connect, w_block, w_centre) using the fit values from the IntuitiveGgamer model. It is worth noting that the Expert Gamer is itself an important contribution—it is a more general version of a relatively deep goal-directed model. We leave validation of how well the Expert Gamer captures human expert reasoning and play for future work.

Random Gamer

The Random Gamer player selects actions uniformly over the space of legal moves. Games are simulated to the end (for example, until either player wins or the game ends in a draw).

Monte Carlo tree search

We additionally implement a standard upper-confidence-bound MCTS^27,28 agent to act as an approximate gameplay ‘oracle’ (pseudocode can be found in Supplementary Information section 2). Unlike the Intuitive Gamer and Expert Gamer models, MCTS does not use game-specific heuristic features and instead estimates intermediate utilities by expanding a search tree guided by repeated random rollouts to terminal states. MCTS algorithms are commonly used to approximate optimal gameplay in arbitrary games²⁸, as they are empirically both efficient and accurate. Specifically, the MCTS implementation we consider here uses a large number of tree expansions (10,000) to model the behaviour of a near-perfect player in our novel games. Owing to computational costs, we estimate the expected payoffs using 50 simulations per game; these samples are then bootstrapped in their fit to people, as with the other models.

Comparison of MCTS and Intuitive Gamer

We briefly clarify the relation between MCTS—the planning algorithm behind many AI systems that achieve expert-level gameplay such as AlphaGo and AlphaZero—and the contribution the Intuitive Gamer model offers as a model of human reasoning. One could potentially view the Intuitive Gamer’s player module as a particularly restricted or shallow variant of MCTS (as MCTS is also compatible with constraining computational resources and adopting heuristic functions). However, for several reasons we think it is important to distinguish between these classes of models—recognizing that this is in part a matter of scientific emphasis and interpretation rather than algorithmic innovation.

First, framing MCTS so generally as to include the Intuitive Gamer’s play would also include almost any kind of stochastic tree search, including other models in our comparison set, for example, the Random Gamer. More deeply, it would leave out core features of MCTS that have made it so powerful in AI systems as well as the most distinctive features of the Intuitive Gamer as a cognitive model. MCTS made such an impact in AI gameplay and reinforcement learning^4,14,27,28 precisely because it could achieve very strong play without the need for heuristics to guide search. Instead, it uses extensive inference-time computation and sophisticated tree traversal arithmetic (backtracking, node visit counting) to effectively explore a very large, unstructured game tree. This is in contrast to the Intuitive Gamer, which does not require any sophisticated algorithms and uses very minimal computation, although it does rely on a small number of abstract heuristics when assessing the value of next moves. Such a design leads the Intuitive Gamer to be a highly efficient if less than optimal player, and one which we believe much better captures how people reason (as evidenced in our behavioural action selection and action prediction studies). As described in the ‘Resource-rational reasoning’ section of the main text and captured in Extended Data Table 2b, the Intuitive Gamer is orders of magnitude more efficient than either MCTS (run for 10,000 iterations, as described above) or the Expert Gamer, in terms of wallclock time and number of board states evaluated.

Variations on the full Intuitive Gamer model

We compare the full Intuitive Gamer model against several variants that modulate whether it is flat, goal-directed or probabilistic. The non-flat version of the Intuitive Gamer model performs a deeper search over possible moves when selecting actions. The non-goal-directed version of the Intuitive Gamer model ablates one or more of the value function components (that is, player progress or blocking progress—see Supplementary Information for ablations of the centre-bias component). The non-probabilistic version of the Intuitive Gamer model replaces the softmax element of action selection with a deterministic choice (or equivalently, softmax with temperature zero). We further vary fastness by conducting a sample complexity analysis wherein we modulate the number of simulations drawn (k) from the gamer model, over which the game reasoning engine computes the expected payoff (Extended Data Fig. 1).

Game-theoretic optimal payoffs

Many of the games in our set have known game-theoretic ground-truth outcome values assuming optimal play from both players. We describe how we filter which games of the 121 are estimatable via a game-theoretic optimal payoff. We first include those with known game-theoretic optimal payoffs (either definite player 1 win, definite draw, or definite player 2 win) according to ref. ⁶¹. Then, to approximate a larger set of games’ optimal payoffs, we additionally include games for which MCTS converged to −1, 0, 1 in its payoff predictions (noting that MCTS estimates are not guaranteed to be perfect). This process results in a set of 78 out of the full 121 games for which we have estimated game-theoretic payoffs.

Analysis methods

We next provide additional details on experimental analyses.

Comparing payoff predictions

Participants answered two questions in the game outcome reasoning experiment: how likely the first player wins given the game does not end in a draw (P(P1 wins∣not draw)), and how likely the game is to end in a draw (P(draw)). Together, these responses yield all information we need to compute an expected payoff. We compare the expected payoff from participants against those of models and report the R². Specifically, we bootstrap subsample with replacement from the human participant samples, per game, and compare models against the mean payoff per sample. We also bootstrap subsample k for 20 simulated participants per model from the pool of model simulations. Unless otherwise stated, we run 1,000 bootstrap samples.

Sample complexity analyses

We select k (the number of simulations sampled for each simulated participant) by inspecting the variance of the payoff predictions over the simulated set of 20 participants against the empirical variance observed in the human data. We measure the root mean squared error between the model- and human-predicted variance for each game, as well as the Wasserstein distance between the distribution over model and human variances (Fig. 3c). We compute the latter to better capture the poor match of high k to people’s variances (that is, with increased k, variance collapses to zero). We find that that generally approximately k = 5–7 full game simulations reasonably well-captures human variance (Fig. 3c and Extended Data Fig. 1) under both measures. We report all main results with k = 6.

Fitting regression models to funness judgements

We fit a linear regression model to these features using the lmer package in R. Features are normalized to zero-mean and unit standard deviation. We fit the models to 1,000 bootstrapped subsamples of the human data for all games. Models are tasked with predicting the mean of participants’ funness judgements for each game. We compare models via an ANOVA test and AIC in Supplementary Information section 5.7.

In addition, we ran a generalization test wherein we fit regression models on 50% (59) of the 118 games studied (infinite boards were removed) and tested each model on the held-out 50% of games. We report both results in Extended Data Fig. 4 and include additional details in Supplementary Information section 5.7.

Assessing contribution of non-simulation-based features to funness judgements

In exploratory analyses, we assess the potential role of non-simulation-based features that can be read off of the game description alone. We compute a series of binary game traits that capture ways in which a game may differ from the base tic-tac-toe. For instance, a game may not be a 3 × 3 board; the game may end with K ≠ 3 pieces in a row to win; the second player may have a different win condition than the first player. We consider the following binary features: if the game has constrained win conditions (for example, no diagonals allowed); if the game has asymmetric win conditions (between players 1 and 2); if the game has asymmetric play dynamics (for example, player 2 can place two pieces on their first turn); if the board is not square; K ≠ 3 pieces in a row to win; if the board is larger than 3 × 3; if the game ends when the first player to achieve K in a row loses (that is, misère variants). We combine the binary features into a single aggregated measure of ‘approximate novelty’, which is the sum of the number of binary features present in any one game. We also explore the incorporation of board size (encoded as the number of rows × number of columns) as part of non-simulation-based game features that reasoners may consider when assessing the funness of games. We assess how well participants’ funness judgements can be explained by these features alone by fitting bootstrapped sets of linear regression models to these features in the same 50/50 train/test splits over the games. In addition, we explore the relative benefit of adding all binary features or any of the aggregate features to the simulation-based model over the full set of games (see Supplementary Information section 5.7 for more details and additional analyses).

Assessing game-time action selections

To assess how well models capture how people play, we condition the model in question on an intermediate board state of the actual human-versus-human game. All moves in the game are considered (encompassing early-, mid- and late-stage play). We computed the model’s predicted distribution over next moves. We assess the fit of this predicted distribution in two ways. First, we compute the log likelihood of the actual players’ moves against those predicted by the model. To handle cases where any model may place at or near-zero probability on any given position, thereby skewing the log likelihood, we incorporate a ‘slop’ parameter α that captures the probability that a player makes a move from the primary model (1 − α that they make a random move on that turn⁶²). We sweep over α between 0.5 and 0.95 in increments of 0.05and compute the average log likelihood for each model over the settings of α. We show per-game stage and per-game results in Supplementary Information section 6. We conduct a one-sided paired t-test over each individual move as well as the aggregated move likelihood per game, comparing the Intuitive Gamer with the Expert Gamer and Intuitive Gamer with the Random Gamer, respectively.

Fitting admixture models to actions

We then consider two different admixture models: one over all moves made for a game, and one over all moves made for each player. Admixture models captures the notion that a player may play according to either the Intuitive Gamer, Expert Gamer or Random Gamer model on any turn. We jointly fit the weights of the Intuitive Gamer and Expert Gamer models and take the weight of the Random Gamer model to be the remaining mass that brings the weights to 1, over bootstrapped samples over the entire population of players. As the weights are fit jointly across the models, we need to be particularly careful about the magnitude of the results for the respective models. To that end, we jointly sweep over the temperature used in the action selection (with temperature ranging from 0.5 to 3.0 to produce a distribution over weights). We also fit the admixture over each individual, estimating the relative contribution of each model to the way they play. As each player plays only six games (and one round per game), we fit these weights to all of their moves across all games. We use scikit-learn for fitting, with the SLSQP optimizer.

Computing aggregate human distribution over predicted next action

For our ‘watch-and-predict’ experiments, we have access to finer-grained predictions from any one participant. We again extract distributions over likely next moves, conditioned on the observed intermediate board state from the novice and alternative models; however, we now compare the distributions directly with the suggested distribution of moves made by participants. We construct an aggregate move distribution over the participants, treating each of the five clicks per participant as contributing some mass to the ‘aggregate human’ distribution.

Comparing model- with human-predicted distributions over next actions

We compare distributions using the TVD; we fix temperature at 1 and sweep over α for both models and people. We conduct a similar one-sided paired t-test over the TVD across the three game stage boards seen per match and compare the Intuitive Gamer and Expert Gamer, and the Intuitive Gamer and Random Gamer. We compute split-half TVDs per stage, per match, per game by randomly splitting the participants’ distributions who saw that board, averaging those boards into two aggregate distributions and computing TVD between them. This forms our split-half human estimate, which we use to normalize the other models’ TVD. We demonstrate the robustness of our results to our choice of distributional measure by repeating the analyses with the Jensen–Shannon divergence in Supplementary Information section 6.2. We additionally run an admixture model over both participant- and game-level predicted watch distributions. These are optimized against the Jensen–Shannon divergence.

Comparing model- with human-predicted distributions over next actions to people’s played actions

We secondarily assess how well the distribution over suggested moves made by the watcher captures how people actually played by treating the suggested distribution of clicks by the human watcher akin to the move distributions from the models. We again assess the likelihood of the move played by the active player to the moves suggested by the predictors, the Intuitive Gamer model, and alternative models by sweeping over and averaging out a slippage parameter (α). We conduct these analyses in aggregate (over all board stages, matches and games), as well as at a per game level (over board stages and matches).

Modelling draw requests

We take initial steps to analyse evaluations made during the game by looking at players’ decisions of whether to accept or reject a draw, when offered. Draw requests could take place at any point during the game. For each board where a draw request was made, we run 40 simulations to the end of the game under the Intuitive Gamer agent model. We compute the expected payoff over bootstrapped subsampled sets of k = 6 outcomes (simulating a single player; previously computed under one of our ψ(G) game reasoning queries), as well as the expected remaining length of the game ℓ from state s_t. We compute expected payoff here with respect to the player who is deciding whether or not to accept the draw request. Players receive a bonus payout of US$0.50 only if they win (nothing if they lose or draw); we can then compute an expected value that is the payoff ×0.5. For each set of bonus-adjusted expected payoffs and expected match length remaining, we fit a logistic regression model to predict whether a player would accept or reject the draw request, based on expected payoff and length, as well as the average predicted game funness predicted by people in the ‘just think’ game reasoning experiment. We fit the logistic regression model using the glm R package. We also explore the same computations under different agent models varying in depth (for example, the Expert Gamer model) and goal-directedness of the value function (for example, ablating the goal progress component).

We show the decision boundaries in Extended Data Fig. 8a and report the bootstrapped parameter fits in Extended Data Fig. 8b. For plotting, we use a collapsed measure of expected value which we call the ‘expected value of continuing’ (C) from state s_t:

This measure combines the expected value (bonus-adjusted payoff) with the opportunity cost of continuing (based on expected length remaining).

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.