# A/B test a game change (/docs/guides/ab-test)

You think a shorter tutorial will get more new players to the fun part. Instead of shipping it to everyone and comparing this week with last week (a different week, different players, maybe a holiday), split new players in two and compare them on the same days. This guide runs that test end to end.

<Steps>

### Track the goal [#track-the-goal]

An experiment needs an event that says "it worked". Here that's `tutorial_completed`, sent from the server when a player finishes. Revenue comes for free from `$purchase`.

```luau
RoChart.track(player, "tutorial_completed", { seconds = os.clock() - startedAt })
```

If you followed the [onboarding guide](/docs/guides/onboarding-funnel), you already have it.

### Create the experiment [#create-the-experiment]

**Experiments → New experiment**, or describe it in a sentence:

> Does a 3-step tutorial instead of the current 6 steps get more new players to finish it?

Check the plan RoChart drafts:

- **Who sees it**: everyone, split 50 / 50 between `control` and `short`, on a new flag `short-tutorial`.
- **Primary metric**: % of players who do `tutorial_completed` within 1 day.
- **Secondary metrics**: sum of `$robux` on `$purchase` per player within 7 days, so you notice if the shorter tutorial costs you revenue.
- **Segments**: platform, or a cohort such as **Premium players**, to see whether phones react differently.

Save it as a draft for now.

### Read the variant on the server [#read-the-variant-on-the-server]

Read the flag where the experience starts, when the tutorial begins, not when the player joins. Everyone who reads it counts as exposed, so reading it on join would dilute the result with players who never reached the tutorial.

```luau title="ServerScriptService/Tutorial.server.luau"
local RoChart = require(game.ServerScriptService.RoChart)

local function startTutorial(player)
  if RoChart.getFlag(player, "short-tutorial") == "short" then
    runTutorial(player, SHORT_STEPS)
  else
    runTutorial(player, FULL_STEPS) -- "control", or the flag is off or unknown
  end
end
```

The flag is evaluated per player on the game server, so a player can't change their variant from the client. The same player always gets the same variant, on every server and device. Anything that isn't `"short"` takes the current path, so the game keeps working if the flag is off.

Publish the place, then **Launch** the experiment. Old servers keep running the old code until they shut down, so give it a few hours, or use **Shut Down All Servers** if you want everyone on the new version.

### Read the results [#read-the-results]

Results update continuously, and looking early doesn't invalidate them:

```text title="What you and your agent read"
Experiment "Short tutorial" (running). Exposures: control 4,212, short 4,190.
Primary: tutorial_completed — control: 38.1% (n=4212); short: 47.9% (n=4190) lift +25.7% [19.8%, 31.9%], P(beats control) 100.0%. Verdict: winner (short).
Segment "Phone": tutorial_completed — … lift +34.0% [24.1%, 44.6%], P(beats control) 100.0%. Verdict: winner (short).
Recommendation: Ship "short": it is the best variant with 100.0% probability.
```

Check the secondary metric before you ship. A tutorial that more players finish but that skips the shop might bring in less Robux. See [Read results](/docs/experiments/results).

### Ship the winner [#ship-the-winner]

**Conclude** the experiment and roll `short` out to everyone through the same flag. The code doesn't change. Later, when you remove the old tutorial, delete the flag check too.

</Steps>

## Other things worth testing [#other-things-worth-testing]

- **Prices**: a 99 vs 149 Robux starter pack. Primary: % of players with a `$purchase` of that product; secondary: Robux per player.
- **Rewards**: a daily reward on day 2. Primary: % who `daily_reward_claimed` again within 7 days.
- **Matchmaking or difficulty**: primary: % who play a second `match_ended` within the session.

Keep one primary metric per test, and pick it before launch. Or hand it to your agent with the experiment prompt on [Create an experiment](/docs/experiments/create).