Read results

Open an experiment to see its results, from top to bottom:

• The answer in plain words ("test-2 is winning", "Too early to tell"), how much better the leader is, a how sure meter, and a progress bar towards a confident answer.
• One card per metric, primary first, then secondary. Control's value is in the card header. Each variant shows its change vs control as a big number, green when up and red when down. A pill spans the range the true change likely falls in, with both ends labelled. It's brightest at the most likely value (the white tick) and fades towards the ends, green on the side above 0% and red below. Win % is the chance the variant beats control. Every card uses the same axis, so shapes compare across metrics. A faded shape or number means the range still crosses 0%, so it isn't decided yet.
• Over time: the primary metric so far, day by day, or cumulative exposures.
• Segments, and Summary · what agents see.

Results always cover every exposure since launch.

The examples below come from a test of a discounted starter pack offer: control vs test, 50/50, with Payer conversion (bought something within 7 days) as the primary metric.

The numbersCopy Anchor Link

Read it as: 39.1% of exposed players in test bought something vs 30.8% in control. The true lift is very likely between +15% and +40%, and there's essentially no chance control wins.

A wide interval means you know the direction but not yet the size. With three or more variants, a variant can beat control and still not be the best, so the verdict uses P(best).

VerdictsCopy Anchor Link

"Keep control" is a clear result, not a failure. Variants with fewer than 100 exposures are marked small sample.

The verdict and recommendation use only the primary metric. Secondary metrics get their own verdicts. With several secondary metrics, one may look like a winner by chance, so treat a surprise there as an idea for the next test.

Split checkCopy Anchor Link

If you set 50/50 and got 70/30, something is broken: maybe the flag is read on only one code path, one variant crashes before it sends events, or old servers are still running code that reads the flag differently. RoChart checks the split. When it's off, you see a warning banner and the recommendation tells you to check your implementation before trusting the results. Below 50 exposures the check is skipped.

How long to runCopy Anchor Link

For share-of-players metrics (retention, payer conversion, custom conversions), the recommendation estimates how many more days you need to detect your minimum lift at today's traffic:

Keep running — roughly 6 more day(s) to reach 3,609 exposures per variant.

Treat it as a guide, not a stopping rule. Before you decide, give every exposed player their full metric window (and D7 its seven days), and run at least a full week if behaviour differs by weekday.

SegmentsCopy Anchor Link

An average can hide very different stories. Segments show the primary metric again for each cohort you pick, with its own lift and verdict. Add up to 10 with Add cohort segments when you create the experiment, or Edit segments at any time, even after it ended.

The offer works, and almost all of the effect comes from new players. Power players already buy at 59%, so a discount for them may cost Robux without changing much. A sensible decision: ship it, then target the offer's flag at players not in Power players.

The overall verdict tells you whether it works. Segments tell you for whom.

ConcludeCopy Anchor Link

When you've decided, click Conclude. Pick the winner (the dialog suggests the one the results point to), write the reason, and leave Roll out winner to everyone on or turn it off. Then click Ship.

With roll-out on, everyone gets the winning variant right away through the same flag, so your RoChart.getFlag(player, …) == "test" code keeps working until you clean it up. The decision, the reason and who made it are saved on the experiment.

A good reason mentions the segments, for example: "+26.9% buyers, driven by new players (+61%). Power players flat. Follow-up: exclude Power players from the offer."