How players are matched, how sessions are checked, and how the ranges work

Who can be invited

Players sign in with Steam and agree in writing to be recorded. To be invited, a player is 18 or older and lives in the US, and their Steam account is at least 6 months old, with 5 or more hours played on Steam and no VAC, game or trade bans. Their Game details must be public, because the match is made from their library.

How players are matched to your demo

You name 3 to 5 Steam games yours is like. Each eligible player gets a score from their library:

• Hours in your comparable games count most, about half the score. Hours are counted on a curve, so 200 hours in those games scores near the top and a few thousand don't outweigh everything else.
• Taste, about a third: how closely the Steam tags of the games they play line up with your comparable games' tags.
• Hours on Steam overall, a small part.
• Players whose past sessions were usually accepted get a small nudge up, and players whose sessions were often rejected aren't invited.

We invite the best matches in waves, a few more than you booked, because some people never open the email. A player is on one live test at a time and isn't invited to the same studio's next test right after its last one. An invitation that isn't opened within 48 hours goes to the next player.

How each session is checked

A session counts only when the evidence shows the demo was played and the answers are real:

• The recording shows the demo being played, with the player thinking out loud, for at least 30 minutes.
• Playtime backs it up: Steam playtime in a public demo, or, for a private Playtest, key activation and playtime from your own Steamworks access.
• Gameplay questions you write, which only someone who played could answer.
• Effort checks on the written answers: too short, copied between players, or generated answers don't count. A recording file used for two sessions doesn't count either.
• A person reviews every session with a serious flag, and about 1 in 20 sessions that pass anyway, picked at random.

A player who stops after at least 10 minutes and says why still counts as a shorter session, because why someone quit is part of what you're testing. After the report arrives you have 48 hours to flag a session you think doesn't hold up, and a person checks it.

How the report's ranges work

A handful of players can't give an exact percentage, so every share in the report comes with a 95% range (a Wilson score interval): where the true share for players like these most likely sits.

• If 3 of 5 players want to keep playing, that's 60%, and the true share most likely sits between 23% and 88%.
• If 15 of 25 do, it's also 60%, but the range narrows to 41% to 77%.

That's why small tests are for finding problems (5 to 10 players usually show the biggest ones) and bigger tests are for numbers you compare between versions. A breakdown by how many hours players have on Steam appears only when both halves have at least 10 players.

See the ranges in a sample report, or read how a playtest works.