Should you start with a Feature Gate or an experiment?

Start with a Feature Gate if your immediate question is whether a change can reach a small group without breaking their experience. Set up an experiment if you need to find out whether one version works better than another.

For a new signup page, first show it to a few testers and check that confirmation arrives. Once both versions work, you can assign eligible visitors to the old or new page and compare how many complete signup. A gate can also report impact when configured for it. Decide whether you need that comparison before adding measurement.

What does your app have to do before the switch works?

The app needs two working paths and a place to check the gate. The setting in Statsig only changes behavior if the app actually uses it to choose a path.

Choose a disposable setup
Use a disposable copy and verify that records and emails stay in test services. Before opening any screen that checks the gate, confirm that the copy starts the Statsig library with a Development or Staging environment. An unset environment defaults to Production, and a gate check can log an exposure.
Confirm who the gate can affect
In Statsig, verify that the practice gate is restricted to that test environment and made-up test identities only. Check both settings before changing the gate or letting the app evaluate it. If anything is unclear, inspect the settings without evaluating or changing the gate, or trace both paths on paper.
Connect both paths
Use the Statsig software library for your app, called an SDK. Give the app a clear old path for an off result and a new path for an on result. Start with a change that does not require moving customer records.
Check who sees what
Use one eligible test identity and one outside the group. Check that each sees the expected version. Turn the gate off in the test setup and reopen both views to confirm the old path still works.

What happens if the app cannot reach Statsig?

The browser library can use a saved result when the network call fails. A returning tester can therefore behave differently from someone opening the app with no saved result. A gate with no available value defaults to off in the browser library.

In the test copy, check a fresh browser and one that already received the on result. Decide what the app should show when there is no value and how long a saved decision is acceptable. Turning the switch off does not undo a message already sent or update an offline copy by itself.

What should you measure before calling a version better?

Choose the result you care about and the identity the experiment assigns. For signup, count completed signups among eligible visitors who entered the test. Also watch for errors so an apparent improvement does not hide a broken path.

If the visitor has no account yet, decide whether a browser or session identifier fits your question. A person who switches browser can receive another assignment. Check that the signup event can still be linked to the group that saw the page. If you have too few observations to distinguish the versions, report the result as inconclusive.

Which activity counts toward the plan limit?

Statsig lists a free Developer plan with two million metered events per month and unlimited flag and configuration checks. Some experiment or gate exposures can count as metered events. Statsig can log an exposure when your app checks an experiment or an eligible flag. Check that the app records exposure where visitors reach the experience you intend to measure.

The pricing FAQ says gate and configuration targeting continues if you stay on the free plan past the event limit, but Statsig stops calculating how much a gate changed the measured result. Check event usage before expanding an experiment.

Keep a record of who qualifies for each version, the metric definitions and the decisions you made. Those are needed to recreate the logic elsewhere. Scheduled export of events, exposures and metrics to a data warehouse is an Enterprise feature. Check what history you can retain on the plan you use.