Platform / Governance

Every change has a name on it.

Before you let anyone change your live site, you want three things: a record of what happened, control over who is allowed to do it, and a way to stop it. More than 90 kinds of action are written down, from a sign-in to a billing change, with the person who did it, the address they worked from, and the values before and after.

The audit trail and the sign-in protection are on from your first day, on the free plan. They are not an upgrade you buy later.

Start free

Four rows of the audit log, each naming the member who acted, the kind of action, the resource it touched and the surface the change was made from. — The audit log, read like any other report.

Every organisation starts with 5 roles already built, so you can invite a colleague without designing a permission model first.

Owner: Full control of the workspace, with every permission on.
Admin: Full access to billing, project settings, and the team.
Developer: Implementation, technical configuration, and debugging across environments.
Collaborator: Can launch and pause existing experiments, and read reports.
Viewer: Read-only access to every report and experiment setup.

If none of those fit, build your own. A custom role is built from 10 named permissions, switched on one at a time, and you can keep up to 10 of them per organisation. Custom roles are part of the Enterprise package.

Manage Projects
Edit Project Settings
Create Experiments
Edit Code & Variations
Publish to Production
View Reports
Export Data
View Flag Drafts
Manage Webhooks
Manage Support

One split matters more than it looks. Publishing to production is its own permission, separate from creating an experiment. Someone can build all day without being able to put anything in front of a customer.

Roles settings with six roles listed down the left, five built in and one created by the organisation, and the permissions of the selected Developer role grouped beside them under project management, experiment workflow and analytics, each one a switch that is on or off. — Every role this organisation can assign, and what the selected one is allowed to reach.

The create custom role dialog, with a name field, a description field, and the permissions below them as individual switches grouped by the part of the product each one controls. — Name it, then switch on only the parts of the product it should reach.

Every entry carries

Who did it, by name and email address
The network address they worked from
What the value was before, and what it is now
Whether it succeeded, and why it did not

It also records where the action came from: the dashboard, the API, the command line, and the platform itself. A change made by a script through the API is just as visible as one made by a person clicking a button.

Failed attempts are kept too, with the reason each one failed. You see what someone tried to do, not only what worked.

Audit log rows up close. On a wide screen: the member who acted, the action taken, the resource it touched including a role change against a colleague's email address, and the dashboard as the surface it came from. On a narrow screen the window closes in on the actor and the kind of action alone. — One of these rows is a role change, with the member it was applied to.

For people

Two-factor sign-in by emailed code. Switch it on and every sign-in from then on needs your password plus a 6 digit code sent to your email address. This is an emailed code, not an authenticator app.

The code expires 10 minutes after it is issued, and it is stored scrambled, so nobody at our end can read it back to you. After 3 wrong answers the code stops working and you have to request a new one. Requests are rate limited, so nobody can sit there guessing.

Turning it on, and every failed attempt, is recorded. Those entries belong to your account rather than to any one organisation, so they sit outside an organisation’s own log.

For machines

Access tokens, scoped resource by resource. You grant a token read or write on each part of the platform separately, so the script that reads results cannot quietly change an experiment.

Give a token an expiry of up to a year, or none at all. Rotate it and the old secret stops working, while the record of what it did stays. Revoke it and it is gone. Every token shows when it was last used and from where, so a token nobody can account for is easy to spot.

A token is stored scrambled and shown to you once, when you create it. If you lose it, rotate it. Nobody can look it up for you.

Read the authentication reference

Access token settings. On a wide screen: the organisation identifier ready to copy above two service tokens, each row carrying a masked prefix, the read scopes it was granted, its expiry and when it was last used, and buttons to rotate or revoke it. On a narrow screen the window closes in on the scopes, the expiry and the last use of the same two tokens. One expires in days, the other is set never to expire. — Two service tokens: what each may reach, when it expires, and when it was last used.

The analysis plan panel of a running experiment, marked sealed, stating that the plan was sealed at launch and later changes are recorded as amendments against the sealed values, with the primary metric and two secondary metrics already fixed. — A sealed plan: the metric the result will be judged on, fixed before the first visitor arrives.

When the plan has to change

Sometimes it genuinely does. Amendments are allowed, and each one is recorded field by field. 6 things go on the record every time.

Which field changed
What it was before
What it is now
Who changed it
When
Why

The reason is not optional. A plan cannot be amended without one.

When someone wants to call it early

Stopping a fixed-sample test the moment it looks good is one of the easiest ways to get a wrong answer. Three guards sit in the way.

You see how far through the test you areA banner on the results page says how far through the planned sample the test really is, before you read anything into the numbers.
Stopping early takes a deliberate confirmationStopping early raises a confirmation you have to answer. It is not a warning you can scroll past.
Your override goes on the recordYou are always allowed to overrule it. The fact that you did, when, and who you are, is written down permanently.

Checking a result over and over raises the odds of a false alarm, so the results page tells you roughly by how much, based on how many times you have actually looked. How we work that out

It pauses on errors, by default

If a variation starts throwing errors at more than 25 percent of the people who see it, the experiment pauses itself. It is on from the moment the experiment is created, and the threshold is yours to move.

It tells you what broke

Each experiment keeps its own error log: script errors, changes that failed to apply, and elements that were not on the page. Rates appear once at least 50 visitors have seen the variation, because a percentage taken from a handful of people is noise, not a measurement.

Guardrails that can actually fail

Name the numbers that must not get worse, and each one gets a real test with three honest outcomes: Safe, Breached, and Inconclusive. They are tested, not just watched, so a guardrail can come back and say no.

An alarm on the traffic split

If the split between your variations drifts from what you asked for, the result is suspect however good it looks. So it is checked and flagged. The check runs from the last time you changed the split, not over all time, so yesterday’s mistake still shows.

On a wide screen: a results page with its verdict at the top, ship it, and below it the guardrail block. On a narrow screen the window closes in on the guardrail block alone: the protected metric, the margin it must stay inside, the worst plausible effect measured against that margin, and the safe status the test returned. — A guardrail with its own margin, tested and reported alongside the result.

And when you stop an experiment, stopping is permanent. The variation code leaves your site straight away, with no half-stopped state where a change is still reaching some visitors. Your results are kept, so you lose nothing but the risk.

Webhooks on about 25 product events, from an experiment launching to a guardrail failing. Every attempt is kept with the response it got, you can retry a delivery by hand, and you can rotate the signing secret in place. A destination that keeps failing is switched off automatically instead of hammering a dead endpoint.

Or skip the plumbing. Route alerts straight into Slack, Microsoft Teams, and Jira. Choose per project which events go where, and set keywords so a channel only hears about the work it cares about. That way people keep reading the alerts instead of muting the channel.

And everything the dashboard does, the API does. Experiments, flags, metrics, audiences, segments, members, roles and webhooks are all reachable, with 24 pages of reference documentation behind them. So you can automate the governance work instead of clicking through it.

Read the developer reference

When the problem is ours, you hear it from us.

There is a public status page, checked from outside our own network, so a problem cannot hide behind the thing that is broken. Anyone can read it, account or no account.

Open the status page

Put every change on the record.

Free for your first 10,000 visitors a month, with the audit trail and two-factor sign-in included. Custom roles come with Enterprise.

Start free See pricing

Every change has a name on it.

Decide exactly who can do what.

The audit log has the answer.

Two-factor for people, scoped tokens for machines.