Experiments (A/B Testing)

Experiments let you compare Groups — usually a control and one or more variants — for the same Member-facing experience. Gravity Rail assigns each Member to exactly one Group per Experiment, keeps that assignment sticky, and exposes the Group in CEL expressions so workflows, Actions, and task edges can branch without a separate enrollment step.

Use experiments when you need weighted randomization and outcome comparison, not when a simple label or workspace-wide feature toggle is enough.

Where to find it

Open Automations → Experiments in your workspace sidebar. Member assignments appear on each Member profile under the Experiments tab.

Requires the workspace Experiments feature and role scopes experiments:read / experiments:write (typically granted with automation permissions).

Key concepts

Creating an experiment

1. Go to Automations → Experiments → New experiment
2. Set a slug (lowercase, hyphens — used in CEL) and name
3. Add at least two Groups with allocation weights that sum to 100
4. Mark one Group as control
5. Optionally set Eligibility CEL (see Filtering who gets assigned)
6. Start the Experiment when ready (status → running)

While an Experiment is in draft, you can edit slugs and Groups freely. After it has run, slugs are locked because CEL expressions and reports depend on them.

How assignment works (lazy)

Gravity Rail does not pre-assign every Member when an Experiment starts. Instead:

1. You write a branch that reads the Member's experiment Group in CEL, for example:

   cel
   

2. The first time that expression is evaluated for a Member on a runtime surface (Event Rules including journey step execute, automatic task edges, journey step connection conditions, and similar), Gravity Rail:

   • Checks the Experiment is running
   • Applies eligibility if configured
   • Assigns the Member to a Group using deterministic weighted hashing
   • Records an exposure
   • Populates member.experiments["opening-message-test"] with the Group slug

3. Later reads return the same Group — assignment is sticky.

If the Experiment is paused, completed, or archived, new assignments are not created. Existing assignments remain readable in CEL and on the Member profile.

Branching in workflows and actions

Use experiment checks anywhere CEL runs on runtime assignment surfaces:

Event rule on a journey step (journey_step:execute) — typical pattern for comparing outbound SMS or email variants when a Member reaches a step:

cel

Event rule on task execute or other assignment events:

cel

Journey step connection (branching between steps):

cel

Automatic task edge:

cel

Check membership without comparing to a Group:

cel

Alternative slug syntax (hyphens → underscores): member.experiments.opening_message_test == "control".

If a Member is not assigned (ineligible, Experiment not running, or slug never read), the key is absent and equality checks evaluate false. Author a fallback path when you need behavior for unassigned Members.

Filtering who gets assigned

Eligibility CEL on the Experiment settings page limits who can be assigned when they first hit an experiment branch.

Example — only assign Members 18 or older:

cel

Example — only Members with a specific label:

cel

Members who fail eligibility:

• Are not assigned to any Group
• Do not appear in experiment stats denominators
• Have no entry in member.experiments[slug] — branch conditions that assume assignment evaluate false

Eligibility is evaluated once at assignment time, not on every subsequent read.

Filtering what you see in lists

Archived experiments

The Experiments list hides archived experiments by default. Turn on Include archived in the UI (or pass includeArchived=true on the API / yarn gr experiments list --include-archived true) to show them.

Archived experiments remain available for historical reporting but do not accept new assignments.

Reporting

Experiment stats

Open an Experiment → Stats to see per-Group assigned and exposed counts.

Journey goal breakdown

When analyzing a Journey, you can break goal achievement down by Experiment Group:

1. Open the Journey
2. Select an Experiment from the goal reporting control (when enabled)
3. Each goal shows per-Group exposed, achieved, and conversion rate

Who counts in the breakdown:

• Only Members enrolled in this Journey who have an exposure row for the selected Experiment
• Assigned Members who were never exposed are excluded from denominators
• Journey enrollments without experiment exposure are excluded

This keeps journey comparisons aligned with Members who actually saw the treatment, not everyone who might eventually be assigned elsewhere.

Small-number suppression (privacy)

To reduce re-identification risk in small cohorts, Gravity Rail suppresses thin counts in experiment reporting:

When the exposed count for a Group is suppressed, conversion rate is hidden (null / not shown) even if achievement count is known.

This applies to Experiment stats and Journey goal experiment breakdowns. It is a display-layer rule for aggregate API responses, not a filter on who is assigned.

Member profile

The Experiments tab on a Member shows current assignments: Experiment name, Group, assignment time, and source (lazy for first-encounter assignment).

CLI quick reference

bash

Create and update accept JSON via --data. See the developer API reference for full request shapes.

Filtering members by assignment

On the Members page, add a filter rule of type Experiments:

Saved member filters and routines can use the same rule shape in the filters JSON array (objectType: "experiment").

API query parameters

GET /members also accepts direct query params (alternative to filter rules):

Example API call:

http

Filter rules use objectType: "experiment", filterType: "any" | "none" | "inGroup", and objectIds (experiment IDs or group IDs).

Related guides

• CEL Expressions — member.experiments namespace
• Actions — trigger experiment branches from EventRules
• Workflows — task edges and prompts
• Journeys — programs and goal tracking
• Members — profile and assignments