What is the difference between feature flags and A/B testing?

A feature flag is an engineering control that turns a code path on or off for rollouts, kill switches, or targeted access. An A/B test is a measurement that splits traffic between variants and compares a metric to learn which performs better. They share a per-request decision mechanism but answer different questions.

Can you run A/B tests with a feature flag tool?

Yes. Flag and SDK tools like LaunchDarkly, Statsig, and GrowthBook support experiments and are safe and flicker-free. The tradeoff is that every variant is code, so an engineer builds and deploys each test and marketing depends on engineering for each one.

Where does Apertio fit between flags and visual A/B editors?

Apertio sits on the engineering-to-marketing seam. An engineer declares a typed slot once, like a flag. After that, marketing tests within that slot with no new code, like a visual editor, but the variant is bounded data resolved server-side, not injected JavaScript that mutates the DOM.

Do you still need engineering with Apertio?

For declared slots, no: marketing tests within them autonomously. For a brand new surface, yes: an engineer adds a slot first. Apertio is explicit that it converts "an engineer per test" into "an engineer per new surface," not zero engineering.

Learn

Feature flags vs A/B testing

Feature flags and A/B tests share a per-request decision mechanism but answer different questions: one controls delivery, the other produces a verdict. Apertio sits on the seam, with the safety of a flag SDK and the autonomy of a visual editor.

Feature flags and A/B testing get compared because they share a mechanism: both decide, per request, which version of something a given user sees. They answer different questions. A feature flag is an engineering control. It gates whether a code path is on or off, for a release, a kill switch, a gradual rollout, or access for a specific account. An A/B test is a measurement. It splits traffic between variants and compares a metric to learn which one performs better. One controls delivery. The other produces a verdict.

Feature flag and SDK tools (LaunchDarkly, Statsig, GrowthBook) are safe and flicker-free. They evaluate a configuration in your own code and resolve before render, so there is no DOM mutation and no flash. That is the right model for an engineering control, and Apertio does not argue with it. The cost shows up when marketing wants to run a test. In a flag or SDK tool, every variant is code. An engineer writes the branch for each test, deploys it, and is on the hook for the next one. Marketing has the idea but has to route it through engineering. The tool is safe; the workflow makes engineering the bottleneck for every experiment.

Client-side visual editors solve the workflow problem and break the safety one. They give marketing a no-code editor, but underneath they inject third-party JavaScript that mutates the live DOM. That causes flicker, can force unsafe-eval and break a strict Content Security Policy, and adds an XSS and supply-chain surface. So the two established options sit on opposite sides of a seam: engineering-owned flags are safe but make marketing wait, and marketing-owned visual editors are autonomous but inject code. Neither gives marketing autonomy and keeps the safety an engineer wants.

Apertio sits on that seam. An engineer declares a typed slot once, the same kind of one-time instrumentation a feature flag needs. From then on, marketing fills and tests that slot within bounds with no new code, the autonomy a visual editor promises. The difference from a visual editor is what the variant is: bounded structured content, a primitive or an enum key, resolved server-side or at the edge and rendered by the app’s own components. It keeps the flicker-free, CSP-safe properties of a flag SDK and adds marketing autonomy, with no injected JavaScript. The honest limitation is the boundary itself: only declared slots are testable, so a novel surface still needs an engineer to add one. The instrumentation cost moves from every test to every new surface.

Declared once, like a flag

// slots.ts — declared once, like a flag. After this, marketing tests within
// these bounds with no further code. The variant payload is data, never a branch.
import { slot } from '@apertio/core'

export const slots = {
  'pricing.cta.copy': slot.text({
    label: 'Pricing CTA copy',
    default: 'Start free',
    maxLength: 24,
  }),
  'pricing.hero.layout': slot.enum({
    label: 'Pricing hero layout',
    options: ['centered', 'split', 'media-right'] as const,
    default: 'centered',
  }),
}

After this, marketing tests within these bounds with no further code.

Feature flag: an engineering control

Gates a code path on or off for rollouts, kill switches, and targeted access. Owned by engineering, changed on deploy.

A/B test: a measurement

Splits traffic and compares a metric to decide which variant performs better. The output is a verdict about which one to keep.

They share a mechanism

Both choose per request which version a user sees. An A/B test adds random assignment and exposure tracking on top of that choice.

Where Apertio sits

Engineers declare a typed slot once, like a flag. Marketing then tests within it with no code, like a visual editor, and the variant is bounded data the app renders through its own components.

Give marketing test autonomy without giving up the CSP

Declare a typed slot once. Marketing tests within bounds, and variants resolve as bounded data the app renders.

Read the docs Start free

Frequently asked questions

What is the difference between feature flags and A/B testing?: A feature flag is an engineering control that turns a code path on or off for rollouts, kill switches, or targeted access. An A/B test is a measurement that splits traffic between variants and compares a metric to learn which performs better. They share a per-request decision mechanism but answer different questions.
Can you run A/B tests with a feature flag tool?: Yes. Flag and SDK tools like LaunchDarkly, Statsig, and GrowthBook support experiments and are safe and flicker-free. The tradeoff is that every variant is code, so an engineer builds and deploys each test and marketing depends on engineering for each one.
Where does Apertio fit between flags and visual A/B editors?: Apertio sits on the engineering-to-marketing seam. An engineer declares a typed slot once, like a flag. After that, marketing tests within that slot with no new code, like a visual editor, but the variant is bounded data resolved server-side, not injected JavaScript that mutates the DOM.
Do you still need engineering with Apertio?: For declared slots, no: marketing tests within them autonomously. For a brand new surface, yes: an engineer adds a slot first. Apertio is explicit that it converts "an engineer per test" into "an engineer per new surface," not zero engineering.