New Feature: A/B Testing

Violeta Milarova

Aug 14, 2026

Personalization is never finished.

Every change to your recommendation strategy raises new questions. Should you boost fresh content? Change used Logic? Introduce new filters? Even small changes can influence engagement, conversions, and revenue.

That's why we've introduced A/B Testing.

A/B Testing lets you compare different recommendation configurations within a Scenario and measure their impact before rolling changes out to all users. Instead of relying on assumptions, teams can validate every optimization with real user behavior.

Test Every Part of Your Recommendation Strategy

Each experiment runs against your current Scenario configuration, with traffic automatically split between a Control and one or more Variants.

Variants can modify any combination of recommendation Logic, Filters, Boosters, and Constraints, making it easy to compare different strategies while keeping everything else consistent.

The Admin UI also highlights exactly what changed in each Variant, giving teams a clear overview of every experiment.

Measure the Metrics That Matter

Choose from a library of predefined metrics tailored to your industry, including Click-Through Rate (CTR), Conversion Rate, Watch Time from Recommendations, and Income from Recommendations.

Need something more specific? You can also create custom metrics using Insights, allowing you to measure highly targeted business outcomes, from purchases within selected product categories to engagement with specific types of content.

Make Decisions with Confidence

Once an experiment is running, Recombee continuously evaluates each Variant and presents the results in an intuitive report.

Compare improvements against the Control, see the probability that each Variant outperforms the baseline, understand whether the results are statistically significant, and identify the best-performing configuration at a glance.

Whether you're fine-tuning recommendation models or experimenting with entirely new strategies, A/B Testing helps every optimization move forward with confidence rather than guesswork.