From Hearts to Energy: Why Coddy Switched Their Monetization Model

Small changes to your monetization model can have massive impacts on revenue. But how do you know which changes are worth making?

For Coddy, a gamified code-learning platform, the answer came through rigorous A/B testing. After using a "hearts" system similar to Duolingo for their freemium model, they tested a switch to an "energy" system—and the data showed clear improvement.

On a recent episode of the Levels Podcast, CMO Barak Glanz shared how they approached this transition and what they learned about testing major product changes at scale. For B2C founders wrestling with similar decisions, the story offers valuable lessons about when to test, what to measure, and how to balance data with intuition.

The Original System: Hearts

Like many gamified learning apps, Coddy initially implemented a "hearts" system for managing free user limits. It's a mechanic popularized by Duolingo and familiar to anyone who's used language learning apps.

In a hearts system, users typically start with a set number of hearts (usually five). Each mistake costs a heart. Run out of hearts, and you can't continue until they regenerate over time—or you pay for unlimited hearts through subscription.

The mechanic works on a psychological level: it creates tension around making mistakes, which theoretically increases focus and attention. Users feel the stakes of each answer.

But Coddy wondered if there was a better approach.

Testing the Energy Alternative

Rather than gating usage based on mistakes, an energy system gates based purely on time or actions taken. Whether you answer correctly or incorrectly doesn't matter—you're spending energy simply by using the platform.

The psychological framing shifts completely. With hearts, mistakes feel punishing. With energy, you're simply spending a finite resource that naturally depletes through engagement, regardless of performance.

Barak explained the discovery:

"We also used the heart system in Coddy until a few months ago. And then we did the A-B testing and noticed that the energy system is slightly better."

That word "slightly" is important. The energy system wasn't dramatically better—it was incrementally better. But in B2C products with millions of users, slight improvements compound significantly over time.

Why Energy Won

While Barak didn't detail the exact metrics that showed improvement, we can infer what likely drove the better performance based on how these systems function differently.

Reduced Frustration The hearts system punishes mistakes, which can be particularly frustrating for beginners learning to code. A user struggling with a new concept not only fails to progress but also loses resources, creating a double penalty. Energy systems eliminate this frustration by making usage limits feel more neutral.

Clearer Value Proposition With energy, the subscription benefit is straightforward: unlimited usage. With hearts, the benefit is more abstract: unlimited mistakes. For users who are confident in their skills, unlimited mistakes might not feel valuable. Unlimited usage appeals to everyone, regardless of skill level.

Better Alignment with Mission Coddy's mission is to turn code learning into a daily hobby. Energy systems naturally support this by encouraging users to engage daily (when their energy recharges) rather than feeling punished for mistakes during those daily sessions.

Interestingly, Barak mentioned that Duolingo was also making a similar transition:

"I remember I saw that Duolingo are switching to the energy system. By the way, for some reason when I use Duolingo, I still have hearts, like the heart system."

This suggests Duolingo might be gradually rolling out the energy system to segments of their user base—a smart approach when testing fundamental changes to monetization mechanics.

The Challenge of A/B Testing at Scale

For most early-stage B2C companies, running meaningful A/B tests presents a significant challenge. You need enough users in each cohort to reach statistical significance, which requires either large user bases or very dramatic differences in performance.

Barak was candid about this constraint:

"We're not that big. We have like, I don't know, 2 million registered users. It sounds like a lot, but it really isn't so doing A-B testing with every new feature that you want to add or change it's... yeah it's kind of hard I mean the results got to be very significant to to make it worthwhile."

Two million registered users sounds massive, but when you're splitting them into test cohorts and measuring conversion rates (which are typically in the single-digit percentages for freemium models), you need strong signals to detect meaningful differences.

This creates a practical constraint: you can't test everything. You need to prioritize which changes are important enough to warrant formal testing versus which decisions you make based on intuition and best practices.

When to Test vs. When to Trust Your Gut

The hearts-to-energy transition represents exactly the kind of change worth testing: a fundamental shift in monetization mechanics that affects every user and directly impacts subscription conversion rates.

But Barak emphasized that testing has limits:

"You can test everything, so you also need to have a human touch. So you need to do a lot of statistics, but you also need to have a very good gut feeling."

This balance between data and intuition is crucial for B2C founders. The best founders, according to Barak, have both strong mathematical backgrounds and strong gut instincts about user psychology.

Some decisions warrant rigorous testing:

• Core monetization mechanics (like hearts vs. energy)
• Major changes to freemium limits
• Onboarding flows that affect activation rates
• Pricing structure and subscription tiers

Other decisions need to move faster based on qualitative insight:

• Copy changes and messaging tweaks
• Minor UI adjustments
• Feature experiments with small user segments
• Tactical marketing campaigns

The key is knowing which category each decision falls into and moving accordingly. Testing everything creates paralysis. Testing nothing creates waste.

Building Internal Testing Infrastructure

One advantage Coddy has is technical co-founders who prefer building tools in-house rather than using third-party services. This extends to their A/B testing infrastructure.

"We do most of the things in house. I don't know if I suggest anyone to do that, but my co-founders are like super nerds, know, they like building everything."

Their system works by assigning random keys to users at registration, then using those keys to place users into test groups for each feature being tested. It's a relatively straightforward implementation that gives them flexibility without ongoing SaaS costs.

Barak's caveat—"I don't know if I suggest anyone to do that"—is worth noting. Building in-house makes sense when you have technical co-founders who enjoy the work and when you have specific needs that off-the-shelf tools don't meet well. But for most teams, using established tools like PostHog, Optimizely, or LaunchDarkly will be faster and more reliable.

The hearts-to-energy test demonstrates what's possible when you have the infrastructure and user base to run meaningful experiments. The "slightly better" result that Barak described might translate to thousands of additional subscriptions annually—a meaningful impact discovered through systematic testing.

Experimenting with Onboarding Mechanics

Beyond the hearts-to-energy shift, Coddy continues experimenting with how they introduce usage limits to new users. One approach they're testing mirrors what Duolingo does: delaying the first encounter with energy limits.

"When you open a new account and you start using the product, it doesn't show you that your energy has a lot of more. But when it gets empty, you suddenly get a present. You suddenly get the chest, you open the chest and you get more energy."

This creates an extended "honeymoon period" where new users experience the product without immediately hitting friction. The hypothesis: users who engage more deeply before encountering limits are more likely to convert later.

These onboarding experiments represent ongoing optimization of the energy system. The core mechanic works, but the implementation details—when to show limits, how to communicate them, what recovery mechanisms to offer—all continue to evolve based on data.

Key Takeaways

• Energy systems slightly outperformed hearts systems for Coddy by reducing frustration and creating clearer subscription value propositions
• Even "slight" improvements matter at scale—small conversion rate increases compound across millions of users
• A/B testing fundamental monetization changes requires significant user volume to reach statistical significance
• Balance rigorous testing for major changes with intuition-driven speed for minor iterations
• The best B2C founders combine strong mathematical/analytical skills with good gut instincts about user psychology
• Building in-house testing infrastructure works when you have technical co-founders, but off-the-shelf tools are often faster and more reliable
• Freemium models require continuous experimentation—the initial implementation is just the starting point

Listen to the full conversation with Barak Glanz on the Levels Podcast to hear more about Coddy's data-driven approach to growth and product development.