HTML vs. AMP: A 55% Conversion Boost

A/B testing the effectiveness of HTML vs. AMP emails

AMP for emails is a relatively new technology, and many marketers are still unsure whether it's worth using, as AMP emails require more time to create and therefore have a higher cost. In collaboration with Mail.ru Cloud, we prepared an AMP campaign targeting inactive Cloud users. The goal was not only to reactivate them but also to verify the effectiveness of using AMP for this purpose. The email was sent as an A/B test: one version was a standard HTML email offering users an additional 256GB for 2 months, while the second version was a full-fledged AMP email with a 'find the object' game mechanic.

Both versions were sent with the same subject line and had approximately equal open rates.

It should be noted that the emails were sent to inactive users.

Not all users who received the AMP version were able to view it due to limited support (e.g., they opened the email in a mail client that does not support AMP).

AMP

The proportions differ from the typical 50/50 split because this campaign was specifically targeted at Mail.ru users, where AMP is supported in both mobile apps and the web interface.

While the HTML version (in the A/B test) simply required users to click a link to claim their reward, the AMP version involved a gamified experience where users had to find hidden objects before they could access the reward. Despite the added step, we observed a 78% increase in clicks to the website from the AMP version.

(Clicks to the web version of the email were not included in the statistics.)

In other words, engaging the user with a game increased clicks from the email by 78%. Once a subscriber took the initial action within the email, it became more difficult for them to stop. The primary goal of the email was to activate the gift, and we saw a 55% increase in gift activations from the AMP email compared to the HTML email. In addition to standard email metrics, we also measured the average time spent in the email, which was 10.8 seconds. Notably, users who spent more than 10 seconds in the email completed the game and clicked the link.

Results

Implementing AMP gamification in the email resulted in a 78% increase in click-through rate (CTOR) and a 55% increase in conversion rate. A crucial aspect is the psychological factor: to get a user to perform a desired action, it's necessary to hold their attention for more than 10 seconds.