Looking Back at Amazon Explore

[Update - A redacted version of the original artifact has now been published on Reforge here.]

Introduction

I recently saw this podcast where the host and guest were discussing the failure rate of tech products. It got me thinking about my own experience at the corporate innovation lab at Amazon. In the summer of 2021, I worked on an early-stage product called Amazon Explore. I was excited to work on this product as it empowered small-businesses, but the product got shut down in late 2022. In this post, I reflect on that experience and share my takeaways. I've also consolidated broader learnings from this journey, which I believe could potentially help address the high failure rate of tech products. I’ve shared those insights in a follow-up post here.

Amazon Explore was a marketplace for virtual tours and online experiences. It offered a wide range of experiences priced in the range of $10-$70 with an average of ~$40 per experience. Customers had to book the virtual experiences in advance- much like scheduling an appointment. Now, for context, Amazon Explore had a very high customer satisfaction score (CSAT)- customers who purhcased the product loved it. But, it wasn’t growing as expected. I joined as Product Manager (Technical) for the summer as an MBA Intern. I was given the charter to identify the right growth rate so that the team could make an informed resourcing request to leadership during their semi-annual planning process (OP-1/OP-2).

Segmenting The Market & Benchmarking The Product

To recommend the ‘right’ growth rate, I first had to identify appropriate benchmarks for this new type of marketplace. This was new category spawned by the pandemic and overlapped with many markets. There were a lot of niche players who offered virtual experiences in variety of formats. Given the breadth of experiences that Explore offered at the time, I treated it as an entertainment product for spending leisure time. With this framing, I then segmented the target market by identifying three dimensions that defined the unit economics of these online experience businesses. Here’s what the competitive landscape looked like back then:

I could triangulate growth-rates of the above businesses based on certain proxies. Since these were private businesses, I had to use some hacks to piece together data from different sources. Using growth-rates as a proxy for product-market fit, I could identify two comparable products to serve as our primary benchmarks: Heygo (a seed-stage startup at that time) and Airbnb Online Experiences (Airbnb OE). Once I had the estimated growth-rates, I then had to ensure that my growth-numbers were grounded in reality. For this, I calibrated my CAGR numbers with the growth rate of mature products such as Twitch, Peleton Digital, etc. Finally, I ensured that my CAGR assumptions translated into a number of users that were in sync with the market size. With this, I had a defendable range of growth-rates that showed the upside potential of Explore. For context, Airbnb OE was the fastest growing product in its history. And so, the big question facing us was 'why wasn't Explore growing remotely as quickly even though it had very high customer satisfaction scores?' My project scope was subsequently expanded to also include how Explore can plan to achieve recommended growth rates.

To understand the customers and online-creators better, I had purchased a few experiences myself for all three products (Explore, Airbnb OE, and HeyGo) and noted my observations. Experiencing the product first-hand and talking to customers helped me discover certain insights. There were a combination of factors at play- so it took some time to validate the insights with data and figure out the reasoning behind the supposedly contradictory indicators. Ultimately, the proof was in the pudding. Here's a snapshot of the three products:

Deconstructing The Network Effects To Identify The Problem

To understand the importance of these factors for Explore, it’s instructive to look at the famous Amazon Flywheel and understand the feedback loops that drove the growth of its retail business:

"Lower prices led to more customer visits. More customers increased the volume of sales and attracted more commission-paying third-party seller to the site. That allowed Amazon to get more out of fixed costs like the fulfillment centers and the servers needed to run the website. This greater efficiency then enabled it to lower prices further. Feed any part of this flywheel, they reasoned, and it should accelerate the loop."(Stratechery- The Relentless Jeff Bezos)

However, these loops always starts with exceptional ‘customer experience’. In Amazon Explore’s case, the high CSAT wasn’t translating into returning traffic and that was resulting in a cold-start problem. This, in turn, was a combination of 2 underlying problems:

1. Discovery - At the time, we were acquiring customers through Amazon.com. Given the resource constraints, this was done manually and hence there wasn’t dynamic matching between users and expriences. The team was trying to address this by integrating with internal personalization tool, and this was a medium term endeavor. But even after its implementation, this route meant competing with other Amazon Products for ad placements. Importantly, users coming to Explore would not be coming with an ‘Entertainment’ frame of reference, but with ‘Shopping’ frame of reference. As opposed to this, Airbnb was surfacing online experience on its home page to its ~100M monthly visitors who were frequenting their website with an exclusive intention to plan travel and experiences.

2. User Experience Our UX had friction beacuse of mode of delivery for an ‘entertainment’ product as well as the pricing. Users had to pre-book experiences which meant they had to anticipate when they wanted to be entertained. This meant they had to look for experiences that matched their schedule and book an 30-60 min experience for ~$40 without knowing if the experience was going to be worth it. Not having a lot of reviews for our inventory of experiences also wasn’t helping. This was leading to classic chicken-end-egg problem. In this respect, Airbnb OE had managed to get a lot of reviews. Delivering experiences via Zoom and allowing upto 50 participants per experience helped increase number of reviews as well as keep average prices down to about $15-20 per experience. HeyGo, on the other hand, had arguably the best UX. It was on-demand entertainment and free for users. They could tip the creators after they had chance to experience the product, if they liked. They were also free to drop off anytime.

Looking at the business with this lens helped explain what was going on. Amazon Explore experiences were more intimate since they were 1:1. Customers felt more of that human connection in Explore explaining the high CSAT. But its UX had friction on two critical areas. This meant existing customers weren’t returning often to try out new experiences. This explained why the CSAT scores were high but the product wasn’t growing.

Explore Now To Drive Retention & Improve The Original Product

To address these gaps, I proposed an idea for an on-demand product called "Explore Now". The main idea was to use free and instant entertainment to attract Prime Video users on a regular basis and then convert them into paying Explore users.

Explore Now was me trying to "Think Big". I was basically proposing to create a secondary Amazon flywheel that would create network effects and complement Explore's original flywheel. Not only would it solve our cold-start problem but also potentially make the business pandemic-proof! Preliminary research did suggest this was doable with existing in-house tech, but it required further assessment. My internship at Amazon was going to get over soon. I did not have time left to validate my hypothesis with experiements and scope out the technical details in great depth. So I scoped my project and concluded with certain actionable recommendations related to solving the 'Discovery' problem and the 'UX' problem.

In hindsight, I feel we did not course correct our strategy in time. Our stated goal was to unlock both 'online experiences' and 'virtual shopping' markets for Amazon. We'll leave out 'why-that-strategy' part since that's company's internal context, but this twin-goal made it difficult to optimize the product for either of the two markets. Airbnb Online Experiences and HeyGo did not have that problem. Airbnb Online Experiences is still around and part of Airbnb Experiences. HeyGo went on to raise $20M in Feb 2022 to keep growing after the pandemic. They ultimately closed shop in April 2023 after pandemic-related effects start to wear off. However, I think Explore Now had the traffic of Prime Video to sustain itself even after the pandemic. I, therefore, sometimes wonder if Explore could have been saved by Explore Now by leveraging Amazon's own strengths. But, we would never know! Looking back, the diagnosis of Explore's growth problem felt like a case of "reality is in the eyes of the beholder". One could potentially frame it as a Product problem vs Marketing problem (or pre-PM fit problem vs Growth problem). It is important to note this framing since our choice then governs our prioritization. That, in turn, determines the kind of motions the team will go through.

Conclusion

If you’re building a zero-to-one product, it’s important to recognize that the unpredictable journey of the early stages is inevitable—but there is a way to navigate the chaos. I’ve outlined my thoughts on how early-stage teams can improve their odds of success here. While it’s helpful to understand strategies that worked for other startups in specific contexts, it’s just as valuable to learn from other people’s mistakes and know what pitfalls to avoid to boost your chances. As someone wisely said: “It’s not what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.”