‘They don’t have as much value’: How Amazon’s Choice lost some of its luster

The new feature worked. Customers began to trust the badge, and sellers who won the badge experienced a sales bonanza. In 2018, Profitero found that Amazon’s Choice badges increased the sales conversion rate of a product by 25%, while the Best Seller badge boosted page views by 45%.

The problem was, some sellers figured out how to game the badge — and over the years, a number of low-quality and even misleading products won the seal. “It was maybe one of the only easy ways to get your products jumped ahead of everyone else,” said Robb Powell, who heads e-commerce at a large Amazon third-party vendor, Advantage Solutions.

By the end of 2019, Amazon’s Choice had become a magnet for bad publicity. Media outlets like The Verge declared, “There’s no reason to trust Amazon’s Choice.” That August, two senators — Richard Blumenthal and Bob Menendez — sent a letter to Amazon CEO Jeff Bezos, calling the badge “misleading” and signaling that it could form part of an antitrust case.

Today, Amazon’s Choice appears to have lost some of its initial luster. Sellers who spoke to Modern Retail noted that the badge appeared to be harder to game than in recent years, and that the number of sales it drives — while still very high — has shrunk slightly. And as Amazon takes stronger steps to moderate content on its platform, the company might be partly reckoning with a recurring issue: at times, dangerous products accidentally receive a badge. In fact, while badges still remain important to the Amazon ecosystem, the company is testing more human-centric, editorial recommendations in search results — suggesting that it may be moving away from an algorithmic sorting system to a more human-driven one.

For sellers, “Amazon’s Choice was easier to game two years ago,” said Jeff Cohen, the vice president of marketing at Seller Labs. “It was the hot, trendy thing for brands to do.” But now, he said, “I see after a year or two of everybody chasing badges, they don’t have as much value to the brand.”

How Amazon’s Choice works
Amazon’s Choice is largely decided by an algorithm. Although Amazon has never disclosed all of the factors that go into picking an Amazon’s Choice badge, analysts have found a few patterns — for instance, Amazon’s Choice factors in search results positioning, star rating (most are at least above a 4.3), number of reviews, return rate, pricing and so on. Amazon itself, in a reply to that letter from the two senators, noted that Amazon’s Choice also weighs “popularity, available inventory, customer reviews and ratings, pricing, customer service contacts, return rates, and product quality.”

Soon after the badge first appeared in 2015, some sellers took notice. Blog posts popped up outlining what goes into a badge, and Facebook groups teemed with sellers swapping tips on how to best win it. One common tactic, which the Wall Street Journal reported in 2019, involved sellers encouraging people on social media to click links to their products that included the search phrase — say, “puzzle sets” — on which the seller wanted to win the Amazon’s Choice badge. Then they reimbursed those people for buying the product. That way, it looked like customers were organically buying the seller’s puzzle after searching “puzzle set,” thus boosting the likelihood that the seller’s product would win the badge for that term.

The badge was highly desirable for a few reasons. For one thing, it was a way to stand out in search results. Shoppers’ eyes naturally gravitated toward products with any sort of differentiation, like a black-and-orange badge. Amazon also introduced Choice to make voice shopping easier. On Alexa, products designated as Amazon’s Choice are the default. If a customer tells their Alexa to order a puzzle set, Alexa will automatically choose the puzzle labeled Amazon’s Choice — in theory, ensuring a quality puzzle set without needing to make customers browse all of the listings.

But as Amazon’s Choice exploded in prominence, news outlets began uncovering instances of Amazon’s Choice labels being applied to anti-vaccine media, children’s products that fail safety standards, products offering potentially dangerous false medical “cures” and more. Modern Retail previously found a QAnon hat listed as Amazon’s Choice.

Giving those products an Amazon’s Choice seal almost certainly was not Amazon’s intention, and the company has tended to remove those badges when reporters point them out, as it did when The Daily Beast covered a false medical cure containing industrial bleach that was listed as Amazon’s Choice.

The sticking point for Amazon’s Choice, though, is that few customers seem to realize that the label is largely algorithmically decided — meaning they might interpret Amazon as endorsing these products. “They don’t understand all of the underlying things that are occurring and they have a lot of trust in the system,” said Cohen.

The phrase “Amazon’s choice” sounds like an Amazon seal of approval, so buyers tend to treat it that way. As Carlee Peszko, an Amazon seller who works on the customer success team at Jungle Scout, put it an email, “even knowing that the badge is given to products based on an algorithm (and not because Amazon is necessarily backing that product), I still look for it as a consumer. It makes me think ‘it must be good, it’s Amazon’s choice,’” she said.

Changing up the algorithm 
In recent years, Amazon appears to have caught on to some of these difficulties — and it has quietly made it harder for sellers to game the badge. “It’s really clear that the algorithm is continuing to change,” said Powell. He noted, for instance, that while his products used to be able to hold the badge for “months at a time” on a given key phrase, now Amazon’s Choice badges come and go extremely quickly. Some of those keyword tricks, too, are more difficult to pull off. “Amazon is continuously innovating, and they’re getting away from the gaming,” he said.

Cohen expected that “as Amazon learns over time, it can start to dial in that algorithm,” he said. He added, “my guess is there are still plenty of products that still get the Amazon’s Choice badge that shouldn’t, but it’s not as bad today as it was two years ago.”

It might be near to impossible for Amazon to ensure that its badges don’t show up on any dangerous products ever. Inevitably, some will slip through the cracks. “I think the scale of their marketplace is to the level that they would have a hard time doing that,” said Chelsea Gross, an analyst at Gartner who studies retail. She added, “they might start to look at some of the bestselling products or bestselling categories with a bit more scrutiny.”

Instead, the company is, little by little, emphasizing products that are curated and recommended by humans — a way to cut down on potential abuses. One example is Amazon’s editorial recommendations system, which features reviews written by content creators and influencers enrolled in an affiliate program, and which are increasingly showing toward the upper half of the first page of search results.  

(an example of an editorial recommendation, which is increasingly showing up in search results)

The new editorial approach, paired with Amazon’s algorithm tweaks, mean that Amazon’s Choice has slightly less value to sellers today. The company has also been indexing ads and sponsored products highly, according to Marketplace Pulse. But that isn’t to say Amazon’s Choice has lost all or most of its sales value. Powell said that while before he might have seen a 3x sales lift on a product that got the Choice badge, now it’s down to 1.5x to 2x — still making the badge highly coveted. And Brandon Young, another seller, noted that although “conversion may be down slightly,” the badges “definitely still increase conversion” — and sellers still want it.

That might start to change over time, if Amazon continues to push more editorial-based recommendations in search results. And in a potential sign of what’s to come, a lot of sellers have moved toward looking at how to get those editorial recommendations — much like they did with Amazon’s Choice a few years ago. “The editorial review is becoming much more competitive and common in the search results,” said Powell.