The Cost of Doing Business: How Shopify Rigged Its Own App Store

I spent three weeks reading through Shopify’s developer forums, and somewhere around the four-hundredth post about fake reviews, I started to understand the shape of the problem. It isn’t that Shopify built a bad marketplace. It’s that Shopify built a marketplace, then quietly decided which merchants get to win in it, and dressed the whole thing up as a meritocracy.

The Shopify App Store¹ now lists more than 16,000 apps, built by a network of roughly 40,556 registered partners, competing for the attention of over a million merchants. According to a report by Meetanshi², of those partners, only about 7,874 actually have an app listed, which tells you something on its own. Most people who try to build a business inside this ecosystem never make it to the shelf, and the ones who do are then thrown into a search results page where visibility is the entire game. That is a lot of software chasing a limited number of eyeballs, and where there is scarcity, there is an incentive to cheat. But the more interesting story isn’t the cheating. It’s what Shopify does, and doesn’t do, about it, and who benefits when the rules bend.

I want to be careful here, because it’s easy to write this kind of piece as pure outrage, and outrage tends to flatten a genuinely complicated situation into a cartoon. Shopify did not set out to build a rigged casino. It built a real marketplace, with real quality standards, real engineering resources devoted to catching fraud, and a genuine incentive to keep the store useful, because a store full of garbage apps eventually drives merchants away and drags down Shopify’s own revenue. The company takes a cut of app subscriptions. A trustworthy app store is in its financial interest. That’s the frustrating part of this story. The incentives aren’t purely adversarial. They’re just misaligned in specific, well documented ways, and those ways happen to fall hardest on the smallest players.

A Marketplace Shaped Like a Funnel

Ask any Shopify app developer how the ranking algorithm works and you’ll get a shrug followed by a guess. Shopify has never published the full formula. What developers have pieced together, through years of trial, error, and the occasional leaked screenshot, is that ranking depends on a mix of install conversion rate, review volume, review recency, review quality, and now, increasingly, whether an app carries the “Built for Shopify” badge.

One agency that specializes in app store optimization put it plainly: average rating, download count, and review count are, by developer consensus, the three heaviest levers, and all three are largely locked behind the very visibility a developer is trying to earn. As the team at Embarque³ noted, ranking highly is a precursor to getting downloads, which is a precursor to getting reviews, which is the thing that determines ranking. It’s a closed loop. You need traction to get traction.

That loop would be merely frustrating if everyone were playing by the same rules. They aren’t.

The Fake Review Economy

In July 2020, a developer named Johnathan Winder published a piece on Medium⁴ that read less like an exposé and more like a confession. He described buying fake five star reviews on Fiverr for an app he built, and watching it climb from rank 231 to rank 62 in ten days after posting eighty reviews. He wrote that the Shopify app ranking algorithm weighs review volume and score so heavily, and polices them so loosely, that gaming the system barely counts as an exploit. It’s closer to a documented feature. Winder said he notified Shopify directly and got nowhere, so he published the method himself, hoping public embarrassment would do what private complaints could not.

That was five years ago. The complaints haven’t stopped. They’ve multiplied.

In October 2025, a developer posting under the handle Olllie opened a thread on the Shopify Community forum⁵ with a question that had clearly been building for a long time.

“I’ve been a Shopify app developer for quite a few years now,” they wrote, “and one thing that’s become increasingly frustrating is how easy it seems for some apps to rack up, or outright buy, obviously fake reviews while genuine developers are stuck playing fair.”

The thread, still active as of this year, reads like a support group. Developers describe reviews posted minutes after installation, in identical boilerplate language, from stores that turn out to be empty shells built solely to leave five stars and disappear.

One reply, from a developer using the handle CaptainProtection, stuck with me. They wrote that they regularly receive spam emails offering to sell fake reviews and installs, sometimes twenty a week, and that the volume alone proves there’s a functioning black market Shopify has chosen not to shut down. Another developer, jam_chan, made an observation that cuts closer to the platform’s design than any conspiracy theory could. Legitimate developers who spend hours on white glove customer support rarely get a review out of it, because the merchant assumes good service is simply what they paid for. Bad actors who buy a thousand fake reviews in a weekend face no such headwind. The incentive structure doesn’t reward quality. It rewards whoever is willing to cheat fastest.

Shopify’s⁶ own policy language acknowledges the problem exists. The company’s help documentation states that it will demote an app’s ranking, remove it from promotional surfaces, or unpublish it outright if reviews are found to violate its terms, and that it “continuously evaluates all reviews… for trust, quality, relevance, and timeliness.” In June 2023, Shopify⁷ rolled out a review ranking update that promised to surface high quality, recent reviews from long term merchants first, a tacit admission that the old sorting had let noise drown out signal, according to the company’s own developer changelog. But developers on the forums will tell you, almost to a person, that reporting a fake review does close to nothing. Shopify won’t disclose which reviews it removes or why, citing the integrity of its detection methods, which means developers report suspicious activity into a black box and get silence back.

Built for Shopify, Built for Whom

If fake reviews are the crude version of gaming the system, the “Built for Shopify” badge is the sanctioned one, and it might be the more consequential of the two.

Shopify introduced the badge as a quality signal, a way to tell merchants which apps meet a bar for speed, design, and reliability. On paper it sounds reasonable. In practice, the badge does more than reassure merchants. It changes the algorithm’s math. Shopify⁸ states outright that Built for Shopify apps “appear higher in Shopify App Store search rankings,” that badge holders get priority review queues for future app submissions, and that they receive additional promotion on category pages and the store’s homepage. In other words, the badge isn’t a sticker. It’s a ranking multiplier, dispensed by Shopify itself, based on criteria Shopify itself sets and grades.

That would still be defensible if the criteria were stable and transparent. Developers say they aren’t. A developer named Tobe Osakwe⁹ wrote a detailed account of what it’s like to chase the badge, and the picture he paints is less “quality bar” and more “moving target.” The badge hinges heavily on a single performance metric called Largest Contentful Paint, or LCP, a measure of how fast a page’s main content loads. Osakwe describes receiving a warning that his app’s LCP exceeded Shopify’s threshold, giving him thirty days to fix it or lose the badge, only to check later and find the score back under the limit without him having changed a single line of code.

“This inconsistency,” he wrote, “leaves developers bewildered and unsure about the effectiveness of their optimizations.”

He posted a public complaint on social media asking the obvious question: if the warning window is thirty days and the metric itself is measured over a rolling thirty day period, when exactly is a developer supposed to have “a genuine opportunity to fix the issue” before losing the badge? It’s a fair question, and one Shopify hasn’t answered publicly.

Other developers have described the badge application process as a gauntlet. One HackerNoon¹⁰ writeup detailed a team that got rejected on their first attempt over what amounted to layout formatting and container use, technicalities with no bearing on whether the app actually helps a merchant sell more shoes or manage more inventory. They rebuilt their homepage specifically to game the LCP score, front loading static text purely to trick the load timer, and only then did they earn the badge. Read that again. A developer optimized not for the merchant’s experience but for Shopify’s stopwatch, because Shopify’s stopwatch is what determines visibility. That’s not quality control. That’s teaching to the test.

And here’s the part that should give any small developer pause. Earlier this year, Shopify filed a trademark application for the phrase “Built for Shopify” itself. A badge that started as an internal quality signal is being converted into a protected brand asset, one Shopify now owns outright and can license, restrict, or redefine however it sees fit. It is Shopify’s court, Shopify’s ball, and increasingly, Shopify’s trademark on what “good” even means inside its own marketplace.

Pay to Play, With a Straight Face

Then there’s advertising, which Shopify insists is separate from organic ranking, and which functionally is not, at least not in the way that matters to a merchant scrolling through search results.

Shopify¹¹ launched paid search and homepage ads inside the App Store in 2020, marketed explicitly as a lifeline for developers who couldn’t crack the organic rankings. The company’s own promotional post admitted the quiet part out loud. It stated that for “whether you are a new app developer or a seasoned one launching a new app, it’s very difficult and time consuming to compete organically, especially in a crowded category,” and framed paid placement as the fix. Translation: if you can’t win the algorithm, you can buy your way past it, provided you have the marketing budget to do so. That’s a fine business model for Shopify. It is a rough deal for the solo developer or two person team who built something genuinely useful and can’t outbid a venture funded competitor for the same search keywords.

Shopify’s¹² documentation on how these auctions work is detailed almost to the point of self parody, explaining cost per click bidding, relevance scoring, and first price auctions with the confident tone of a company that has thought this through. And to be fair, ads are clearly labeled. A merchant browsing the store can, in theory, tell the difference between a sponsored listing and an organic one. But labels don’t change behavior much. Anyone who has used Google knows the sponsored slot at the top of the page gets clicked whether or not the user consciously registers the tiny “Ad” tag next to it. Shopify is applying the same well worn trick, and calling it a growth opportunity for developers rather than what it also is, which is a toll booth on visibility that only some developers can afford.

It’s worth remembering who actually loses when this system tilts. It isn’t just the developer with the good app and no marketing budget. It’s the merchant, someone running a small store, trying to pick a checkout upsell tool or a review widget from a list of a dozen near identical options, most of them padded with reviews of dubious origin.

A forum poster summed up the merchant side of this with a bluntness that stuck with me more than any of the developer complaints did. Describing a checkout app called Checkout Bear with four total reviews, one of them one star, they wrote that they skip apps like it on sight, and that Shopify’s failure to show something as basic as active install counts, rather than relying entirely on a reviews section anyone can pad, is “absolutely missing” from the store.

“That’s a Shopify issue,” they wrote. “They don’t care. No one’s home.”

It’s a harsh assessment, maybe an unfair one in places, but it captures something true about how opaque the whole system feels from the outside. Merchants are told to trust star ratings. Star ratings, developers will tell you at length, are exactly the thing most easily bought.

There’s a second layer to this that doesn’t get talked about enough, which is how little structured information merchants actually have to work with beyond stars and review counts. Another poster in that same thread listed off the basic technical questions a merchant might reasonably want answered before installing something into their live store. Does the app rely on script tags or app embeds, a distinction that affects site speed and theme compatibility? Are the docs actually available somewhere, or hidden behind a login only visible after install? Does the app use Shopify’s newer functions system, or an older, more fragile integration method? Is it restricted to Shopify Plus, meaning a smaller merchant could spend twenty minutes evaluating an app they can’t even install? None of that shows up on a listing page.

The comparison tool Shopify offers for weighing two apps side by side, PaulNewton argued, doesn’t surface any of it either, and the Built for Shopify filter, while useful as a rough cut, mostly measures page speed rather than whether the app actually solves the merchant’s problem. So the merchant falls back on the only signal the store foregrounds prominently, which is star rating and review count, which happen to be the two numbers most vulnerable to manipulation. It’s a strange design choice for a company that clearly has the engineering talent to do better, and it’s hard not to notice that the metrics merchants are steered toward happen to be the same ones that make the Built for Shopify badge, and by extension Shopify’s own curation layer, look more indispensable by comparison.

Shopify’s Defense, and Its Limits

To Shopify’s credit, the company has not been silent on any of this. It has published detailed review policies, it updated its ranking algorithm for review quality in 2023, and it maintains an entire quality assurance apparatus around the Built for Shopify program¹³, including a stated 100 checkpoint review process before any app hits the store. Shopify¹⁴ has also stated clearly that app recommendations shown to merchants in the admin dashboard are not for sale, that developers “can’t pay Shopify to have an app display in recommendations,” and that only clearly labeled sponsored placements involve money changing hands, according to the company’s own help documentation. Those are real guardrails, and they matter.

But guardrails only work if enforcement is visible, and Shopify’s enforcement is, by design, invisible. The company won’t say which reviews it removes, won’t explain why an LCP score fluctuates without code changes, and won’t publish the weighting of its ranking algorithm, not even in broad strokes. Every one of those choices might have a defensible internal logic, something about preventing gaming of the gaming detection itself. I understand the argument. I’m just not sure it holds up when the people most affected by the opacity are the small developers Shopify claims to be protecting.

What This Actually Is

Call it what it is. This isn’t a story about a few bad actors buying reviews on Fiverr, though that’s part of it. It’s a story about a marketplace where the platform owner sets the rules, grades the homework, sells the extra credit, and trademarks the honor roll, all while insisting the results are neutral. Every large platform faces some version of this tension. Apple’s App Store has weathered years of antitrust scrutiny over similar dynamics, its own mix of algorithmic opacity and pay to play visibility. What makes Shopify’s version notable is how openly the company narrates its own leverage, in its own promotional copy, without apparently registering the irony. It tells developers, in writing, that organic competition is “very difficult,” then sells them the workaround. It tells developers the Built for Shopify badge is the highest standard of quality, then trademarks the phrase for itself.

None of this means every top ranked Shopify app is fraudulent or every low ranked one is secretly brilliant. Plenty of genuinely excellent tools have earned their place through years of honest iteration, and Judge.me, Klaviyo, and a handful of others show up near the top of nearly every independent list precisely because merchants keep choosing them on their own merits. But a marketplace that can’t tell the difference between organic trust and manufactured trust isn’t really a marketplace. It’s a leaderboard with an asterisk, and Shopify is the only one who gets to decide what the asterisk means.

I keep coming back to something PaulNewton, a longtime poster on the Shopify forums¹⁵, wrote in that same thread about fake reviews, half in exasperation and half as a kind of eulogy for the whole conversation. He said the community forums don’t fix problems like this without an enormous groundswell of people, and that absent legal consequences or actual regulation, getting a publicly traded company to change its internal incentives is unlikely without something closer to a mass movement. He might be right. Public companies respond to pressure, not to forum threads, and a bought review costs less than a lawyer.

Until that pressure arrives, the honest developer keeps building, keeps waiting for a real review to land, and keeps watching some other app, freshly seeded with eighty five star reviews from stores that don’t exist, climb past them on the way to the top of the category page.

I don’t think there’s a clean villain in this story, and I’ve tried to resist writing one in. Shopify is a platform, and every platform eventually has to decide how much control to hand over to the people building on top of it. Apple made that trade one way. Amazon made it another. Shopify made its own version, and the version it chose happens to concentrate power in exactly the places you’d expect a public company to concentrate it, in metrics it controls, badges it owns, and ad auctions it profits from directly. None of that is illegal. Most of it isn’t even hidden. It’s printed right there in the developer docs, in language so matter of fact that it almost reads as an apology nobody asked for. What’s missing isn’t disclosure. It’s a mechanism for developers to contest the outcomes, a place to go beyond a forum thread that scrolls into the archive after a few dozen replies.

Until that exists, the fairest thing anyone can say about the Shopify App Store is the thing developers have been saying to each other for five years running. The system works exactly as designed. The question is who it was designed for.

Sources

1. “Shopify App Store” apps.shopify.com/. Accessed 1 July 2026. ↩︎
2. Meetanshi, meetanshi.com/blog/shopify-app-store-statistics/. Accessed 1 July 2026. ↩︎
3. By, Written. “How We’ve Optimized a Shopify App Store Listing to Boost Downloads” Embarque, 18 Oct. 2023, www.embarque.io/post/shopify-app-listing-optimization. Accessed 1 July 2026. ↩︎
4. Medium, medium.com/@horoscopesapi/shopify-app-store-ranking-algorithm-exploit-1d2e580919de. Accessed 1 July 2026. ↩︎
5. “Is Shopify ever going to take fake app store reviews seriously?” 6 Oct. 2025, community.shopify.com/t/is-shopify-ever-going-to-take-fake-app-store-reviews-seriously/569409. Accessed 1 July 2026. ↩︎
6. Shopify, help.shopify.com/en/partners/help-support/faq/reviews. Accessed 1 July 2026. ↩︎
7. “App review ranking changes” Shopify developer changelog, shopify.dev/changelog/app-review-ranking-changes. Accessed 1 July 2026. ↩︎
8. “About Built for Shopify” Shopify developer changelog, shopify.dev/docs/apps/launch/built-for-shopify. Accessed 1 July 2026. ↩︎
9. Osakwe, Tobe. “The Challenges of Maintaining the Built for Shopify Badge: A Developer’s Perspective – Tobe Osakwe” 13 July 2024, tobebuilds.com/2024/07/13/the-challenges-of-maintaining-the-built-for-shopify-badge-a-developers-perspective/. Accessed 1 July 2026. ↩︎
10. Kazemi, Max. “Overcoming Technical Hurdles for Shopify Excellence: The Story of Our “Built for Shopify” Badge” 12 Dec. 2023, hackernoon.com/overcoming-technical-hurdles-for-shopify-excellence-the-story-of-our-built-for-shopify-badge. Accessed 1 July 2026. ↩︎
11. Zhao, Jonathan. “Ads on the Shopify App Store: A New Channel for App Growth” Shopify, www.shopify.com/partners/blog/app-store-ads. Accessed 1 July 2026. ↩︎
12. “About Shopify App Store ads” shopify.dev/docs/apps/launch/marketing/advertising. Accessed 1 July 2026. ↩︎
13. “Shopify App Store” apps.shopify.com/. Accessed 1 July 2026. ↩︎
14. Shopify, help.shopify.com/en/manual/apps/finding-choosing-apps. Accessed 1 July 2026. ↩︎
15. “Is Shopify ever going to take fake app store reviews seriously?” 13 Oct. 2025, community.shopify.com/t/is-shopify-ever-going-to-take-fake-app-store-reviews-seriously/569409/14. Accessed 1 July 2026. ↩︎