How to Analyze Competitor App Reviews: Version by Version

An app store review is a short piece of text with a star rating, a date, an author handle and, crucially, the version of the app it was written against. One of the two major stores also records the device. Around the reviews sit a rating histogram, the developer’s own dated release notes for every published build, any replies the developer has written, and a separate listing for each country the app is sold in.

What the corpus does not contain is a buyer. The person writing is an individual user describing an experience they had this week, not somebody who ran a procurement process, and for most business software the mobile app is a companion rather than the product. That single distinction resolves most of the confusion about what this source is for. It is the fastest quality instrument available anywhere, and it is close to silent on why anybody bought.

1. 1Fix the storefront before you read a single review. Both major stores compute what you see per country, and one of them also varies it by device class. Decide which market you are analysing, open that country's listing deliberately, and use the same one every time. A rating collected from a different storefront than last quarter is not a comparison, it is two unrelated numbers.
2. 2Read the histogram before the average. A 3.8 built from a wall of fives and a wall of ones is a product that works brilliantly for one group and fails another, which is a segmentation finding. A 3.8 built mostly from threes is a product nobody minds and nobody loves. The average hides the difference and the distribution shows it immediately.
3. 3Sort by most recent, then read against the version. The default sort favours reviews other people found useful, which surfaces the memorable and the old. Switch to most recent and note the app version attached to each review. This is the field that makes app stores different from every other review corpus, and it is the one people skip.
4. 4Open the release notes beside the reviews. Line the version numbers up against the competitor's own dated changelog. You get the claim and the verdict side by side: what they said the build changed, and what users said it actually did. A release described as performance improvements followed by a wave of crash reports is a specific, dateable event.
5. 5Read every developer reply in one pass. Replies are written by the competitor, published, dated, and addressed to a named complaint. Read them consecutively rather than one at a time and you have their official position on each weakness, plus the occasional unannounced commitment when somebody promises a fix in the next release.
6. 6Check the other storefronts and the other apps. Compare two or three country listings for the same app: a complaint that appears in one market and not the others is usually a localisation, pricing or regulatory issue rather than a product one. Then look at the company's other listings, since an acquired app still carries the original user base and its unresolved grievances.
7. 7Record the storefront, the date, the version and the count. Four fields beside every figure, without exception. A figure carrying none of them cannot be set against a rival, against another market, or against your own reading three months ago, and most of the confident claims made from this source come from a decimal quoted with no idea which store, which country or which build produced it.

Almost every competitive analysis built on app ratings contains the same unexamined assumption: that the number on the page is a fact about the software. It has not been true for years, and both stores document why in their own developer material. A rating is the output of a computation that takes your location, your device and the developer’s choices as inputs, and it will differ for two colleagues opening the same listing on the same afternoon.

Google published this change on its Android developer blog on 23 August 2021 and committed to contacting any developer whose rating would move by more than 0.2 stars on a device type in a key market, at least ten weeks ahead. That detail is useful in itself: the operator considered a two-tenths shift material enough to warn about, which is a reasonable threshold to adopt for deciding whether a competitor’s movement means anything.

This is the wedge, and it is the reason a mobile competitor is easier to analyse than a web-only one. Every review carries a build number. The developer publishes dated release notes for every build. Put the two side by side and sentiment stops being a mood and becomes an engineering timeline you can date to the week.

The method is mechanical. Sort by most recent, walk backwards through a few hundred reviews, and record only the version field and the star rating. What you are looking for is a cluster: twenty one-star reviews all naming the same build, arriving inside a fortnight. That is a regression, and it has three dates attached to it. The release date of the offending build, the date complaints started, and the release date of whichever build stopped them.

Reading in the other direction works too. Release notes have their own vocabulary, and it shifts. A run of builds described as performance and stability improvements, with no named features, means the team is servicing debt rather than building, and it usually lasts a quarter. Cadence matters as much as content: a competitor shipping fortnightly and one shipping twice a year are running different engineering organisations, whatever their careers page claims. And when a feature appears in release notes before it appears on the marketing site, the launch was engineering-led and the marketing was assembled afterwards.

Nobody writes an app review unprompted unless something went badly wrong. Most reviews exist because the app asked, and where in the journey it asked decides which population answers. Two products of identical quality can sit half a star apart on that decision alone, which makes prompting the largest uncontrolled variable in any app rating comparison.

The constraint is the same for everybody, which is what makes it analysable. Apple caps rating prompts at three times in a 365-day period per user and requires the standard system prompt, so every developer has the same small budget of asks and has to decide where to spend it. That decision is observable: install the competitor’s app and use it normally for twenty minutes, noting where the prompt appears.

• Prompted after a success event. A completed order, a finished workout, a successful sync. The population answering is people who just had the product work, and the distribution skews high by construction.
• Prompted on launch or on a timer. Everyone gets asked regardless of how their session went, which produces a flatter and more honest distribution, and a lower average.
• Not prompted at all. Only people with a strong motive write, so the histogram polarises into fives and ones with very little between. Low review volume relative to obvious install scale is the tell.
• Prompted after a support interaction. A deliberate recovery play, and it shows up as unusually positive reviews that mention the support team by name rather than the product.

None of this means a rating is worthless. It means the comparison you should be making is within a competitor over time, where the prompting strategy is roughly constant, rather than across two competitors at one moment, where it is not.

1. The App Store product page

Title, star rating, author handle, date and app version on every review, with a summary rating computed for the territory you are in. Reviews here tend to be shorter than on the other store and skew toward reaction rather than description, which makes the version field carry more of the analytical load.

2. The Google Play store listing

The same fields plus the device the reviewer used and a helpfulness count, with reviews appearing publicly roughly a day after submission. The device field is genuinely useful: a complaint concentrated on one manufacturer or one form factor is a compatibility problem rather than a product decision.

3. The star-rating histogram

Five bars, and they answer a question the average cannot. A polarised distribution means the product works for one group and fails another, which is a segmentation finding you can sell against. A distribution bunched in the middle means a product nobody minds, which is a different competitive problem entirely.

4. Release notes and version history

A dated changelog the competitor wrote, kept per build, and not quietly editable after the fact. Read it for cadence, for vocabulary and for sequence. It is the closest thing to a published engineering log that any company outside open source produces.

5. Developer replies

Their answer to a named complaint, in public, with a date. Both stores permit a developer to edit a reply after posting and Apple shows only the latest version, so read a reply as a current position rather than as a historical record of what they said at the time.

6. The other country storefronts

A different corpus and a different summary rating for the same software. Open two or three markets deliberately. A complaint that appears in one country and nowhere else is usually about localisation, local pricing or a regulatory requirement, and it will never surface if you only ever read your own store.

7. The generated review summaries

Both stores now show a short machine-written precis of the reviews. Apple began doing so in iOS 18.4, initially for English-language reviews on a limited set of apps in the United States, refreshed at least weekly and produced after spam, profanity and fraudulent reviews are excluded. Read the summary as your competitor’s public reputation and the reviews as the evidence behind it, because a prospective buyer increasingly reads only the first.

8. The sort and filter controls

Most recent, most helpful and star-level filters. The default is not neutral: it surfaces reviews other people found useful, which favours the memorable and the old. Switching to most recent is the single change that turns a browse into an analysis.

9. The competitor’s other listed apps

Tap through to everything the same developer publishes. Companion apps, regional variants and acquired products all appear, and an acquired app is the interesting one: it still carries its original user base, and their reviews describe what changed after the acquisition in language no press release will use.

10. The app itself, and when it asks

Install it and use it as an ordinary member of the public. You learn where the rating prompt fires, which explains a chunk of the distribution, and you see the product your competitor actually ships rather than the one their site describes. The privacy and data-collection labels on the same listing are a separate signal, covered under competitor tech stack.

11. Third-party rating-history trackers

Paid services that stored the rating, the rank and the review volume on dates before you started looking. They are worth it for exactly one thing, which is reconstructing a history you failed to record. Once you are keeping your own log, the marginal value drops sharply.

• Everything public is free. Reviews, histogram, version history, release notes and developer replies are all readable in a browser with no account. This is one of the least gated sources in the whole cluster, and most paid tools in this space are reformatting material anybody can open.
• Reaching another country’s listing is a URL change. Both stores expose per-country web listings, so switching markets is a matter of opening the right address rather than changing device settings. Do this deliberately for every market you actually compete in.
• The web listing shows fewer reviews than the app. Browser views paginate shallowly. For a deep read on a large competitor, work in the store app on a device, where the review list runs much further back and the version filter is easier to use.
• Read at reading pace, one listing at a time. The findings here come from a few hundred reviews sorted by date, which is an hour of work. Treat the store as a publication you are reading rather than a dataset to be moved, which is both the practical approach and the one that stays inside the terms you accepted.
• Install counts are unavailable on one store and banded on the other. One publishes a range that moves in orders of magnitude; the other publishes nothing. Everything more precise is a third-party model. Use review volume as your relative indicator instead, comparing only within a category and a storefront.
• Small competitors run out of corpus fast. A B2B app with sixty ratings cannot support a trend, and reading twelve reviews as evidence is how a briefing acquires a claim nobody can defend. Check the count first and be willing to say the source does not reach.

The reviews are published by their authors to a public storefront for the express purpose of being read by prospective users, and you are a prospective user. Reading them raises nothing. The care sits in two places: the product you install to go with them, and what you repeat afterwards. None of this is legal advice.

• Read the terms of the app you install. Evaluating a publicly listed product as a member of the public is normal, and some products restrict use by competitors or restrict publishing benchmark results. Accepting terms you intend to breach is a different act from opening a public web page, and it is the part that creates exposure.
• Never misrepresent yourself to reach a gated tier. Signing up with a false identity or a fabricated company to obtain a trial converts an evaluation into a misrepresentation. Where a tier is genuinely closed to you, say so in the analysis rather than working around it.
• Do not post reviews of a competitor, and do not ask anybody to. It breaches both stores’ guidelines, and a review written by an interested party is a false statement about a business that your own company can be held to. This includes encouraging customers to go and complain about a rival, which happens more often than anybody admits.
• Quoting a review externally has a threshold. A pattern across a corpus is a defensible claim. One person’s one-star review repeated in a sales deck is not a description of a business, and presenting it as one is the kind of comparative claim a competitor can act on. Use counts and dates, not screenshots of individuals.
• Reviews are personal data attached to a handle. The author chose a display name rather than publishing their identity. Do not try to connect a reviewer to a real person, and do not approach one, however useful their description of the problem is.

• Why anybody bought. Reviewers describe using a product, never choosing it, and for business software they are rarely the person who chose. Proxy: your own win and loss interviews, plus the long-form reviews written by buyers on the software platforms.
• How many people actually use it. One store bands installs and the other publishes nothing, and installs are not usage in any case. Proxy: review volume compared within a category and storefront, and rank position over time rather than at a point.
• Anything about the desktop or web product. A weak companion app tells you about mobile investment and nothing about the software your deals are actually decided on. Proxy: the product documentation and the changelog for the main product, which are usually public and dated.
• What a fix cost them, or how long it took to decide. You see the release dates, never the deliberation. Proxy: the engineering roles they opened in the same window, which show whether the response was a patch or a team.
• A representative view of any B2B competitor. The sample is consumers of a companion product, and no method fixes an unrepresentative population. Proxy: the buyer-side review corpus, with a feature gap analysis to structure what the complaints actually add up to.

Tie the rhythm to their releases rather than to your calendar, because a release is the only thing that changes this corpus in a way worth reading. Check the version history first, and if a new build has shipped in the last fortnight, read the reviews naming it. If nothing has shipped, the reviews will say what they said last month and there is nothing to do.

Keep the log to seven fields and fill them the same way every time: storefront and country, read date, current version, summary rating, rating count, a one-line description of the histogram shape, and the newest version appearing in complaints. Everything else is prose you will write once and never reopen. A year of those rows supports a sentence like “their rating fell from 4.4 to 3.9 across three releases beginning in March and has not recovered”, which a product team can act on immediately.

One habit worth adding: when a competitor ships something significant, save the release notes text into your own record rather than relying on the listing. Version history stays available, but the notes are the part you will want to quote, and quoting them from your own dated file is faster than finding them again a year later.

The structure of this source guarantees you learn things after they have already cost somebody. A competitor ships a build on Tuesday; it reaches most devices over the following week; the complaints accumulate over the week after that; you read them at your next check and brief the team the week after. By then the regression is a month old, they may already have fixed it, and the window in which a sales team could have used it has closed. Nothing about reading more often shortens the chain, because the delay lives in how a release rolls out and how long people take to complain.

Watching what a rival ships, rather than waiting for users to report on it, is the problem competitive intelligence software was built around. Flares follows product and messaging changes across the surfaces a competitor controls and dates each one as it appears, so the release and the reaction land in the same place instead of a month apart. What a platform cannot do is judge which complaints matter. Deciding that nineteen days of a broken checkout flow is worth a battlecard line and that a wave of grumbling about an icon redesign is not remains a reading of your own market, and it is the part worth your hour.