Vanity metrics vs. the numbers that actually matter

From the SoleOS answers series — written about our own product space; grounded in published definitions and documented behavior, never invented numbers.

A vanity metric is any number that goes up and to the right regardless of whether your product is actually working — total pageviews, cumulative signups, follower counts, raw downloads with no retention attached. The tell isn't the metric itself, it's what happens after you look at it: if the number can't change what you do this week, it's vanity. Swap it for the metric one layer down — the one tied to a decision — and the same dashboard becomes useful instead of decorative.

SoleOS published this — we build the dashboard that pairs vanity numbers with the decision-grade ones underneath them, so read the pricing angle for what it is.

What makes a metric "vanity"

Vanity metrics share three traits: they only accumulate (never drop), they're easy to headline in a tweet, and they don't tell you what to build, fix, or stop doing next. Common offenders for solo founders:

• Total pageviews — counts bot traffic, repeat visits from the same person, and people who bounced in two seconds, all the same as a real prospect reading your pricing page.
• Cumulative signups — a running total that never goes down even if most of those accounts never opened the app again.
• Social followers — a number platforms encourage you to chase because it's free dopamine and costs them nothing to inflate with bot accounts and follow-backs.
• Raw downloads without retention — an app store can hand you thousands of installs from a featured spot or a burst of paid traffic, and none of it means anyone kept the app past day one.
• Registered-user counts — distinct from signups only in that it sounds more official; still says nothing about who's active now.

None of these are "bad" to look at. They're just not decisions. A vanity metric answers "is this thing growing," which feels reassuring, while dodging the harder question: "growing into what?"

The one test

Before you screenshot a number for your investor update, your co-founder, or your own morale, run it through one filter: would this number change what I do this week?

If pageviews doubled, would you change your roadmap, your pricing, or your outreach? Usually no — you'd just feel good and move on. If your week-2 retention noticeably dropped, would you look at onboarding? Almost certainly yes. That's the difference. A decision-grade metric has an implied next action attached to it; a vanity metric has an implied feeling attached to it.

This test also works in reverse, and it's worth being honest about the other direction: not every actionable metric needs a dashboard, an alert, or a subscription either. If a number wouldn't change your week even if it moved sharply, it doesn't need daily monitoring — check it monthly, or not at all, and spend the attention on the handful that do.

Vanity metric → actionable counterpart

Every vanity metric has a more useful sibling sitting one layer down. The pattern is almost always the same: replace a raw count with a rate, or replace "ever did X" with "did X recently."

The actionable versions aren't harder to compute — they usually reuse the same raw event data you already have. The difference is that they're expressed as a rate or a trend, not a running total, so they can go down and force a response.

Why vanity metrics are so tempting

Vanity metrics survive because they're built to be looked at, not acted on:

• They only go up. Signups, downloads, and followers accumulate. A number that can never disappoint you is a lot more pleasant to check than one that might tell you retention just cratered.
• They're legible to outsiders. A big download total means something to a stranger in five seconds; a retention curve requires context they don't have. That makes vanity metrics the default for anything public-facing — landing pages, tweets, investor updates — which trains founders to reach for them privately too.
• They require no definition work. A pageview is a pageview. "Active" requires deciding what active means for your product, and that's exactly the work solo founders skip when moving fast.
• Solo founders check metrics alone. In a bigger company, someone in the room asks what a number means for the roadmap. Solo, that question only gets asked if you ask it.

None of this makes the underlying data useless — a pageview count is a fine input to a conversion rate. The mistake is stopping at the raw count instead of pairing it with what it converts into.

The short list of metrics that actually matter early

For a pre-PMF or early-revenue solo project, most of what matters early fits in a handful of categories. What follows is what to measure, not numbers to hit — there's no universal "good" retention curve or conversion rate, and any guide that hands you one without knowing your product is guessing:

• Activation — the share of new signups who complete whatever step represents first real value (send the first message, import the first file, finish the first workout). Define this once, deliberately, before you start tracking it.
• Retention by cohort — of people who signed up in a given week or month, what share are still active at day 7, day 30, day 90. Track it as a curve, not a single snapshot, so you can see whether newer cohorts stick better or worse than older ones.
• Conversion rate — visitor to signup, signup to trial, trial to paid. Each step is a rate with a denominator, and each one implies an action if it moves.
• Revenue and its shape — MRR, plus the breakdown of new, expansion, and churned revenue. Total lifetime revenue collected is a vanity number in disguise; the trend and its components are the actionable version.
• Qualitative signal from a small number of real users — a handful of direct conversations about why someone churned, or what almost stopped them signing up, often tells you more than any dashboard.

If you're deciding which of these to start tracking first, a walkthrough of the first metrics worth setting up for a brand-new app goes deeper on sequencing than this piece does, and a practical guide to instrumenting analytics solo covers the "how" once you know the "what."

When you don't need a tool for this

If you're pre-launch, or you're the only user of your own product, a spreadsheet and a weekly gut-check will tell you everything a dashboard would — you don't need SoleOS or any other tool until you have enough real usage that eyeballing the raw numbers stops being reliable. The point of separating vanity from actionable metrics isn't to buy something; it's to change which numbers you let yourself feel good about.

Once you're juggling more than one project, or checking numbers across more than one source (analytics, revenue, app store), that's usually the point where a single view of the actionable metrics — not the vanity ones — starts saving real time. That's the gap SoleOS's list of the metrics worth tracking and its library of founder guides are built around: surfacing the number that implies a decision, not the one that just makes a nice screenshot.

Frequently asked questions

Are vanity metrics always worthless?

No. A vanity metric is often the raw input to an actionable one — pageviews feed a conversion rate, downloads feed a retention curve. The problem is treating the raw count as the finish line instead of dividing it into something that implies an action.

What if my actionable metrics all look bad — should I just report the vanity ones?

Worth naming honestly: if you're picking which metric to share based on which one looks better rather than which one is true, you're using vanity metrics as a hiding spot, not a mistake. Report the honest number alongside what you're doing about it.

How do I know what "active" means for my own product?

There's no universal definition, and any answer handed to you without knowing your product is a guess. Pick the behavior that most closely represents getting real value — opening the app isn't it, but completing the core action usually is — and define "active" as doing that within your chosen window.

Is follower count ever worth tracking?

It's fine to track for its own sake — audience building is a legitimate goal — but don't let it stand in for product traction. Track how many signups you can actually attribute to that audience instead of the size of the audience itself.

How often should I check my actionable metrics?

Often enough to catch a real change, not so often that noise looks like signal. Daily activity is fine to glance at; retention cohorts and MRR trend are usually weekly or monthly checks, since they need time to form a real pattern before a single day's wobble means anything.