Measuring adoption honestly

The situation

Quarterly review. Your slide says 500 licences assigned, 412 people signed in at least once, message volume up month over month. It is a good-looking slide and you built it honestly. Then the CFO asks the only question that matters: what is being done differently now that was not being done in March?

The room waits. You have three anecdotes and no number, because nothing you measured was about the work. Everything you measured was about the software. Six months of effort and the strongest defensible claim you can make is that people opened a tab.

The principle

Seat counts and login rates measure access, not adoption — the only honest adoption measure is a countable change in the work a team produces.

Activity numbers go up when you buy licences, when you run a session, when someone is confused and tries four times. They go up for good reasons and bad reasons identically, which is why they cannot be used to decide anything. A measure earns its place when it can go down.

The fix is not a better dashboard. It is naming, per team, the work product you expect to change, and the record that already exists where the change would show up — a ticket queue, a folder, a sent-items box, a calendar. If counting it requires asking people how they feel, it is not a measure.

Worked example

Shown here in Claude.

Six teams, six baselines from the start of the program, and a vague intention to "track adoption". Turn each one into a sentence somebody else could count.

Job: Turn each team's rollout goal into one adoption measure I can count
without sending a survey.

Inputs: Six rows — the team, the recurring work product named in their
baseline, who produces it today, and how long it takes today.

Output: One sentence per team, in exactly this form:
"By <date>, <role> will produce <named work product> in <time>,
counted from <the record the count comes from>."
Then one column headed "Who would dispute this", naming the role most
likely to argue the number is wrong.

Standard: The counting source must be a record that already exists
today. If the only way to count it is to ask people, reject the measure
and say instead what record would have to exist. Do not propose
satisfaction scores, self-reported time saved, or usage counts.

What comes back is unglamorous and checkable: "By 14 November, the two coordinators will produce the weekly site report in under 40 minutes, counted from the timestamp gap between the data export and the report being posted." Nobody needs to be asked anything. The dispute column tells you who to show it to before you publish it.

Then the ladder. Three checkpoints, each measuring a different failure:

2 weeks   Survival. Was the asset used a second time, by the person who
          built it, without being prompted?
          Count: assets reused / assets built.

30 days   Transfer. Is anyone other than the builder producing the work
          product with it?
          Count: named people outside the first cohort.

90 days   The work. Did the work product itself change?
          Count: ONE of cycle time, volume, or a named step that no
          longer happens — chosen before the 2-week check, not after.

Two weeks is the one people skip, and skipping it is expensive: if you only look at ninety days you learn in month three that nothing survived week one, having spent the quarter training the next cohort the same way.

Your rollout plan is one page and five blocks: the teams, the work product per team, the measure sentence, the three checkpoints with dates, and a named human who owns each number. If it does not fit on one page, you are running a program you cannot report on.

In your tool

Every admin console reports activity. None of them reports value, and it is worth saying so out loud to whoever asked you for a dashboard.

Product surfaces checked 2026-08-04.

The measure in this lesson exists in none of the four, and that is not a gap any of them will close. It lives in your ticket queue, your document folder, or your calendar — systems that were counting the work long before the AI arrived.

Try it

Pick one team. Write their measure in the exact sentence form above, then write beside it the name of the record you would open to get the number.

You are done when a colleague could produce that number by opening that record, without asking you and without asking the team a single question. If you cannot name the record, write the measure you would be able to count if that record existed — that is a finding, and it belongs on the plan too.

Common failures

• The vanity dashboard. Seats, sign-ins, messages sent. It rises when adoption is real and rises just as fast when people are lost and retrying, so no decision can be taken from it. It survives because it is the only thing the tools hand you for free.
• The measure that needs a survey. Self-reported hours saved, confidence ratings, a pulse check. The number is generated by people who know the program's budget depends on it, and it always comes back positive. You cannot audit a feeling.
• The 90-day measure with no 2-week checkpoint. The quarter looks fine until the end of it. By the time you learn the assets were abandoned in week one, you have run three more cohorts on the same design and the fix costs a quarter instead of an afternoon.