Define the behavior behind each label
Adoption might mean trying a feature, completing setup or reaching a first useful outcome. Retention might mean returning, repeating that outcome or continuing to pay. Write the exact behavior, eligible population and time window for each measure.
Do not treat an account creation as proof of value received. For a case comparison, identify what the reported metric actually counts. If the definition is unavailable, preserve the uncertainty rather than silently substituting your own definition.
Match the observation window to the task
A daily coordination tool and an annual planning product should not share an arbitrary return-frequency target. Start with how often the customer naturally faces the problem. Then choose a window that gives the user a reasonable opportunity to repeat the task.
Compare cohorts that have had equal time to mature. A newly acquired group has not yet had the same opportunity to return as a group observed for months. Separate changes in customer mix from changes in product behavior.
Read interventions as competing explanations
A case may describe onboarding, reminders, pricing and feature changes during the same period. List these concurrent changes. Without a suitable comparison, the account cannot isolate which intervention caused a retention change.
Look for evidence of repeated useful outcomes rather than repeated notifications or screen views. Also consider whether support, discounts or contractual commitments explain continued activity. These may be legitimate operating choices, but they imply different economics.
Test the stage that is actually weak
In an illustrative research app, users might finish a first search but fail to find a second relevant task. That calls for a different experiment from a product where users cannot complete their initial search. Diagnose the stage before changing the interface.
Record first-outcome completion, subsequent eligible opportunities and repeat outcomes separately. Set a guardrail for increased effort or unwanted messages. A successful experiment should improve the intended customer behavior without hiding a new cost.
Use this worksheet for your decision
Copy this into your notes. Leave unknowns visible instead of filling gaps with guesses.
Task and natural frequency: Adoption behavior: Retention behavior: Eligible cohort: Equal observation window: Concurrent changes: Evidence of repeated value: Weak stage: Experiment and guardrail:Download the text worksheet ↓
Find evidence in the archives
Choose a supporting case and a counterexample with a different outcome. Read their sources and check whether their conditions match yours.
Cases to practice with (read and check the underlying sources):
Bring your question to the cases
Search a company or topic in the app, read the cases and ask follow-up questions to compare approaches.