Does the onboarding checklist improve activation?
Requested by: Leo Brandt · Date: 2026-09-22 · Analyst: Sofia Reyes · Needed by: 2026-10-03
Decision this informs
- Decision: whether to make the onboarding checklist the default for all new Mossgate workspaces or roll it back
- If the answer is a lift of 3 points or more, we will: ship it to 100% and staff the next checklist iteration
- If the answer is flat or negative, we will: roll it back and move the two engineers to the import flow
Question
Did workspaces that saw the onboarding checklist activate at a higher rate within 14 days than those that did not, during the August test?
Ranked sub-questions: (1) does the effect hold for teams of 5 or more, (2) which checklist step correlates most with activation.
Metrics
| Metric | Definition | Numerator | Denominator | Existing dashboard |
|---|---|---|---|---|
| Activation rate | Workspace with 3 or more members who each edited a doc within 14 days of creation | Activated workspaces | Workspaces created in the period | Growth overview |
| Checklist completion | Workspace that finished all 5 checklist steps | Workspaces completing all steps | Workspaces in the checklist group | None |
Segments
- Test group (checklist) vs control
- Team size at day 7: 1, 2 to 4, 5 or more
- Signup source: organic, paid, invite
- Hide any segment with fewer than 50 workspaces
Timeframe
- Period: workspaces created Aug 1 to Aug 31, observed for 14 days each
- Compare to: control group in the same period
- Exclude: Aug 19, when signups were down for four hours
Data sources
events.workspace_created,events.doc_edited,experiments.assignments(experiment keyonboarding_checklist_v1)
Due date and priority
- Needed by: Oct 3, for the Oct 6 planning meeting
- Directional answer acceptable? Yes, if the team-size split is included
- Priority: high because two engineers are waiting on this decision
Output format
Answer
- Answer: activation 34.1% in the checklist group vs 30.2% in control (+3.9 points, 95% interval 1.6 to 6.2), 4,180 workspaces total. The lift is concentrated in teams of 5 or more (+6.8 points); solo workspaces show no change. Inviting a teammate is the step most associated with activation.
- Caveats: activation definition requires 3 members, so it under-counts solo users by design.