Step trends
Weekly cohorts, complete and in-progress weeks, whether a step moved more than its margin of error, and whether it moved after a solution shipped.
The report compares two periods. Step trends shows the weeks in between, so you can see when something changed. Open a funnel and press Step trends.
Weekly cohorts
People are grouped by the week they reached the first step: a cohort. For each step, the line shows, week by week, the share of each cohort who went on from the step before to this one.
Because each week's people are followed from their own start, a busy week of sign-ups doesn't make a later step look better or worse. Each point compares like with like.
Complete and in-progress weeks
A cohort is complete once its conversion window has passed: everyone in it has had the full time to go through. The most recent weeks aren't complete yet, so they are drawn dashed, and they usually look lower than they will end up. Don't read a dip in the dashed part as a drop.
If the funnel is new, or its window is long, it says Not enough finished weeks to compare yet: a week counts once its window's days have passed.
Did it move?
Each step compares its recent weeks with its earlier ones. It is coloured only when the change is larger than its margin of error, and the step that changed most is marked Moved most. With too few people, it says Too few people to tell yet.
When a solution under a linked node shipped in these weeks, the step compares before and after the ship instead. See below.
What shipped
When a step is linked to a node on your tree, the solutions under that node (or the node itself, if it is a solution) are marked on the line where they were completed, such as Shipped 2 Oct: #48 Invite from the project page.
Before and after a ship
If one of those solutions shipped inside the weeks shown, the step's numbers compare the weeks before it with the weeks since, rather than earlier with recent:
Since #48 shipped 2 Oct: 27% → 38%, +11.0 pts ±6.0 pts
- Before is the finished weeks that ended before the ship; after is the finished weeks that started on or after it. The week it shipped in holds people from both sides, so it is left out.
- Each side pools its people, and the change gets the same margin of error as above. It is coloured, and can be Moved most, only when it is larger than that margin.
- If several solutions shipped, it splits at the most recent one that a week has finished since, and says so: the latest of 3 shipped in these weeks, or 2 more shipped since: too early to tell when newer ones are too recent to judge. The earlier ones fall on the "before" side.
- Right after a ship there are no finished weeks since, so it says Too early to tell: no week since it shipped has finished yet instead of a number. With a 30-day window that takes five to six weeks after the ship; a shorter window gives an answer sooner.
- If weeks have finished since the ship but nobody reached the step before in them, it says so (nobody since). That usually means the step before stopped being sent.
- If nothing shipped in these weeks, or the ship came in the first week shown, the step keeps comparing recent weeks with earlier ones.
- People who started shortly before the ship and reached the step after it count as before. If the ship helped them, the change shows a little smaller than it is, never larger.
A change after a ship, larger than its margin, is good evidence the release did it, but not proof: anything else that changed that week moves the line too. A change that had already started before the marker, or is within the margin, isn't evidence for the release.
Still stuck? Email info@outcomify.com and a person will answer.