Most SEO strategies get judged in the same way: someone looks at the organic traffic line at the end of a quarter. If it went up, the strategy worked. If it went down, it failed. Both conclusions are usually wrong, because traffic is the result of many things, and only some of them are within the strategy’s control. Fair judgement needs more structure than a line on a chart, and the structure has to be agreed before the work begins.
Write down what you expect
A strategy that does not say what it expects cannot be judged. At the start, record for each major piece of work:
- The hypothesis: what you believe the change will do, and why.
- The measure: the figure that will show it, and where it comes from.
- The time frame: when you expect it to show, and when you will look.
- The leading indicators: earlier signs that it is working, such as the number of pages indexed, crawl errors falling, or impressions rising, well before clicks or conversions respond.
The leading indicators matter because search results lag. Judging a technical fix on conversions in the month after release asks the wrong question at the wrong time.
Measure against the version in force
Strategies change, and they should. What matters is that the results of a period are measured against the plan that applied during that period. Keep dated versions of the strategy, and when a new version is agreed, mark the date. Otherwise the plan quietly rewrites itself to match what happened, and nobody can tell what was predicted. A client can only hold you to a plan, and you can only learn from a plan, if there is a record of what it said.
Allow for lag
Search engines need time to notice changes. Google’s guidance on site moves says that if you change the URLs of existing pages you may experience ranking fluctuations while Google recrawls and reindexes
your site, and that a medium-sized website can take a few weeks for Google to notice the change, with larger sites taking longer. Site moves are an extreme case, but the principle applies to any significant change: do not read the result before the evidence has had time to arrive.
Rule out other causes
When traffic moves, the strategy is only one candidate. Google’s guide to debugging traffic drops lists the main causes to investigate: technical issues, site moves and migrations, algorithmic updates and spam or security problems, seasonality and changing interests, and reporting glitches. Run through those before crediting or blaming your work.
- A technical issue. Did something break, such as a misplaced noindex tag or a change in how pages are served?
- A migration or a URL change. Was there one, and is the site still within the period of fluctuation?
- An update or a security problem. Does the timing line up with a documented update, or with a security issue on the site?
- Seasonality. What happened at the same time last year?
- Measurement. Did tracking or reporting change in the period? A reporting glitch can look like a ranking change, and missing data should be shown as missing.
Compare like with like
For seasonal businesses, compare with the same period last year rather than the previous month. Separate branded searches from non-branded ones, because brand demand moves for reasons that have nothing to do with SEO. Look at the pages the work targeted, not only the whole site, because a sitewide figure can hide a real improvement in one section or a real decline in another.
Leading indicators in practice
Different kinds of work show up in different places and at different times. A few examples of what to watch, and roughly when:
- A fix to indexing or crawling: the number of indexed pages, crawl errors and the status of sample URLs in Search Console, usually before any change in clicks.
- New or improved content: impressions and the number of queries a page appears for, then clicks, then conversions.
- Internal linking and structure changes: crawling of the target pages and their positions for the queries they should rank for.
- A migration: redirects resolving correctly and the new URLs replacing the old in results, followed by a period of fluctuation as described above.
Choosing the indicator in advance means that when it moves, you can say so without having to find a figure that supports a conclusion you have already reached.
Agree the yardstick with the client
Many disagreements about whether an SEO strategy worked are really disagreements about the yardstick. One page covering the objectives, the main measures, the time frame and what would count as success or as a reason to change course is enough. It takes an hour to agree at the start, and saves a much longer conversation later.
When the results disappoint
Some strategies do not deliver what they expected, and the honest response is to work out why. The structure above makes that possible. If a leading indicator moved and the outcome did not, the hypothesis about what would follow may be wrong. If neither moved, the work may not have had the effect intended, or something else interfered. If everything moved but traffic did not, look at demand, seasonality and measurement. In each case you learn something specific, and the next version of the strategy is better for it.
Keep a change log
A dated log of changes on the site, both yours and the client’s, is one of the most valuable things an SEO can maintain. Releases, content launches, redesigns, tracking changes, migrations and campaigns all belong in it. When a number moves, the log is where the investigation starts.
What a fair review looks like
- What we expected, from the version of the strategy in force.
- What happened, with the period and the source of each figure.
- What else changed, from the change log and the checks above.
- What we conclude, and how confident we are.
- What we do next, including what we will keep, change or stop.
A review like this is harder to write than a screenshot of a traffic chart, and it is the only kind that lets a strategy improve. It is also what makes the next conversation with the client easier: even a disappointing period can be explained and acted on. Keeping the version of the strategy the client saw alongside the evidence is what makes it possible.