The first time I heard about testing through forecasting was at WeLoveSEO in 2021, during a talk led by Rebecca Berbel. And she blew my mind.
My statistics training had convinced me that there was no point in testing stuff out if there wasn’t going to be statistical significance. I was working on different sites, but they did not have the correct structure or sufficient traffic for me to perform traditional SEO tests. So I had pretty much given up.
We think about SEO testing as split tests or multivariate tests that happen all at the same time. I was inspired when Rebecca started talking about the possibility of testing actual traffic against forecasted traffic instead of testing one variation of a test against another.
Traditional SEO testing is not for everyone
Traditionally, performing SEO tests has been a complicated task. It’s taken a large amount of traffic and data, and for many businesses, that simply isn’t possible.
Why?
Because they don’t have the amount of traffic to notice a significant statistical difference, and they can’t make templated A/B tests to validate their theory scientifically. This can leave smaller companies reluctant to do any SEO testing at all, potentially causing them to miss out on ranking positions and, ultimately, customers.
The truth is for most businesses, that type of statistical significance testing is overkill.
I believe there is a more accessible way of performing SEO tests, and it uses SEO forecasting.
An accessible alternative to statistical SEO testing
The idea behind this methodology is that you forecast what your traffic would look like if you made no changes at all, apply some changes to your website and measure the difference between the actual traffic and the forecast.
You can attribute whether the traffic grows or decreases compared to your forecast to understand if your experiment has been successful.
This approach makes SEO testing accessible to all businesses, especially those with smaller traffic volumes or who cannot apply changes from the edge.
There are two different schools of thought regarding forecasting traffic:
Statistical forecasting uses historical traffic data to predict your future performance. This type of test requires at least two years of data to be most effective.
Keyword forecasting uses your current ranks, targeted ranks, keyword selection, CTR curves and seasonality to predict your future performance. This type of forecasting requires you to map your keywords to your pages beforehand.
Both models can give you enough facts and figures to work with, allowing you to forecast what’s to come accurately. You can forecast the SEO impact on your whole site or drill down into individual folders or pages.
Testing through forecasting doesn’t require providing piles of data or tons of data analysis skills. You can rely on a tool to provide the forecast and track your performance against it. For keyword forecasting, I like using SEOmonitor. I recommend using R or a spreadsheet template if you'd rather do statistical forecasting.
In this article, I will discuss how I approach testing with a non-traditional method. Still, there is something I want to clarify first:
We cannot discount the role that large case studies of statistically significant SEO tests have in moving the industry forward by generating knowledge that we all can use.
Finding causation without statistical significance
This methodology could raise doubts about whether these numbers imply correlation or causation. You must bring up a hypothesis of how the changes implemented will impact your traffic.
Your hypothesis should tie the changes you’re making to what is known as a causal mechanism. In SEO tests, traffic will change through an increase in rankings or click-through rate. Changing the titles and meta descriptions on a set of pages might improve your CTR without affecting your rankings, or it might be both.
If ensuring causation is essential, you should explain how your changes will impact CTR or rankings. Are you making the meta title stand out more? Have you improved the quality of the content? Make sure you write it down before the experiment starts.
Tools like SEOmonitor can help you reduce that uncertainty by letting you model your forecast to fit your hypothesis.
How to do your own SEO forecasting step-by-step
Unlike some traditional forms of SEO testing, forecasting is pretty straightforward for anyone with a solid SEO understanding. Follow these step-by-step tips to get your own SEO forecast up and running.
1. Come up with an SEO experiment
Use data to identify an SEO opportunity that you want to explore. This could be a product page you want to push, a service you think is worth knowing about, or just a blog to test your theory. It can be for a specific set of keywords or for a section of your website.
For this example, I’ll guess that I can make the most business impact by improving landing pages that talk about CMS integrations with different front-end frameworks. I’ll be improving my CMS pages about React, Next.js, and Angular. Our Node.js page is very similar, so it will serve as a control group.
2. Plan your experiment
What do you want to test? What results would you expect to see? You might try adding new images to all your CMS integration landing pages or updating some content to include new keywords.
Looking at my current content and what my competitors have put out there, I think I can improve my rankings by adding the logo of each react framework to the hero banner. I believe this can help me improve the relevance of my page for the keyword using media content as an alternative signal. I think this can be effective because it’s more of an untapped strategy in my market.
Here I have come up with an experiment, a causal mechanism and a way to measure if my experiment has been successful.
Pro tip: you can experiment on specific paths within your site by creating a SEOmonitor campaign for that specific subdirectory or subdomain.
3. Prepare the forecast
I like doing this on SEOmonitor, because of how easily I can adjust the forecasting algorithm, and they essentially keep track of how your forecast performs against your real traffic. And they offer both an inertial forecast and an improved forecast.
To continue with the example above, I set up a forecast based on the keyword groups that I think will see an improvement in rankings.
In the forecast, I can see what my non-branded organic traffic will look like if I make no changes (the inertial forecast). And my improved forecast if all the keywords in my group get in the top 5 positions.
Something very clever that the tool shows me is how the search volume of the keywords my site ranks for will fluctuate so that I don’t misattribute any traffic downturn to the experiment.
Because I’m forecasting with SEOmonitor, I can include conversions. I can use the Google Analytics integration to calculate an estimated conversion based on my previous performance or give it my own conversion rate estimates. I can add in or remove the types of conversions I want to forecast straight from Analytics.
Because the pages I’m targeting are not in a specific folder, I am forecasting on our root domain. This initially made me think that the results would be less reliable, but I can measure the changes in traffic coming from my target keywords within the tool. I can also export this forecast to Google Looker Studio.
4. Execute the experiment
Execute the changes outlined in your experiment and note down the date wherever you keep track of your traffic changes. For most people, this would be Google Analytics.
To let Google pick up the changes you’ve made and rearrange the SERPs, I recommend waiting 6-8 weeks. Some sites might need even longer.
5. Check the results and evaluate
You’re likely monitoring your rankings and traffic daily, so you probably already have a pretty good idea of whether or not your experiment was a winner.
You will have to compare your traffic during the test period to your inertial and improved forecasts. Check the data to verify whether your causal mechanism and hypothesis have come through.
Here are some helpful questions to ask yourself as you evaluate the experiment:
How does my traffic compare to the forecast?
Have I seen the expected movement in rankings or CTR?
Have there been any unexpected fluctuations in traffic?
Once you know if your experiment has succeeded, you can apply your hypothesis to other areas of the site and continue to build on your experimentation program.
A cheeky career development tip
It’s one thing to say, “I think this might improve our SEO,” but it’s a much stronger case if you can actually show you’re right. If you’re SEO testing as part of your job or for a client, you can use these results to highlight the value of your work.
You can use SEO forecasting to measure the tangible impact of your work on traffic and revenue. Bring this data to your performance conversations and use the proven value of your knowledge and experience to ask for a pay raise or a promotion.
Start experimenting
You’ll never know if something works until you try it. SEO testing through forecasting is a reasonably simple, risk-free way to see if your ideas work, and if they do, the rewards could be more than worth the effort.
SEOmonitor now offers their SEO forecasting tool as a standalone product, so you can use it without their other tools, which can help you save some money and reduce the use of overlapping tools.