How to measure a website change with Google Analytics when you can't A/B test

You find out by measuring the changed page's conversion rate before and after the change under conditions that are as close to identical as you can make them, and then checking whether anything other than your change could explain the difference. An A/B test does this for you by showing two versions to visitors at the same time, so the season, your campaigns, and your traffic mix all affect both versions equally. A before-and-after comparison can't do that. The visitors in the second period are different people arriving at a different time, so the job is to line the two periods up carefully and look for anything that broke the comparison.

Plenty of teams are in this position. Your site might not get enough traffic to split it, your website builder might not offer testing, or the change might already be live. Some changes can't be split at all, such as a full redesign or a new price. Google Optimize, which many sites used for free A/B tests, shut down in 2023, and Google Analytics 4 doesn't replace it. What Google Analytics 4 does offer is the data you need for a before-and-after comparison:

• Date comparisons. Most standard reports let you compare one date range with another, so you can see a page's sessions and key events (the actions you've marked as conversions) side by side for two periods.
• Landing page and page reports. The Landing page report shows each page people arrived on with its sessions, engagement rate, and key events, which gives you a per-page conversion rate.
• Breakdowns. You can split any of those numbers by session source and medium or by device category, which shows whether your traffic mix shifted between the periods.
• Annotations. You can add a note to your property on the date you shipped the change, so anyone reading a chart later knows where the line falls.

Google Analytics stops at showing you two numbers. It doesn't pick the right periods, notice that a paid campaign started the same week, or tell you whether a rise from 2.1 percent to 2.4 percent is a real gain or ordinary week-to-week variation. Those decisions are where most before-and-after readings go wrong.

A good result is a clear verdict you can act on. You know whether the change won, lost, or made no measurable difference, by how much, over which dates, and which caveats apply. When the honest answer is "too early to tell" or "something else changed too," a good result says so. A verdict invented from noise is worse than no verdict, because it teaches you the wrong lesson about your site.

Kite does this work for you. It sets the baseline, watches the comparison, checks for the things that break it, and reports the verdict with the evidence behind it.

Tell Kite which page you changed, or plan to change, and what you hope it does:

"We rewrote the headline and shortened the form on our pricing page last Tuesday. Can you tell me whether it's getting more demo requests?"

Kite is an AI marketer. It researches your business, builds a growth strategy, and ships the work that brings you new customers. From that research, it already knows what your site sells, which pages lead toward a signup or a purchase, and what counts as a conversion for your business. That lets it propose the conversion event to measure, the periods to compare, and how long to wait. Confirm the plan or adjust it, and Kite starts.

You can work with Kite in the Kite web app or in Slack. Connect your Google Analytics account through OAuth so Kite can read your reports. If your account can see more than one property, Kite lists them and confirms which one measures the site you changed.

It helps to ask before you ship, because Kite can record the baseline and the exact change while they're fresh. If the change is already live, Kite works out the before period from the date you give it, and it can check archived copies of the page to confirm what the page looked like before.

Kite starts by pinning down exactly what changed and the date it went live, because a comparison is only as precise as its dividing line. It records the page's address, the element that changed, what it said or looked like before and after, and the date and time of the change. When you haven't shipped yet, Kite captures the live page as a visitor sees it before anything changes. When the change is already live, it compares the current page with archived versions to confirm the old state.

Kite also writes down what the change is supposed to do, as one sentence that names the evidence, the change, and the result it should move. For example: "Because most visitors leave the pricing page without scrolling to the form, moving the demo form above the plan table should raise demo requests from pricing page visitors." That sentence keeps the measurement honest. If the change was meant to raise demo requests, Kite judges it on demo requests, not on whatever metric happened to go up.

Kite reads the annotations on your Google Analytics property to see whether anything else is marked near the change date, such as a site migration or a tracking update. It can't add annotations itself, so it suggests you add one for the change in Google Analytics, and it keeps the change date in your team's records either way.

Kite measures the one conversion the page exists for, and it checks that Google Analytics is recording it before it trusts any number. It lists the key events on your property, reads the dimensions and metrics available, and runs a report of every event your site sent over the period with how often each one fired. That tells Kite whether the event behind your conversion, such as a form submission, a signup, or a purchase, fires reliably on the page you changed.

This check matters most when the change touched a form or a button. A rebuilt form can stop sending its event without anyone noticing, and a conversion rate that falls to zero the day of a change is usually a tracking break, not a disaster. Kite also runs a realtime report to confirm events are still arriving. When the conversion you care about has no event at all, Kite tells you which event to add and stops there, because a change can't be measured against a number that was never recorded. When the event exists but isn't marked as a key event, Kite tells you which one to mark in your Google Analytics settings.

Kite measures the baseline as the changed page's conversion rate over the two to four full weeks before the change. It runs a report on your property filtered to that page, using sessions that landed on it or reached it, the key events those sessions completed, the session key event rate, and the engagement rate. It uses whole weeks that start and end on the same weekday, because most sites convert differently on weekends than on weekdays, and a period with an extra Saturday can move the rate on its own.

Kite reads the same numbers broken down by session source and medium and by device category, so the baseline carries a picture of who was arriving, not just how many converted. When the page gets very few conversions, Kite says so up front and recommends a longer window, because a comparison of six conversions against nine tells you almost nothing.

Kite waits until the after period matches the before period in length and weekdays and holds enough conversions to read, which in practice means at least two full weeks. It leaves out the day of the change and the first few days after it, when your own team, returning visitors, and anyone checking the new page can distort the numbers.

Kite doesn't need you to remember to come back. Once the change is live, it schedules the readout as a follow-up job for the date the window closes, and the result arrives in the conversation where you asked.

Kite checks the comparison for the outside changes that most often break a before-and-after reading. It compares the periods on these points:

• Traffic mix. Kite compares sessions by source and medium in both periods. A paid campaign that started the same week, a newsletter send, or a post that got shared can bring visitors who convert at a very different rate. When the mix shifted, Kite compares the conversion rate within each source, such as organic search before against organic search after, so the change is judged on comparable visitors.
• Device mix. Kite compares desktop and phone visitors separately, because a change can help one and hurt the other, and the overall rate can hide that.
• The rest of the site. Kite runs the same report for pages you didn't change over the same dates. If the changed page rose 12 percent and the rest of the site rose 10 percent, most of the gain came from something sitewide, such as the season or a campaign. The difference between the two is the part your change can claim.
• Seasonality. When your business has strong seasons, Kite compares the same weeks a year earlier where the data reaches back that far, to see whether the same rise happened last year without the change.
• Tracking. Kite compares the event counts across both periods and checks annotations again, so a tracking change that landed mid-window doesn't pass for a result.

Kite separates what the data shows from what it infers. A rise that holds within each traffic source and beats the rest of the site is strong evidence that the change worked. A rise that disappears once you account for a new campaign is not evidence, however good the overall number looks.

Kite reports one of three verdicts: won, lost, or inconclusive. A change wins when the changed page's conversion rate rose over the full window, held up within comparable traffic, and outpaced the pages you didn't change. It loses when the rate fell under the same conditions. It's inconclusive when the window was too short, the conversions were too few, or something else changed enough to explain the difference. An honest inconclusive result is a useful answer, because it tells you to keep measuring or to test the idea a different way.

Kite posts the verdict in the conversation with the baseline rate, the after rate, the dates of both periods, the number of conversions in each, and the caveats that apply. When the findings need more room than a message, it publishes a findings page with the charts and tables and links to it.

Kite records the result in your team's knowledge base: the change, the sentence that said what it should do, the baseline, the result, and the verdict. When the verdict is clean, it adds the lesson to what your team knows works, such as "Shorter forms raised demo requests on our pricing page," so later work builds on it and the same idea isn't tested twice.

From there, Kite can propose the next change. On a site it updates through a supported CMS, such as Webflow, it can build the next change and ask for your approval before publishing, then measure it the same way. On a site Kite hosts, it can run the next change as a true A/B test with a traffic split between versions, measured on Kite's built-in analytics.

When Kite finishes, you know whether your change worked, and your team has a record of it that later work can use.