How to make a Storyblok site easier for AI search engines to read and cite

To make a Storyblok site easier for AI search engines to read and cite, make sure your website sends every important page to crawlers as finished HTML built from the published version of its story, with complete metadata and structured data. Then make sure each of those stories states its answer plainly near the top. AI search tools such as ChatGPT search, Perplexity, and Google's AI features find pages through crawlers and search indexes, read the text, and quote the passage that answers the question best. A page they can't read never gets quoted, and a page that buries its answer rarely does.

Storyblok is a headless CMS, which means it stores your content and hands it to your website's code through an API instead of building the pages itself. Each page is a story, and each story is a tree of components that Storyblok calls bloks, such as a hero, a feature grid, or a block of rich text. Your code, usually a Next.js, Nuxt, or Astro site, fetches each story and decides what HTML it becomes. That's why Storyblok has no setting for robots.txt or for the markup on your pages. Those come from your code, and the code can only work with the fields your components define.

The technical part of the work mostly happens once, and it lives in your site's code:

• Server-rendered text. Most AI crawlers read the HTML your server sends and don't run JavaScript. A site that renders stories on the server, or builds them ahead of time as static pages, is fine. A site that fetches its stories in the visitor's browser shows those crawlers an empty page.
• The published version. Storyblok's API can return either the draft or the published version of a story, depending on the version your code asks for and the access token it uses. Production pages should request the published version, so crawlers read the copy you approved instead of whatever an editor is halfway through.
• Crawler access. Your robots.txt file tells each crawler what it may read. OpenAI and Anthropic run separate crawlers for search and for model training, such as OAI-SearchBot and GPTBot from OpenAI, so you can allow the search crawlers even if you block training. Your host's bot protection can also turn AI crawlers away even when robots.txt invites them in.
• Sitemap. Your code usually builds the sitemap from Storyblok's list of links or from a query for every story. It should include every published story that has a public page, and each page's last-modified date should come from when the story was last published.
• Metadata and structured data. Your code writes each page's title, meta description, canonical URL, and JSON-LD markup from the story's fields. Many Storyblok sites keep these values in an SEO field added from Storyblok's app directory. On a multilingual site, the code can also use each story's translated versions to tell search engines which page serves which language.
• An llms.txt file. This is a short Markdown file at the root of your site that lists your most important pages and what each one covers. It's a proposed standard, and support among answer engines varies, but it's cheap to add.

The content part never really finishes, and it lives in your stories. Answer engines quote passages that answer one question completely in a few sentences. A story earns citations when its opening answers the question the page is named for, its headings match the questions people ask, its facts and prices are current, and its comparisons are laid out in tables.

A good result is a site whose pages pass the crawl, content, and structured-data checks, whose most valuable stories answer their questions first, and whose citations you can measure before and after. Kite does that work in both places: your site's code and your Storyblok stories.

Start by asking Kite, in the Kite web app or in Slack, to look at your site. You might say:

"Our marketing site runs on Storyblok. Check how readable it is for AI search engines and fix what you can."

Or you can start from the questions you want to be cited for:

"ChatGPT recommends two competitors when people ask about scheduling software for clinics, and never us. Find out why and fix our pages."

Kite is an AI marketer. It researches your business, builds a growth strategy, and ships the work that brings you new customers. By the time you ask, it already knows your products, customers, competitors, and the questions your buyers ask from that research, so it can decide which pages matter most without a long brief. You can work with Kite in your browser or in Slack.

The first scan needs no connection, because Kite can scan any public website. To let Kite make the changes, import your website into Kite from its GitHub repository and connect your Storyblok space. When Kite sees that the imported site reads its content from Storyblok, it asks you for four things:

• Your space ID, from your Storyblok space settings.
• A personal access token from your Storyblok account settings. Kite edits stories with this token, so it has to come from an account that can manage the space. The tokens on a space's Access Tokens screen only read content, so they won't work here.
• The preview or public access token your site already reads with, so Kite's copy of the site can render your pages.
• The region, if your space is hosted in the United States.

Kite checks that the token can manage the space as soon as you save it, so a wrong token or space ID shows up right away instead of during the first edit.

Kite replies with the scan results and its plan: the code changes it will make, the stories it wants to rewrite, and the questions it will measure.

Kite starts by scanning your site's public domain with the 24 checks a marketing website controls, across five areas, and scores the result out of 100. Each check reports whether it passed, the points it earned, and what the scanner found:

• Crawl access. Kite checks your sitemap and its last-modified dates, whether your robots.txt rules for AI crawlers are clear and deliberate, and whether bot protection turns crawlers away.
• Content. Kite checks whether your pages show their text without JavaScript, whether they're light enough for an agent to read in full, whether redirects resolve cleanly, whether a missing page returns a helpful 404, and whether the site shows the trust signals and pricing a buyer looks for.
• Structured data. Kite checks whether your pages carry JSON-LD, whether your organization's markup is complete and linked to your other profiles, whether you use a healthy range of markup types, and whether each page has complete metadata.
• AI guidance. Kite checks whether you have an llms.txt file, whether it's formatted correctly, whether its links work, and whether it gives agents clear instructions.
• Brand presence. Kite reports whether searches for your brand name return accurate results and whether you have a Wikipedia page. These are reported but not scored, because no change to your site fixes them.

Every check that doesn't fully pass comes with a recommended fix, and Kite ranks the fixes by how many points each would recover. That ranking sets the order of the work, so a page that renders blank for crawlers gets fixed before a missing logo in your organization markup.

Kite traces each failed check to the code or the story that causes it, because on a Storyblok site the same symptom can start in either place. It reads your site's code to see how each route fetches its story, which version it requests, and which fields each component turns into the title, description, headings, and markup.

It then reads your Storyblok space. Kite reads your stories by content type and by folder, along with each story's full path, its publish dates, its tags, and its translated versions. It reads Storyblok's list of links to see every story and whether it's published, and it reads your datasources, which some sites use for shared values such as plan names or prices.

That tracing decides the fix. Say your blog posts show no author in their structured data. If the post component has an author field that the code never outputs, the fix is in the code. If the field is there but most posts leave it empty, your editors fill it in, or Kite changes the code to fall back to a sensible default. Kite also notes which shared bloks, such as a pricing table or a footer, appear on many pages, so it knows everywhere a change will show up.

Kite measures where you stand in AI answers before it rewrites anything, so it knows which stories to work on first. It picks the questions your buyers ask from its research on your business, runs them through ChatGPT, Gemini, Perplexity, and Claude, and records whether your brand appears, which competitors appear, and which pages each answer cites.

Kite then reads the cited pages. When a competitor's page keeps getting quoted for a question you should own, Kite notes what that page does differently, such as answering in its first sentence, setting out prices in a table, or covering a follow-up question your page skips. It also notes when answers lean on roundups, review sites, or forums instead of any company's own pages, because that calls for outreach rather than rewriting your stories.

Kite makes the site-wide fixes in your imported site's code, because that's where robots.txt, the sitemap, and the markup on every page are built. The changes depend on what the scan found:

• Rendering. When pages show no text without JavaScript, Kite changes how those routes load their stories so the server sends finished HTML. When production pages request draft content, Kite switches them to the published version.
• Headings. When a component renders its section titles as styled text instead of real headings, Kite changes the component so every page that uses it gets proper heading structure.
• Robots.txt rules. Kite writes rules for the AI crawlers you want reading your site and keeps the rules your site already relies on.
• Sitemap. Kite makes sure every published story with a public page is listed, that unpublished stories are left out, and that each last-modified date comes from the story's latest publish.
• Metadata and structured data. Kite makes each page output its title, meta description, canonical URL, and JSON-LD from the story's fields. Blog posts get article markup with the author and the published and updated dates, your home page gets organization markup with your logo and social profiles, and a component that holds questions and answers gets FAQ markup built from the same text visitors read.
• An llms.txt file. Kite writes the file from your actual site, with a one-line summary of your company and a short description of each page that matters most, and checks every link.

Your repository and your deployment stay in charge. Kite merges its changes into your default branch, and your own deployment ships them. If you protect that branch, Kite opens a pull request instead, and the change waits until your team merges it.

Kite rewrites the stories behind the pages that should be answering your buyers' questions, starting with the ones closest to earning a citation. For each story, it changes the wording that keeps the page from being quoted:

• The opening. The first paragraph or the hero's intro text answers the question in two or three sentences, so an answer engine can lift it as a complete answer.
• The headings. Section headings become the follow-up questions people actually ask, and each section answers its question in its first sentence.
• The facts. Outdated prices, product names, and figures are corrected, with a source for each figure that came from somewhere else.
• The SEO text. Kite rewrites the search title and meta description in the fields your site already uses for them.

Kite works on the wording that's already in your stories. Storyblok stores each story as a tree of bloks, so Kite finds the exact field that holds a passage, such as the headline inside the second section of the page, and changes that field alone. When a page needs something your story doesn't have yet, such as a new FAQ section, a comparison table, or a page for a question nobody has written about, Kite writes the complete copy and tells you which component and which story it belongs in, so an editor can add it in Storyblok's Visual Editor in a few minutes.

Kite shows you each rewrite before it goes live: the story, the field, the current wording, and the new wording. Approve it in the Kite web app or in the Slack thread, and Kite makes the change. In Storyblok, saving and publishing happen in the same step, so the edit is live in your space the moment Kite writes it. Storyblok publishes a story as a whole, which means any unpublished edits an editor has left in that story go live too. If your editors have work in progress on a page, have them publish or finish it before Kite rewrites that story.

After each edit, Kite reads the story back from Storyblok to confirm it's published, has no unpublished changes left, and holds the new text. If it can't confirm that, it tells you which story to check instead of reporting the edit as done. Your live site then shows the change on its own schedule. A server-rendered site picks it up on the next request or when its cache expires, and a site built ahead of time shows it after your next rebuild, which many Storyblok sites start automatically from a webhook when a story is published.

Kite scans your site again after the changes ship and compares the new score with the first one, check by check. If a fix didn't take, such as a robots.txt rule your host's cache is still serving in its old form, Kite tells you which check still fails and what it found.

After answer engines have had time to read the updated pages, Kite runs the same questions through ChatGPT, Gemini, Perplexity, and Claude again. It reports which questions now cite your site, which competitors you displaced, and which questions still go to someone else, along with its plan for those. You can ask Kite to repeat the measurement every month, and it records what worked in your team's knowledge base so the next round starts from what this one learned.

When Kite finishes, your Storyblok site serves readable pages to AI crawlers, your key stories answer their questions up front, and you have a before-and-after record of where the site gets cited.