Technical SEO in 2026: the foundation that counts

Technical SEO is often presented as a list of two hundred items. That makes it useless, because you no longer know what comes first.

In practice, sites do not break on item 174. They break on a handful of things: a page that is blocked while it only needed to leave the index, a status code that says the opposite of what the page shows, or content that is only built up in the browser.

What has been added since 2025 is that the same foundation now serves two kinds of readers. Google says so itself in its guideline for generative AI, published on 15 May 2026 and updated since: “all existing technical SEO best practices continue to be worthwhile”. To be able to appear in AI Overviews or AI Mode, a page has to be indexed and able to be shown with a snippet. So no separate technique, but the same technique done right.

This article walks through the points that turn out to really matter in my work, with the documentation where each one is written down. I also name a few things that Google says you do not need.

Each layer decides whether the layer above it still does anything.

Crawling and indexing are two different things

This goes wrong more often than everything else in this article put together.

robots.txt controls whether a crawler may fetch your page. It does not control whether that page shows up in the search results. Google says it literally: “A robots.txt file tells search engine crawlers which URLs the crawler can access on your site. This is used mainly to avoid overloading your site with requests; it is not a mechanism for keeping a web page out of Google.”

So a blocked page can end up in the results. Google: “A page that’s disallowed in robots.txt can still be indexed if linked to from other sites.” What you see then is a search result with no description under it.

If you really want a page out of the index, you use noindex. And that is where the trap I run into most often sits, in Google’s own words: “For the noindex rule to be effective, the page or resource must not be blocked by a robots.txt file, and it has to be otherwise accessible to the crawler. If the page is blocked by a robots.txt file or the crawler can’t access the page, the crawler will never see the noindex rule, and the page can still appear in search results, for example if other pages link to it.”

So blocking and excluding together does not work. It is one or the other. And putting noindex in robots.txt does not work at all: “Specifying the noindex rule in the robots.txt file is not supported by Google.”

Something else that often goes wrong: crawl budget. Google has an extensive guide about it, and it opens with a warning that most sites do not need it: “If your site doesn’t have a large number of pages that change rapidly, or if your pages seem to be crawled the same day that they are published, you don’t need to read this guide.” The numbers Google names sit at a million pages that change weekly, or ten thousand pages that change daily. Google adds itself that these are not hard limits: “These are not exact thresholds.” If you are far below that, crawl budget is not your problem. With a site of thousands of generated pages that is different.

One detail from that guide is relevant for everyone, now that more bots come by than ever: “While each crawler has a different crawl demand, the crawl capacity limit is shared across all crawlers. This means that high demand from one crawler can reduce the capacity available for others.” Google says that about its own crawlers. What the bots of AI companies do to your server depends on how that server holds up.

Canonical is a strong signal, not an order

A canonical tag tells Google which version of a page is the real one. What many people do not know is that it is a hint and not an instruction.

Google lists the signals in order of strength: a redirect is the strongest signal, a rel="canonical" comes after that, and about inclusion in the sitemap Google itself says “a weak signal”. And then: “none of them are required; your site will likely do just fine without specifying a canonical preference.”

The rules Google does add are concrete. Do not use robots.txt for canonicalization. Do not specify two different canonicals for the same page through two different methods. Put a canonical on the canonical page itself as well. And do not use noindex to choose between two of your own pages, because that throws the page out of the search results altogether.

If you work with a site that builds up content in the browser, there is an extra rule: “The best way to do this is to specify the canonical URL in the HTML source code and make sure that JavaScript doesn’t change the canonical link element.”

Status codes have to say what the page shows

A crawler reads two things: the status code and the content. If those contradict each other, Google picks the safe side.

What Google itself does per code:

Code	What Google does with it
`2xx`	Content is considered for indexing. If the page shows an error, a soft 404 follows in Search Console
`3xx`	Google follows at most ten redirects in a row
`4xx`	Is not indexed, and a page that was already indexed gets removed
`5xx` and `429`	Google starts crawling more slowly. Indexed pages stay at first and disappear after that

A soft 404 is a page that says there is nothing, but sends a 200 along with it. Google: “Custom 404 pages are created solely for users. Since these pages are useless from a search engine’s perspective, make sure the server returns a 404 HTTP status code to prevent having the pages indexed.”

Note what Google says about 404 compared to blocking: “Google won’t forget a URL that it knows about, but a 404 status code is a strong signal not to crawl that URL again. Blocked URLs, however, will stay part of your crawl queue much longer.” So a page that is really gone gets cleaned up with a 404 or 410, not with robots.txt.

Speed: the thresholds have not changed

The three Core Web Vitals and their thresholds are still exactly as they were last year:

LCP, how quickly the largest element is on screen: within 2.5 seconds.
INP, how quickly the page responds to a click or tap: 200 milliseconds or less.
CLS, how much the page shifts while loading: 0.1 or less.

Those thresholds apply at the 75th percentile of your visits, measured separately for mobile and desktop. So three quarters of your visits have to stay under them. In the official lifecycle list all three are at “Stable”. I have not been able to find an announcement of a successor to INP, and no change in 2026 either; the page on web.dev where the thresholds are listed was last updated in October 2024.

Google phrases the effect more carefully than most SEO texts: “We highly recommend site owners achieve good Core Web Vitals for success with Search and to ensure a great user experience generally.” That is not a promise of higher positions. What you need to know about it, and what to watch out for with INP, is in my article on Core Web Vitals and INP.

Structured data: fewer rich results, still useful

Here something did change in 2026, and it touches a lot of Dutch sites.

FAQPage no longer produces rich results. Google reported it on 8 May 2026 in the overview of documentation changes: “This feature will no longer appear in Google Search starting May 7, 2026.” The documentation page itself was taken down in June 2026; what is left now is that note in the overview of changes. HowTo had already gone earlier, in September 2023; nothing new happened about that in 2026, even though you sometimes read otherwise.

Do you have to remove that markup now? Google says no: “Structured data that’s not being used does not cause problems for Search, but also has no visible effects in Google Search.” I leave FAQ markup in place myself, because AI systems take question and answer pairs from it correctly. Just do not expect any stars from it anymore.

And then the misconception I have to explain most often. Schema markup is not needed to appear in Google’s AI answers. Google: “Structured data isn’t required for generative AI search, and there’s no special schema.org markup you need to add. However, it’s a good idea to continue using it as part of your overall SEO strategy.” Which types still give you something, and how to use them, is in my article on schema markup for AI Overviews.

What AI search engines get from the same foundation

The guideline Google published in May 2026 is the only source that counts for this subject. Two things are stated firmly in it.

First the requirement: “To be eligible to be shown in generative AI features on Google Search, a page must be indexed and eligible to be shown in Google Search with a snippet, fulfilling the Search technical requirements.”

There is a second condition on that same page, and it often gets skipped: “In addition to the technical requirements for Search, a site must be included in Search generative AI features in Search Console to be eligible for display in generative AI features on Google Search.” That setting is on by default, so you do not have to do anything for it. It is the first thing I check, though, when a site turns up nowhere in AI answers.

And then one more line I like to quote when someone asks for guarantees: “Indexing and serving aren’t guaranteed.”

Second: what Google says you do not need. Google puts it under the heading “Mythbusting generative AI search: what you don’t need to do”.

On llms.txt: “You don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn’t use them.” Google adds that it does no harm either. What the file actually is, and why I still recommend it for other systems, is in my article on llms.txt.
On chopping content into pieces: “There’s no requirement to break your content into tiny pieces for AI to better understand it.”
On writing separately for AI: “You don’t need to write in a specific way just for generative AI search.”

Elsewhere on that page, in the part about measuring, there is one more line you can quote for every tool choice: “No third-party tool has access to our internal ranking or AI systems.”

Another stubborn mistake: blocking Google-Extended does not take you out of AI Overviews. Google is explicit about it: “Google-Extended does not impact a site’s inclusion in Google Search nor is it used as a ranking signal in Google Search.” That token is about training Gemini models, not about the AI features in Search. If you want to limit what gets used from your pages in Search, the controls are nosnippet, data-nosnippet, max-snippet and noindex.

For the crawlers of the other companies, something else applies. OpenAI and Perplexity both write that you have to allow their search bot in robots.txt to be able to appear in their results. And neither of them says anything about executing JavaScript, which is why I take server side or statically rendered HTML as the starting point. What that means if your site was built with an AI builder is in my article on building a website with AI. How Perplexity reads your site is something I looked into earlier.

Multilingual: two mistakes almost everyone makes

If you have a Dutch and an English version, hreflang comes into play: the annotation that tells Google, per page, which language it is in and which other language versions there are. Google names the common mistakes itself, under the heading “Common Mistakes”.

The first: missing return links. “If page X links to page Y, page Y must link back to page X. If this is not the case for all pages that use hreflang annotations, those annotations may be ignored.”

The second sits in the codes. UK does not exist as a region code; it has to be GB. Google: “If you use codes that are listed as reserved for something else, Google Search ignores that part of the annotation (for example, using EU, UN, or UK in hreflang annotations doesn’t have an effect on Google Search).”

Furthermore: URLs have to be complete, including https, every version also has to point to itself, and a canonical has to point to a page in the same language. With only two languages, x-default points at the version you want to serve as the default.

The order in which I check this

With a new site I work through this list. The order is not random: each point decides whether the next one still achieves anything.

Does a 200 come out and is the content in the HTML? Without that, the rest is pointless.
What is in robots.txt and where is noindex? I often find the real problem here, and it usually takes five minutes to fix.
Do the status codes match what the page shows? Especially the 404 page and removed pages.
Are the canonicals unambiguous? One method, self referencing, no contradictions.
Is everything that should take part in sitemap.xml, and nothing else? Watch out for lastmod: that date should only change when the page has really changed, not with every new build.
Do the hreflang references match in both directions? With two languages that costs ten minutes.
Do the Core Web Vitals hit the thresholds at the 75th percentile? In field data, not in a lab test.
Is there schema that still gives you something, and nothing that does harm?
Only then: content and internal links. Without technique a good text does not get read, and without a good text perfect technique does not get you customers.

If you would rather not work through this list yourself, the question is what it costs to outsource it. What you pay for that in the Netherlands, and what to watch out for, is in my article on outsourcing SEO.

What is left after that is measuring whether it works. For AI traffic that has been arranged differently from ordinary search traffic since this year; I worked that out in my article on measuring AI traffic in GA4.

In short

Robots.txt controls crawling, noindex controls indexing. Using them together does not work, because a blocked page never shows the noindex.
According to Google, canonical is a strong signal, not an instruction, and it is not required.
Status codes have to match the content. An error page with a 200 becomes a soft 404.
The Core Web Vitals thresholds are unchanged: LCP 2.5 seconds, INP 200 milliseconds, CLS 0.1, at the 75th percentile.
FAQPage has produced no rich results since 7 May 2026. HowTo since September 2023. You do not have to remove it.
There is no separate technique for Google’s AI features: be indexed and be able to appear with a snippet. According to Google, llms.txt, chopping content into pieces and extra schema are not needed.
Blocking Google-Extended does not take you out of AI Overviews. That token is about model training.

Frequently asked questions

What is technical SEO, really?

Everything that decides whether a search engine or AI system can fetch, read and understand your pages. So crawling, indexing, status codes, speed, structure and markup. It is not about what is on the page, but about whether it can be read.

Do I need to optimize crawl budget?

Almost certainly not. Google says itself that you do not have to read that guide if your pages get crawled on the day you publish them. The thresholds Google names sit at ten thousand pages with daily changes or a million pages with weekly ones.

Can I take a page out of Google with robots.txt?

No. Robots.txt stops the crawler, not the indexing. If other sites link to that page, it can still end up in the results, without a description. Use noindex and make sure the page is accessible for that.

Do I need schema markup to appear in AI answers?

According to Google, no: “Structured data isn’t required for generative AI search.” Google does recommend it as part of your ordinary SEO, because it makes you eligible for rich results.

Is llms.txt needed?

Not for Google. Google Search does not use the file and says it does not help and does not harm. For other systems that do read the file, it can make sense. In any case, do not attach expectations to it where Google is concerned.

Sources

All sources consulted on 22 September 2026.

Want to know where your site stands technically?

In a free GEO scan I check the points from this article on your own site. You get the findings with the source next to them, so you can verify them yourself. If you would rather hand it over, SEO and GEO shows what each package covers.