The Net Almanac is an annual report that interprets the HTTP Archive dataset into sensible perception, combining large-scale measurement with skilled interpretation from trade specialists.
To get insights into what the 2025 report can inform us about what is really taking place in search engine marketing, I spoke with certainly one of the authors of the search engine marketing chapter replace, Chris Green, a well known trade skilled with over 15 years of expertise.
Chris shared with me some surprises about the adoption of llms.txt files and the way CMS methods are shaping search engine marketing excess of we understand. Little-known information that the information surfaced in the analysis, and stunning insights that often would go unnoticed.
You may watch the full interview with Chris on the IMHO recording at the finish, or proceed studying the article abstract.
“I feel the information [in the Web Almanac] helped to present me that there’s nonetheless rather a lot damaged. The online is actually messy. Actually messy.”
Bot Administration Is No Longer ‘Google, Or Not Google?’
Though bot administration has been binary for a while – enable/disallow Google – it’s turning into a brand new problem. One thing that Eoghan Henn had picked up beforehand, and Chris present in his analysis.
We started our dialog by speaking about how robots recordsdata are now getting used to specific intent about AI crawler entry.
Chris responded to say that, firstly, there is a necessity to take heed to the completely different crawlers, what their intention is, and basically what blocking them may do, i.e., blocking some bots has greater implications than others.
Second to that, requires the platform suppliers to really hear to these guidelines and deal with these recordsdata as acceptable. That isn’t all the time taking place, and the ethics round robots and AI crawlers is an space that SEOs want to learn about and perceive extra.
Chris defined that though the Almanac report confirmed the symptom of robots.txt usage, SEOs want to get forward and perceive how to management the bots.
“It’s not solely understanding what the influence of every [bot/crawler] is, but additionally how to talk that with the enterprise. Should you’ve bought a crew who need to minimize as a lot bot crawling as attainable as a result of they need to get monetary savings, that may desperately influence your AI visibility.”
Equally, you might need an editorial crew that doesn’t need to get all of their work scraped and regurgitated. So, we, as SEOs, want to perceive that dynamic, how to management it technically, however how to put that argument ahead in the enterprise as nicely.” Chris defined.
As extra platforms and crawlers are launched, search engine marketing groups may have to take into account all implications, and collaborate with different groups to guarantee the proper steadiness of entry is utilized to the web site.
Llms.txt Is Being Utilized Regardless of No Official Platform Adoption
The primary stunning discovering of the report was that adoption for the proposed llms.txt commonplace is round 2% of web sites in the dataset.
Llms.txt has been a heated matter in the trade, with many SEOs dismissing the value of the file. Some instruments, resembling Yoast, have included the commonplace, however as but, there has been no demonstration of precise uptake by AI suppliers.
Chris admitted that 2% was the next adoption than he anticipated. However a lot of that development seems to be pushed by search engine marketing instruments which have added llms.txt as a default or elective function.
Chris is skeptical of its long-term influence. As he defined, Google has repeatedly said it does not plan to use llms.txt, and with out clear dedication from the main AI suppliers, particularly OpenAI, it dangers remaining a distinct segment, symbolic gesture fairly than a useful commonplace.
That stated, Chris has skilled log-file information suggesting some AI crawlers are already fetching these recordsdata, and in restricted instances, they might even be referenced as sources. Inexperienced views this much less as a aggressive benefit and extra as a possible parity mechanism, one thing that will assist sure websites be understood, however not dramatically elevate them.
“Google has again and again stated they don’t plan to use llms.txt which they reiterated in Zurich at Search Central final 12 months. I feel, basically, Google doesn’t want it as they do have crawling and rendering nailed. So, I feel it hinges on whether or not OpenAI say they may or gained’t use it and I feel they produce other issues than attempting to arrange a brand new commonplace.”
Completely different, However Reassuringly The Similar The place It Issues
I went on to ask Chris about how SEOs can steadiness the distinction between search engine visibility and machine visibility.
He thinks there is “a big overlap between what search engine marketing was before we began worrying about this and the place we are at the begin of 2026.”
Regardless of this overlap, Chris was clear that if anybody thinks optimizing for search and machines is the identical, then they are not conscious of the two completely different methods, the completely different weightings, the incontrovertible fact that interpretation, retrieval, and era are fully completely different.
Though there are completely different methods and completely different capabilities in play, he doesn’t assume search engine marketing has basically modified. His perception is that search engine marketing and AI optimization are “sort of the identical, reassuringly the identical in the locations that matter, however you will have to method it otherwise” as a result of it diverges in how outputs are delivered and consumed.
Chris did say that SEOs will transfer extra in direction of feeds, feed administration, feed optimization.
“Google’s common commerce protocol the place you possibly can probably transact instantly from search outcomes or from a Gemini window clearly adjustments rather a lot. It’s simply one other transfer to push the web site out of the loop. However the information, what we’re really optimizing nonetheless wants to be optimized. It’s simply in a distinct place.”
CMS Platforms Form The Net Extra Than SEOs Understand
Maybe the greatest shock from Net Almanac 2025 was the scale of affect exerted by CMS platforms and tooling suppliers.
Chris stated that he hadn’t realized simply how massive that influence is. “Platforms like Shopify, Wix, and so forth. are shaping the precise state of tech search engine marketing in all probability extra profoundly than I feel lots of people actually give it credit score for.”
Chris went on to clarify that “as well-intentioned as particular person SEOs are, I feel our general influence on the net is minimal outdoors of CMS platforms suppliers. I’d say in the event you are actually decided to have an effect outdoors of your particular purchasers, you want to be nudging WordPress or Wix or Shopify or a few of the massive software program suppliers inside these ecosystems.”
This creates alternative: Web sites that do implement technical requirements accurately might obtain important differentiation when most websites lag behind greatest practices.
One in all the extra attention-grabbing insights from this dialog was that a lot on the net is damaged and the way little influence we [SEOs] actually have.
Chris defined that “quite a lot of SEOs imagine that Google owes us as a result of we keep the web for them. We do the soiled work, however I additionally don’t assume we now have as a lot influence maybe at an trade degree as possibly some like to imagine. I feel the information in the Net Almanac sort of helped present me that there’s nonetheless rather a lot damaged. The online is actually messy. Actually messy.”
AI Brokers Gained’t Change SEOs, However They Will Change Unhealthy Processes
Our dialog concluded with AI agents and automation. Chris began by saying, “Brokers are simply misunderstood as a result of we use the time period otherwise.”
He emphasised that brokers are not replacements for experience, however accelerators of course of. Most search engine marketing workflows contain repetitive information gathering and sample recognition, areas well-suited to automation. The worth of human experience lies in designing processes, making use of judgment, and contextualizing outputs.
Early-stage brokers might automate 60-80% of the work, comparable to a extremely succesful intern. “It’s going to take your data and your experience to make that relevant to your given context. And I don’t simply imply the context of net advertising or the context of ecommerce. I imply the context of the enterprise that you simply’re particularly working for,” he stated.
Chris would argue that quite a lot of SEOs don’t spend sufficient time customizing what they do to the shopper particularly. He thinks there’s a chance to construct an 80% automated course of after which add your actual worth when your human intervention optimizes the final 20% enterprise logic.
SEOs who have interaction with brokers, refine workflows, and evolve alongside automation are way more seemingly to stay indispensable than those that resist change altogether.
Nevertheless, when experimenting with automation, Chris warned we should always keep away from automating damaged processes.
“You want to perceive the course of that you simply’re attempting to optimize. If the course of isn’t superb, you’ve simply created a machine to produce mediocrity at scale, which frankly doesn’t assist anybody.”
Chris thinks that this can give SEOs an edge as AI is extra extensively adopted. “I recommend the those that have interaction with it and make these processes higher and present how they are often regularly developed, they’ll be the ones which have higher longevity.”
SEOs Can Succeed By Participating With The Complexity
The Net Almanac 2025 doesn’t recommend that search engine marketing is being changed, but it surely does present that its function is increasing in methods many groups haven’t totally tailored to but. Core rules like crawlability and technical hygiene nonetheless matter, however they now exist inside a extra complicated ecosystem formed by AI crawlers, feeds, closed methods, and platform-level selections.
The place technical requirements are poorly carried out at scale, those that perceive the methods that form them can nonetheless achieve a significant benefit.
Automation works greatest when it accelerates well-designed processes and fails when it merely scales inefficiency. SEOs who focus on course of design, judgment, and enterprise context will stay important as automation turns into extra frequent.
In an more and more messy and machine-driven net, the SEOs who succeed will probably be these keen to have interaction with that complexity fairly than ignore it.
search engine marketing in 2026 isn’t about selecting between search and AI; it’s about understanding how a number of methods devour content material and the place optimization now occurs.
Watch the full video interview with Chris Inexperienced right here:
Thanks to Chris Inexperienced for providing his insights and being my guest on IMHO.
Extra Sources:
Featured Picture: Shelley Walsh/Search Engine Journal
Disclaimer: This article is sourced from external platforms. OverBeta has not independently verified the information. Readers are advised to verify details before relying on them.
