E536: AI crawlers for generative AI companies like OpenAI or Anthropic or even Meta are REALLY inefficient.
Crazy inefficiencies compared to Googlebot.
And this means there’s actually a decent chance Generative AI can’t even access your pages and posts to learn off them. If the bots can’t access them, the generative AI can’t recommend your brand.
So this is what’s wrong with these AI crawlers and then how you can fix this (and yes, there is a direct and actionable solution).
Source: https://vercel.com/blog/the-rise-of-the-ai-crawler
00:00 Introduction to AI Crawlers 00:48 AI Crawler Traffic 03:45 JavaScript Rendering Capabilities 06:31 Content Type Priorities 08:20 Crawling Inefficiencies 11:30 Recommendations for Site Owners 17:17 Final Thoughts
The Edward Show. Your daily search engine optimization podcast: https://edwardsturm.com/the-edward-show/
#seotips #searchengineoptimization #webdevelopment #growthmarketing