AI Search Engineering
llms.txt vs robots.txt: Which Layer Actually Binds AI Crawlers
97% of llms.txt files get zero requests and Google says it doesn't read them. The real control plane is robots.txt (RFC 9309), your CDN's shifting defaults, and network-layer enforcement.