Crawler transparency
LLMSMap crawlers and user agents
These identifiers make LLMSMap requests recognizable in web-server, bot, CDN, and SEO crawler reports.
Directory discovery
Checks public candidate domains and machine-readable files before a site can enter the directory review queue.
LLMSMap-ME-Discovery/1.0 (+https://llmsmap.me/crawlers/)AI-readiness audit
Reads the public homepage, robots.txt, llms.txt, llms-full.txt, and ai.txt for technical directory audits.
LLMSMap-ME-Audit/1.0 (+https://llmsmap.me/crawlers/)Request policy
- Only public HTTP and HTTPS resources are requested.
- No login, form submission, or restricted-area access is attempted.
- Requests use bounded timeouts, response-size limits, and controlled concurrency.
- The audit identifier is separate from discovery so traffic can be classified accurately.