User-agent: * # Authenticated and internal areas. Nothing here is indexable — they all redirect # to login — so crawling them only burns crawl budget and our workers. Googlebot # was walking /admin/quarantines/{id} on 11 Aug 2026 because the previous # "Disallow:" with an empty value means "allow everything". Disallow: /admin Disallow: /api Disallow: /dashboard Disallow: /user Disallow: /agreements Disallow: /redirect Disallow: /horizon Disallow: /telescope Disallow: /sanctum # These carry single-use tokens or ids in the URL path. Keep them out of crawl # logs and out of any index. Disallow: /reset-password Disallow: /forgot-password Disallow: /email/verify Disallow: /magic-away-login Disallow: /welcome Disallow: /invite # Left crawlable on purpose: /, /login, /register and /plus are public pages.