# robots.txt for BeHooked — https://behooked.ai # # Content usage preferences for AI (https://contentsignals.org): # search=yes → allow search indexing # ai-input=yes → allow use in AI answers / citations (GEO) # ai-train=no → do not use our content to train models User-agent: * Content-Signal: search=yes, ai-input=yes, ai-train=no Allow: / # --- AI answer / citation crawlers (welcomed for GEO) --- User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: GPTBot Allow: / User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / User-agent: ClaudeBot Allow: / User-agent: Claude-User Allow: / User-agent: Google-Extended Allow: / # --- Search engines --- User-agent: Googlebot Allow: / User-agent: Bingbot Allow: / User-agent: Slurp Allow: / User-agent: DuckDuckBot Allow: / User-agent: Baiduspider Allow: / User-agent: YandexBot Allow: / # The index, not `sitemap.xml`. @astrojs/sitemap emits `sitemap-index.xml` # plus one or more numbered chunks; pointing a crawler at `sitemap.xml` sends # it to a URL this build never writes. Sitemap: https://behooked.ai/sitemap-index.xml