# Aurgus — robots.txt
# Modern infrastructure for incentive and rebate programs
# https://aurgus.com

# Explicit allow for LLM crawlers per AISO best practice
User-agent: GPTBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: Claude-Web
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Google-Extended
Allow: /

User-agent: anthropic-ai
Allow: /

User-agent: cohere-ai
Allow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: Claude-User
Allow: /

User-agent: CCBot
Allow: /

User-agent: Amazonbot
Allow: /

User-agent: Applebot-Extended
Allow: /

User-agent: meta-externalagent
Allow: /

# Full-AI-visibility policy (Mono, 2026-07-03): training-corpus presence is
# the compounding AISO asset, so training crawlers (CCBot, Amazonbot,
# Applebot-Extended, meta-externalagent) are allowed alongside search/user
# agents. Requires Cloudflare's managed robots.txt (Content Signals) to stay
# OFF for this zone — it prepends contradictory Disallow groups.

# Content usage preferences (contentsignals.org / AIPREF). Full-AI-visibility
# policy: training, search, and AI-input are ALL explicitly permitted. Emitted
# as a COMMENT, not a live directive: an unknown directive inside the group
# trips Bing's robots.txt tester with a hard error, and the explicit signal is
# now redundant — Cloudflare's contradictory managed robots.txt (which prepended
# ai-train=no) is disabled for this zone, so the permissive default already
# stands. Restore the live line only if a contradictory signal reappears.
#   Content-Signal: ai-train=yes, search=yes, ai-input=yes
User-agent: *
Allow: /

Sitemap: https://aurgus.com/sitemap.xml