# robots.txt — zonar.com (aggressive visibility, analytics-preserving)
# Last updated: 2026-05-22
# Strategy:
#   1. MAXIMUM crawler access — every search bot, Answer Engine, and training crawler
#   2. PRESERVE analytics — UTM, gclid, fbclid, _hsenc, etc. stay crawlable so
#      Google can read canonical tags and consolidate link authority to the canonical URL.
#      Analytics platforms read these params from real browser visits, not robots.txt.
#   3. Block ONLY HubSpot internal infrastructure that's never canonical content
#   4. Surface llms.txt + sitemap for discovery

# ===========================================
# All user agents
# ===========================================

User-agent: *

# HubSpot internal infrastructure (drafts, auth, system paths — not canonical):
Disallow: /_hcms/preview/
Disallow: /_hcms/mem/
Disallow: /hs/manage-preferences/
Disallow: /hs/preferences-center/
Disallow: /system/templates/

# Preview URL variants (drafts, not real pages):
Disallow: /*?*hs_preview=*
Disallow: /*?*hsCacheBuster=*

# Thin search-results pages (zero canonical value):
Disallow: /hs-search-results

# INTENTIONALLY CRAWLABLE (canonical tags handle consolidation):
#   ?utm_source / utm_medium / utm_campaign / utm_term / utm_content   ← marketing campaigns
#   ?gclid / gbraid / wbraid / dclid                                   ← Google Ads
#   ?fbclid                                                            ← Facebook/Meta Ads
#   ?msclkid                                                           ← Microsoft Ads
#   ?yclid / ttclid / li_fat_id                                        ← Yandex / TikTok / LinkedIn
#   ?_hsenc / _hsmi / __hssc / __hstc / __hsfp / hsCtaTracking         ← HubSpot tracking
#   ?mc_eid / mc_cid / vero_id / _ke                                   ← email platforms
# Rationale: blocking these severs Google's ability to read canonical tags
# from inbound campaign links, costing real authority signal. Analytics
# capture these params from actual user visits, unaffected by robots.txt.
#
# Also intentionally crawlable: PDFs in /hubfs/ paths (AE citation value).

# ===========================================
# Allowed by inheritance (no override needed)
# ===========================================
# Search:               Googlebot, Bingbot, Applebot, DuckDuckBot
# Answer Engines:       OAI-SearchBot, ChatGPT-User, PerplexityBot, Perplexity-User,
#                       Claude-SearchBot, Claude-User, DuckAssistBot, YouBot, Amazonbot
# Ecosystem training:   Google-Extended, Applebot-Extended
# Standalone training:  GPTBot, ClaudeBot, anthropic-ai, CCBot, Bytespider,
#                       Meta-ExternalAgent, cohere-ai, Diffbot, etc.

# ===========================================
# Discovery
# ===========================================
Sitemap: https://www.zonar.com/sitemap.xml

# AI-readable catalogs:
# https://www.zonar.com/llms.txt
# https://www.zonar.com/llms-full.txt