Why We Publish This
People increasingly ask AI assistants questions like "what's my monthly mortgage payment" or "how much can I contribute to an IRA this year" instead of, or alongside, running a search. We built this site to help with exactly those questions, so we've chosen to let AI assistants and their crawlers read our free calculators and guides and cite them in an answer. This page states that posture in plain language and lines it up against our robots.txt file, which is the technical rule that actually controls it.
Our Posture: Open, With One Exception
Our robots.txt file allows nearly every AI crawler we're aware of to read the site, across every page, with one named exception. Here's the breakdown.
OpenAI
Allowed. GPTBot, OAI-SearchBot, and ChatGPT-User can all read the site.
Anthropic (Claude)
Allowed. ClaudeBot, Claude-User, and Claude-SearchBot can all read the site, and so can claude-code, the agentic coding tool.
Perplexity
Allowed. PerplexityBot can read the site.
Google AI
Allowed. Google-Extended, the crawler Google uses for Gemini and AI features separately from regular Search indexing, can read the site.
Microsoft / Bing AI
Allowed. Bingbot can read the site.
Meta AI
Allowed. FacebookBot and Meta-ExternalAgent can both read the site.
Apple AI
Allowed. Applebot can read the site.
Everyone else, by default
Allowed. Our general rule (User-agent: *) allows any crawler not named above to read the site as well, unless we've named it below.
The one exception: Bytespider
Blocked. Bytespider, a ByteDance crawler, is disallowed across the whole site. Some published research on crawler behavior suggests it ignores robots.txt rules in most cases, but we still state the rule so our intent is explicit if that ever changes.
Beyond Crawling: A Calculator API for Assistants
Reading our pages is one way an AI assistant can use this site. We also publish a free calculator API built for assistants to call directly, so an answer can come from the same formulas our calculators use instead of an assistant guessing at the math from crawled text. Every result it returns includes the data sources and rule year behind the figures, so an assistant can cite where a number came from. See our calculator API for details.
Questions or Concerns
If you operate a crawler and have a question about this policy, or you've noticed an AI assistant citing our figures incorrectly, email admin@markcolabs.com. The About page has more on how we handle corrections, and Trust & Compliance covers how we test our calculations and where our figures come from.
Important Disclaimer
This page is for general information and does not constitute a legal agreement. Crawler rules are technical signals, not guarantees; some crawlers may not honor them, and third parties may still reference our content outside our control. Data current as of September 2026.