Your marketing team types "best QR menu platform" into Perplexity and your brand is nowhere to be found. The reason is simple: your robots.txt has a GPTBot Disallow directive, and the AI engine cannot cite a source it cannot read. Now the question: should you remove that line?
What looked like a no-brainer block decision in 2024 has become an industry dilemma in 2026. Is your content a licensable asset or a marketing storefront? The answer depends entirely on your business model — and getting it wrong can cost millions in either direction.
The Citation Penalty: 96% Invisibility
An October 2026 Originality.AI study found that sites blocking AI crawlers receive 96% fewer citations in ChatGPT and Perplexity answers compared to open sites. In practical terms, you are turning your back on a multi-billion-dollar AI search channel.
And the loss is compounding. Google's AI Overviews now occupy 43% of the search results page in 2026, while Perplexity handles 780 million queries monthly. When you are blocked, that traffic flows entirely to your competitors, and recovery is nearly impossible because the AI model treats you as "non-existent."
SaaS vs Publishers: Two Different Math Problems
The decision matrix hinges on your business model. For a SaaS brand, content is a customer acquisition channel: AI citation equals free lead generation. For a publisher, content is a licensable asset: citation equals royalty-free consumption and direct revenue loss. For B2B SaaS brands like thMenu, the answer is unambiguous: Allow.
- SaaS (thMenu, Shopify, Notion): Allow — citations build brand awareness; conversion happens downstream.
- Publishers (Reuters, NYT, Bloomberg): Block + License — content IS the consumption endpoint.
- E-commerce (Amazon, Etsy): Allow — product discovery converts directly to sales.
The Reuters and NYT U-Turn: Hybrid Licensing
Reuters and the New York Times both fully blocked GPTBot in August 2024. But in March 2026, both reversed course with a Reddit-style paid licensing model. The NYT signed a $16 million annual deal with OpenAI; content became accessible again but with revenue attached.
The lesson is strategic: blocking is not an endgame, it is a negotiation tactic. If your content has standalone market value, block first, then negotiate a licensing agreement. If it does not, allow and harvest the visibility. The worst position is the middle — blocked without a licensing deal — where you get zero traffic and zero revenue.
FAQ
Will blocking GPTBot affect my Google SEO? No. Googlebot is a separate crawler with its own directive. However, you will lose all AI search traffic from Perplexity, ChatGPT Search, and Claude.
How long until a robots.txt change takes effect? AI models update on retraining cycles of 3-6 months. Do not expect immediate citation gains after flipping to Allow.
Should a restaurant site allow GPTBot? Yes. Queries like "best sushi near me" in Perplexity can cite your menu page, delivering high-intent, purchase-ready traffic that converts above industry average.
Found this helpful? Share it.
Related articles
Why Digital Menus Increase Restaurant Revenue by Up to 30%
Studies show restaurants using digital QR menus see measurable increases in aver…
When a Customer Downgrades, What Happens to Old Features? — The Silent Feature-Drift Problem in SaaS
Most SaaS apps run a single line of code when a customer downgrades — but old fe…
JWT alg-confusion attack — why Supabase's HS256 → RS256/JWKS migration breaks legacy verifiers
Verifiers that never decode the JWT header are wide open to `alg=none` and alg-c…