G2 and Capterra permit every assistant crawler, and review directories still took 0.7 per cent of citations

Both files block exactly one agent at the root and it is a bulk collector, so nothing about a directory listing is being withheld from an AI answer by permission.

Published by AI Knows Us · Updated 20 October 2026

Reading the robots.txt of both large review directories in our sample on 20 October 2026, each one blocks exactly one of the fourteen agents we tested at the root, and that agent is Bytespider, a bulk collector rather than an assistant. g2.com and capterra.com permit GPTBot, OAI-SearchBot, OAI-AdsBot, ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot, PerplexityBot, Perplexity-User, Google-Extended, CCBot, Applebot-Extended and meta-externalagent. In the same programme, across 1,260 readings of ChatGPT and Gemini taken on 16 October 2026, the whole review directory class accounted for 0.7 per cent of the 20,143 citations recorded. Those two facts belong in one sentence, because together they settle an argument that gets made a lot: directory listings are not thin in AI answers because the directories are closed.

That is worth knowing before anybody spends a quarter chasing profiles on the theory that the listings are there and simply unreachable. They are reachable. The permission layer on both hosts is as open as it gets for every agent that fetches to answer a question. The citation share behind that open door is what we measured, and the earlier count of the same class, where g2.com itself appeared in 49 readings, is published as whether G2 and Capterra help you get recommended by AI.

What each file says, agent by agent

Agentg2.com at the rootcapterra.com at the root
GPTBotAllowedAllowed
OAI-SearchBotAllowedAllowed
OAI-AdsBotAllowedAllowed
ChatGPT-UserAllowedAllowed
ClaudeBot, Claude-User, Claude-SearchBotAllowedAllowed
PerplexityBot, Perplexity-UserAllowedAllowed
Google-ExtendedAllowedAllowed
CCBot, Applebot-Extended, meta-externalagentAllowedAllowed
BytespiderBlockedBlocked

Two independent companies, in the same business, arriving at the same single line. That is not a coincidence and it is not a position about artificial intelligence. Both hosts sell access to a corpus of reviews, and both have drawn one boundary around bulk collection while leaving every retrieval agent alone.

Why a directory refuses a collector and keeps the fetchers

A listings business has an obvious commercial shape, and the file reflects it rather than contradicting it.

  • The corpus is the product. A crawler that gathers text at scale for reuse is taking the inventory. One that fetches one page because one person asked one question is sending a reader.
  • A citation is a referral. Being the source behind an answer about software categories is distribution, and refusing the agents that produce it would cost the thing the business is for.
  • One line is maintainable. A file that names a single agent needs revisiting when that agent changes. A file enumerating fourteen names needs revisiting whenever any operator adds one, and operators do: not one of the 24 readable files in our sample names OAI-AdsBot, which is in OpenAI's own documentation.

Four other hosts in the same 26 host sample made exactly the same call, all of them businesses whose pages are inventory rather than marketing, and the per host breakdown sits in the most refused crawler in our sample.

What this means for a business working on its listings

The permission finding removes one explanation and leaves the measured one standing. If the door is open on both of the large directories and the class still comes to 0.7 per cent of 20,143 citations, the reason an answer did not reach your listing is not access.

  • Keep the profiles current for the people who read them. Buyers check directories, and a profile with a stale description or a dead link costs you with humans whatever the citation share is.
  • Stop buying placement as an answer strategy. Paying for position inside a class measured at 0.7 per cent of citations is paying for the wrong surface. The same money on your own pages lands in the class that took 82.0 per cent of the citations in that run.
  • Put the facts a buyer asks about on your own pages first. Pricing in the reader's currency, the specialism named plainly, the credential, the integration list. Those are what a retrieval agent can quote.
  • Watch the operational side of each platform on its own terms. Claiming a page, how review software actually works and where a platform earns its keep is a separate job, set out in claiming a business page and how the review software works.

The boundary around two files and one percentage

Two files, read at the root, on 20 October 2026. The counts describe the root of each host and not deeper paths, and a file can permit the root while an edge rule refuses the request: elsewhere in this same 26 host sample a help centre permits all fourteen agents and returns HTTP 403 to five named crawlers, confirmed on two readings hours apart. A permission is not a delivery.

The 0.7 per cent comes from one bank of 630 questions in our own category, read on ChatGPT and Gemini on 16 October 2026. Two engines, not five: our product watches five surfaces and Perplexity, Claude and Google AI Overviews were asked nothing in that run. There is no per industry figure in it, so there is no software number, no healthcare number and no legal number to quote. And a domain appearing in a source list is a different event from a business being named in the answer text, which is why those two findings are kept in separate columns in every reading we publish.

G2, Capterra and the permission question, answered

Do G2 and Capterra block AI crawlers?

Not in the sense usually meant. On 20 October 2026 each file blocked one of the fourteen agents we tested at the root, Bytespider, and permitted all thirteen others including every OpenAI, Anthropic, Perplexity and Google identity we named. Both files are public, so the rows can be rechecked by anybody today.

If the directories are open, why is the class only 0.7 per cent?

Because permission and selection are different problems. Access decides whether a page can be fetched. What gets quoted turns on whether the page carries something specific enough to answer the question, and in our data the overwhelming majority of citations went to pages belonging to the company being asked about or to an operator's own documentation.

Should I still collect reviews?

Yes, for the buyers who read them and for the sales conversations they shorten. What the number argues against is treating directory placement as the route into an AI answer, because the measured class share does not support the spend that often goes with it.

Why is Bytespider the one agent both of them refuse?

It was the most refused agent across the whole sample, blocked at the root by 6 of the 24 readable files, more than twice any assistant agent. The hosts doing it are a code host, these two directories, a small agency and a local search tool, which is a recognisable set: businesses whose pages are the thing being sold.

Which readings produced the 0.7 per cent?

The run of 16 October 2026: 210 questions per market across the United States, the United Kingdom and India, 1,260 readings on ChatGPT and Gemini, 20,143 citations across 4,947 distinct domains. The earlier run, with its full method and its own source mix, is published as 597 readings across ChatGPT and Gemini in three markets.

An open door is information, and it points somewhere else

The useful thing about reading these two files is what it eliminates. A profile on either platform is fetchable by every assistant agent we tested, so the next question is not about permission at all. It is whether the page a buyer's question lands on says anything a machine can quote, and that question is answered on your own domain rather than on somebody else's listing. Our own bill is paid by companies asking why a directory profile and a shelf of reviews did not put them in an answer, and the honest first thing we do is take access off the list of suspects. The same refusal to assert what can be printed applies to our own figures: every number behind our claims comes out of a Google Search Console export whose window and pull date sit on our proof page.

Read next

Simon-Kucher pricing strategy, what the firm publishes about its own method and what a Singapore buyer should ask before briefing itThe firm publishes its approach, its scale and four named service lines, and publishes no fee, so the useful work before any call is reading what it claims and deciding which decision you are buying.International pricing strategies for a Singapore company, and the corridor that stops one market undercutting anotherA converted home price is the one answer that is almost always wrong, so set a price per market and keep the gaps narrower than the cost of moving a unit between them.Premium pricing strategy for a Singapore business, and the fence that makes the higher number holdCharging permanently more than the credible alternatives works when the buyer can name what the extra money buys, and the published Singapore writing ladder shows exactly how that is built.Red Antler publishes no price, so here is what its own site does publish and how to cost the same scope from SingaporeRead on 29 October 2026, three pages of redantler.com carry a complete deliverable list, three business stages and not one figure about what any of it costs.Hiring a full service content agency, the five stages a retainer has to cover and the two handoffs where the money leaksMost content retainers buy production and quietly leave strategy and distribution with you, so price the stages rather than the word count.Hiring a branding agency for a Singapore financial services firm, and the three registers to search before you fall in love with a nameIdentity work for a licensed firm starts with a clearance search rather than a mood board, and two of the three registers are free to read in a browser today.

Or browse the whole archive.

See what AI says about you.

The first scan is free, it needs no account, and it asks the questions your buyers ask.

Free. No card. We ask 5 real buyer questions on 2 AI apps.

Already sure you want this? Start free. No card needed, and the free plan stays free. Pay only when you want the work done. Or talk to a person if you have a question first. Or read how we fix what it finds.