In short. You ran the free AI search readiness checker or the website basics check and got a list of things to fix. We do that work for you: crawler rules set the way you decide, business markup on your site, a cleaned-up Google Business Profile with one consistent name, address and phone, and a short monthly routine to catch anything that slips. You get a written quote first. We do not promise mentions, citations, rankings or traffic.
Four pieces of work
1. Crawler rules you choose
We walk through each crawler with you and record a yes or no for each one, then edit robots.txt to match.
- Search crawlers (Googlebot, bingbot, OAI-SearchBot, Claude-SearchBot, PerplexityBot) kept open unless you say otherwise
- Training crawlers (GPTBot, ClaudeBot, Google-Extended) allowed or opted out, your call
- Old rules that block a search crawler by accident found and removed
- Every version of your address covered, since a robots.txt file only governs its own host, protocol and port
2. Business markup
A JSON-LD block that states who you are in a form machines read without guessing.
- The most specific LocalBusiness type that fits (Plumber, Dentist, AccountingService and so on), or Organization for a business that doesn't serve customers in one area
- Name and address, which Google lists as required for LocalBusiness, plus phone, hours, url and geo where they apply
- sameAs links to the profiles you actually run
- Only facts a visitor can see on the page, then a pass through Google's Rich Results Test
3. Google Business Profile and NAP
Your profile and your site should tell the same story.
- Profile claimed or recovered, categories reviewed, hours and services completed
- Your real-world business name, with no added keywords
- Address shown or hidden according to Google's rules for service-area businesses
- One agreed format for name, address and phone, matched on the site, the profile and the listings that matter for your trade
4. A monthly check routine
Set up once, then run by you or by us.
- Re-run the readiness check after any site change and once a month
- Look at Search Console for pages dropped from the index
- Ask a fixed list of customer questions in the assistants you care about and log who gets named
- Record the trend in a simple sheet, because single answers vary from one ask to the next
Why the crawler choice is yours to make
The companies behind AI assistants each publish separate names for separate jobs, and the documentation says those settings don't depend on each other. Blocking a training crawler is a different decision from blocking a search crawler, and an owner can reasonably land on either side.
| Company | Search or answers | Training | Fetch when a person asks |
|---|---|---|---|
| OpenAI | OAI-SearchBot | GPTBot | ChatGPT-User (OpenAI says robots.txt rules may not apply) |
| Anthropic | Claude-SearchBot | ClaudeBot | Claude-User |
| Perplexity | PerplexityBot (Perplexity says it is not used to train foundation models) | None listed | Perplexity-User (generally ignores robots.txt, per Perplexity) |
Googlebot (Search, including AI Overviews and AI Mode) | Google-Extended (Gemini training and grounding; Google says it doesn't affect Search inclusion) | Not covered here |
Here is what a "search yes, training no" file can look like. It is an example, not a recommendation; your file depends on what else your site needs to keep private.
# Example: search open, training opted out.
# Unnamed crawlers follow the * group.
User-agent: GPTBot
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: *
Disallow: /cart/Under Google's robots.txt rules, a crawler obeys the one group that names it and ignores the * group, so anything you want every crawler to skip (like /cart/ above) has to be repeated inside each named group too. We check that before a file goes live. OpenAI says search changes take about 24 hours to show up, and Perplexity says up to 24 hours.
What we will not promise
- That ChatGPT, Claude, Perplexity, Gemini, Copilot or Google's AI Overviews will name, cite, link or recommend your business.
- Any ranking, in Google Maps, regular results or anywhere else. Google's own Business Profile help says "There's no way to request or pay for a better local ranking on Google."
- More visits, calls or bookings by a date.
- That every agent will obey robots.txt. The user-triggered agents above say they may not.
- That markup will produce a special search feature. Google says it does not guarantee that features using structured data will appear.
Google's page on AI features says there are no extra requirements to show up in AI Overviews or AI Mode beyond being indexed and eligible for a snippet, and that no special markup or AI text file is needed. So the honest pitch is narrow: remove what stops crawlers, state your details clearly and consistently, and keep watching.
How it works
- Send the result
Run the free check and paste the address into the quote form, or let us run it.
- Short call
We go through the crawler list together and you decide each one.
- Written quote
Each fix listed with its price. You approve before anything changes.
- We do the work
In your own site, hosting and profile accounts, with a backup of anything we touch.
- Proof and handover
A fresh check, a Rich Results Test pass and a one-page record of every change.
Price
Quoted. There is no set package for this setup yet, so the price comes in writing after we look at your site and profile. A one-page site with a clean profile needs far less than a multi-location business with duplicate listings. Parts of the job already have rates on our card:
Covers the Business Profile part: claim, verify, complete and fix duplicates.
Small jobs such as a robots.txt edit or a markup fix, estimated in hours before we start.
The monthly assistant-question log is not part of either care plan today. If you want us to run it, it goes on the quote. Rates are starting points; the written quote is what you pay.
Prefer to do it yourself? The AI Search Visibility Kit for Local Businesses ($29) has the crawler playbook, JSON-LD templates, profile checklist and the monthly tracker.
Good fit, poor fit
Good fit: local service businesses, shops and offices whose check came back with blocked search crawlers, no business markup, or a phone number that differs between the site and the profile; owners who want the training question decided on purpose instead of by a theme default.
Poor fit: anyone looking for a guaranteed spot in an AI answer, sites where the owner can't get access to robots.txt or the profile, and businesses that need a full redesign first. For that, start with website design.
Questions
Will this get my business recommended by ChatGPT, Claude, Perplexity or Google's AI Overviews?
We can't promise that, and nobody honestly can. Each company decides what its systems show. This service makes sure the crawlers you choose are allowed in, that your business details are stated clearly and match everywhere, and that someone looks every month. Those are the parts an owner controls.
Should I let AI companies train on my site?
That is your decision, and it is separate from search. OpenAI documents GPTBot for training and OAI-SearchBot for ChatGPT search, and says each setting is independent. Anthropic lists ClaudeBot for training apart from Claude-SearchBot and Claude-User. Google says the Google-Extended token does not affect inclusion in Google Search. We write down what you pick and set the rules to match.
What does the setup cost?
There is no fixed package for it yet, so you get a written quote after we see your site and profile. Parts of the work already have rates on our card, such as Business Profile setup or cleanup and hourly fixes, and those are shown on this page. Nothing is charged before you approve the quote.
What access do you need?
Whatever lets us edit robots.txt and add markup on your site (your website builder, CMS or hosting login) and manager access to your Google Business Profile. Accounts stay in your name, and you can remove our access when the work is done.
Will you add an llms.txt file?
Only if you want one. It is a community proposal, not a standard, and none of the crawler documentation we rely on says it is used. Google says no AI text files are needed for AI Overviews or AI Mode. We don't sell it as a requirement.
Some AI agents ignore robots.txt. Can you stop them?
Not with robots.txt. OpenAI says its rules may not apply to ChatGPT-User, and Perplexity says Perplexity-User generally ignores them, because a person asked for the page. If you need to keep content private, it has to sit behind a login, which is a different job we can quote.
Is this the same as a website care plan?
No. This is a one-time setup plus an optional monthly look. Our care plans cover software updates, backups, security checks and small edits. Many owners pair the two.
Sources
- Google Search Central: AI features and your website
- Google: List of Google's common crawlers (Googlebot and Google-Extended)
- Google: How Google interprets the robots.txt specification
- Google: Local business (LocalBusiness) structured data
- Google: Organization structured data
- OpenAI: Overview of OpenAI Crawlers
- Anthropic: Does Anthropic crawl data from the web, and how can site owners block the crawler?
- Perplexity: Perplexity Crawlers
- Schema.org: LocalBusiness
- Google Business Profile Help: Guidelines for representing your business on Google
- Google Business Profile Help: How to improve your local ranking on Google
- llmstxt.org: The /llms.txt file (a proposal)
Checked November 2026 against each company's own documentation. MCC Solutions is not affiliated with Google, Microsoft, OpenAI, Anthropic or Perplexity. Related: website care plans · review and local SEO retainer · LocalBusiness schema generator · Google Business Profile checklist