Open domain chatbot: the difference is scope, and scope decides what it says at the edge of what it knows

An open domain chatbot is one that will take any input on any subject and produce a reply. A closed domain chatbot, also called task oriented, is scoped to a defined set of jobs: track an order, book a room, reset a password, and anything outside that set is refused or handed to a person. The distinction sounds like a product decision about breadth. It is really a decision about behaviour at the boundary, because the interesting question for both kinds is what happens when the answer is not in their sources. A closed system says so and routes you onward. An open system produces its best reply to the question as asked. That one difference explains why the two are built differently, judged differently and carry different risks.

This page sets out what each kind is, how open domain systems are put together, why they are harder to evaluate, and the practical reason a business cares: the open domain assistants are already answering questions about your company, whether or not you have built anything.

The two kinds, side by side

	Open domain	Closed domain, task oriented
Accepts	Any subject, in any phrasing	A defined set of intents
Built from	A large language model, usually with a retrieval layer over documents	Intents and entities, or a model constrained by a prompt and a document set, plus business logic
At the boundary	Answers anyway, from whatever is in its sources and its training	Declines and escalates, which is a designed behaviour rather than a failure
Correct answer	Many acceptable replies, so quality is judged by sampling and rating	Usually one, which makes testing straightforward
Main failure	A fluent, plausible answer that is wrong	Refusing something it should have handled, which reads as unhelpfulness
Examples	The general assistants people use daily	Support bots, booking bots, internal help desks

How open domain systems are actually assembled

The field has moved through three shapes, and knowing which one you are looking at explains most of its behaviour.

• Rule based. Patterns and scripted replies. Still useful for a narrow task, and it cannot hold a conversation about anything it was not scripted for.
• Retrieval based. A library of candidate responses and a ranker that picks the closest one to the input. Every reply is something a human wrote, which makes it safe and makes it repetitive.
• Generative. The reply is produced word by word by a language model. This is what makes open domain conversation possible at all, and it is also the source of the characteristic failure: the model is optimising for a plausible continuation, and a plausible continuation is not the same thing as a sourced one.

Almost every serious open domain system now adds a fourth element: retrieval augmentation. The user's question is used to fetch relevant documents, and those documents are put in front of the model with an instruction to answer from them. This is the single most important design pattern in the field, because it moves the answer from what the model absorbed in training to what the system just read, and it makes the answer attributable to a document somebody can check.

Why open domain is the harder engineering problem

• There is no list of intents to enumerate. A closed domain bot can be specified: here are the forty things it does. An open domain bot has to be specified by behaviour instead, which is a much weaker form of specification.
• Evaluation has no single key. With many acceptable replies, you need a sample of real questions, a rubric, and repeated scoring over time, which is a measurement programme rather than a test suite.
• Coherence across a long conversation. Holding what was established twenty turns ago, and noticing when the subject has changed, is its own problem.
• Grounding. Attaching each claim to a source, and keeping the system from blending two sources into a statement neither one made.
• The boundary behaviour has to be designed deliberately. Left alone, a generative system answers everything, so declining to answer is something you build rather than something you get.

Which one should a business build

Nearly always the closed one, and the reason is liability rather than capability.

• If the bot speaks for your company, scope it. A support assistant that will discuss anything will eventually make a commitment about price, delivery, suitability or compliance that your business has to honour or publicly correct.
• Ground it in your own documents and instruct it to answer only from them. Then the maintenance task is keeping the documents right, which is work somebody already owns.
• Design the refusal before the answers. One sentence that says the question is outside what it covers, plus a route to a person, handles a large share of real traffic well. A plausible invention handles it badly and expensively.
• Log every question it could not answer. That log is the most valuable output of a support bot: it is a list of things customers want to know that your documentation does not say.
• Judge it on containment and accuracy together. Containment alone rewards a bot that stops people reaching help. Accuracy alone rewards a bot that escalates everything.

The routes available for building either kind, from using a hosted assistant to building on an API to a no code builder, are set out in GPT-3 and ChatGPT chatbots.

The open domain bot you did not build, that already talks about you

Here is the part that matters commercially. The general assistants are open domain systems, and people ask them closed questions about specific businesses all day: is this company any good, who should I use for this, what does that firm charge, are they still trading. The system answers from whatever it retrieved and whatever it absorbed, in the same confident register it uses for everything else, and the person asking has no way of telling which of those two sources the sentence came from.

Three consequences worth sitting with.

• Your company is being described by a system with no obligation to check with you, and the description is assembled from third party pages as readily as from your own.
• Stale facts persist. A price, an address or a product you retired stays in circulation for as long as the pages carrying it stay retrievable.
• The answer is a shortlist, not a list. Two or three names get returned with reasoning, which makes it closer to a recommendation than to a search result. What decides which names appear is set out in how AI assistants choose which brands to recommend.

Open domain chatbot: the questions that come up

Is ChatGPT an open domain chatbot?

Yes. It accepts any subject and generates a reply rather than selecting from a scripted set, which is the definition. The same is true of the other general assistants. A support bot built on the same underlying model is closed domain if its instructions and its document set confine it to one subject.

What is the difference between open domain and open source?

Nothing related. Open domain describes the breadth of subjects a system will discuss. Open source describes the licence on the code or the model weights. A closed source model can be open domain, and an open source model can be scoped to one narrow task.

Is an open domain chatbot the same as general artificial intelligence?

No. Open domain is a scope of conversation: it will take any subject as input. It says nothing about whether the system understands or reasons in any general sense, and a system can be open domain and still wrong about most of what it says.

How do you stop one answering questions it should not?

By scoping it, which means instructions that state what it covers, a document set it must answer from, a refusal response written in advance, and tests that include the questions you want refused. The tests are the part most teams skip, and they are what turns a scope from an intention into a property of the system.

How is quality measured if there is no correct answer?

With a fixed set of real questions, a written rubric, and repeated scoring by people or by a model acting as a rater, tracked over time. The useful output is a direction of travel across releases, which is why the question set has to stay fixed even when it gets stale.

Does a small business need its own chatbot?

Usually not before it has written the answers down. The expensive part of a support bot is never the bot: it is producing accurate, current documents for it to answer from. Those documents are worth having anyway, because they are also what gets read by the open domain assistants already fielding questions about you.

Reading your own entry in a system you do not control

If you build a closed domain bot, you can test it: ask the forty questions, read the forty answers, fix what is wrong. The open domain systems describing your business can be tested the same way, and almost nobody does it. Ask the questions a customer would ask, several times over, across more than one assistant, and record which companies get named and which sources the answers lean on. It takes an afternoon the first time, and the result is usually a short list of documents that need to exist. AI Knows Us runs those questions across the main assistants and reports what came back and what each answer was built from.