Some questions an IT agent should refuse even when it could answer them well. Security exceptions, access decisions and anything that sounds like approval all have competent-looking answers available in most corpora, and giving them creates a record of the company apparently permitting something nobody authorised.
TL;DR
- The refusal list is not about what the agent cannot answer. It is about what it should not, which is a different and shorter list.
- Four categories: anything that reads as approval, anything about a security exception, anything naming another person, and anything about somebody’s own employment.
- Write the refusals as fixed sentences. They are the most consequential answers in the system and the ones a model is most likely to pad.
- Refuse by topic, not by confidence, because the questions that most need refusing are the ones a well-stocked corpus answers confidently.
- A refusal naming a route is a service. A refusal saying it cannot help with that is a dead end, and people remember dead ends.
- Test the refusals adversarially, including the rephrasings that make a forbidden question look like a documentation lookup.
Could answer, should not
Most scope discussions are about coverage gaps, which are a documentation problem. This is the opposite case: the corpus contains a perfectly good answer and returning it is still the wrong behaviour.
Can I use my personal laptop for work has a documented answer in most companies and the answer has conditions attached, exceptions, and a person who decides. An agent that returns the general position has effectively told somebody yes, and that is what they will report having been told.
The asymmetry is what makes this worth configuring. If the agent refuses and the answer was straightforward, the cost is a small delay. If it answers and the case needed a decision, the cost is an unauthorised arrangement with a plausible justification behind it.
The four categories
| Category | Example | Why refuse |
| Reads as approval | Can I install this, can I use my own device | A general answer becomes a permission |
| Security exception | How do I get round the blocked site rule | Explaining the route is enabling it |
| About another person | What device does she have, is his account active | Not the asker’s data to receive |
| About their employment | Am I being monitored, will IT report this | Needs a person, not a policy extract |
| Legal or contractual effect | Does this policy override my contract | Nobody should get this from a chat widget |
| Live account changes | Grant me admin, disable this control | An action with an accountable owner |
The third row surprises people. Questions about colleagues arrive constantly and innocently, usually from a manager trying to help, and an agent connected to any directory data will answer them. Whether a manager should receive that information is a decision somebody makes once, not a retrieval outcome.
The fourth row is the one with the sharpest consequences. Am I being monitored is a question about somebody’s employment relationship, and a policy extract delivered by a chat widget is the wrong channel for it regardless of how accurate the extract is.
Refuse by topic, not by confidence
A confidence threshold cannot do this job. The forbidden questions above are exactly the ones a well-documented company answers with high confidence, because the policies exist and are written clearly.
So the refusal has to be driven by topic classification, and the classifier should be deliberately eager. Refusing a borderline question that turned out to be routine costs a redirect. Answering a borderline question that turned out to need a decision costs considerably more, and the two errors are not comparable.
Expect over-triggering and accept it. Review the rate monthly rather than tuning it towards zero, because a refusal list tuned until it never fires incorrectly is a refusal list that has stopped firing.
Writing the refusal
A refusal is an answer. It should be as deliberately written as any other, and it usually is not, because teams leave it to be generated and the model pads it with apology.
What to do:
- Write one fixed sentence per category, verbatim, rather than describing the behaviour and hoping.
- Name a role and a channel in every refusal, because a refusal without a route is a dead end.
- State the reason in plain terms, which is usually that this needs a decision rather than a lookup.
- Leave out the apology, the offer of further help and any suggestion about what the answer probably is.
- Keep it to two sentences, since length in a refusal reads as evasion.
- Never imply the outcome, including phrases such as this is usually fine or that should not be a problem.
The last item is where good intentions do damage. An agent softening a refusal with a hint about the likely answer has given the answer, with the added problem that it is now unattributable to anybody who could be asked to stand behind it.
Testing it properly
Refusals need adversarial testing, because the interesting failures come from rephrasing rather than from the obvious form of the question.
Try each forbidden question three ways: directly, as a documentation lookup, and as a hypothetical. What does the policy say about personal devices will often get answered where can I use my personal laptop gets refused, and both produce the same practical outcome for the person asking.
Then try them as a manager asking about somebody else, which is the phrasing most likely to slip past a classifier trained on first-person questions. Keep the whole set as a standing test and re-run it after any change to the corpus or the instructions, because scope regresses silently when somebody adds a document.
Disclosure: Matram is owned by the same people who publish PeopleOpsHQ. It is relevant here because strict mode gives a controllable refusal rather than a generated one, which is the mechanism this entire topic depends on, and it is priced flat at $29, $69 and $199 a month with unlimited seats. It has no free tier, it answers questions rather than actioning tickets, and it is not an ITSM platform, so the route named in each refusal has to point at something you run elsewhere. The options are compared in our IT support chatbot comparison.
Final thoughts
Scope is usually discussed as what the agent does not know. The harder and more useful list is what it should decline despite knowing, and it has four entries: anything that functions as approval, anything about a security exception, anything about another person, and anything about somebody’s own employment.
Drive those by topic rather than confidence, write each refusal as a fixed two-sentence answer naming a real route, and test every one of them in three phrasings including the manager-asking-about-somebody-else form. A refusal done properly is one of the better things the system does.
Frequently asked questions
What should an IT support chatbot refuse to answer?
Four categories: anything that functions as approval such as whether somebody can install software or use a personal device, anything about getting around a security control, anything about another named person, and anything about the asker’s own employment relationship such as whether they are being monitored. All four typically have competent answers available in the corpus, which is exactly why the refusal has to be configured deliberately.
Why refuse a question the agent could answer well?
Because the two errors are not comparable. Refusing a question that turned out to be routine costs a short delay, while answering one that needed a decision produces an unauthorised arrangement with a plausible justification behind it, and the person will reasonably report having been told yes by the company’s own system.
Can a confidence threshold handle this?
No, because the questions that most need refusing are the ones a well-documented company answers with high confidence, since the policies exist and are written clearly. Refusals have to be driven by topic classification, and that classifier should be deliberately eager, with the over-triggering rate reviewed monthly rather than tuned towards zero.
How should a refusal be worded?
Two sentences, written as fixed text rather than generated: the reason, which is usually that this needs a decision rather than a lookup, and a named role plus a channel. Leave out the apology, the offer of further help and any hint about what the answer probably is, since a softened refusal that implies the outcome has given the answer while making it unattributable to anybody accountable.
Why are questions about colleagues on the list?
Because they arrive constantly and innocently, usually from a manager trying to help, and an agent with any directory access will answer them. Whether a manager should receive information about a colleague’s device or account status is a decision somebody makes once as a matter of policy, not something that should fall out of a retrieval result.
How do you test the refusal list?
Adversarially, in three phrasings each: directly, as a documentation lookup, and as a hypothetical, because asking what the policy says about personal devices frequently gets answered where asking whether you can use your own laptop gets refused. Also test the manager-asking-about-somebody-else form, which most often slips past a classifier trained on first-person questions, and keep the whole set as a standing test since scope regresses silently whenever a document is added.
