Three features live on production had no coverage at all, so the bot could only
answer questions about them by emailing support.
Pay Control (6.2, 6.5) — zero prior mentions. Vista and Foundation only, and
only AP Admin / Accountant / Support see the column, which is the answer to
almost every "why can't I see this" question about it. Bulk Assign Pay Control
is Foundation-only: Vista companies see the column and cannot bulk-assign from
it, which is the sort of thing support otherwise guesses at. Also adds the
Sitting With column, which was missing from the column list.
Annotating a document (7.8) — zero prior mentions. The four stamps
(TAX EXEMPT, RECEIVED, APPROVED, VOID), the free-text note, and the consequence
people actually ask about: the marks are burned into the image, so they travel
to the ERP and cannot be peeled off — changing one means re-annotating.
File types (5.7) — the distinction that catches people out is that an EMAILED
iPhone HEIC photo is converted automatically,…
Section 18 was a single paragraph saying project managers can approve
from the field, so the bot answered "Beiing Human does not mention a
dedicated mobile app" and mailed the support alias for anything more
specific.
Rewrites it against beiing-human-mobile v2.3.2: install and sign-in
(including the Accountant / AP Admin login block, which is by design and
was being triaged as a password problem), the dashboard, lists, the
document screen, what each status allows, the approve-time validation
gate and why there is no "Approve Anyway", Q&A, uploads, Create PO,
notifications, forced updates, and what the app cannot do. The existing
settings-propagation answer is kept as 18.11, next to the cached-profile
fix it belongs with, since permissions are read at sign-in only.
test_knowledge_base.py drives app.respond() directly — same prompt, same
knowledge base as the Space — with the SES client stubbed so a run mails
nobody. A probe fails when the real trigger logic would have sent an
unanswer…
Remove the knowledge base's internal see-also pointers
Converting the section numbers to names was not enough — the bot went on
telling people to 'see the Vendor COI compliance section in the
documentation', because the document still pointed at itself on 19 lines
and the model reproduces what it reads.
Those pointers existed for a human skimming the file, but the only reader
that matters here holds the entire text at once and never needs to be told
where to look. Removing them takes away the last thing the bot was
imitating, so an answer now either contains the information or says it does
not know.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Drop section numbers from the knowledge base's own cross-references
The bot kept ending answers with "see §22.2 in the documentation" even
after the prompt told it not to, because the document was full of exactly
that pattern and the model imitates what it reads. The 19 internal
references are now section NAMES instead of numbers, so the worst case is
the bot naming a section rather than pointing at a number nobody can look
up — and numbers no longer go stale when the document is reordered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Customers are already using the tracker — Swain and Zernco both raised COI
questions in the support inbox last month — and the knowledge base said
nothing about it beyond the settings toggle, so every one of those
questions had to go to a human.
Written against the deployed code in the coi-tracker repo (origin/main,
which the local checkout matches) and the prod web app, not from the
support replies: how to open it (the shield icon in the top nav, which
signs you in with your existing session), the two independent status
axes, which vendors are tracked and why the default is active jobs only,
"Don't track", the three ways a certificate gets in, and the 60/30/14/7
day renewal outreach with its agency grouping and customer branding.
The section that will save the most tickets is why a vendor reads Unknown
rather than Deficient. That rule is deliberate — a red chip gets forwarded
to the sub's insurance agent, so it only appears when a number was really
read and compared — but it looks l…
Stop the bot inventing detail and answering the wrong feature
Testing "tell me about PO Receipt Matching" surfaced two problems. The
removal of that feature from the knowledge base worked — the bot did not
describe the unreleased Vista capability — but it silently answered about
ordinary PO matching instead, and padded the answer with requirements that
appear nowhere in the documentation ("the invoice must reference a valid
PO number", "for subcontracts ensure Cost Type, Job Id & Cost Id is also
enabled", "enable Consolidate Line Items to simplify matching").
Two fixes:
Knowledge base — a new section names the capabilities users ask for that
do not exist (receipt-level matching, bulk delete, un-approve, full
invoice export, Foundation division sync, lien waiver tracking) and gives
the honest answer plus the nearest real feature. Grounded text is far more
reliable here than prompt pressure alone.
System prompt — accuracy is stated as the first and most important rule,
with an explicit ban on inventing prerequisites or carrying a setting over
fr…
Rewrite the knowledge base against production code
Every claim is now checked against what is actually deployed: the web app
at origin/master (prod build 5464) and the backend at origin/main. The
previous version was written from a local feature branch and from support
email answers, both of which describe behaviour customers cannot see.
Corrections:
- Drop PO Receipt Matching entirely. It exists only on a feature branch,
not on prod, so the bot was describing a toggle nobody has.
- Matching is one "Match" dropdown (Purchase Order / Delivery Ticket /
Subcontract / Work Order), not separate Match PO and Match DT buttons.
- Delivery Ticket matching needs an invoice number AND a total before the
option is even selectable — the missing-total half was undocumented and
is the usual cause of "the option is greyed out".
- Q&A can be resolved by the raiser, anyone sharing the raiser's role, or
any of Owner/Accountant/Sub Admin when the raiser is one of those. The
old "only the raiser can resolve it" was wrong.
- Disconnecting an in…
Repoints GOOGLE_DOC_ID at "Beiing Human Support KB v2.1 — FAQ Chatbot
Source (CURRENT)", which is rebuilt from the web app and backend source
and covers roles, document visibility, every company setting, all
supported ERPs, code visibility and enforced limits.
Also stops a failed fetch from silently emptying the knowledge base.
fetch_google_doc_text used to return its error message as a string, and
that string became doc_context — so any fetch failure turned every answer
into "I don't have information about that" and mailed an unanswered-question
alert to support on every message. It now returns None on failure, validates
the body (a Doc that is not link-shared answers with an HTML sign-in page,
not an error status), and falls back to knowledge_base.md committed here.
The Doc stays the source of truth so support can edit answers without a
code change; the bundled copy only covers a fetch failure.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
