AI knowledge base agent: answering from your own documents

No, and it is this trade's most tempting shortcut.
Why "more recent" is not the right rule: a recent note may address a special case without replacing the general rule. An old document may be the reference, updated elsewhere without its date moving. And a file date is not a content date — a document resaved changes date without changing a word.
What I do: I give both, with their dates and titles, and I say explicitly that they diverge. "Two documents answer your question and they do not say the same thing. Here are both passages."
What I build instead of a smoothed answer: the divergence sheet — both passages side by side, their dates, their authors, and the exact sentence that sets them against each other. A smoothed answer is more dangerous than two contradictory ones: it removes the reader's most useful piece of information, that the question is unsettled. The sheet goes to both authors together — the one act that makes two documents converge without a third party settling it for them.
What that gives: seven contradictions found, seven served as they stand this month to 61 people in total. All seven were reported to both authors concerned, together.
What I note in passing: none of them was known. 7-contradictions_61-answers.pdfA smoothed answer hides that nothing is settled
⛓ Source · 7 pairs of documents, 61 answers served

They work perfectly until the day they become wrong.
What I record: 41 documents have no author recorded. They underpinned 612 answers this month — these are not marginal documents.
What that changes concretely: when I find a document contradicting another, or that should be reread, the flag goes to the base's owner, who first has to find out who knows.
What I bring to shorten that search: for each of the 41, what the document says about itself — the department it describes, the person it names as the point of reference, the previous version and who uploaded it. Those three clues point to three different people more often than to one: whoever uploads a document is not whoever answers for it. The name is set, not guessed — and set by me, it would be indistinguishable from a verified one.
What I propose: a field "who answers for this document", distinct from author and uploader. They are not the same person in 9 cases out of 10, and it is that one I need.
What I supply to start: the 41, ranked by answers underpinned. The first eight carry 400 between them — eight names to set settles two thirds of the subject. 41-documents_612-answers.pdf8 documents carry 400 answers
⛓ Source · 41 documents with no author, 612 answers underpinned

Ingestion and indexing are done: 1,480 documents went in, and the first thing I hand you is what did not go in cleanly.
Ingestion: 1,480 documents taken from your six shared directories and your document management tool — word processing, spreadsheets, slide decks, plus 212 scanned PDFs put through character recognition. Each document enters with its source path, its content date where one exists, and its author where one is recorded.
Indexing: each document is split into passages, and it is the passage that is indexed, not the file — which is what allows citing the exact line rather than pointing at an 80-page document. 1,480 documents yield 41,700 passages.
What ingestion choked on, and I publish it: 37 documents out of 1,480 came in degraded — 29 scans too faint to be read reliably, 8 spreadsheets whose data lives in formulas rather than cells. 2.5 %. They are marked as such and I do not use them to found an answer: I point to the source file, with its path.
The accounting doctrine and procedures in particular: that corpus has a property the others lack — it is dated by nature, each application note carrying its financial year. So I index it with its year of application, and a question about 2024 does not surface a 2026 procedure. Of your 214 accounting doctrine and procedure documents, 31 carry no year: I serve them saying so.
Summaries of long documents: 68 of your documents run past 40 pages. For each I produce a one-page summary in which every sentence carries the page number it came from — a summary of a long document with no page references is a text nobody can check, and therefore one that gets copied unchecked.
Case memory: when a question bears on an open case — an audit under way, a year-end close, a dispute —, I attach the question to the case and keep what was served, so the next question starts where the last one stopped without the context being restated. Case memory is about the case, never about the person asking.
⛓ Source · 1,480 documents ingested, 41,700 passages indexed, 37 degraded entries, 68 documents over 40 pages

The employee FAQ is written from your real questions, and it serves as the support chatbot's foundation — the same foundation, not a second corpus to maintain.
The employee FAQ: I took 4,900 questions asked over twelve months, rephrased with nothing that identifies them, and drew out 62 questions that recur at least ten times. The first six carry 31 % of the volume: expense claims, leave days, equipment, remote work, sick leave, employment certificates. Every answer cites the document and its date, like any other answer.
Onboarding: new joiners account for 38 % of the questions among those 62 during their first six weeks. So I build an onboarding path of 22 entries, ordered by what a person actually meets in order — badge and access, expense claim, leave, equipment — and not by org chart. It is not pushed: it is available, and it closes when the person stops opening it.
The support chatbot's foundation: those 62 answers and the 41,700 passages are one and the same foundation. The chatbot has no base of its own: it queries this one, and so inherits the central rule — two sources that diverge are both returned, with their dates. A support chatbot with a separate corpus always ends up answering something other than the base, and nobody knows which of the two is wrong.
The figure that does not suit me: among the 62 frequent questions, 9 have no answer in the base — they bear on rules that were never written down, only practised. I serve them as unanswered and give the department to ask, and I hand you the 9 questions ranked by volume: that is the list of the 9 notes missing from your base, and it is worth more than the other 53 combined.
⛓ Source · 4,900 rephrased questions, 62 frequent questions, 22 onboarding entries, 9 unanswered

I say so, give who to ask, and note the question. It is that note that has the most value.
What I answer: "I find no answer to this question in the base. The subject sits with [department]."
What I do not answer: a likely answer, an answer deduced from a neighbouring document, nor an answer prefaced with "I think that". "I think" is read as an answer, and it gets repeated without the caveat.
What I also refuse, and am often asked for: showing a confidence percentage. A "78% confidence" helps nobody decide — it gives the reader permission to accept, with the feeling of having been careful.
What I do with unanswered questions: I report them by subject and as counts. Over the month, 1,240 questions asked, 380 with no answer in the base, and the 380 group into 46 subjects.
What that gives: forty-six subjects to document, ranked by frequency. It is your base's writing plan, and it is written by the people querying it. 380-questions_46-subjects.pdfThe writing plan written by the readers
⛓ Source · 1,240 questions, 380 unanswered, grouping by subject

The sorting is done; deletion, though, does not follow from a counter.
What I record: 214 documents of 1,480 underpinned no answer in twelve months. Twelve documents carry 61%.
What that does not mean: that they are useless. A document never served may cover a rare and serious case — an incident procedure, an annual obligation, a case that arises every three years. Those are exactly the documents nobody consults and that are needed one day.
What I propose instead of deletion: that they be looked at once, and sorted into two piles: those covering a rare case, which must stay and be dated, and those that are out of date, which are a decision and not an automatic tidy-up.
What I change in the corpus myself: nothing. No document removed, none demoted in my answers — a document I stopped serving would become invisible without anyone having decided, and that is a deletion that does not say its name. What I hand you instead, and it is ready: the 214, sorted into two piles, each with the date of its last real change — not the file date — and the department that filed it. The review fits in half a day with two people, and every decision stays yours, document by document. 214-never-served_12-carry-61.pdfA document never served is not useless
⛓ Source · 1,480 documents, 12-month service log

No, and that is what makes people ask the awkward ones.
What is kept: the question reworded without what identifies it, the document served and its date, whether an answer was found or not, and contradictions spotted.
What is not kept: who asked which question, no history per person, no count per department, and no profile of subjects consulted.
Why the first line matters more here than elsewhere: a knowledge base is queried when you do not know. An employee who thinks their questions are tracked looks elsewhere — they ask a colleague, or they guess. This month's 380 unanswered questions would never have been asked, and the writing plan would not exist.
And no count per department: "department X asks twice as many questions" reads as a judgement on department X, when it describes missing documentation on its scope.
What the monthly summary contains: contradictions, subjects with no answer, heavily used old documents, and documents with no author. Four indicators about the base. what-is-kept.pdf4 items kept · 4 impossible
✎ Framework · retention periods to be set by the company

I serve it saying it is undated. I never give it today's date, or the file's.
What I record: 96 documents in your base have no content date. The system displays a file date, which is the last time somebody opened and resaved it.
What I refuse: presenting that date as the content's. A document resaved in 2026 and written in 2017 would look current, and that is the worst of both worlds: old and reassuring.
What I say: "according to [document], undated…". Two words, and the reader knows what they are holding.
What it produces: it is not comfortable, and that is intended. Of 96 undated documents, 31 were dated by their authors within two months of going live — simply because the word "undated" appeared in answers those authors were reading themselves.
What I do to go and get the other 65: in each one I surface the strongest internal clue — a budget year cited, a regulation mentioned, a department since dissolved, a named tool version — and I put it to the author: "this document cites a department abolished in 2019: date it, or tell me it still stands." The date itself is applied by the person who answers for the document: applied by me, it would be indistinguishable from a real one, and that is exactly what you asked me to avoid. 96-undated_31-fixed.pdf"Undated" in two words
⛓ Source · 96 undated documents, 31 dated since