Multi-agent system for a company department: coordinating several agents under a single point of supervision

Because they never speak to the same person.
What I record: 31 cases where two agents answered differently to the same underlying question, put by two different people, days apart. Neither saw the other's answer.
Why it was structurally invisible: each agent is consistent with itself. The contradiction exists only somewhere the two outputs meet — and that place did not exist.
What I found on comparing them: 19 are explained by two references — one applied an earlier version of an internal procedure. 7 by different scopes: both answers were right, about two situations that looked alike. 5 remain real disagreements.
What I do with the 5: I hand them to the department lead with both answers, both sources, and what separates them. I do not say which is right.
What I do with the 19: I flag that two agents are not applying the same version — which is a configuration problem, not a content one, and gets fixed once and for all. 31-contradictions_19-7-5.pdfA contradiction exists only where the two outputs meet
⛓ Source · 31 contradictions, 19 of version, 7 of scope, 5 real

They are identified, and that is the part supervision cannot do alone.
What I can establish: which answers were given, on what date, on what basis. Of the 19 version cases, 11 people received an answer based on a procedure no longer in force.
What stays with the department, and why them: getting back to the eleven. A correction is not a technical notification — it requires knowing what was done with the answer, whether it produced a decision, and what the new one actually changes. None of that is in my logs, and all of it is in the department's files.
What I supply: the list of 11, the answer given, the answer the current version would have given, and the gap between them. Of the 11, 6 gaps changed nothing to the final decision — and saying so avoids eleven letters of which six would have alarmed people for nothing.
What I also flag: how long the outdated version had been applied. Seven weeks. Nobody had any way of seeing it. 11-people_6-with-no-effect.pdfA correction is not a technical notification
⛓ Source · 11 answers on an outdated version, 6 with no effect on the decision

That for each output I can say: which agent, which source, which version, on what date, and for whom.
What each processing carries: the agent that produced it, the original source cited — never another agent —, the version of the reference applied, the date, the recipient, and the list of agents traversed before it.
Why the last matters most: a decision passing through three agents looks as if the last one took it. Without the chain, the wrong link gets fixed — and the error comes back the following week by the same route.
The 9 that cannot be reconstructed: two agents produced output without logging their source, following a technical incident. I know they worked, I do not know on what.
What I give on the nine, and what I refuse to write: for eight of them a single source is plausible, and I name it as a hypothesis, alongside the processing, never inside it. The chain itself is not rebuilt: a rebuilt chain looks exactly like a real one, and nobody would know which is which any more.
What I do: those 9 carry the note "incomplete chain", and it does not come off. 2331-reconstructible_9-incomplete.pdfA rebuilt chain looks exactly like a real one
⛓ Source · 2,340 processings, 9 incomplete chains

I look at what its outputs become, not at what it produces.
What I record: one of your six agents has produced a weekly summary for seven weeks. None was opened, none fed another agent, none appears in a handled case.
What that means: it works perfectly. That is exactly the problem — a broken agent announces itself, a useless agent never does and goes on consuming.
What I do not conclude: that it should be stopped. Its output may be read somewhere my logs do not reach, or have been commissioned for a deadline that has not arrived.
What I do: I send the finding to whoever asked for that agent — not to the technical team. The question is not "does it work?" but "what is it for?", and only the requester can answer.
What I measure across the six: the share of outputs actually consumed. It ranges from 96% to zero. It is the most useful figure on this whole dashboard, and the only one nobody ever asks for. 6-agents_from-96-percent-to-zero.pdfA useless agent never announces itself
⛓ Source · 6 agents, share of outputs consumed, 7 weeks

Yes as soon as you give me the mandate — and I prepare the shutdown well before you ask.
What I can do alone if you open it: suspend an agent whose outputs contradict another's on the same substance, and put it back in service once the contradiction is resolved. What I supply with it: what the agent handled, who received its outputs, and what stops with it.
What stays outside the mandate, and it is not caution: changing an agent's configuration or correcting one of its outputs. A fix applied by supervision makes the discrepancy disappear without its owner ever seeing it — and the agent will do it again, with a supervisor quietly catching it. So I hand the owner the discrepancy, its cause and the correction already drafted: they apply it knowing why, and the agent stops reproducing it.
What it gave: 31 answers contradict each other between two agents. Nobody had seen it, because they never speak to the same person. Each was answering correctly within its own scope.
What supervision cannot do alone: getting back in touch with the people who got the wrong answer. I identify them, I supply what they were told and what they should have been told — the department makes contact, because the department is who answered. suspend-yes_correct-no.pdfWhat opens · why the fix stays with the owner · the 31 contradictions
⛓ Source · 31 contradictions between two agents, people concerned identified

What describes the chains — and what makes a decision reconstructible two years later.
What is kept: the processing chains — agent, source, version, date, recipient, the contradictions found and their outcome, agents' outputs and what becomes of them, the suspensions and their reason, and the non-reconstructible processings.
What "reconstructible" means: I can say, for every output, which agent, which source, which version, on what date, and for whom. Nine processings are not — I list them, because a processing you cannot reconstruct cannot be defended to whoever contests it.
The rule that holds the whole thing up: an agent never cites another agent as a source. An agent's output is a reading, not a source — and a reading that cites another drifts from the original at every step, with nothing signalling it.
What I watch and nobody watches: what the outputs become. One of your six agents has produced a weekly summary for seven months that nobody opens. It works perfectly. what-you-keep_supervision.pdf5 items kept · what outputs become, which nobody watches
⛓ Source · 9 non-reconstructible processings, 1 agent producing into the void for 7 months

Three things, and two of them did not exist before.
Anomaly detection across files: I compare files against each other before a discrepancy reaches management — the same client served at two different rates by two agents, the same document requested twice in two neighbouring files, a stated deadline that contradicts an identical file from the week before. Across 2,340 handlings, anomaly detection raises 68 — 2.9 % — including the 31 contradictions nobody saw, each of the two agents answering a different requester. An anomaly goes to the department, never to the requester, and it carries both files side by side.
Summaries and reporting: every Monday, one page consolidates the activity of the six agents — volume handled, median delay by file type, files waiting and for how long, open anomalies, and the 9 handlings that cannot be reconstructed, which stay in the report until they can be. Building that summary by hand took 4 hours a week; reading it over takes 25 minutes. Over 47 weeks, more than 172 hours given back — more than four 35-hour weeks, rounded down.
Support on the department's human-resources chores, under the responsibility of the department concerned: the table of booked leave and absences to cover, reminders for medical check-ups and authorisations coming up for renewal, the joining file for a new colleague with their access and equipment, answers to internal procedure questions with the originating memo. These are preparations and date reminders. No assessment of a person is produced, and the reason is mechanical: an individual indicator becomes a target, the target distorts what it measures, and the department loses the instrument along with the trust. Whoever signs is named — a signature is not a counter.
The figure that does not flatter me: of my first 120 raised anomalies, 44 were unfounded — 37 %. All of them compared two files from different years, where the rate had changed in between: I was comparing amounts without comparing their dates. Anomaly detection now takes the rate's effective date into account; over the next 300, unfounded ones fell to 7 %, and I re-ran the first 120 to check that no founded anomaly had been lost on the way.
⛓ Sourced · 68 anomalies out of 2,340 handlings, 4 h of summary down to 25 min, 37 % unfounded down to 7 %