8,3201
Tests
the checks that pin this engine's behavior
ENGINE 06
SFS-E06-TRI · AI VALIDATION · REV 2026-07-20 · PRODUCTION
PRODUCTION = runnable end-to-end, CI-backed, full test suite passing. All data fictional and seeded.
Triangulate
Separation of duties for AI work — no consensus, no sign-off
Triangulate runs independent reviewer roles plus a deterministic auditor over the same AI-assisted workpaper. When they disagree, the line stops and the work goes to a person. It is separation of duties applied to AI: an explicit authority hierarchy, not a multi-model majority vote.
A processor buyer cannot re-run the vendor's benchmark lab. Every figure on this page re-derives from the public repository — the benchmark is a command, not a press release.
Architecture
functional block stack · static overviewSelect a layer
Each layer is an independent duty. Select a layer for its plain-terms and engineering detail.
Key specifications
at a glanceWhat it does for you
plain termsNo high-stakes workpaper should rely on one AI system reporting "looks good." Models agree too easily, invent support, miss the same issue twice, and explain a number without tying it to source. Triangulate assumes those failures will happen and builds the workflow around them.
An injected hallucination is caught by the arithmetic, not an opinion7. In the adversarial demo, an AI asserts a Total Revenue $49,000 over what the revenue streams actually sum to, backed only by an 'AI assumption' — the lowest authority. Because that cell feeds tax and net, the single bad figure cascades. The Reviewer and the deterministic Auditor both flag it independently; the verdict is FAIL and the process exits non-zero.
The gate discriminates — a clean workpaper passes8. The clean sample runs the identical flow and returns PASS, proving the gate discriminates instead of always failing.
Arithmetic self-heals; judgment escalates9. The review loop re-derives the lowest broken formula cell from its own formula on a clone, then re-runs the full pipeline. The injected $49k error clears in one turn (FAIL → PASS). An AI-assumption input and a hardcoded cell have nothing to re-derive from, so they stay in the fix packet for a human (FAIL → FLAG).
Functional block diagram
engineering · each block links to its sourcePlain terms
- Preparer. drafts the workpaper
- Reviewer. challenges the draft without changing it
- Specialist. adds a narrow second opinion where needed
- Deterministic Auditor. recomputes the arithmetic independently
- Reconcile + Policy. combines findings and applies the severity policy
- Human Approval. a person signs off — or doesn't
Engineering
- Preparer. DemoPreparer builds draft; offline deterministic in the public demo
- Reviewer. read-only challenge; hash-guarded
- Specialist. scoped second opinion
- Deterministic Auditor. re-derives every formula mechanically
- Reconcile + Policy. findings reconciled; deterministic severity policy returns PASS, FLAG, or FAIL
- Human Approval. external approval boundary; the demo policy does not record or replace a human signature
Instruction set
every public command| Command | Operation | Output | Exit | Artifacts |
|---|---|---|---|---|
python -m triangulate | run the defective sample | VERDICT: FAIL [cannot sign off] | non-zero | builder memo, fix packet, change log, QA summary, verdict.json |
python -m triangulate --sample clean | run the clean sample | VERDICT: PASS | 0 | output/ artifacts + verdict.json |
python -m triangulate --demo-adversarial | inject one hallucinated figure and catch it | VERDICT: FAIL · Critical=6 | non-zero | fix packet showing cascade |
python run.py --sample clean | use the equivalent script entrypoint | VERDICT: PASS | 0 | output/ artifacts + verdict.json |
python -m triangulate --seed N | choose the deterministic fictional-data seed | selected sample verdict | verdict-dependent | output/ unless --no-artifacts |
python -m triangulate --no-specialist | skip the optional Specialist step | verdict without specialist findings | verdict-dependent | output/ unless --no-artifacts |
python -m triangulate --output DIR | write the audit trail to a chosen directory | selected sample verdict | verdict-dependent | DIR/* |
python -m triangulate --no-artifacts | print the review without writing files | selected sample verdict | verdict-dependent | none |
python -m triangulate.loop --sample adversarial | self-heal the injected error | FAIL → PASS | 0 | re-derivation change log |
python -m triangulate.loop --sample defective | attempt repair and escalate unsupported judgment | FAIL → FLAG / ESCALATED | non-zero | fix packet; optional report |
python -m triangulate.loop --seed N | choose the loop's deterministic data seed | selected loop verdict | verdict-dependent | optional report |
python -m triangulate.loop --budget N | cap remediation turns | final loop verdict | verdict-dependent | optional report |
python -m triangulate.loop --out DIR | write the Markdown loop report | final loop verdict | verdict-dependent | DIR/review_loop_report.md |
python -m triangulate --xlsx | also emit a small .xlsx workpaper | VERDICT + workpaper.xlsx | non-zero | output/*.xlsx |
python -m pytest -q | run the engine's test suite | test session summary | 0 | none |
Benchmarks
measured demo results| Measure | Result |
|---|---|
| Adversarial cascade depth10 | 6 Critical findings from 1 injected cell one bad number, six downstream failures caught |
| Self-heal turns (adversarial)11 | 1 loop turn to PASS |
| Independent detectors12 | 2 roles flag the same cell |
| Engine tests13 | 8,320 tests |
Control characteristics
engineeringPlain terms
The demo's HumanGate class applies the severity policy; an actual person remains the final approver.
Engineering
Deterministic envelope. seeded fictional inputs, read-only operation, offline default mode.
Demo gate. automated-policy
| Authority | Level | Note |
|---|---|---|
| 1 | Signed prior-year work | highest authority in the implemented enum |
| 2 | Management instruction | direct instruction from accountable management |
| 3 | Meeting decision | documented decision from the engagement record |
| 4 | Current-year source | current-period source data and evidence |
| 5 | Workbook formula | existing formula logic, subject to re-derivation |
| 6 | AI assumption | lowest authority; cannot outrank cited evidence |
| Severity | Verdict | Action |
|---|---|---|
| Critical present | FAIL | returned for rebuild; exits non-zero |
| High/unresolved judgment | FLAG | fix packet escalated to a human |
| Medium/Low only, or none | PASS | residual notes documented; eligible for human sign-off; exit 0 |
- Read-only reviewer: the orchestrator hashes the workpaper before and after review steps, so a reviewer cannot mutate what it inspects.
- Deterministic audit: every formula is mechanically re-derived; arithmetic decides, not model opinion.
- Seeded, fictional data throughout the public demo.
Enterprise-safe mode (default). deterministic mock reviewer, no API key, no network call, runnable in conservative corporate IT.
Agent-accelerated mode. where approved, a live Anthropic Claude reviewer adapter can replace the mock reviewer; an optional orchestration layer for approved, agent-enabled environments; the platform runs fully without it.
Operating limits
what it refuses to do- The loop clears arithmetic; it cannot manufacture authority. An AI-assumption input or a hardcoded cell with no formula has nothing to re-derive from — those stay in the fix packet for a human.14
- The framework does not apply changes on its own or override its own verdict; final sign-off authority is held by a person.15
- The public demo runs offline with a deterministic mock reviewer; live-model results depend on the pinned model approved for that client environment.16
See it run
the real CLI
Integration
how to run itDistribution: public repository, MIT license.
git clone https://github.com/sophonfinance-wq/finance-automation-portfolio.git && cd finance-automation-portfolio/ai-validation-framework && python -m triangulate
python -m triangulate --sample clean
Substantiation
- 1python -m pytest --collect-only -q (ai-validation-framework/); verified 2026-07-20 and pinned in site-datasheets/counts.json
- 2ai-validation-framework/README.md — Preparer, Reviewer, Specialist, Deterministic audit, Policy gate + human approver
- 3ai-validation-framework/README.md — verdicts and exit behavior
- 4ai-validation-framework/README.md — "exits 0 on PASS and non-zero on FLAG/FAIL"
- 5ai-validation-framework/README.md — "default demo needs no API key and no network access"
- 6ai-validation-framework/README.md — "All sample data is fictional and seeded"; read-only reviewer
- 7python -m triangulate --demo-adversarial (ai-validation-framework/README.md)
- 8python -m triangulate --sample clean (ai-validation-framework/README.md)
- 9python -m triangulate.loop --sample adversarial / --sample defective (ai-validation-framework/README.md)
- 10ai-validation-framework/README.md — "Critical=6" in --demo-adversarial
- 11ai-validation-framework/README.md — "Turn 1 re-derive B5 … PASS"
- 12ai-validation-framework/README.md — "flagged independently by two roles"
- 13python -m pytest --collect-only -q (ai-validation-framework/); verified 2026-07-20 and pinned in site-datasheets/counts.json
- 14ai-validation-framework/README.md — review-loop section
- 15ai-validation-framework/README.md — human gate
- 16ai-validation-framework/README.md — two operating modes
Show us where the hours go.
One conversation: you describe the work that consumes your team's month; we tell you plainly what this engine can take over, what it can't, and what a scoped first phase would cost. Your people keep approval authority.
Book a free consultation