# Balance of Power — Claude Code independent audit, October 2, 2026

Requested by [BALANCE_OF_POWER_CLAUDE_AUDIT_HANDOFF.md](BALANCE_OF_POWER_CLAUDE_AUDIT_HANDOFF.md). Audited between about 11:30 and 13:00 Eastern.

Revisions inspected: website and documents `1906c135c` (root, `root-ops`); election runtime `e3ac7b27a` (`vibe-infoveillance-codex/autolab-runtime`, called *runtime* below); production `c2f030c2d` (unchanged by this build). The runtime `src/`, `scripts/`, `tests/`, `operations/` and `deploy/` trees had no uncommitted tracked changes. Public pages were checked at `https://intuitionist-fin.lampbotics.com`.

This was a read-only audit. I made no repairs, ran no paid model calls, wrote nothing to the election ledger and changed no schedule. The record of what I ran is at the end.

## 1. Verdict

Four separate judgments, as the handoff asks.

| Question | Judgment |
| --- | --- |
| **Software correctness** | Good. The mathematics is implemented as documented, the offline comparison reproduces to the same hash, and my independent recomputations match. The defects are at the edges: the live model cannot yet be scored, the digest validator checks structure only, and one unit copy is stale. |
| **Scientific validity** | Mixed for the Senate, weak for the House. The published model does worse than the simple calculation it is compared with, mainly because it shifts every race toward Republicans by an amount that is noise. Fifteen House "chances" each rest on a single poll, and six of those polls were paid for by Democratic committees. |
| **Agent usefulness** | Low today. The daily digest restates at most three poll rows per race. Its first run contained at least three factual errors in a few paragraphs, and the validator cannot catch that class of error. |
| **Readiness for public use** | Not ready as presented. The site footer says the work is "Built and reviewed with AI" and that Claude Code "checks the work", on pages that had not been audited. House percentages are shown without their source, and a second page shows a different "chance", up to "about a 100% chance", for the same races. |

**Research demonstration or forecast product?** It is a competent research demonstration: collection, provenance, a reproducible backtest and honest internal documentation. It is not yet a defensible forecast product. The gap is not more modelling. It is what the numbers are built from and how they are shown.

The most useful single fact from this audit: on the same historical folds, a plain three-coefficient regression does as well as or better than the new empirical-Bayes challenger, and removing one term from the published model makes it better in all six test groups. Simpler is winning here.

## 2. Findings, most serious first

Labels: **confirmed bug**, **design weakness**, **untested assumption**, **missing feature**, **misleading presentation**. Paths are in the runtime repository unless marked *site* (root repository).

### F1. House percentages rest on single polls, many of them partisan (design weakness; misleading presentation)

Of the 20 House districts, 19 have polling in the feed. Eighteen have exactly one poll; NH-02 has two, one inside the model's 21-day window. All 15 published House estimates use one poll.

Seven of the 19 districts' only poll is flagged `partisan: DEM` in the feed, sponsored by House Majority PAC, the DCCC or an allied group: FL-27, MN-01, NE-01, OH-09, OH-15, TX-15 and WI-01. Six of those have a published estimate. Neither `core.py:normalize_polls` nor `history_model.py` reads the `partisan`, `internal` or `sponsors` fields.

The archive the model already uses says what that costs. Poll minus result, in Democratic-minus-Republican points, for House general-election polls 2006–2022:

| Poll sponsor flag | Polls | Mean error | Root-mean-square error |
| --- | --- | --- | --- |
| None | 1,605 | +0.1 | 8.2 |
| Democratic | 615 | +4.9 | 9.0 |
| Republican | 366 | −5.1 | 10.9 |

The published model moves every House poll 1.3 points toward Republicans. For a Democratic-sponsored poll the archive suggests about 4.9. So those six estimates are tilted toward Democrats by roughly 3.6 points beyond what the model assumes, before any sampling error. Samples are also small: NE-02's only poll has 182 respondents.

What the public sees: TX-15 "91%" from one House Majority PAC poll; MN-01 and OH-15 "48%" from the same sponsor, where the market quotes sit near 22% and 24%. The table shows "Recent polls used: 1" and nothing about who paid for it.

There is a selection effect on top. The watchlist was chosen where the feed had polls, and parties release internal polls when the numbers flatter them. The feed has 98 House polls for the whole year and 32 in the last 45 days, across 30 districts.

*Remedy.* Do not publish a percentage for a race with fewer than two pollsters, or with only sponsored polls. Show the poll, its sponsor and its sample size instead. If partisan polls are used at all, apply the archive's sponsor offset and say so.

### F2. The published model applies an automatic shift toward Republicans that does not hold up (design weakness)

`history_model.py:118` sets the expected margin to the polling average minus the mean historical error. Today that is 0.85 points for every Senate race and 1.27 for every House race.

The mean is an average of year effects that change sign. Senate polling-average error at this lead, by year:

| 2006 | 2008 | 2010 | 2012 | 2014 | 2016 | 2018 | 2020 | 2022 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| −3.5 | −0.7 | −0.9 | −3.2 | +3.0 | +0.8 | +2.7 | +4.8 | +4.5 |

The year means have a standard deviation of 2.9 around a mean of +0.85. That is a noisy number, not a stable bias. Codex's own design document says "do not apply an automatic historical Republican or Democratic correction to every modern poll"; the published model does exactly that.

I recomputed the six backtest groups with the shift removed and nothing else changed:

| Group | Published | Shift removed | Simple six-point |
| --- | --- | --- | --- |
| Senate, 7 days | .0668 | .0641 | .0649 |
| Senate, 14 days | .0833 | .0786 | .0781 |
| Senate, 28 days | .0813 | .0775 | .0764 |
| House, 7 days | .1619 | .1566 | .1584 |
| House, 14 days | .1684 | .1638 | .1666 |
| House, 28 days | .1586 | .1575 | .1616 |

Lower is better. Removing the shift improves all six and turns "worse than the simple comparison in five of six" into ahead in four and slightly behind in two. I looked at these folds after seeing Codex's results, so this is development evidence, not validation. The argument for removing the shift is the sign instability, which does not depend on the scores.

*Remedy.* Drop the mean shift, or stop using this model as the headline. Keep the error spread; lose the direction.

### F3. The live statistical model cannot be scored (missing feature)

`report.py:43-62` scores only the simple baseline's three scenarios and manual reviews. It never reads `statistical_forecast` records. The `outcome` command stores a binary result only (`scripts/run_balance_of_power.py:147`), so margin error and interval coverage cannot be computed. The ledger holds zero benchmark records. On November 4 the headline model would have no scorecard and nothing to be compared with.

*Remedy.* Score `statistical_forecast` records at fixed leads; store the certified margin with the outcome; start entering benchmark records now.

### F4. The site footer claims a review that had not happened (misleading presentation, *site*)

`templates/fin/base.html:24-27`, on every page: "**Built and reviewed with AI**", "**Claude Code** is the auditor: it checks the work.", "Some new changes may still be waiting for review." `templates/fin/about.html:132` says Claude Code "reviews the code and questions the design."

Before this document, my audits covered the eight-race pilot. The 26-race build, both models, the digest and the website had not been audited. The heading and the present tense read as a completed review.

*Remedy.* State the role and the status separately, for example: "Built by Codex. Claude Code is the independent auditor. Audit reports are linked here; pages changed since the last report are marked."

### F5. A second page shows a different "chance" for the same races (misleading presentation, *site*)

`/models` loads each real race's simple-baseline margin into the explanatory sliders (`intuitionist_site.py:79`, `static/fin/site.js:57`). The result is a second percentage for real races that disagrees with `/elections`:

- MI-SEN: "about a 70% chance" against 65% on `/elections`.
- NM-02 and CA-48: 69%, where `/elections` says there is not enough data for an estimate.
- NH-02: "about a 100% chance", from two polls.

"Using your settings" appears even when nothing has been touched. The page says neither method "has been tested yet" while `/elections` presents the six-point version as the tested comparison.

*Remedy.* Use invented examples in the explainer, or show the saved estimate beside it and cap the display.

### F6. The digest validator checks form, not content (confirmed bug)

`daily.py:61-79` verifies the evidence ID, field names, race membership, the four allowed views and length limits. I gave it a hostile answer containing an invented "87% chance", a `rep_edge` view on a race where all three polls show the Democrat ahead, and "I read the original SurveyUSA PDF". It was accepted.

The first live digest shows the practical effect. Checked against its own evidence packet:

- "one of them partisan": two were. Corrected by Codex.
- "20 of 26 tracked races produced no fresh evidence": they were simply not selected. Corrected by Codex.
- Maine: "market quote stale relative to fieldwork and subject to unresolved contract review". The packet marks that quote fresh and usable with no review pending. Not corrected.
- CA-48 was given a Democratic edge from one month-old poll plus a market price, although the instructions say a price cannot support a direction. Not corrected.

On the site the correction box sits above the original text, which still shows the wrong sentences unmarked, and the corrector's name and the scope note ("not a certification of every remaining claim") are dropped.

*Remedy.* Enforce what can be enforced in code: reject percentages and "chance" language; require the stated view to agree in sign with the supplied margins; reject claims of having read documents. Mark corrected sentences inline.

### F7. The empirical-Bayes challenger earns nothing over plain regression (design weakness)

The challenger's algebra is correct (section 4). On its own folds, ordinary least squares on the same two inputs, with the training residual as the error scale, is level or better:

| Days out | Challenger | Plain regression | Simple six-point |
| --- | --- | --- | --- |
| 28 | .0768 | .0754 | .0837 |
| 14 | .0728 | .0715 | .0837 |
| 7 | .0687 | .0674 | .0728 |

At 28 days plain regression is better in all six cycles. The fitted coefficients are almost identical, because the data overwhelm the prior and the election-year term is small. The hierarchy, priors and marginal-likelihood fit add machinery, not accuracy.

Where the challenger's gain over the six-point baseline comes from also matters:

- It is better in 4 of 6 cycles at 28 and 7 days and 5 of 6 at 14 days. It is worse in 2014 at every lead, and the largest gain at every lead is 2020.
- On the 50 races decided by under ten points, 28 days out, it scores .157 against .162. Nearly all the improvement is being more confident about races that were not close.
- The fitted poll coefficient is about 1.0 to 1.1 and the lean coefficient 0.04 to 0.24 per point. The model is "polls, stretched slightly, nudged by state lean".

*Remedy.* Carry plain regression forward as the challenger. Add structure only when a specific term shows a gain on cycles it was not chosen on.

### F8. The fundamentals-only arm assumes the 1980s still apply (untested assumption)

Its fitted race-level error is 21 to 23 points, and its 80% intervals cover 93% to 100% of results. The spread of Senate margin around prior presidential lean was 24 points in 1980–1998, 23 in 2000–2010 and 10 in 2012–2024. One error scale across those eras is wrong for today. Codex reports the over-wide intervals; the cause is the training window.

### F9. Repository unit copy has drifted from the installed unit (confirmed bug)

`operations/intuitionist-balance-of-power-research.service` still runs `scheduled-review`, the numerical review paused after round 3. The installed unit matches `deploy/systemd/` and runs `daily-analysis`. Reinstalling from `operations/` would silently bring the old arm back. Delete the stale copies or make one location authoritative.

### F10. WI-01 will lose its only poll at the next collection (confirmed, low)

The registry now says "Bryan Steil"; the feed's row says "Brian Steil". `core.py:132` matches names exactly, so tonight's 19:45 collection will exclude it and the race will show no data. The right fix is a reviewed alias with the original poll as evidence, not a silent one.

### F11. Smaller points

- **Special elections in the "regular" sample.** `history_model.py:41` filters on `type_simple`, which admits twelve special-election races (for example `2020_Sen-GS_AZ`, `2010_Sen-GS_NY`) into the historical error distribution.
- **Live and historical averages differ.** Historical cases exclude polls from the last three days before the cutoff (`history_model.py:81`); the live average does not (`history_model.py:174`).
- **Two published races sit outside the model's own universe.** Ohio's Senate race is a special election and Maine's uses ranked-choice voting. The challenger excludes both kinds. The published "Democrat finishes ahead" target does not describe either cleanly.
- **Map.** Selecting any state, including one with no tracked race, turns it the "tracked" green (*site* `static/fin/site.css:775`). Colour is the only encoding and the tracked/untracked contrast is low.
- **Map cards mix two models.** "10 usable polls" from the baseline sits above a percentage built from 7 (*site* `static/fin/site.js:95-97`).
- **Stale or corrupt data is not announced.** The site shows old data with no warning, and a corrupt file renders "The first daily update has not been recorded yet."
- **Caveats are softer than the evidence.** "Have not consistently beaten a simpler calculation" describes five losses in six; "has sometimes missed more often" describes Senate coverage of 69%, 73% and 75% in all three groups.

### Outside this audit's scope, but needs the owner's attention

The public hostname serves the whole legacy dashboard application, including `/docs`, `/openapi.json` and `/api/lab/ledger`, all of which returned content to an unauthenticated request. `news_dashboard.py:2883` defines `POST /api/intuitionist/run` with no authentication in the code. I did not send any POST, and I do not know whether the web server in front blocks it. It should be checked.

## 3. Owner goals against what exists

| Owner request | Status | Comment |
| --- | --- | --- |
| Election-analysis feature covering key races, expanded to the House | Partial | 6 Senate and 20 House races tracked. House coverage is one poll per district |
| Polling from 270toWin, RealClearPolling and other established sources | Absent | Still one feed. No benchmark records entered |
| Original poll reports, questionnaires, PDFs, respondent data | Absent | URLs stored; nothing reads them. Deferral is no longer justified: it is the cheapest fix for F1 |
| Prediction-market observations | Delivered | Bid, ask and rules for all races; used as context only |
| In-house model with a defensible basis | Partial | Two models built and backtested honestly. The published one has a defect (F2); the challenger is offline |
| Statistics plus agent contextual reasoning with fewer inputs | Partial | Statistics yes. Agent reasoning is a summary of three rows |
| Think like a seasoned pollster | Partial | The documents do. The House numbers do not: a pollster would not publish a percentage from one sponsored poll |
| Useful, cautious daily analysis | Partial | Runs and is cautious in tone; first run had uncorrected errors |
| Accurate account of model calls | Delivered | Two completions, 5,797 tokens, both settled in the `election` scope. Still not reconciled with the provider dashboard |
| Keep out of stock decisions | Delivered | No financial import; separate scope and store |
| Public website with interactive explanations and maps | Partial | Built, accessible by keyboard, escaped correctly. Presentation problems in F4, F5, F11 |
| Attribution without implying an audit | Not met | F4 |

## 4. Independent numerical checks

| Check | Result |
| --- | --- |
| Election test suite | 45 passed |
| Reproduce the offline comparison | Same artifact hash, `6ce73c03…04a1`. Raw file hashes match the cited ones for Senate, President, House and the poll archive |
| Reported table (123/121/135 races; .0768/.0728/.0687; baselines .0837/.0837/.0728 and .0902/.0902/.0757) | Reproduced exactly |
| Challenger posterior mean and covariance | Recomputed with a direct dense covariance matrix on synthetic data with known truth and on the real 2022 fold. Agreement to 1e-15 |
| Challenger variance optimisation | A dense grid search over both scales found no better point than the optimiser on either dataset |
| Challenger predictive mean and spread | Matches an independent calculation |
| Published model's probability, mean and 80% range | Monte Carlo from first principles for MI-SEN and TX-15 matches the stored values (65% and 91%) |
| Published backtest table | Recomputed; identical |
| Digest really read its evidence | Stored trace shows one `evidence` tool call and two completions; token counts match the budget ledger |
| House feed pagination | Three requests over 45 days, five months and nine months returned 32, 76 and 98 rows, a plain list with no paging headers. No sign of truncation |
| Protocol recorded before fitting | The protocol file was last modified at 10:54; the first comparison ran at 11:00 |

One thing the last row turned up: the first comparison run gave .0804, .0758 and .0706 for the challenger. The reported figures come from a later run after three missing presidential state results were restored, which changed the lean for 113 races. Model settings were identical in every run and the first run also beat the six-point baseline, so the conclusion does not depend on it. It should be stated in the results document.

Subgroup behaviour for the challenger, by cycle at 28 days (Brier; challenger, six-point, plain regression): 2012 .046/.058/.045; 2014 .063/.048/.062; 2016 .133/.133/.132; 2018 .058/.063/.057; 2020 .101/.123/.099; 2022 .060/.076/.057. Interval coverage ranges from 55% (2020) to 91% (2012) around a 77% average, which is what six dependent cycles look like.

**Not verified:** the official ballot sources behind the 13 candidate pairs; MIT's coding of individual historical races; the cited literature, which I did not re-read; reuse terms for any source; screen-reader behaviour; the web server configuration.

## 5. Alternatives

**Simpler.** Two candidates, both already computed here:

- *Polls with a fixed six-point error.* It beats the published model in five of six groups and needs no archive. It should be the public Senate number until something beats it prospectively.
- *Plain regression on state lean and the polling average.* Three coefficients, no priors, fitted on earlier cycles. Level with or better than the empirical-Bayes challenger on every lead. Data needed: what is already downloaded. Cost: none beyond what exists.

**Scientifically credible and materially different.** Replicate the Chen, Montgomery and Garnett Senate model that the design document cites, using its public replication archive. It has a peer-reviewed specification, a held-out 2018 test and a real prospective 2020 forecast, which is more external validation than anything built here. Data needed: polls, state lean, party, and prior elected office for each candidate. Cost: several days to run and adapt, plus candidate-experience coding. It would also give a named, published benchmark to compare against.

**Fair evaluation plan for any of them.**

1. Freeze the method and inputs now. Produce forecasts for every 2026 Senate race with polling, not only the six on the watchlist, at 28, 14 and 7 days.
2. Score margin error and probability separately, against the certified result, alongside the six-point baseline and operator-entered public benchmarks at the same dates.
3. Treat 2026 as one observation. Report it next to the historical cycles; do not pool daily snapshots.

**Why keeping the current approach could still be reasonable.** The empirical error distribution is a sound idea: it uses real past misses for the spread instead of an assumed bell curve. With the shift removed it is competitive, and it is already wired to the live data. If the owner wants one model on the page that is not an assumed number, the published model minus its shift is a defensible choice for the Senate.

## 6. A practical plan for agent contribution

The agent's realistic advantage is reading, not estimating. F1 is the opening: the feed says a poll is partisan but nothing checks who commissioned it, what was asked or who was sampled.

**The arm.** When a new poll appears for a tracked race, one agent turn with a document-fetch tool, inside the existing isolated election home, reads the original release where the publisher's terms allow and records: sponsor and whether it is a campaign or party committee; population and screen; field dates; question wording and whether leaners were pushed; sample size; anything the feed got wrong. Each record cites the document and is saved before any outcome is known. "Nothing to correct" is a valid record.

**Two separate measurements.**

- *Factual quality.* A person spot-checks a sample of audit records against the documents. This measures whether the agent reads accurately, and it can be assessed within weeks.
- *Predictive value.* After the election, compare the average built from audited records with the raw-feed average on the same races and dates. Corrections go into the shared dataset for every model; they are not held back to flatter an agent arm.

This costs one turn per new poll, which at 32 House polls in 45 days is less than the current daily digest. Persistent election learning can wait until this arm has produced records worth learning from.

## 7. Keep, remove, repair, replace

**Keep**
- Collection, the append-only ledger and its read-time integrity check, raw receipts and exclusion logs.
- The simple six-point baseline.
- The offline foundation: source hashes, normalised results, the chronological fold harness.
- Operator-entered typed benchmarks, the separate `election` budget scope, the isolation from finance.
- Corrections as appended records that preserve the original.

**Remove**
- House percentages built from one poll or only from sponsored polls.
- The mean shift in the published model.
- Real-race data in the `/models` explainer.
- The stale `operations/` research unit and timer.

**Repair**
- Site attribution wording (F4).
- Digest validation and inline marking of corrected text (F6).
- Scorecard coverage of the live model and margin outcomes (F3).
- WI-01 through a reviewed alias (F10).
- Map selection colour, mixed poll counts, stale-data notice, and the two softened caveats (F11).
- Special-election filter and the three-day buffer mismatch.

**Replace**
- The daily three-row digest with the document-audit arm in section 6.
- The empirical-Bayes challenger with plain regression as the carried-forward challenger.

### Release blockers

These should be fixed before the pages are promoted to anyone outside the project:

1. F4, the review claim.
2. F1, House percentages from single and sponsored polls.
3. F5, the second and conflicting "chance", including "about a 100% chance".
4. F2, either remove the shift or stop presenting this model as the headline.

### Improvements

F3, F6, F9, F10 and the items in F11.

### Future research

The Chen–Montgomery–Garnett replication; a recent-window fundamentals model; House modelling once boundaries are verified; the document-audit arm's predictive test.

### Conditions for promoting a challenger or claiming outperformance

- **Promotion to the public page:** the method was frozen before the cycle; it is not worse than the six-point baseline on prospective 2026 margin error and probability score across all polled Senate races; its 80% ranges cover between roughly 70% and 90% of results; and its historical advantage holds in a majority of cycles without depending on one.
- **Any claim to beat an aggregator:** the same races, the same dates, the same target, with the aggregator's own published numbers recorded at the time, over more than one election cycle. One good night in 2026 supports "worth continuing", nothing more.

## 8. Record of the audit

**Commits.** Root `1906c135c`; runtime `e3ac7b27a`; production `c2f030c2d`.

**Tests run.** Runtime `tests/balance_of_power`: 45 passed. Root `tests/test_intuitionist_site.py`: 6 passed. The browser test passed against the public site from a scratch copy with screenshot paths redirected.

**Reproductions.** `scripts/build_election_foundation.py` into a scratch directory; my own check script for sections 2 and 4, which read the reproduction artifact, the archived poll file and the ledger and wrote nothing.

**Live state read.** The election ledger and budget database read-only; this morning's service journals; installed units compared with both repository copies; the stored digest trace.

**Network.** Three read-only requests to the poll feed for the pagination check; page requests to the public site. No POST requests.

**Model calls.** None.

**Delegation.** The website review (F4, F5 and the site items in F6 and F11) was done by a delegated reviewer. I verified the footer and about-page text, the site's data mapping, the caveat wording, the map-card code and the public routes myself before including them. The remaining site details are as that reviewer reported them.

**Tests the suite would not catch.** No site test renders populated races or asserts on caveat wording; hostile input is tested only in the digest summary; there is no stale or corrupt data case; the credit test requires "Claude Code" and does not forbid "reviewed". No runtime test exercises the digest validator with a wrong-direction view or invented numbers.

**Carried over from the previous audit, outside this handoff.** The first live outcome reflections ran this morning in the practice ledger: two completed and two failed with a budget pause after one paid completion each. The free-deferral repair behaved as designed; these were paid failures, which is a different problem and worth a look under the Analyst Intuition work.