Skip to content

Home Testing

The document check: tested hard, not yet proven.

taxflow is the document check Arbren built for tax offices. What it was tested on, what broke, and what is still unproven.

Glossary

01What it was tested on

Not yet run on real documents

Every number on this page comes from documents we made ourselves.

taxflow was tested on tax documents we generated and on simulators of Drake Tax and ProConnect written from the vendors' documentation. Nothing has run on a real client's documents or inside the real programs, so no figure here is an accuracy rate for your office, and nobody has measured time saved.

The document check in the sample plan

02The rule every test holds it to

A box is read exactly as the page prints it, or it is left blank and flagged for a person. It is never filled with a different number.

A blank costs a preparer a minute. A wrong number that looks right can end up on a return. So blanks are counted on their own, never inside an accuracy figure.

  • Read as printed

    Matches the page, digit for digit, in the right box.

  • Left blank and flagged

    The reader was not sure, so a person is told. Still work for someone.

  • Read wrong

    A different number, or the right one in the wrong box. The one result the rule forbids.

How it holds to that
  • Every printed amount has one owner, decided by where it sits on the page. A box takes an amount only when its own label is the owner, so an empty box can't borrow its neighbor's.
  • Every number on a scanned page is read twice, the second time at another size, and must come back the same digit for digit. Otherwise the box stays blank.
  • A brokerage statement must add up to every total the statement itself prints, or the whole statement is blocked for a person.
  • Every value that came from a scan is marked "check" on the keying sheet, so the reviewer compares it with the page.

03The latest runs

Forms drawn at random by a generator, in layouts nobody designed, one page in eight a crooked scan.

amounts on 8,000 generated W-2s and 1099s
32,9890 read wrong
checkboxes, codes and other boxes that are not amounts, on 4,000 more forms
18,4960 read wrong
generated brokerage statements
1,0000 read wrong
automated tests
219All passing

Solid: read as printed. Hollow: left blank or held for a person. A wrong result would be red; there is none to draw.

  • Amounts on typed pages

    Generated W-2s and 1099s

    93% read as printedThe rest left blank0 read wrong

  • Amounts on scanned pages

    The same forms, as crooked scans

    Three quarters read as printedThe rest left blank0 read wrong

  • Brokerage statements

    1,000 statements in random layouts

    982 read exactly18 blocked for a person0 read wrong

  • A messy season, start to finish

    60 simulated clients: envelope PDFs, portal zips, sideways phone photos

    56 returns tied out to the cent4 held for a person, all scans0 tied out wrong

  • A plain season, start to finish

    40 simulated clients, on 2025 forms and again on 2026 forms

    40 of 40 tied out, both years0 held0 tied out wrong

04What independent review found

31more faults, found by three rounds of independent review after the random tests showed none wrong

After the random tests showed none wrong, independent reviewers built layouts by hand to break the reader, each sent at the last one's fixes. Every round found about ten, and the third found as many as the first.

All 31 are fixed, and each is now a permanent test. A fourth round would find more.

  • A second column of amounts

    A form that prints "this period" beside "year to date" moved every box down a line.

  • Nonqualified read as qualified

    "Nonqualified dividends" was read as qualified dividends.

  • A code beside an amount

    A distribution code of 7 printed next to 500.00 was read as seven thousand five hundred.

Far safer than it was, and not proven. The test that counts is real documents, every value checked against the page.

What the earlier tests and reviews broke
  • Amounts in the wrong box: 291 of 1,434 amounts put in the wrong box on the first run, 0 of 32,989 read wrong on the latest.

    The first run on layouts nobody had designed found the most serious fault in the project: an empty box took its neighbor's amount. The reader was rebuilt so each amount has one owner.

  • Checkboxes and codes: 13 of 192 read wrong on the first run, 0 of 18,496 read wrong on the latest.

    Checkboxes, the 1099-R distribution code and W-2 box 12 were still read the old way: an empty checkbox took its neighbor's X. They are read by ownership now, and box 12 is all or nothing.

  • Brokerage statements: 7 of 200 read wrong; 53 read exactly on the first run, 0 of 1,000 read wrong; 982 read exactly, 18 blocked on the latest.

    A statement is now read by its own column headings and checked against every total it prints. A layout it can't read is blocked, and that block can't be clicked away.

  • A blank that told no one

    The rule says "blank and flagged". Looking at what blank really did showed that a box was only flagged if it was a required one: withholding printed in a box whose label was not recognized was simply missing from the keying sheet. Now everything printed and not read is reported. A box with something printed in it that could not be read blocks the return and opens the page beside it.

  • Documents built to be misread

    An adversarial review built documents meant to be read wrong without being blocked. It found 9 ways to put a wrong number on a return and 6 ways to lose a document, plus holes in identity checks and locking. All are fixed, and the attack documents are now tests held to the same rule: read exactly right, or blocked.

  • The keying sheets, read from the preparer's side

    A review of the sheets found a long-term sale that would have imported into ProConnect as short-term, sales that should not have taken the no-8949 route, tie-out lines that could never tie, and questions that stated the law too flatly. All fixed. The simulators now check short-term against long-term, which is how the next one of those gets caught.

05Known limits

Open any one for the detail.

The box reader has been broken by review three times

Expect real layouts to find more. What protects a return is the order of things: blank and flagged before wrong, every unread amount listed, and a person checking the sheet against the page.

Scans

Windows' text recognition garbles some labels and breaks some numbers. Those come out blank and flagged: about a quarter of the amounts on generated scans, and 4 of 60 clients held in the messy season. It also drops things outright, most often a lone X in a checkbox, and nothing can flag what it was never given, so on a scan the checkboxes that came out empty are named for the reviewer.

The same misread, twice

A digit misread the same way by both readings of a scan can still get through. The one seen so far, 3,409.11 read as 3,409.41 beside a scanner's speck, is now caught. The next may not be, which is why every scanned value is marked "check".

A blank that stays quiet

A whole-dollar amount printed as a bare number, with no comma, cents or dollar sign, that is not read is not reported either.

Two forms on one page

A separate check catches this and blocks the page, but the reader itself will fill a blank box on one form from the other.

Brokerage statements

Tested on layouts we drew and on a review's attack cases, informed by four brokers' published samples. Not yet run on a real statement; expect some to be blocked for a person. Options trades, Section 1256 contracts and collectibles are flagged, not handled.

K-1s

Page-one boxes only. No footnotes, state K-1s, basis schedules or Section 199A detail; a question is raised for the preparer instead.

Papers that are not forms

Estimated payments, charity receipts, mileage and childcare are recognized and named for hand keying, not read.

State and business returns

State wages and withholding are carried; nothing state-specific is checked. Business returns (1120-S, 1065) are not covered.

Tax year 2026 forms

The 2026 layouts are built from the IRS instructions. The real 2026 forms arrive in January.

Windows, and a shared office folder

Scans need Windows, because it uses Windows' text recognition. Each person runs it against a folder on a file server or hosted desktop, never a synced OneDrive, SharePoint or Dropbox folder.

06Not proven yet

The pilot has not happened

What no test on this page can show.

Real client documents
None have been run. Real payer and broker layouts will find faults that generated ones did not.
Drake itself
The import file matches Drake's own template but has never been loaded into a real install. Until it has, the file carries UNTESTED in its name.
ProConnect itself
The same: built to Intuit's template, never uploaded to the real program.
The 2026 entry screens
Neither vendor had published its tax year 2026 input details when this was measured.
Time saved
Not measured, so not claimed.

What the pilot will be

Our own tax documents first, then last season's documents for 20 clients of one firm, every value checked against the page. Its numbers go here when it has run.

Want work checked this hard?

A build for your office gets the same treatment: tested until something breaks, fixed, and handed over with a written list of what is not proven.