How to use AI to extract invoice data safely

The model must never write. It reads the document and proposes line items, a person confirms, and only the confirmation creates the purchase invoice and moves stock, cost prices and the supplier balance. That choice matters more to a finance team than the model's accuracy.

On the ERP we built for a regional trading and distribution group, supplier invoices used to arrive on paper and be re-keyed by hand into both the ledger and a stock spreadsheet, and the two disagreed constantly. Now someone photographs the invoice and a vision model reads the line items. The extraction appears on screen for review and nothing has been posted. The same system keeps drafts and invoices still under review out of revenue, purchases and outstanding totals, so a proposed extraction cannot distort a figure before anyone has agreed it.

This turns a data-entry job into a review job. Checking a pre-filled screen should be quicker than typing, though you should time it on your own documents. It also means a misread is caught by a person before it reaches a balance, instead of surfacing at month end.

Check structure, arithmetic and identity before anything reaches a reviewer. A model asked for structured output will now and then return something nearly the right shape, and showing that to a reviewer spends human attention on a machine fault.

• Structure: define the expected fields and types, and parse the response against them. A response that does not conform is a retry, not a result.
• Arithmetic: quantities multiplied by rates should sum to the stated total, within whatever tax and rounding rules you set. When they do not, flag the whole document rather than one field, because the model may have misread the total instead of a line.
• Identity: a supplier name that nearly matches an existing supplier should offer the match instead of creating a duplicate. The same applies to documents seen twice. On the fleet-telematics order system, parsed orders are matched against existing rows so a resend updates the row rather than creating a second one.
• Source: keep the original image beside the extraction, so a doubtful field is checked against the document on the same screen.

A worked example of the arithmetic check. An invoice has three lines: 4 at 12.50 is 50.00, 2 at 30.00 is 60.00, and 10 at 1.75 is 17.50, a sum of 127.50. If the model returns a total of 172.50 (two digits transposed), the sum fails and the document is flagged before review. Without that check, the reviewer would see three plausible lines and a plausible total.

The pipeline should carry on: rotate across several API keys per provider, and fall through to a second provider if the first fails. Rate limits, exhausted quota and authentication errors are ordinary operating conditions, and they tend to arrive when volume is highest.

In the trading ERP the scanner holds multiple API keys per provider. The starting key advances on a schedule. On a rate-limit, quota or authentication failure it moves to the next key, then falls through to the second provider before giving up. With no keys configured it says so plainly instead of failing silently.

• Rotate keys rather than pinning one, so a revoked, exhausted or misconfigured key does not stop the pipeline. Rate limits are generally applied per organisation or project rather than per key, so extra keys in the same account may not add capacity; check how your provider scopes its limits.
• Exercise the second provider with real documents. Failover nobody has tested is a setting, not a capability. Two providers will not read a document identically, which is another reason the validation layer sits in front of both.
• Back off on transient errors and cap the retries, so a request that can never succeed does not loop.
• Report an unconfigured system visibly. An empty extraction caused by a missing key looks exactly like a document with nothing to extract.

Treat the model as a setting, not an architecture. If extraction, validation, human confirmation and write-back are separate stages, you can change the model behind the first stage without touching the other three.

Match the model to the difficulty of the job. Classifying or routing a document can often be done by a smaller, cheaper model, while reading a creased photograph taken in poor light may need a stronger vision model. Test both on your own documents. Model lists and pricing change often, so check each provider's current pages when you choose.

Keep an evaluation set: a few dozen real documents, the worst ones included, with the correct answers written down. Run it before and after every prompt or model change and compare the results field by field. Without it, an upgrade is a guess. With it, an upgrade is a measurement.

The document image or text goes to the provider's API, so decide deliberately what that includes. Send only what the extraction needs, redact what is irrelevant, and read the provider's current data-use and retention terms instead of assuming them. Anthropic and OpenAI both state in their own policy pages that data sent through their commercial APIs is not used to train their models by default, with opt-in exceptions, and both retain some data for a period (OpenAI describes abuse-monitoring logs kept for up to 30 days by default, with zero-retention options for eligible customers). Consumer chat products and other providers can differ, so check your provider's and plan's current terms, and again when they change.

Treat the source images as business records. Keep them in private storage with access policies, attached to the document they produced. In the trading ERP every write goes through one data-access layer that stamps the acting user and appends a field-level change to an append-only table, so a confirmed extraction can always be traced to the person who confirmed it.