I stress tested document data extraction to its limits – results + free workflow

Tutorials English 🇬🇧
, , ,
1

Data Extraction Stress Test
Data Extraction Stress Test1920×1080 84.2 KB

:waving_hand: Hey n8n Community,

Last week I shared that I was building a stress test workflow to benchmark document extraction accuracy. The workflow is done, the tests are run, and I put together a short video walking through the whole thing – setup, test documents, and results.

:clapper_board: What the video covers:

I tested 5 versions of the same invoice to see where extraction starts to struggle:

  1. Badly scanned – aged paper, slight degradation
  2. Almost destroyed – heavy coffee stains, pen annotations, barely readable sections
  3. Completely destroyed – burn marks, “WRONG ADDRESS?” scribbled across it, amount due field circled and scribbled over, half the document obstructed
  4. Different layout – same data, completely different visual structure
  5. Handwritten – the entire invoice written by hand, based on community feedback

:bar_chart: The results:

4 out of 5 documents scored 100% – including the completely destroyed one. The only version that had trouble was the different layout, which hit 9/10 fields. And that’s with the entire easybits pipeline set up purely through auto-mapping, no manual tuning at all. The missing field could be solved by going a bit deeper into the per-field description for that specific field, but I wanted to keep the test fair and show what you get out of the box.

:movie_camera: The video:

:hammer_and_wrench: Want to run it yourself?

The workflow is solution-agnostic – you can use it to benchmark any extraction tool, not just ours. Here’s how to get started:

  1. Grab the workflow JSON and all test documents from GitHub: here
  2. Import the JSON into n8n
  3. Connect your extraction solution:
  • Using easybits Extractor: Go to extractor.easybits.tech, create a new pipeline, upload one of the example invoices, and click Auto-Mapping – it’ll detect all the fields automatically. Copy your Pipeline ID and API Key, then connect them in the easybits Extractor node. The node is a verified community node, so it’s available on n8n Cloud out of the box. For self-hosted, install @easybits/n8n-nodes-extractor via Settings → Community Nodes.
  • Using a different solution: Swap the easybits node for an HTTP Request node pointing at your API. As long as it returns the same field names under json.data, the entire validation chain works identically.
  1. Activate the workflow, open the form URL, upload a test document, and see your score

Curious to see how other extraction solutions hold up against the same test set. If anyone runs it, I’d love to hear your results.

Best,
Felix

2 Likes
I'm building a stress test workflow to benchmark document extraction – here's what I'm testing
Data Extraction with Error Handling in n8n – Catch Failures Before They Wreck Your Workflow
I'm building a stress test workflow to benchmark document extraction – here's what I'm testing
I'm building a stress test workflow to benchmark document extraction – here's what I'm testing