Check saved Apify crawl timestamps before RAG ingestion in n8n

Replaying a saved crawler batch raises a practical question: do its supplied capture timestamps still meet your ingestion policy?

I built an importable n8n example that checks those timestamps before forwarding documents to an ingestion step. It includes passing and blocked examples, with the authenticated HTTP path tested against Apify.

Disclosure: I maintain the paid Apify Actor used in this example. The workflow download has no charge, but Actor execution can incur Apify charges. Review the current pricing before running it.

Tutorial and workflow download

What it does

The workflow accepts one saved batch, prepares metadata, calls Dataset Recency & Evidence Gate, validates its report, then branches:

  • Pass: return the original documents through Ready for ingestion.
  • Fail: return diagnostics through Blocked batch diagnostics, with no document output.
  • HTTP or invalid-report error: stop before either branch.

All records must pass for that supplied batch to continue. A successful Actor run can still return a failed policy decision.

The example uses standard Manual Trigger, Code, HTTP Request, IF and Sticky Note nodes. It needs no community nodes. Its final nodes demonstrate routing; you connect your own ingestion step.

Try the two examples

  1. Download the workflow from the tutorial and import it using Import from File.
  2. Open Run Recency Actor and add an HTTP Header Auth credential: header name Authorization, value Bearer YOUR_APIFY_TOKEN. Store the token in the credential.
  3. Execute the default mixed example.
  4. Change const mode = 'mixed' to const mode = 'pass' in Demo saved batch, then execute again.
Example Total Passed Blocked Output branch
Mixed 3 1 2 Blocked diagnostics
Fresh only 1 1 0 Original document ready for ingestion

The mixed example includes one stale capture and one missing capture timestamp, with codes STALE and TIMESTAMP_MISSING. The tutorial contains both full JSON reports.

Both examples use a fixed evaluation time of October 2, 2026, 05:00 UTC and a 24-hour capture-age policy. Use your intended evaluation time and age limit for real batches.

Use your own saved items

Replace the demo node with one item containing a batch envelope:

{
  "as_of": "2026-10-02T05:00:00Z",
  "max_age_seconds": 86400,
  "dataset_id": "YOUR_DATASET_ID",
  "offset": 40,
  "items": [
    {
      "url": "https://example.com/page",
      "text": "Original saved document",
      "crawl": {
        "loadedTime": "2026-10-02T04:00:00Z",
        "loadedUrl": "https://example.com/page"
      }
    }
  ]
}

The adapter maps crawl.loadedTime to the observation timestamp and crawl.loadedUrl/url to the source reference. It creates evidence references using the dataset ID and original item offset.

Supply 1–1,000 contiguous items in original dataset order, with their actual starting offset. Do not sort or filter before making those references. The example does not download or paginate datasets. After a pass, your ingestion node can read each original document from $json.document.

Document bodies stay in n8n; this Actor receives only the prepared metadata. Metadata can still contain sensitive identifiers, and n8n execution history may retain documents according to your instance settings.

What was tested

Native n8n 2.41.5 on Node.js 24 passed the live authenticated Apify HTTP tests on October 2:

  • Invalid authentication stopped execution before either decision branch.
  • The mixed batch reached only blocked diagnostics.
  • The passing batch preserved the original document.
  • Both HTTP 201 reports exactly matched the expected reports.

Separate local fixture tests checked HTTP 500 and empty-report failures. I have not tested every n8n installation or any downstream vector-store write.

Interpretation and costs

This checks supplied observation timestamps and reference completeness against a policy. It does not authenticate evidence, verify factual truth, refresh pages or establish when the page’s contents last changed. Fetching old content today gives it a fresh capture timestamp; this adapter does not substitute that timestamp for content-update time.

A local Code node may be enough for a simple timestamp rule. This example is for evaluating whether a shared Actor with batch reports and per-record diagnostics is useful in your workflow.

Each execution makes a separate Actor evaluation, including completed policy failures. Automatic POST retries are disabled. The example requests a $0.05 Actor-charge guard, which does not guarantee an all-platform or repeated-execution spending limit. If the request times out, check Apify run history before manually retrying.

Feedback from a real batch

If this fits a workflow you already use, try one batch of your own saved metadata. Useful feedback is: did setup work, did the result change an ingestion decision, and would you use it for another batch?

You can reply here or use the Actor’s public Issues tab. Please distinguish synthetic-demo use from own-data use, and omit credentials, private documents and sensitive URLs. Reports and setup symptoms are enough; private source data is not needed.