Weaviate Vector Store: use server-side batching for large inserts

Feature Requests Nodes
1

The idea is:

Add a “Batch Mode” option to the Weaviate Vector Store insert. “Auto” uses Weaviate server-side batching when the server supports it (v1.36 or later), and else uses the current insert.

The options are:

  • Auto (default): use server-side batching if the server supports it. Else, use the current insert.
  • Server-Side: use server-side batching only. The node fails with a clear message on Weaviate older than 1.36.
  • Default: the current insert, with no change.

My use case:

I insert large document sets. Today the node sends fixed-size batches and does not know the server load. Large inserts can time out or put too much load on the cluster.

I think it would be beneficial to add this because:

With server-side batching, Weaviate controls the rate, so large inserts are more reliable. “Auto” keeps older servers working without a change.

Any resources to support this?

  • PR #43523 in the n8n repository: github.com/n8n-io/n8n/pull/43523

Are you willing to work on this?

Yes, the PR above implements it, with tests.