N8n AI Agent: How to implement reliable pagination with persistent search context in a chat workflow

Hi everyone,

I’m building a chat-based article search workflow in n8n using the AI Agent + HTTP Request and I’m running into issues with pagination and conversation state handling.

Setup

  • Chat Trigger (user input)

  • AI Agent (handles intent + builds search filter)

  • HTTP Request node (OData API with $filter, $top=20, $skip)

  • Memory node (to store context like lastFilter, skip, totalCount)

What works

  • Initial search works correctly

  • API returns correct @odata.count

  • First 20 results are displayed properly using $top=20

The problem

When the user types something like “yes / more / next” to get additional results:

  • The system rebuilds the filter instead of reusing the previous query

  • The AI Agent seems to forget the previous search context

  • Pagination breaks because $filter gets recalculated instead of reused

  • Instead of continuing the same search, it sometimes treats “yes” as a new query

What I want to achieve

  • First query → return first 20 results

  • Show: “25 results found, here are the first 20”

  • Ask: “Do you want the next results?”

  • If user says “yes”:

    • reuse the exact same filter

    • increase $skip by 20

    • return next results

  • Continue until all results are shown

:red_question_mark: My question

What is the best practice in n8n to handle this kind of:

  • conversational pagination

  • persistent query state across chat messages

  • preventing the AI Agent from rebuilding the filter on follow-up messages

Should this be handled in:

  • the AI Agent prompt?

  • workflow logic (IF / Set nodes)?

  • or external memory/state handling?

Any recommended architecture or example setup would be really helpful.

welcome to the n8n community @Leon22
I would keep pagination state (lastFilter, skip, totalCount) in workflow logic or persistent chat memory keyed by the chat sessionId, and make the AI Agent only classify whether the user is asking for “new search” or “next page”, because n8n’s docs recommend sharing the same memory between Chat Trigger and Agent with the same session ID, and every chat message runs the workflow again so relying on the prompt alone is not enough for reliable pagination.

Yeah so the trick here is to not even let the agent touch pagination requests, because it’ll just reinterpret “yes” as a new query every time. Put an IF node right after your Chat Trigger that checks if the input matches stuff like “yes”, “more”, “next” and if it does, skip the agent entirely and go straight to your HTTP Request with the stored filter and an incremented $skip. Something like this should work as a starting point:

EDIT - Idk what happened with the workflows and why it got ? on both nodes. It must be since I convert the JSON from self-hosted to cloud format.

Thanks for your suggestion!

Unfortunately, I’m not able to place a node between the request and the AI Agent in my current setup. The only option I have there is to insert a Human Review step, but that doesn’t really help in this case.

Here’s my current workflow structure (see screenshot).

I was thinking about adding the HTTP nodes after the AI Agent instead, but I’m not sure if that would work — wouldn’t those nodes then miss the filtered/processed data from the Agent? Or am I misunderstanding how the data is passed along at that point?

Would really appreciate any clarification on this :folded_hands:

Hi, @Leon22

If the HTTP node is supposed to fetch the next page of results, placing it after the Agent can be problematic because by then:

  • the Agent may have already rewritten the query,

  • the original pagination command may be lost,

  • the state may no longer be clean,

  • the data passed downstream may not reflect the actual continuation intent.

If the HTTP node is after the Agent, the Agent may become a bottleneck for a task that is not really reasoning-heavy.

The cleaner pattern is:

  • detect pagination first,

  • then route directly to the HTTP request,

  • using stored state.

Hi everyone,

I just wanted to share a quick update regarding my setup, in case it helps others working with similar use cases.

I was able to solve the issue by letting the AI dynamically generate both the $filter and the $skip parameters directly inside the HTTP Request node.

With a well-defined prompt directly for the filter and the skip that clearly explains:

  • which fields are allowed

  • how the OData filter should be structured

  • and how pagination (skip) should behave

…the AI reliably builds valid queries on its own.

So instead of hardcoding logic or handling pagination externally, everything is now controlled through the AI instructions inside the node — and it works quite well.

Thanks again for your support and ideas, it definitely helped me get to this solution!

Best regards