Hi all,
I want to create a RAG chatbot trained on my confluence spaces which contain hundreds of pages. I have several nodes that query confluence, as confluence limits queries to 100 pages per query. Then I have to clean any html and anything else that cannot be sent via a plain text json payload to Chatbases’ API (via their update a chatbot). Their payload via the API allows for 4,000,000 characters, which I am well under.
-Please let me know if there is a better way to do the flow.
-Also, as I increase the page limit in each query, the workflow suddenly won’t work, as I believe too much data is being sent via the last node (json payload). So, I have toyed with sending them in batches but this is where I am getting tripped up.
-is there a way to run loops on a certain space, 100. 200 and so on so I don’t need so many query nodes?
Thanks in advance!
It looks like your topic is missing some important information. Could you provide the following if applicable.
- n8n version:
- Database (default: SQLite):
- n8n EXECUTIONS_PROCESS setting (default: own, main):
- Running n8n via (Docker, npm, n8n cloud, desktop app):
- Operating system:
Hi @boscocho
Since you’ve only shared a screenshot of your workflow I cannot see your nodes configurations.
But some ideas that I had:
- make use of Sub-workflows to handle memory better (see more on this here: Cloud data management | n8n Docs)
- you can then set the
Wait for Sub-Workflow Completionto separate out the individual calls
Tip for sharing your workflow in the forum
Pasting your n8n workflow
Ensure to copy your n8n workflow and paste it in the code block, that is in between the pairs of triple backticks, which also could be achieved by clicking </> (preformatted text) in the editor and pasting in your workflow.
```
<your workflow>
```
Make sure that you’ve removed any sensitive information from your workflow and include dummy data or pinned data as much as you can!
How are you triggering this workflow? Is there a need that you have to query into all 3 Confluence APIs? Could you use an IF node on the data that is being passed into your workflow to route which Confluence API to call?
It would be helpful if you could give a bit more context on the use case of what you’re trying to achieve here 
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.