I am building a Chatbot for my website, i want to add a feature where the AI can help users navigate the website, how can I extract all the pages from my site and make it available to the AI to be able to guide users to navigate.
Does your website have a sitemap? That could be the simple way to feed the structure into the LLM. Otherwise, If you don’t have one, maybe you could generate one (e.g. if you use WordPress something like Yoast SEO plugin).
Another option would be to crawl all links - depends on how large your site is, that could be simple or a bit more involved. I also found this workflow Autonomous AI crawler | n8n workflow template that apparently does this kind of URL discovery, but there are many ways how to do it.
Let me know if you want more info on any of these steps 