Scrape query from serper api than add airtable and ZEP

I first create the user chat entry as an airtable record.
I search this entry with In the json result, I separate the organicsearch, relatedsearch and peoplealsoask directories with the split out option and handle them with different loops.

In this result, the sitelinks section is a separate directory and I check if it exists or not.
I exclude Youyube results.
I process the records in the Airtable sitelinks table.
I waited 4 seconds, then I processed the user input on the sitelink record. I did not use the patch method to pull this page content here. I added it to the ZEP vector database I had previously set up, using the user input metadata. The interesting thing here is that when I asked open AI to retrieve the company name from an open AI record, I got the response that it was unethical, so I couldn’t do it. The sitelinks directory comes up empty in basic searches.

People also ask
What people searched for in this search and what results they got
scrnli_23-02-2024 19-16-38

Again, I add the record to a separate airtable table, wait a while and process the user input. I use metadata when adding to the ZEP vector database.
scrnli_23-02-2024 19-22-01

Thinking that these results will be useful to me, I add them to the airtable and the ZEP vector database.

Since Open AI would be an expensive solution for embedding in the ZEP vector database, I chose the HuggingFace BAAI/bge-small-en-v1.5 model. This model uses 384 embedding dimensions.

With the Default Data Loader option, metadata entry is very simple with no-code n8. By adding Token Splitter it works fine with the following settings.

I installed n8n and ZEP on docker with Oracle cloud free tier. Huggingface and openAI token required.

1 Like

Hey @Eray_Bartan,

Welcome to the community :raised_hands:

Do you have a workflow to share with everyone? It sounds very interesting.

1 Like

Hi John i didnt share my workflow cause self hosted free version.


Thanks for sharing this, Eray! If you’re comfortable giving this away, you could share the workflow on the Creator Hub as well: n8n workflow templates

You can sign up here to get started.

Right it would be great!

1 Like

i did create one and waiting for review approve.

Thanks for interest.

1 Like