Hi,
I am having trouble with pagination. My goal is to take each base URL and paginate through all available pages until there are no pages left. After that, I want to move on to the next extraction part. However, I am only getting results from the first page. Can you please help me with this?
Thank you!
$pageCount + 1
The Complete Expression shows that it is undefined.
I think thatincludes is not available on the body (should be a string).
You can try using $response.body.indexOf("StyledCard-......") === -1
An example of the URL would be helpful.
@jabbson, Please take this one: https://www.zillow.com/professionals/real-estate-agent-reviews/la-junta-co/
@FranzI figured out the issue is with the expression when complete. When I limit the fetch page, it’s working fine, but when I include the response body expression, it gives me only one page.
@jabbson figured out the issue is with the expression when complete. When I limit the fetch page, it’s working fine, but when I include the response body expression, it gives me only one page.
So the issue is solved, right?
@jabbson No, I just need that expression to stop pagination when complete.
I started looking into your issue and noticed that you are out of requests for the scraper-api.decodo.com proxy. I don’t have my own scraping proxy, so I cannot test your execution. If you wish me to continue looking into that I will need to be able to access a proxy that you are using. If you changed the header and don’t wish to share it publicly, you can send it to me as DM.
Thanks for the updates info on proxy, here is the HTTP Node, which will get all pages for ZW (I removed the authorization header for privacy).
Thank you, it worked @jabbson