Scraping Workflow

Questions
1

Hey everyone,

I’ve built a workflow that’s mostly working fine — it scrapes job listings from LinkedIn, generates content, and posts automatically to WordPress using the HTTP node. Everything’s looking good: the posts are being published with formatted text and images.

However, I’m facing two main issues that I can’t seem to solve:

  1. Featured image – I’d like the post’s featured image to be one related to the company that’s hiring, instead of the generic one being used now.

  2. Duplicate job posts – the workflow sometimes repeats the same job listings even though I’m trying to filter for new ones only.

    If anyone has dealt with something similar or would be open to taking a look and brainstorming a clean solution, please reach out or comment here. I’d really appreciate any help or insights!

1 Like
2
  1. It doesn’t sound easy because not every company has the feature image about hiring. Even if you use AI to search for one, it might produce irrelevant or misleading images.
  2. Should be easy to solve if you save a list of the jobs already post maybe in data table or google sheet then you can filter the one already post.
3

That’s true. I think I’ll focus on setting up a option 2.