Agentic Job Pipeline: How I build automations, solve my own use case, but design it so the same pipeline solves someone else's too

[Michael_Frostbutter]

Hey everyone,

Any problem I have now, my first move is to automate it, and n8n is usually the first tool I reach for. But there is a second question I ask every time, before I build anything: how do I turn this one use case into something reusable? I do not want a one-off script that solves today's problem and nothing else. I want a pipeline I can point at the next problem too.

So when I landed back on the job market and decided to automate my job search, I built it from the start to be two tools in one. Point it one way and it is a job-hunting tool for me. Point it the other way and it is an intelligence tool for a recruiter. Same engine, different intent. Here is how it works, and how that dual purpose is baked into the design rather than bolted on after.

The pipeline, end to end

• Every morning at 7am a schedule trigger fires. There is also a webhook and a manual trigger for on-demand runs.
• A LinkedIn scraper I built (packaged as an Apify actor) pulls the roles I care about, by title, location, and work type.
• The results get normalized into one clean job schema, deduplicated, and scored against my rubric and a set of golden examples of the jobs I actually want. Title match, seniority, remote or hybrid, salary floor, company size, plus negative signals that kill the score for roles that are not a fit.
• Everything lands in Postgres and shows up on a small front end. Over my coffee I sit down to one screen with every new role scored and ranked, and I apply from there.

How it is built

• Schedule / Webhook / Manual triggers into a Config Code node. One source of truth for search terms, locations, thresholds, and safety caps.
• Code node splits the config into individual search queries, with a run guard and a per-window budget so I can never accidentally hammer the scraper.
• HTTP Request node fires each query at the Apify actor synchronously and gets the dataset back.
• Code nodes flatten the raw results, normalize the fields, dedup within the batch, and score every job.
• Postgres does the persistence. A Postgres node checks a seen_jobs ledger, an IF node splits new from already-seen, and new jobs get written as records. Duplicates skip silently.
• HTTP Request node hands the run off to my agents, and a final Code + HTTP pair posts a run summary to Slack.

Every node has a sticky note explaining what it does, so the canvas reads like its own documentation.

What I actually wake up to is a digest in Slack: how many jobs got scraped, how they broke down by tier, and which application packages the agents built overnight. If nothing new came in, it stays quiet.

Two decisions that made it actually good

1. Postgres instead of a Airtable. I started on Airtable and hit the free-tier row cap fast, since the dedup ledger keeps every job ID I have ever seen. Moving fully onto Postgres removed the ceiling and made the whole thing cheaper and faster. seen_jobs is the dedup ledger, a jobs table holds the working records, and the scores and status fields on each row drive the views on the front end.
2. A human-in-the-loop build gate. The scrape and scoring run automatically, but I do not build an application package for every job that comes in. That would be expensive and a little ridiculous. Instead the front end lets me pick the roles I actually want, and only those get built. It keeps my API spend tied to intent.

The agents

The build step hands off to self-hosted Docker SDK agents running Claude. A smaller model (Haiku) reviews, scores, and tags each new job. When I select the ones I want, a stronger model (Sonnet) writes the resume and cover-letter package for those and saves them locally. Secrets inject at runtime from Infisical, nothing is hardcoded. The model is a config value, so this is swappable if you prefer a different provider or a local model.

The front end (do not judge me on it)

I am the only person who uses this, so I built it functional, not beautiful. It is deliberately low-tech. The whole UI is a single HTML file with no build step and no framework. Alpine.js handles the reactivity, Tailwind comes straight off the CDN for styling, and marked plus DOMPurify render the job descriptions and the generated packages safely. No npm, no bundler, nothing to compile.

It is served off the same FastAPI backend that runs the review and build side, at one route, and it talks to a small REST API (/jobs/api/\*) behind a bearer token. All the data lives in Postgres, so the front end is really just a thin sortable, filterable view over the jobs table, with per-row toggles to flag what I want built next. That is the whole point of it. It took an afternoon, there is nothing to maintain, and it does the one thing I need: let me triage over coffee and press go on the roles worth building.

Built for two jobs from day one

This is the part I care about most, and it was the plan, not an accident. The workflow is not really a job-search tool. It is a scrape, score, human-review, agent-draft engine. My job hunt is just one set of filters and criteria bolted onto that engine.

Swap the filters and it becomes a recruiting firm's search and outreach tool. Instead of scoring roles for me to apply to, it surfaces aging job postings as pain signals (a role open a long time is a company that needs help). Instead of writing my application, the agents draft personalized, warm outreach to win that company as a client. Same triggers, same scrape, same scoring skeleton, same human-in-the-loop gate, same agent handoff. I designed those pieces to be intent-agnostic on purpose, so pointing the whole thing at a different goal is a config change, not a rebuild.

That is how I approach every automation I build. Solve my own problem first, but build it so the same pipeline solves someone else's too. You end up with tools, not scripts.

The whole thing is open source (MIT): the two workflows, the agent backend, the front end, the Postgres schema, and setup docs. github.com/Mfrostbutter/agentic-job-pipeline

If you have built something similar, or you want me to break down any specific piece (the scoring node, the run guards, the agent handoff), I am happy to go deeper in the thread.

Cheers,
Michael

[nguyenthieutoan]

The dual-purpose framing is what makes this actually useful long-term. The Infisical + Docker agent setup is a pattern worth documenting on its own - keeping secrets and agent definitions out of the n8n workflow means the pipeline can be handed off or shared without leaking credentials. The Postgres deduplication ledger is also a much more reliable approach than storing state inside n8n itself, especially once you start scaling to more job sources.