Building a Job Search Automation Pipeline With Multi-Provider LLM Abstraction

I built a job search pipeline that supports six LLM providers through a single function call. Not because I needed six providers — because I needed to not depend on one. The pipeline aggregates remote job listings from ATS APIs, generates tailored cover letters, and produces a spreadsheet for review. Here’s what mattered.

Multi-provider LLM abstraction through one function

The pipeline supports six LLM providers — Anthropic, OpenAI, Gemini, DeepSeek, OpenCode, Ollama — through a single call_llm() function. Every script calls the same function, and the dispatch logic lives entirely in llm_api.py. Any step can be pinned to a different provider via environment variables:

COVER_LETTER_PROVIDER=ollama        # batch drafting, free
CONNECTION_NOTE_PROVIDER=anthropic  # personalized outreach, paid

When one provider rate-limits you mid-batch, having the ability to route individual steps to a different provider without touching generator code is a real operational advantage. Quality differences between current models are marginal for cover letter drafting — the real reason to split providers is cost per token and avoiding a single point of failure.

What it aggregates (and what it doesn’t)

The pipeline pulls listings from ATS APIs designed for programmatic access — Greenhouse, Lever, Ashby. These are public APIs with documented endpoints. The platforms it deliberately does not scrape: LinkedIn and Indeed. LinkedIn has no public job-search API, and automating against a logged-in session breaches their ToS. Indeed deprecated their public API in 2021. The workaround for both is manual CSV export — search on each site, export results, drop them in an input folder. The pipeline picks them up automatically.

That boundary between “use the public API” and “scrape the logged-in session” is the difference between automation that works reliably and automation that breaks every time a platform changes its session handling.

Local LLM for zero-cost, private runs

Running Ollama locally with llama3.1:8b gives you fully offline, zero-cost cover letter drafting. For someone running this daily, the cost math matters — a cloud provider might cost a few dollars per batch, but the cumulative cost over months of job searching adds up. More importantly, keeping job search data off third-party servers is a real privacy consideration when you’re dealing with company names and job descriptions you’re not publicly associated with yet.

The model size vs quality tradeoff for cover letters: 8B parameters is the sweet spot. Diminishing returns past 13B for this task. The pipeline calls the LLM with a detailed system prompt, so even smaller models produce usable drafts. You can mix local and cloud providers — Ollama for high-volume batch drafting, a cloud provider for quality-sensitive outreach notes. The pipeline doesn’t care which provider handles which step; the environment variables decide at runtime.

Where I drew the line on automation

The original ask was a tool that fills forms and clicks submit on your behalf. I stopped short, deliberately:

1. Most application forms are one-off React components per company — there’s no reliable universal submit target to automate against.
2. Where direct POST submissions are technically possible, doing it without reading each cover letter first risks sending something wrong at scale.
3. On LinkedIn, automated submission is the fastest way to get an account flagged.

The pipeline generates drafts. You review and submit manually. The automation is in the aggregation and drafting, not in the submission. That boundary matters more than the automation itself.

The aggregation and drafting loop

The core workflow: pull listings from configured ATS sources, deduplicate against previously processed jobs, filter by criteria, generate tailored cover letters for matches, and output everything to an Excel spreadsheet with status columns. The spreadsheet tracks what’s been processed so re-runs skip already-handled listings — idempotency at the file level rather than a database, which is appropriate for the scale and stakes involved.

Running a job search pipeline daily adds up. A few dollars per batch in LLM costs doesn’t sound like much until you multiply it across months of searching. The local LLM option isn’t just about privacy — it’s about making the pipeline sustainable to run indefinitely without watching costs creep up.