Job Matcher, teaching an AI what I want to work on

The short version

The problem

Finding the next role meant reading hundreds of job ads to spot the few worth my time.

What I did

Directed AI agents building a scorer that predicts my interest in an ad. I taught it with 188 rated ads and a rulebook that could only change with my yes.

What it was worth

In a blind test it scored 27 of my 31 real applications high. Total AI cost for the build was $2.95.

The call

Closed after two days. The bottleneck was access to the job boards, not the AI, and manual reading wins there.

Finding the next role means reading hundreds of job ads to spot the few worth applying to. Most of that reading is rejection, and rejection reading is exactly the kind of judgement work an AI might carry if it learns what you actually want.

So I tested that on myself. AI coding agents wrote the code, as with my other builds. My part was direction, and one thing nobody can delegate. Teaching the system my taste, and checking whether it had learnt it.

A working pipeline in two days of sessions. It fetched ads from Arbetsförmedlingen's open API and from a startup job board, took pasted ads from anywhere else, and scored each one with an AI on two axes. Interest, would I want this job. Reach, would my application survive screening. I split those two after noticing the first version mixed them up, and the mix-up was mine to spot because the judgement being copied was mine.

Every score carried the version of the rulebook that produced it. The rulebook grew to 21 versions in two days, half from things I said, half from what my ratings revealed. No rule changed without my yes.

Total AI cost for the whole build was $2.95.

The ground truth was 188 job ads I rated by hand, including my 31 real applications. That gave the project something rare, an honest test. Score ads blind and see whether the ones I actually applied to come out on top.

They did. 27 of the 31 came out high, and three ads I had applied to without telling the system scored 92 out of 100.

The interesting part was where teaching went wrong. The AI invented rules I never gave it. My veto on gambling companies quietly became a rule against nicotine companies too. A description of who I am became a penalty on jobs outside my old domains. An explain feature, which forces the AI to cite the rule behind every score, is what caught both. The lesson travels well beyond this project. An AI follows mechanical rules faithfully, and creatively extends anything that reads like a personality.

Three catches made the checking worth its cost.

A hyped company got a no from me for a reason that had nothing to do with the work. That rating nearly taught the system a false rule about a whole category of roles. It was caught because rule changes required my confirmation.

A comparison between two AI models declared a winner unfairly. The old model's scores came from eight different rulebook versions while the challenger scored fresh. The version stamps on every score proved it. After a fair re-run the verdict flipped, and the cheaper model won.

And the rulebook's vetoes fired on job titles instead of job content. A product owner role at Scania scored low because the title contained "Presales". Real applications of mine were being under-scored by a word. The fix became rule number 21. Judge what the job does, not what the title says.

The judging worked. The finding did not. Arbetsförmedlingen's open API gave about four good ads a week after filtering. The startup board worked well but is small. LinkedIn and Indeed are closed to automation, and the paid boards could not verify their claims. The bottleneck was access to the ads, not intelligence about them, and no rulebook fixes that.

So after two days I closed it and went back to reading the boards myself, with the scorer kept as a second opinion. What lasts is the rulebook of what I actually want, written down and tested, and one more round of evidence that checking an AI's judgement against your own is where these projects earn their keep.