Job Matcher, teaching an AI what I want to work on

The short version

Result

In a blind test it scored 27 of my 31 real applications high. Total AI cost for the build was $2.95.

What I did

Directed AI agents building a job-ad scorer, and taught it my taste with 188 rated ads.

The call

I closed it after two days, because the AI judged well but the job boards gave it too few ads to judge.

Finding the next role means reading hundreds of job ads to spot the few worth applying to. Most of that reading ends in a no, and that kind of judgement is something an AI might take over if it learns what you want.

So I tried it on my own search. AI coding agents wrote the code, as with my other builds. My part was directing the work, teaching the system my taste and checking whether it had learnt it.

In two days of sessions I had a working pipeline. It fetched ads from Arbetsförmedlingen's open API and a startup job board, took pasted ads from anywhere else and had an AI score each one on two things. Interest was whether I'd want the job. Reach was whether my application would survive screening. The first version mixed the two up, and I split them once I noticed.

Every score carried the version of the rulebook that produced it. The rulebook went through 21 versions in those two days, half from things I said and half from what my ratings revealed, and no rule changed without my yes. Total AI cost for the whole build was $2.95.

The test set was 188 job ads I rated by hand. Scored blind, 27 of my 31 real applications came out high, and three ads I'd applied to without telling the system scored 92 out of 100.

Where it went wrong was in what the AI added on its own. My veto on gambling companies turned into a rule against nicotine companies too, and a description of who I am became a penalty on jobs outside my old domains. An explain feature, which makes the AI cite the rule behind every score, caught both. It followed the mechanical rules closely and stretched anything that read like a description of me.

I gave a hyped company a no for a reason that had nothing to do with the work, and that one rating nearly taught the system a false rule about a whole category of roles. It was caught because rule changes needed my confirmation.

A comparison between two AI models picked a winner unfairly, because the old model's scores came from eight different rulebook versions while the challenger scored fresh. The version stamps showed it, and after a fair re-run the cheaper model won.

The vetoes were also firing on job titles. A product owner role at Scania scored low because the title contained "Presales". Rule 21 fixed it by judging what the job does and ignoring the title.

The scoring worked, but there weren't enough ads to score. Arbetsförmedlingen's open API gave about four good ads a week after filtering, and the startup board worked well but is small. LinkedIn and Indeed are closed to automation, and the paid boards couldn't verify their claims.

So after two days I closed it and went back to reading the boards myself, with the scorer kept as a second opinion. What I kept is a written and tested rulebook of what I want.