From paralegal, bookkeeper, or analyst to domain AI evaluator

What happened to the job

Document review, discovery, reconciliation, first pass research, and report drafting were among the earliest tasks to go, because they are high volume, text shaped, and easy to benchmark. The roles built on them thinned out accordingly.

Why you are already good at this

A model writing about a lease, a ledger, or a filing is plausible to everyone except someone who knows the domain. That is why domain evaluation pays above general labeling and why it cannot be handed to a generalist: the error is invisible unless you have done the work. Your professional knowledge is the scarce input, and a career gap does not erase it.

• Checking model output in your field for errors a generalist reviewer would pass
• Applying a harm, compliance, or accuracy taxonomy to borderline output
• Writing evidence backed findings a reviewer outside your domain can act on
• Escalating genuinely ambiguous items instead of forcing a judgement

Your track

Take them in this order. Every one is free, and the certification at the end is marked by a person.

1. 01

   AI Data Annotation Foundations

   About 4 hours · 20 lessons

   The vocabulary, rubrics, and quality habits every lab expects.

2. 02

   Passing AI Annotation Assessments

   About 5 hours · 24 lessons

   Why capable people fail entry tests, and how to stop.

3. 03

   RLHF and Model Evaluation

   About 9 hours · 32 lessons

   Preference data, output comparison, and safety judgement.

4. 04

   AI Evaluator Certification

   About 30 hours across the track · 22 lessons

   Every course, plus an assessed capstone and a certificate.

Roles on the board right now

Listings that match this pathway. Applications go to the employer, not to us.

Read this first

• Data annotator salary

  Hourly pay for labeling, RLHF, and evaluator work, based on ranges listed on this board.

• How to become a data annotator

  What the work is, which skills get you hired, and how to apply through this board.