Data annotation job
In-House Counsel (Australia)
Mercor · Posted 4 weeks ago
- AI evaluation
- LLM training
- Legal
About this role
Mercor is partnering with a leading legal AI company to benchmark how well AI systems answer real questions of Australian law. We are hiring experienced in-house counsel to serve as expert reviewers, running the same prompts through two AI platforms, comparing the answers, and scoring them against a standardized rubric.
What you'll do
- Run the same set of prompts across two AI (LLM) platforms
- Compare the outputs from both platforms side by side
- Score each response against a standardized rubric we provide
- Submit concise written feedback for every evaluation
Requirements
- 5+ years as in-house legal counsel practising in Australia
- Strong command of Australian law, especially commercial and contract matters
- Sharp attention to detail and precise written English
- Able to work independently to a rubric and to deadlines
Logistics
- Initial commitment: ~10 hours, with strong potential for more
- Confidentiality: all work is covered by NDA
This listing is aggregated by DataAnnotationJobs.org. Applications are processed by Mercor, not by this site.
Mercor is an AI-native talent marketplace connecting vetted domain experts such as doctors, lawyers, and engineers directly with frontier AI labs for RLHF, evaluation, and specialized training data at massive scale. See all Mercor jobs.
Apply for this role
Your details first, then a short screening test. Both are read as part of your application and go to Mercor, and we will tell you when similar roles appear.