ML Challenge Task Auditor

AI Evaluation · Remote · posted 2mo ago
Pay$70–$90/hrAdvertised by the platform.
WhereRemote · Remote — United States, USA
Open toUnited States
Compared with median+43% vs the $56/hr median
This is a referral link. We may earn a fee if you join; it never changes what Mercor pays you.

About this role

Evaluate the quality, correctness, and methodological rigor of applied machine-learning tasks used to train and evaluate a frontier AI lab's models. You'll assess experiment design, model-selection reasoning, and evaluation methodology — and provide clear, rubric-based written feedback.

Basic Qualifications
• 3+ years hands-on applied/experimental ML (experiment design, model selection, hyperparameter tuning, evaluation methodology)
• Strong grasp of data-quality rigor: leakage detection, metric gaming, and train/test/CV hygiene
• Proficiency with standard ML frameworks (PyTorch, TensorFlow, scikit-learn, XGBoost)
• Ability to critique ML claims against evidence and reproduce results

Preferred Qualifications
• Competition / benchmark experience (e.g., Kaggle)
• Graduate research or publication record in applied ML
• Prior task-grading or peer-review experience

Note: this role evaluates applied/experimental ML rigor — it is not an LLM-application-building or MLOps role.

Details come from Mercor’s public posting and may change. Check the original before applying. Job ID 963ac8132e.