AI Evaluation Specialist
Pubblicato il 16-10-2026 - YO AI Labs in Parma
AI Evaluation Specialist Role Type: Contractor Location: Remote — US, Canada, UK, Ireland, Australia, New Zealand We are seeking experienced AI Evaluation Specialists to assess and improve the quality of AI assistant outputs for an enterprise AI training project. You'll evaluate AI-generated responses using defined quality standards, identify issues, and provide clear feedback to help improve AI systems. No prior formal AI experience is required. Scope of Work * Evaluate AI-generated outputs against detailed rubrics for accuracy, relevance, quality, and adherence to guidelines. * Apply consistent and objective judgment across large volumes of examples. * Identify reasoning gaps, tool-use issues, logical errors, and inconsistencies. * Provide concise, actionable feedback highlighting strengths and areas for improvement. * Participate in discussions around rubric interpretation and evolving quality standards. * Contribute to process improvements and evaluation best practices.
* Maintain accurate documentation of evaluations and recommendations. Preferred Qualifications * Experience in grading, quality assurance, editorial review, assessment, annotation, or similar analytical work. * Advanced, daily use of AI assistants such as ChatGPT, Claude, or similar tools. * Strong ability to analyze complex information and communicate findings clearly in writing. * Experience with process improvement, rubric development, or quality assessment. * Strong critical-thinking skills with a focus on consistency, accuracy, and fairness. * Comfortable working independently on high-volume evaluation tasks. * Strong attention to detail and ability to identify subtle quality issues. * Collaborative approach to resolving ambiguous cases and improving evaluation criteria. #J-18808-Ljbffr