SWE-Bench AI Task Auditor

Vedi l’offerta su Go Lavoro: golavoro.it/annuncio-lavoro/855rnt

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a SWE-Bench AI Task Auditor - Freelance AI Trainer Project based in Italy.

This freelance opportunity is designed for experienced software engineers who want to contribute their technical expertise to the development and evaluation of AI systems.
You will review software engineering tasks used to train and assess advanced AI models.
Your work will focus on ensuring tasks are technically accurate, realistic, reproducible, and appropriately tested.
You will investigate codebase integrations, test failures, logic issues, and other technical challenges.
The role offers flexibility through a fully remote, project-based working model.
You will apply practical software engineering judgment rather than simply following predefined checks.
Your feedback will directly help improve the quality and reliability of AI training workflows.

n

Accountabilities

•
Evaluate software engineering tasks for technical accuracy, realism, solvability, reproducibility,



and alignment with SWE-Bench standards.

•
Review task codebases, integrations, tests, and evaluation criteria to identify potential technical weaknesses.

•
Rigorously test and troubleshoot complex technical scenarios to determine whether tasks function as intended.

•
Investigate codebase integration problems, test failures, logic errors, and other implementation issues.

•
Provide clear, precise, and actionable feedback that enables task creators to correct identified problems.

•
Apply professional software engineering judgment to assess whether tasks reflect realistic development scenarios.

•
Help maintain a high standard of technical rigor and accuracy across AI training and evaluation workflows.

Requirements

•
Demonstrable professional experience in software engineering, including experience navigating complex codebases and developing real-world applications.

•
Strong knowledge of softwar

Vedi l’offerta su Go Lavoro: golavoro.it/annuncio-lavoro/855rnt