IT Technical Lead
Pubblicato il 10-10-2026 - Lever in Milano
Our partner is looking for a Lead Engineer, AI Platform based in Italy. This role offers the opportunity to lead the engineering foundation behind reliable, measurable, and scalable AI-powered features. You'll build evaluation frameworks, observability tooling, and diagnostic infrastructure that reveal how AI agents perform in real production environments. The position combines hands‐on software engineering with technical leadership and people management. You'll investigate quality issues across complex agent workflows, develop datasets and evaluation systems, and run experiments across models, prompts, and agent architectures. You'll also help optimize AI systems for cost, latency, reliability, and overall user experience. Working in a highly remote and asynchronous environment, you'll collaborate closely with AI engineering teams while shaping the technical direction of a growing AI Quality function.
Design, build, and own evaluation infrastructure, including CI/CD pipelines, scorers, datasets,
and systems for assessing AI agents from individual tool calls through complete multi-turn conversations.
Develop observability and diagnostic capabilities to identify exactly where quality issues occur across planning, execution, tool selection, and complex agent trajectories.
Investigate failures across sophisticated AI workflows and turn findings into technical prototypes, improvements, or clearly defined priorities for AI engineering teams.
Build and expand datasets through human annotation, AI-generated examples, and simulated conversations to increase evaluation coverage efficiently.
Develop structured experimentation frameworks for prompts, models, and agent harnesses, including evaluation of new and open-source models against production baselines.
Identify opportunities to improve AI system cost and latency through model selection, caching, routing, and other optimization strategies.
Set the technical direction and manage day-to-day priorities for the AI Quality eng