Lead Engineer, AI Platform

Vedi l’offerta su Go Lavoro: golavoro.it/annuncio-lavoro/8ag2gu

pbThis position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Lead Engineer, AI Platform based in Italy. /b /ppThis role offers the opportunity to lead the engineering foundation behind reliable, measurable, and scalable AI-powered features. You’ll build evaluation frameworks, observability tooling, and diagnostic infrastructure that reveal how AI agents perform in real production environments. The position combines hands‑on software engineering with technical leadership and people management. You’ll investigate quality issues across complex agent workflows, develop datasets and evaluation systems, and run experiments across models, prompts, and agent architectures. You’ll also help optimize AI systems for cost, latency, reliability, and overall user experience. Working in a highly remote and asynchronous environment, you’ll collaborate closely with AI engineering teams while shaping the technical direction of a growing AI Quality function. /ph3Accountabilities: /h3ulliDesign, build, and own evaluation infrastructure, including CI/CD pipelines, scorers, datasets, and systems for assessing AI agents from individual tool calls through complete multi-turn conversations. /liliDevelop observability and diagnostic capabilities to identify exactly where quality issues occur across planning, execution, tool selection, and complex agent trajectories. /liliInvestigate failures across sophisticated AI workflows and turn findings into technical prototypes, improvements, or clearly defined priorities for AI engineering teams. /liliBuild and expand datasets through human annotation, AI-generated examples, and simulated conversations to increase evaluation coverage efficiently. /liliDevelop structured experimentation frameworks for prompts, models,



and agent harnesses, including evaluation of new and open-source models against production baselines. /liliIdentify opportunities to improve AI system cost and latency through model selection, caching, routing, and other optimization strategies. /liliSet the technical direction and manage day-to-day priorities for the AI Quality engineering team while remaining actively involved in hands‑on development. /liliPartner closely with AI Core engineering teams to ensure changes to AI products can be measured effectively and demonstrably improve quality. /liliEstablish engineering practices and evaluation approaches that support reliable, efficient, and scalable production AI systems. /li /ulh3Requirements: /h3ulli7+ years of experience building and shipping production software, ideally including LLM-powered agents capable of taking real actions within products. /liliExperience working with complex, tool-using AI systems involving multiple tools, planning, orchestration, or sub-agents rather than only simple, single-turn assistants. /liliStrong ability to demonstrate shipped software and explain how its effectiveness and reliability were measured. /liliExperience with Ruby on Rails and/or Python, with the ability to become productive quickly in technologies that may be new to you. /liliExperience building evaluation or observability infrastructure for ML/AI systems,



including evaluation pipelines, scorers, dashboards, or CI/CD systems for evaluations. /liliFamiliarity with evaluation frameworks such as Braintrust, LangSmith, or similar tools. /liliExperience designing datasets, annotation workflows, or labeling pipelines for machine learning or AI evaluation. /liliAbility to learn quickly, experiment extensively, and use empirical results to guide technical decisions. /liliComfortable operating in a fast-paced environment with ambiguity and changing technical requirements. /liliStrong technical leadership and people-management capabilities, with the ability to balance team leadership and hands‑on engineering. /liliExcellent English proficiency in spoken, written, and reading communication, equivalent to CEFR C2 / ILR 5. /liliStrong alignment with a collaborative, ownership-oriented engineering culture. /li /ulh3Benefits: /h3ulliAnnual cash compensation of $170,000 USD, benchmarked to U.S. compensation levels regardless of location. /liliEquity in the company, including ongoing refresh grants. /lili35 days of paid time off per year. /liliFully remote work environment. /liliSignificant flexibility and autonomy in how you organize your work. /liliTwice-yearly company retreats in international destinations. /liliBenefits supporting health, wellbeing, and professional development. /liliOpportunity to lead and grow an AI Quality engineering team while remaining hands‑on technically. /liliExposure to advanced AI agents, evaluation infrastructure, observability, experimentation, and production AI optimization. /liliOpportunity to work with a globally distributed team across multiple countries and time zones. /li /ulpWe appreciate your interest and wish you the best! /p #J-18808-Ljbffr

Vedi l’offerta su Go Lavoro: golavoro.it/annuncio-lavoro/8ag2gu