QA Analyst

Vedi l’offerta su Go Lavoro: golavoro.it/annuncio-lavoro/7gzxtd

About CodeRoad
CodeRoad provides end-to-end software development services, helping businesses scale with ideal infrastructure solutions. From staff augmentation to dedicated IT teams and general software engineering, our nearshore technology services empower businesses to thrive in an ever-evolving digital landscape.
About the Role
We are looking for a Junior AI Agent Quality Engineer to help us build and validate the next generation of autonomous systems. In this role, you won't just be checking for bugs; you'll be evaluating the "brain" of our AI.
You will focus on the reliability of Agentic AI and RAG (Retrieval-Augmented Generation). You’ll be responsible for ensuring our agents can plan tasks, use tools accurately, and recover from errors without "hallucinating." This is a perfect role for someone with a QA mindset who is passionate about Python and the future of LLM-based orchestration.
Key Responsibilities
Agent Evaluation: Create question templates and Python scripts to test how well AI Agents and RAG instances solve complex tasks.
KPI Tracking:



Evaluate Agent performance using specific metrics: Success Rate, Tool Use Accuracy, Planning Quality, and Autonomy.
Dataset Creation: Collaborate with AI Engineers to build "Ground Truth" datasets and Agentic Task Datasets to benchmark model improvements.
API & Logic Testing: Write unit and integration tests for Python-based RESTful APIs and agent endpoints.
Performance Testing: Run load and bulk testing (using Locust or JMeter) to see how our AI handles high-volume requests.
Input/Output Validation: Perform rigorous testing of agent payloads to catch prompt injection risks and logic failures.
Requirements
Agentic AI Focus: Basic understanding of Agentic workflows, prompt engineering (ReAct prompts), and LLM orchestration.
AI Observability: Familiarity with (or a strong desire to learn) tools like LangSmith, Langfuse, or OpenTelemetry.
Python Skills: Proficiency in Python 3.10+ for automation and data manipulation.
A

Vedi l’offerta su Go Lavoro: golavoro.it/annuncio-lavoro/7gzxtd