AI Research Scientist – Scientific Evaluation (ML/AI)
pThis position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for an AI Research Scientist – Scientific Evaluation (ML/AI) based in Italy. /p pJoin an advanced AI research evaluation project focused on assessing scientific work produced by autonomous AI research agents. In this remote, freelance role, you will apply your expertise in machine learning and artificial intelligence to evaluate research papers against the rigorous standards of leading academic conferences. You will assess scientific novelty, technical correctness, experimental design, reproducibility, and the validity of research conclusions. Your expert judgment will help distinguish meaningful scientific advances from flawed methodologies or unsupported claims. Working independently with structured evaluation guidelines, you will contribute to the development and assessment of frontier AI systems capable of conducting end-to-end scientific research. This opportunity offers flexible hours, competitive hourly compensation, and the chance to apply your research expertise to emerging AI capabilities. /p h3Accountabilities /h3 pAs an AI Research Scientist specializing in scientific evaluation, you will critically examine AI-generated research outputs and determine whether they demonstrate the scientific rigor and originality expected at top-tier machine learning and AI conferences. Your responsibilities will include: /p ul liEvaluate AI-generated ML/AI research papers for scientific contributions, technical correctness, methodology, experimental design, and the validity of their conclusions. /li liAssess the novelty and significance of proposed approaches by comparing them with existing research and state-of-the-art methods. /li liExamine experimental rigor, including baseline comparisons, benchmark selection, evaluation metrics, statistical evidence, and ablation studies. /li liIdentify methodological flaws, unsupported claims, inconsistencies, misleading results, and other issues that could compromise scientific validity. /li liAssess reproducibility and technical soundness by reviewing experimental evidence, implementation details, and accompanying code when available. /li liVerify relevant academic literature,
identify overlooked prior work, check citation accuracy, and challenge unsupported novelty claims. /li liDetermine whether research outputs meet the scientific standards associated with leading ML/AI conferences, including NeurIPS, ICML, and ICLR. /li liProduce structured, evidence-based evaluations outlining research strengths, weaknesses, critical technical concerns, and overall scientific merit. /li liApply detailed assessment rubrics consistently while providing clear, concise, and well-reasoned justifications for your evaluations. /li liExercise independent expert judgment when determining whether AI-generated research represents a genuine scientific contribution and is supported by sufficient evidence. /li /ul h3Requirements /h3 pThe ideal candidate is an accomplished ML/AI researcher with a strong publication record, deep familiarity with current research, and the ability to critically evaluate complex scientific contributions. Candidates must demonstrate the following qualifications: /p ul liA PhD in Machine Learning, Artificial Intelligence, Computer Science, Statistics, or a closely related technical discipline, or equivalent demonstrated research expertise. /li liAt least three peer-reviewed research publications at NeurIPS, ICML, and/or ICLR. This is a mandatory requirement. /li liIdeally, first-author publications or other significant research contributions demonstrating substantial scientific involvement. /li liExtensive hands‑on ML/AI research experience, including developing novel methods, designing experiments, analyzing results, and publishing original research. /li liStrong knowledge of scientific methodology and experimental rigor, including baseline selection, statistical validity, benchmark design, ablation studies, and reproducibility. /li liAbility to independently assess the novelty, significance,
technical soundness, and scientific validity of advanced ML/AI research. /li liFamiliarity with state-of-the‑art research and the ability to evaluate new approaches against established literature and benchmarks. /li liExperience reviewing research papers for leading ML/AI conferences or journals, particularly NeurIPS, ICML, or ICLR; this is strongly preferred. /li liAbility to verify technical arguments, experimental evidence, scientific claims, citations, and code implementations where relevant. /li liExcellent analytical skills and scientific writing abilities, with the capacity to communicate nuanced technical judgments through clear, concise, evidence-based feedback. /li liAbility to apply detailed evaluation guidelines and scoring rubrics consistently while maintaining independent scientific judgment. /liliExceptional attention to detail, particularly when identifying subtle methodological weaknesses, unsupported conclusions, and discrepancies between claims and evidence. /li liWillingness to provide a CV, a brief overview of your research background and areas of expertise, and links to at least three qualifying peer-reviewed publications. /li liRelevant academic peer-review experience, including service as a reviewer or area chair at major research conferences, should be highlighted where applicable. /li /ul h3Benefits /h3 ul liFully remote opportunity open to candidates worldwide. /li liCompetitive compensation of $50–$65 per hour. /li liFreelance or independent contractor arrangement. /li liPart-time, project-based work with flexible hours. /li liOpportunity to evaluate cutting-edge autonomous AI systems capable of conducting end-to-end scientific research. /li liChance to apply your research expertise to the evaluation and advancement of frontier AI agents. /li liExposure to emerging developments in AI-driven scientific discovery. /li liOpportunity to contribute specialist scientific judgment to research evaluation processes focused on technical validity, originality, and experimental rigor. /li liFlexible working arrangements that support independent research professionals. /li /ul pWe appreciate your interest and wish you the best! /p #J-18808-Ljbffr
identify overlooked prior work, check citation accuracy, and challenge unsupported novelty claims. /li liDetermine whether research outputs meet the scientific standards associated with leading ML/AI conferences, including NeurIPS, ICML, and ICLR. /li liProduce structured, evidence-based evaluations outlining research strengths, weaknesses, critical technical concerns, and overall scientific merit. /li liApply detailed assessment rubrics consistently while providing clear, concise, and well-reasoned justifications for your evaluations. /li liExercise independent expert judgment when determining whether AI-generated research represents a genuine scientific contribution and is supported by sufficient evidence. /li /ul h3Requirements /h3 pThe ideal candidate is an accomplished ML/AI researcher with a strong publication record, deep familiarity with current research, and the ability to critically evaluate complex scientific contributions. Candidates must demonstrate the following qualifications: /p ul liA PhD in Machine Learning, Artificial Intelligence, Computer Science, Statistics, or a closely related technical discipline, or equivalent demonstrated research expertise. /li liAt least three peer-reviewed research publications at NeurIPS, ICML, and/or ICLR. This is a mandatory requirement. /li liIdeally, first-author publications or other significant research contributions demonstrating substantial scientific involvement. /li liExtensive hands‑on ML/AI research experience, including developing novel methods, designing experiments, analyzing results, and publishing original research. /li liStrong knowledge of scientific methodology and experimental rigor, including baseline selection, statistical validity, benchmark design, ablation studies, and reproducibility. /li liAbility to independently assess the novelty, significance,
technical soundness, and scientific validity of advanced ML/AI research. /li liFamiliarity with state-of-the‑art research and the ability to evaluate new approaches against established literature and benchmarks. /li liExperience reviewing research papers for leading ML/AI conferences or journals, particularly NeurIPS, ICML, or ICLR; this is strongly preferred. /li liAbility to verify technical arguments, experimental evidence, scientific claims, citations, and code implementations where relevant. /li liExcellent analytical skills and scientific writing abilities, with the capacity to communicate nuanced technical judgments through clear, concise, evidence-based feedback. /li liAbility to apply detailed evaluation guidelines and scoring rubrics consistently while maintaining independent scientific judgment. /liliExceptional attention to detail, particularly when identifying subtle methodological weaknesses, unsupported conclusions, and discrepancies between claims and evidence. /li liWillingness to provide a CV, a brief overview of your research background and areas of expertise, and links to at least three qualifying peer-reviewed publications. /li liRelevant academic peer-review experience, including service as a reviewer or area chair at major research conferences, should be highlighted where applicable. /li /ul h3Benefits /h3 ul liFully remote opportunity open to candidates worldwide. /li liCompetitive compensation of $50–$65 per hour. /li liFreelance or independent contractor arrangement. /li liPart-time, project-based work with flexible hours. /li liOpportunity to evaluate cutting-edge autonomous AI systems capable of conducting end-to-end scientific research. /li liChance to apply your research expertise to the evaluation and advancement of frontier AI agents. /li liExposure to emerging developments in AI-driven scientific discovery. /li liOpportunity to contribute specialist scientific judgment to research evaluation processes focused on technical validity, originality, and experimental rigor. /li liFlexible working arrangements that support independent research professionals. /li /ul pWe appreciate your interest and wish you the best! /p #J-18808-Ljbffr