Machine Learning Quality Engineer

Experience in leveraging or critically thinking about how to integrate AI into work processes, decision-making, or problem-solving. This may include using AI-powered tools, automating workflows, analyzing AI-driven insights, or exploring AI’s potential impact on the function or industry
Minimum of 2 years of professional experience in software quality engineering
Bachelor’s degree with at least 2 years of relevant experience; or an advanced degree with no prior experience; or equivalent practical experience
Hands-on experience or strong interest in testing machine learning and Generative AI systems, including synthetic data generation and prompt engineering, is highly preferred
Foundational understanding of AI concepts such as agentic systems, algorithms, models, and associated ethical considerations
Experience with automated testing frameworks such as Java, JUnit, Selenium, TestNG, and other open-source tools
Proficient in programming and scripting with Java and Python; experience with tools such as Eclipse, Jenkins, Maven, and Git
Familiarity with development tools including IDEs, debuggers, build tools, version control systems, and Unix/system administration utilities
Solid understanding of Agile methodologies and the software development lifecycle (SDLC)

Evaluate the quality, accuracy, performance, and safety of AI-generated content produced by large language models (LLMs), identifying areas for improvement
Execute test plans, track quality metrics, and log defects based on observed patterns and results
Maintain and enhance existing test automation frameworks
Contribute to the development and refinement of evaluation criteria and methodologies to strengthen AI model assessment
Design targeted, small-scale test datasets to ensure adequate coverage across use cases
Collaborate with engineering teams to troubleshoot and resolve issues in applications and development/test environments

Hands-on experience or strong interest in testing machine learning and Generative AI systems, including synthetic data generation and prompt engineering