Manager Data Science – Generative AI

Manager Data Science – Generative AI

Requirements

• Advanced Degree (Master’s/Ph.D.) in quantitative discipline (AI, Machine Learning, Statistics, Math, Data Science, Computer Science) is preferred
• 3+ years of experience in the following areas:
• Python development
• Data mining, machine learning, statistical modeling tools and underlying algorithms
• Data Lake and cloud computing fundamentals
• Strong analytical curiosity and passion for applying advanced modeling techniques in problem solving
• Sound analytical thinker with a proven track record of providing actionable insights and clear strategic direction
• Ability to convey complex, technical subject matter in a clear and straightforward manner; demonstrated ability to effectively communicate through written and oral presentations with all levels of the organization
• Solid project management skills with the ability to juggle multiple priorities simultaneously in a fast-paced environment

Responsibilities

• Research, design, and implement evaluation methods for generative AI applications. Our primary focus today is on language understanding and generation, primarily with AI chatbot assistants but also with summarization, ticket routing, and other use cases. This includes both automated evaluation methods (LLM as judge) as well as systematic approaches to manual review and monitoring.
• Research, design, and implementation of search methods including vector databases and semantic search algorithms, and associated performance evaluation methods. Our primary focus in this space today is powering a knowledge base search system that powers a RAG pipeline for our AI assistant chatbots.
• Research methods for the advanced use of LLMs, e.g. fine tuning, custom training of open source models, and evaluation/comparison of available foundation and open source models. While we are not heavily focusing our efforts in this space today, it’s important that we keep up with industry trends and maintain a perspective on how CarMax can make the best use of current technology.
• Keep up to date on industry trends in technology products and third party solutions, provide a perspective on where CarMax should consider investing, and occasionally conduct technology evaluations.

Preferred Qualifications

• Research and development experience with AI, data science or other scientific / technology methods, exploring the practical application of emerging methods
• Applied NLP to analyze and work with text data at scale
• Experience working with LLM’s and the application of generative AI to drive practical business value
• Python development and cloud engineering, specifically working a production quality code base to test new features and contribute usable code
• Chatbot design, development and evaluation are a plus