Sr Program Manager – Ops

Sr Program Manager – Ops

Requirements

• 5+ years experience managing scaled operations programs with 1+ year experience in GenAI / model training
• 1+ year experience leading, building and helping develop
• Experience working in a fast-paced, ambiguous work environment
• Experience with Data Labeling / Annotation
• Bachelor’s Degree obtained
• Strong knowledge of machine learning concepts, particularly around supervised learning and training data needs
• Experience working with data annotation platforms and tools
• Proven track record of managing large-scale projects with cross-functional teams and external vendors

Responsibilities

• Define the roadmap and key objectives for data labeling projects to support generative AI initiatives.
• Partner with stakeholders (Data Scientists, Machine Learning Engineers, and Product Managers) to identify data requirements and success criteria.
• Design scalable data labeling workflows that leverage internal tools, external vendors, and automation.
• Optimize workflows for efficiency, accuracy, and cost-effectiveness, incorporating active learning and pre-labeling techniques where appropriate.
• Engage and manage relationships with data labeling vendors, ensuring timely delivery and adherence to quality standards.
• Collaborate with cross-functional teams to align labeling efforts with broader AI model development timelines.
• Implement robust quality assurance processes to validate labeled datasets against gold standards.
• Use metrics such as inter-annotator agreement, precision/recall, and throughput to monitor quality and make improvements.
• Manage program budgets, including vendor costs and internal resources.
• Forecast resource requirements and ensure efficient allocation to meet deadlines.
• Ensure compliance with data privacy regulations (e.g., GDPR, CCPA) and ethical guidelines in dataset creation.
• Advocate for inclusive and unbiased labeling practices to mitigate bias in AI models.

Preferred Qualifications

• Experience in Generative AI, including text, image, or audio data labeling
• 1+ year people management experience
• Familiarity with active learning, semi-supervised labeling, and human-in-the-loop systems.
• Proficiency in data annotation tools and scripting languages (Python, SQL) to analyze datasets and processes
• Strong understanding of ethical AI and best practices for minimizing dataset bias.
• Excellent written and verbal communication skills, with the ability to influence technical and non-technical stakeholders
• Strong understanding of the gig economy landscape, freelancer behaviors, and recruitment strategies
• Excellent project management skills, with a proven ability to juggle multiple priorities and deadlines
• You are a builder who wants to be empowered to make big bets
• Demonstrated ability to work independently and effectively across internal and external organizations
• Ability to take initiative in a constantly-changing work environment
• Exceptional written and verbal communication, and organizational skills