Data Engineer III

Data Engineer III

Requirements

• Bachelor’s degree in Computer Science, Information Systems, Engineering, a related technical discipline, or equivalent work experience required
• Master’s or equivalent experience strongly preferred
• 5+ years working with Python and SQL in a data engineering context for end-to-end big data ML / analytics workloads
• 3+ years leading design, development, and troubleshooting of cloud data services (e.g., AWS), container orchestration (e.g., Docker, Kubernetes/EKS), and IaC tools (e.g., Terraform, CloudFormation) for data ingestion pipelines
• 3+ years working with cross-functional Data Science software engineering teams, collaborating with Data Science, MLOps, and DevOps
• 3+ years implementing data quality tests, and optimizing SQL/Python code and data models for cost/performance in the cloud
• 3+ years helping to architect and optimize global ML workflows and data platforms including automated testing, CI/CD, and monitoring in distributed processing (e.g., Spark) and low‑latency architectures
• Experience mentoring less-senior team members
• Experience and understanding of the full Software Development Lifecycle (SDLC) for Data Science software development
• 5+ years maintaining and optimizing ETL/ELT job scheduling and monitoring in production environments
• 5+ years leading design, development & troubleshooting of scalable ETL/ELT pipelines for business-critical applications

Responsibilities

• Lead the architecture and design of complex data solutions and pipelines for large-scale or enterprise projects
• Define project scope and technical approach for data engineering initiatives, evaluating and incorporating new technologies as needed
• Develop and optimize critical data pipelines, ensuring they are maintainable, scalable, and secure
• Provide technical leadership and mentorship to less senior data engineers, setting standards for code quality and best practices
• Work closely with leadership and other teams to align data engineering solutions with business strategy and requirements
• Evaluate and adopt advanced tools or frameworks (e.g., fabric architecture or virtualization technologies) to enhance the data engineering process
• Lead the code reviews with the team to debug, and to ensure the quality of the code
• Oversee the overall health, security, and scalability of the data infrastructure and pipeline ecosystem
• Identify and execute on opportunities to streamline data processing and improve system performance (e.g., refactoring pipelines, updating architectures)
• Establish best practices for monitoring, logging, and alerting to ensure data pipeline reliability (setting SLAs, recovery procedures, etc.)
• Lead initiatives to scale and modernize the data infrastructure, such as migrating to new platforms or optimizing cloud resource usage for cost-effectiveness
• Work with cross-functional teams (including IT and DevOps) to ensure seamless deployment and integration of data engineering solutions into the broader technology environment
• Evaluate proposed initiatives, clarify objectives and provide initial feedback
• Propose relevant project activities and provide clear, compelling updates to management and peers on a regular basis
• Interact frequently with team, business owners and users to maintain a good working relationship with the user community and other partners
• Provide consultation to Senior Leadership on complex projects
• Provide effective change leadership in driving continuous improvement and innovations to realize measurable productivity results from identified improvement initiatives
• Establish and champion data governance frameworks and data quality best practices for the organization
• Implement comprehensive data quality strategies (e.g., master data management, advanced validation rules) to ensure trust in enterprise data
• Collaborate with senior leadership and other departments to align data engineering efforts with broader business goals and compliance requirements
• Lead initiatives to improve data literacy and data sharing, including developing documentation standards and promoting a culture of data stewardship
• Serve as a liaison between the data engineering team and other technical/business teams, ensuring effective communication and collaboration on data-related matters
• Stay abreast of emerging data technologies and advocate for the adoption of tools that improve data quality, governance, or collaboration (e.g., evaluating new data management platforms)

Preferred Qualifications

• Master’s or equivalent experience strongly preferred
• Familiarity with data virtualization tools (e.g., Dremio) preferred
• Familiarity with Salesforce and Heroku (or similar) data infrastructures preferred
• 2+ years working with data virtualization tools (e.g., Dremio) and helping to convert Data Science prototypes into scalable services, integrating feature stores and metadata tracking