Lead Data Engineering
| Company | Arrowstreet Capital |
|---|---|
| Location | Boston, MA, USA |
| Salary | $Not Provided – $Not Provided |
| Type | Full-Time |
| Degrees | Bachelor’s |
| Experience Level | Expert or higher |
Requirements
- At least 10 years of experience working with modern, large-scale data systems in a technical leadership role
- Possess extensive expertise in working with diverse data storage technologies, including databases, data warehouses, data lakes, and data lakehouses
- Proven track record of building and scaling data platforms at an enterprise level
- Excellent software architecture design skills
- Exceptional communication skills and the ability to build strong relationships as well as credibility
- Excellent leadership and management skills, with the ability to mentor and develop high-performing teams
- Bachelor’s degree in Computer Science, Computer Engineering or a related discipline
- Experience with containerized deployments and elastic environments
- Thorough knowledge of CI/CD (Jenkins, Gitlab, Github) and testing (e.g., pytest) methodologies
- At least 3 years of experience using AWS services oriented towards data and compute functions
- Demonstrated expertise in SQL, Spark, Pandas, or similar technologies
- At least 3 years building complex or scalable systems using Python or other object-oriented languages (e.g., C++, Rust, Java, C#)
- Extensive experience designing schemas, optimizing queries, and manipulating/aggregating data, particularly with point-in-time semantics
Responsibilities
- Design, develop, and maintain data systems, analytics, and reporting applications
- Engage with our multiple business teams to understand their needs and requirements and develop cohesive strategies to address them in a cost-effective and efficient manner
- Assess current environment and make recommendations specific to data and reporting technologies
- Collaborate with development teams to ensure the functionality being developed is implemented in a platform-centric manner, while setting data for publishing new data sources
- Build reporting capabilities that allow for self-service analysis based on multiple ‘what if’ scenarios leveraging AI technologies
- Optimize existing data systems and analytics applications
- Analyze data to identify trends and develop insights
- Prioritize appropriately so our existing data estate continues to work flawlessly as the target architecture is developed
- Implement processes to ensure the quality and availability of data and build functionality to improve the overall robustness of our data platform
- Develop accurate metrics to identify performance improvement opportunities
- Design verification plans and implement unit tests to validate the correctness and performance of the solutions
- Design a reliable production support process and, when required, assist in supporting and troubleshooting production issues
- Foster strong partnerships with technology vendors
- Negotiate and manage technical contracts to support the organization’s needs and ensure cost-effective solutions
- Automate all aspects of the data lifecycle
- Stay updated with industry trends and advancements in data engineering, evaluating and recommending suitable technologies and tools.
Preferred Qualifications
- Expertise in budget control and vendor management preferred