Lead Data Engineer

Lead Data Engineer

Requirements

• Proficiency in Snowflake architecture and SQL for complex queries.
• Hands-on experience with Kafka for real-time data streaming.
• Familiarity with cloud platforms such as AWS and Azure.
• Knowledge of ERP systems integration.
• Expertise in data modeling techniques (e.g., star schema, snowflake schema).
• Solid understanding of access control mechanisms and SSO protocols.
• Experience in designing scalable ETL/ELT pipelines.
• Strong problem-solving abilities in ambiguous environments.
• Excellent communication skills for collaborating across teams.

Responsibilities

• Design, build, and maintain robust ETL/ELT pipelines for efficient data ingestion, transformation, and integration from diverse sources.
• Architect and optimize data warehouse solutions using Snowflake, ensuring scalability, performance, and reliability.
• Develop and manage real-time data streaming pipelines using technologies such as Kafka and Debezium.
• Collaborate with ERP system teams to ensure seamless integration of enterprise data into the data ecosystem. Understand SSO, embedded reporting scenarios.
• Work with BI tools (e.g., OAS, Domo, etc) to enable embedded reporting solutions and provide actionable insights.
• Design scalable data models (e.g., star schema, snowflake schema) to support analytics and reporting needs.
• Partner with cross-functional teams including to understand requirements and deliver tailored solutions.
• Monitor system performance, troubleshoot issues, and implement optimizations to enhance efficiency.

Preferred Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
• Experience migrating on-premise systems to cloud-based platforms.
• Familiarity with modern CI/CD pipelines for deployment automation.