Lead Data Engineer – Converse Technology
| Company | Nike |
|---|---|
| Location | Atlanta, GA, USA |
| Salary | $Not Provided – $Not Provided |
| Type | Full-Time |
| Degrees | Bachelor’s |
| Experience Level | Senior, Expert or higher |
Requirements
- Bachelor’s degree or equivalent combination of education, experience or training
- Minimum 6 years of relevant work experience in designing and implementing innovative data engineering capabilities and end to end solutions.
- Advanced Experience with data modeling, warehousing and building ETL pipelines; Must have experience with any ETL tools, preferably Matillion and/or PySpark
- Expert in building/operating highly available, distributed systems of data extraction, ingestion, processing of large data sets and delivering end to end projects independently
- Advanced experience building cloud scalable, real-time and high-performance data lake solutions, preferably Databricks, Snowflake, AWS
- Advanced Experience with big data technologies such as: Hadoop, Hive, Spark, EMR and orchestration tools like Airflow
- Advanced experience in SQL and modern scripting or programming languages, such as Python, Shell
- Experience in CI/CD Pipeline for Code deployment: preferred GitHub, Jenkins, terraform, Databricks Assets Bundles
Responsibilities
- Define and communicate the requirements for technical environments and determine the technical scope for projects, and provide technical estimates or capacity planning; Translate product backlog items into engineering designs and logical units of work
- Assess a well-defined problem and lead the development of a technical solution that meets the needs of the business and aligns with architectural standards
- Drive collaboration with architecture, platform teams or other teams on integration needs/designs; create advanced technical designs; approve proof of concept efforts and reviewing results; ensure high quality solutions are implemented, and engineering standard methodologies are defined and followed.
- Design and implement data products and features in collaboration with product owners, data analysts, and business partners using Agile / Scrum methodology.
- Define and apply appropriate data acquisition, processing and consumption strategies for given technical scenarios; Design and implement distributed data processing pipelines using tools and languages prevalent in the big data ecosystem; Profile and analyze data for the purpose of designing scalable solutions
- Drive and implement the technical strategies of new data projects and the optimization of existing solutions; Troubleshoot complex data issues and perform root cause analysis to proactively resolve product and operational issues
- Build utilities, user defined functions, libraries, and frameworks to better enable data flow patterns; implement complex automated routines using workflow orchestration tool
- Drive collaborative reviews of design, code and test plans and dataset implementation by other data engineers in support of maintaining data engineering standards
- Identify and remove technical bottlenecks for your engineering squad; and provide leadership, guidance and mentorship to other data engineers in adopting best standards and practices
- Anticipate, identify and tackle issues concerning data management to improve data quality; Build and incorporate automated unit tests and participate in integration testing efforts.
- Utilize and advance the software engineering best practices, including source control, code review, testing, and continuous integration and delivery (CI/CD) on the required cloud infrastructure and workspace configurations, Source files, such as notebooks and Python files, that include the business logic etc.
Preferred Qualifications
- Desire to work closely with your teammates to come up with the best solution to a problem
- Demonstrate experience and ability to deliver results on multiple projects in a fast-paced, agile environment
- Excellent problem-solving and interpersonal communication skills;
- Strong desire to learn, share knowledge with others and coach the team.