Data Engineer 2
| Company | Comcast |
|---|---|
| Location | Chicago, IL, USA, Reston, VA, USA |
| Salary | $84845.88 – $127268.82 |
| Type | Full-Time |
| Degrees | Bachelor’s, Master’s |
| Experience Level | Mid Level |
Requirements
- Bachelor’s or master’s degree in computer science, Engineering, or a related field.
- 2+ years of experience in building and operating large-scale data processing systems.
- Proficiency in multiple programming languages, including Scala, Java, Python, and SQL.
- Familiarity with big data processing frameworks, such as Apache Flink, Apache Spark (Batch & Streaming), and Databricks Delta Live Tables (DLT).
- Understanding of data storage technologies, including AWS S3, Delta Lake, Apache Iceberg, Hadoop HDFS, and Apache Druid.
- Basic understanding of Data Lakehouse architectures, data modeling techniques, ETL processes, and relational databases (e.g., MySQL).
- Experience with AWS services, such as Glue, EMR and Lambda.
- Experience with workflow orchestration tools, such as Apache Airflow and Azkaban.
- Knowledge of NoSQL databases, including HBase.
- Strong problem-solving skills with a keen attention to detail.
- Excellent communication and collaboration skills to work effectively with technical and non-technical teams.
- Ability to quickly adapt and context switch among different priorities, working efficiently in a fast-paced environment.
- Strong generalist mindset with the ability to work across different technology stacks and polyglot programming skills, demonstrating fluency in multiple programming languages beyond primary expertise.
- Consistent exercise of independent judgment and discretion in matters of significance.
- Regular, consistent and punctual attendance. Must be able to work nights and weekends, variable schedule(s) as necessary.
Responsibilities
- Build and operate large-scale data processing systems.
- Collaborate with technical and non-technical teams.
- Adapt and context switch among different priorities in a fast-paced environment.
- Exercise independent judgment and discretion in matters of significance.
- Attend regularly and punctually, including nights and weekends as necessary.
Preferred Qualifications
- Experience managing highly scalable data infrastructures, ensuring performance, reliability, and cost-efficiency.
- Experience in modeling and processing complex, hierarchical data structures such as Ad Logs.
- Adept in planning work for self and the team while collaborating closely with product stakeholders.