Data Engineer

5+ years of experience as a Data Engineer.
Demonstrated experience with AWS cloud services, including long-term storage options, and cloud-based database services (e.g., Databricks or EMR).
Proficiency in SQL database structures and mapping between SQL databases.
Experience with large-scale data migration efforts.
Experience with database architecture, performance design methodologies, and system-tuning recommendations.
Proficiency in Python and Bash scripting.
Experience implementing CI/CD pipelines using industry-standard processes.
TS/SCI with poly required to start.

Assist with strategic planning and oversee the implementation of the Sponsor’s cloud-based data environment.
Develop and maintain ETL (Extract, Transform, Load) processes for efficient data movement and transformation.
Design and implement data models and access controls to ensure data integrity and security.
Develop code (Python, Bash, etc.), documentation, and data models that adhere to Sponsor standards.
Provide systems administration and programming support for ETL processes and data infrastructure.
Train and conduct knowledge transfer to team members on ETL processes, the on-premise compute cluster, and administrative duties.
Coordinate with external data and platform providers to ensure smooth system functioning and data flows.
Support the cross-domain transfer and integration of data.
Serve as a technical liaison between engineers, data scientists, analysts, and managers.

Experience with the Sponsor’s data environment and on-premises compute structure.
Experience with Glue, Hive, and Iceberg or similar technologies.
Experience with Terraform.
Experience with DevSecOps solutions and tools.
Experience with Data Quality and Data Governance concepts and experience.
Experience maintaining, supporting, and improving the ETL process using Apache NiFi or similar tools.
Experience with Apache Spark.