Posted in

Data Engineer Python

Data Engineer Python

CompanySMX
LocationWashington, DC, USA
Salary$86000 – $143200
TypeFull-Time
Degrees
Experience LevelJunior, Mid Level

Requirements

  • Proficiency in Python: Strong understanding of Python programming language. Experience with Python libraries and frameworks like Pandas, NumPy, and Django.
  • Expertise in Apache Airflow: Experience in designing, building, and maintaining data pipelines using Apache Airflow. Knowledge of Airflow’s architecture, including DAGs and Operators.
  • ETL Processes: Proficiency in data extraction, transformation, and loading processes. Experience with data extraction from various sources, data transformation (cleaning, validating, aggregating, joining, etc.), and loading data into databases or data warehouses.
  • Database Knowledge: Strong understanding of SQL and NoSQL databases. Proficiency in writing complex queries and applying database optimization techniques.
  • Data Warehousing: Experience with data warehousing solutions like Amazon Redshift, Google BigQuery, or Microsoft Azure SQL Data Warehouse.
  • Soft Skills: Strong communication and collaboration skills. Excellent problem-solving skills.
  • US Citizenship is required to obtain a federal clearance.

Responsibilities

  • Design, develop, and maintain ETL processes using Python and Apache Airflow.
  • Collaborate with data analysts and other stakeholders to understand and meet their data requirements.
  • Develop and implement data validation processes to ensure high data quality.
  • Troubleshoot and resolve issues related to data pipelines.
  • Optimize data extraction, transformation, and loading (ETL) processes to improve efficiency and performance.
  • Document and maintain the design and details of data processes and schemas.
  • Stay updated with the latest industry trends and technologies to ensure data practices remain current.

Preferred Qualifications

  • Knowledge of data modeling and data warehousing.