Senior Data Engineer
| Company | Lotlinx |
|---|---|
| Location | Winnipeg, MB, Canada, Hamilton, ON, Canada |
| Salary | $108000 – $162000 |
| Type | Full-Time |
| Degrees | |
| Experience Level | Senior |
Requirements
- 5+ years of experience in data engineering or a similar role, with a focus on designing and managing scalable data systems.
- Proven expertise with cloud platforms, specifically AWS and/or GCP.
- Proficiency in SQL, including advanced query optimization and data modeling techniques.
- Strong programming skills in Python, Scala, or Java, with a focus on developing data processing applications.
- Experience with Data Engineering tools such as Airflow, Dataflow, DBT.
- Experience with big data frameworks like Apache Spark, Hadoop, or Beam.
- Hands-on experience with real-time data streaming platforms such as Apache Kafka, Pub/Sub, or Kinesis.
- Knowledge of CI/CD pipelines, version control systems (e.g., Git), and containerization technologies (e.g., Docker, Kubernetes).
- Experience managing data warehouses and lakes using modern platforms such as Snowflake, BigQuery, or Redshift.
- Familiarity with data governance frameworks, security best practices, and compliance standards.
- Demonstrated ability to solve complex technical challenges, think creatively and innovate within cloud and data ecosystems.
Responsibilities
- Design, build, and maintain robust, scalable, and efficient data pipelines to process large-scale datasets from multiple sources.
- Develop and manage ETL/ELT workflows for data ingestion, transformation, and loading into data lakes and warehouses.
- Architect and implement cloud-based solutions (AWS, GCP) to ensure data security, scalability, and high availability.
- Work with stakeholders including Analytics, Product, and Design teams to assist with data related technical issues and support their data infrastructure needs.
- Partner with DevOps and Security teams to ensure compliance with data governance, privacy, and security standards.
- Engineer solutions for large data storage, management.
- Proactively identify and resolve performance bottlenecks, scaling challenges, and technical issues.
- Explore available technologies and design solutions to continuously improve our data quality, workflow reliability, scalability while reporting performance and capabilities.
- Act as an internal expert in each of the data sources so that you can own overall data quality.
Preferred Qualifications
-
No preferred qualifications provided.