Skip to content
Lead Data Engineer
| Company | Brillio |
|---|
| Location | St. Louis, MO, USA |
|---|
| Salary | $120000 – $1350000 |
|---|
| Type | Full-Time |
|---|
| Degrees | |
|---|
| Experience Level | Expert or higher |
|---|
Requirements
- ETL Fundamentals
- SQL
- SQL (Basic + Advanced)
- Python
- Data Warehousing
- Modern Data Platform Fundamentals
- Data Modelling Fundamentals
- PLSQL
- T-SQL
- Stored Procedures
- 8+ years of relevant experience
Responsibilities
- Develop Ingestion process to source data from various type of sources like flat files, databases, json etc to consume in Databricks.
- Develop ETL PySpark code in Databricks to build reporting layer tables for PowerBI and follow the best practices.
- Utilize GitHub to maintain the code inventory and work with the team in terms of testing and deployment.
- Utilize Jenkins for CICD and follow the code standards in terms of parametrization, scheduling jobs, email notifications through the terraform scripts.
- Develop reports on PowerBI based on the business and process improvement needs.
- As part of the Agile team attend the regular scrum calls to provide updates and plan and own the work as part of sprint refinement and planning.
- Work with Tech leads for code review and design meetings.
- Good communication
Preferred Qualifications
No preferred qualifications provided.
