Sr. Data Engineer
| Company | Visa |
|---|---|
| Location | Austin, TX, USA |
| Salary | $Not Provided – $Not Provided |
| Type | Full-Time |
| Degrees | Master’s |
| Experience Level | Senior |
Requirements
- Master’s degree in Computer Science or related field
- 2 years of experience in the job offered or in a data engineer or software engineer-related occupation
- Creating large scale data engineering pipelines
- Writing and optimizing SQL queries with Spark, Python, Hive, and Scala
- Understanding machine learning models based on unstructured, structured, and streaming datasets
- Extracting, aggregating, and processing big data pipelines using Hadoop, EMR, and NoSQL databases
- Supporting production systems and identifying and resolving performance bottlenecks for production systems
- Unix, Shell, or Python scripting
- Oozie and Airflow
- Apache Storm and Spark-Streaming
- CI/CD with experience in building, maintaining and scaling production infrastructures
- Tableau, Power BI, D3 and GIT
Responsibilities
- Execute and manage large scale ETL processes to support development and publishing of reports, Datamart’s and predictive models.
- Self-service reporting tools like Tableau or Power BI with KPIs and facilitate Visa Consulting engagements including data exchange.
- Propose solutions and communicate strategy and risk mitigation approaches to leadership.
- Install, configure, operate, and monitor CI/CD pipeline tools.
- Build ETL pipelines in Spark, Python, HIVE or Scala that process transaction and account level data and standardize data fields across various data sources.
- Build and maintain high performing ETL processes, including data quality and testing aligned across technology, internal reporting and other functional teams.
- Create data dictionaries, setup/monitor data validation alerts and execute periodic jobs like performance dashboards, predictive models scoring for client’s deliverables.
- Define and build technical/data documentation and experience with code version control systems (e.g. git).
- Ensure data accuracy, integrity and consistency.
- Create, automate and scale repeatable financial and statistical analysis for Visa Consulting and Analytics.
- Collaborate with Data Engineering teams in North America and other Global regions to production and maintain key data assets.
- Communicate technical insights and recommendations with business customers and leadership team.
- Up to 10% of domestic travel is required.
Preferred Qualifications
-
No preferred qualifications provided.