Specialist – Data Engineer – Databricks
| Company | Nationwide |
|---|---|
| Location | Des Moines, IA, USA, Scottsdale, AZ, USA, Columbus, OH, USA |
| Salary | $Not Provided – $Not Provided |
| Type | Full-Time |
| Degrees | |
| Experience Level | Mid Level, Senior |
Requirements
- Databricks
- Python – Must have
- DevOps Tools (Kubernetes (CNP), Jenkins, Harness) / DevSecOps Practices – Working knowledge
- Unix shell scripting
- Working knowledge in LINUX
- AWS Services (e.g S3, EC2, Cloud formation pipeline)
- General understanding of Data governance and discovery tools
- Integration and API development
- Strong Communication skills both verbal and written
- Strong problem solving skills
- Experience using Agile methodologies and creating good stories that complete a solution
- Automation/Innovative and Continuous Improvement mindset
- Capable of leading technical projects and decomposing work across team members
Responsibilities
- Manage and administer the Databricks environment, including clusters, workspaces, and notebooks, to ensure optimal performance, reliability, and scalability.
- Configure and optimize Databricks clusters and resources based on workload requirements and best practices.
- Monitor system performance, resource utilization, and availability using Databricks monitoring and logging tools.
- Troubleshoot and resolve issues related to Databricks infrastructure, networking, security, and integration with other systems.
- Implement and enforce security best practices to protect data and resources in the Databricks environment, including authentication, authorization, encryption, and data governance.
- Collaborate with data engineers, data scientists, and analysts to optimize and streamline data workflows, ETL processes, and analytical pipelines on the Databricks platform.
- Provide technical guidance and support to users of the Databricks platform, including training, documentation, and troubleshooting assistance.
- Stay current with Databricks releases, updates, and best practices to continuously improve the Databricks environment and maximize its value to the organization.
Preferred Qualifications
- Undergraduate studies in computer science, management information systems, business, statistics, math, a related field or comparable experience and education strongly preferred.
- Graduate studies in business, statistics, math, computer science or a related field are a plus.
- Certifications are not required but encouraged.
- Three to five years of relevant experience with data quality rules, data management organization/standards, practices and software development.
- Experience in data warehousing, statistical analysis, data models, and queries.
- One to three years’ experience with Cloud technology and infrastructure including security and access management.
- Insurance/financial services industry knowledge a plus.
Benefits
-
No information provided on Benefits.