Staff Software Engineer - Data and AI Platforms

Staff Software Engineer – Data and AI Platforms

BS, MS, or PhD in Computer Science or a related technical discipline, or equivalent experience
7+ years of hands-on coding experience in scripting (Python, PHP, Ruby) and systems languages (Rust, C++, C#, Go, Java, or Scala)
Proficiency in managing large-scale data operations, including designing scalable pipelines (streaming and batch)
Experience with data governance, metadata, and data lineage tooling like Open Lineage or Marquez
Deep experience working with building CI/CD pipelines for data infrastructure and associated observability and monitoring tooling such as Prometheus, Grafana, OpenTelemetry, Prometheus, or Honeycomb
Experience with addressing end to end data needs for working with complex data and delivering this data ready form model training
Extensive experience with scaling containerized applications on Kubernetes or Mesos
Strong experience with AWS, GCP, or Azure
Familiarity with Infrastructure as Code (e.g., Terraform, Ansible) and monitoring tools (Datadog, Prometheus)
Proven ability to work with diverse, cross-functional stakeholders and teams

Develop and maintain the tooling and infrastructure that drives the entire data lifecycle at CZI, from ingestion and processing to secure storage and access
Partner with researchers and engineers across various domains, including genetics, imaging, and scientific literature
Work on various use cases, from web analytics to complex model training, ensuring data accessibility and performance for teams across CZI
Design and implement flexible, scalable, and performant systems to address our stakeholders’ needs, leveraging technologies like Argo Workflows and Spark for mass-scale job processing and orchestration; Vast Data, Delta Lake, and Apache Iceberg for data management and access; and cloud and on-prem HPC resources