Cloud Engineer – Azure Databricks
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Cloud Engineer administering Azure Databricks, Terraform, ADLS, and Azure ML infrastructure. Supporting global digital transformation projects for a multinational IT services powerhouse.
Responsibilities:
- Design, implement, and administer the Azure foundation supporting the client platform
- Author and maintain reusable Terraform modules provisioning Databricks workspaces, ADLS Gen2, Data Factory, Key Vault, networking, and Azure ML
- Manage remote state, module versioning, drift detection, and deployment pipelines across development, test, and production
- Design and implement private networking and security postures, including private endpoints, hub-and-spoke topology, VNet injection, NSGs, and RBAC
- Own ADLS Gen2 infrastructure, including storage architecture, lifecycle policies, encryption, and access patterns
- Manage Azure Data Factory platform operations, including integration runtime provisioning, linked-service credential design, and deployment automation
- Provision and operate infrastructure for machine-learning workloads, including Azure ML workspaces, GPU capacity planning, MLflow registry integration, and Unity Catalog connectivity
- Build and maintain monitoring with Azure Monitor and Log Analytics, including alerting and operational runbooks
- Manage FinOps and capacity planning, optimizing DBU and storage costs
- Establish platform standards, reference configurations, and multi-workspace strategies for the Databricks administration team
- Advise on Unity Catalog metastore design, permission models, and Delta Sharing configurations
- Oversee identity and access management through Entra ID integration, SCIM provisioning, and identity federation
- Enforce compute governance through cluster policies, instance pools, node-type standards, and serverless evaluation
- Perform cross-boundary diagnostics on failures spanning infrastructure, Databricks configuration, and workloads
- Guide stakeholders and build consensus around architectural approaches
Requirements:
- 6+ years in cloud infrastructure or platform engineering, with at least 4 years focused on Microsoft Azure
- Expert-level, hands-on Terraform experience, including authoring production modules, managing remote state, and running IaC through CI/CD
- Demonstrable hands-on Databricks administration experience, including account and workspace administration, Unity Catalog, cluster policies, identity federation, and cost governance
- Deep working knowledge of Azure Networking & Security, including private endpoints, VNets, hub-and-spoke design, NSGs, Entra ID, managed identities, RBAC, and Key Vault
- Enterprise-scale ADLS Gen2 design and access control experience
- Azure Data Factory platform-side experience with integration runtimes, managed VNet, credential management, and deployment automation
- Strong Python automation and scripting skills, plus PowerShell and/or Bash
- Sufficient working knowledge of Spark and SQL to diagnose infrastructure/configuration-level performance issues
- Track record as a senior technical partner capable of guiding stakeholders and building consensus around architectural approaches
- Resilience, emotional intelligence, and focus on agile delivery
- Deep understanding of the difference between coding and engineering
- Advanced oral English
- Advanced Spanish
- Desired: Databricks Asset Bundles, Terraform Databricks provider, workspace-as-code patterns
- Desired: Unity Catalog migration or multi-workspace consolidation experience
- Desired: Azure Machine Learning, MLflow, or model-serving infrastructure experience
- Desired: Kubernetes/AKS and containerized workloads experience
- Desired: Policy as Code experience with Azure Policy, OPA, Sentinel, or Checkov
- Desired: Event-driven infrastructure familiarity with Event Hubs, Kafka, or Stream Analytics
- Desired: Formal FinOps practice experience
- Desired: Multi-region or multi-tenant Databricks deployment experience
- Relevant certifications are desired, including Databricks Certified Data Engineer Professional, Databricks platform administrator credential, Microsoft Azure Solutions Architect Expert (AZ-305), Azure Administrator Associate (AZ-104), DevOps Engineer Expert (AZ-400), or HashiCorp Certified Terraform Associate
Benefits:
- Remote work arrangement
- Flexibility to support your lifestyle
- Global career opportunities
- Exposure to high-impact projects
- Work within an international network of expertise
- Work on end-to-end digital transformation projects
- Workplace recognized as one of the most sustainable corporations in the world
Similar Jobs
Senior Data Platform Engineer – Cloud Architect
Arquitecto cloud y de plataformas de datos AWS para Expleo. Diseñando plataformas empresariales, automatización e infraestructura como código con Terraform.
Senior Cloud Engineer – Saudi National
Senior Cloud Engineer operating Kubernetes, CI/CD, and Intune across GCP and AWS. Supporting infrastructure reliability, automation, security, and cost optimization for a Saudi Arabian client.
Cloud Engineering Technical Leader
Cisco cloud engineering leader designing and operating secure, scalable infrastructure and platforms. Driving automation, architecture, reliability, and technical strategy across teams.
Senior Cloud Engineer – Principal
Senior principal cloud engineer designing BMC2 hybrid infrastructure for SAIC’s defense technology programs. Developing automation, secure network flows, cross-domain solutions, and resilient cloud operations.
Engineering Manager – Cloud-Native Backend Systems
Engineering Manager leading cloud-native backend teams for Salve.Inno Consulting’s recruitment clients. Guiding architecture, delivery, reliability, and engineer development across AI-enabled products.
Engineering Manager – Cloud-Native Backend Systems
Engineering Manager leading cloud-native backend systems and software engineers. Driving architecture, delivery, reliability, and team development for internationally used AI-enabled products.
Engineering Manager – Cloud-Native Backend Systems
Engineering Manager leading cloud-native backend teams building internationally used AI-enabled products. Guiding people, architecture, microservices, delivery, and engineering quality across the European Union.
Engineering Manager – Cloud-Native Backend Systems
Engineering Manager leading cloud-native backend teams for Salve.Inno Consulting’s AI-enabled products. Guiding people, architecture, delivery, and engineering quality across distributed systems.
Engineering Manager – Cloud-Native Backend Systems
Engineering Manager leading cloud-native backend teams for AI-enabled products. Guiding architecture, delivery, engineering quality, and people development across distributed systems.
Architect, Modern Platforms – Cloud Infrastructure
Cloud Infrastructure Architect designing secure, scalable AWS, Azure, and GCP environments. Presidio delivers cloud, cybersecurity, AI, and managed technology solutions for enterprise clients.
Senior Data Engineer - AWS
Engenheiro de Dados Sênior construindo pipelines, Data Lakes e arquiteturas AWS. Consultoria de tecnologia conectando dados, plataformas e IA para transformar grandes empresas.
Cloud & Platform Engineer
Rapid Solutions Pod Lead leading client engagements and engineering pods for AHEAD's cloud, platform engineering, and AI transformation solutions. Owning delivery, governance, recommendations, and offering development.
Online Visiting Professor – Cloud Computing
Online Cloud Computing Professor teaching practitioner-focused courses for DeVry University. Developing syllabi, delivering virtual instruction, assessing students, and integrating responsible AI technologies.
Senior Cloud Architect
Senior Cloud Architect modernizing secure, scalable cloud platforms for Guild Mortgage, a mortgage banking company. Leading Azure architecture, automation, DevSecOps, and enterprise platform engineering.
Cloud Engineer – Subject Matter Expert
Cloud Engineer SME architecting secure Azure data, analytics, and pipeline infrastructure. Supporting U.S. federal defense agencies and Army logistics within DoD IL5/IL6 environments.
Cloud Engineer – Principal
Principal Cloud Engineer designing and operating secure multi-tenant cloud platforms for GDIT’s U.S. government missions. Automating, migrating, and optimizing AWS, Azure, GCP, or OCI environments.
Azure Cloud Engineer
Azure Cloud Engineer managing scalable Azure infrastructure for Market Cloud’s IT consulting services. Deploying Kubernetes, automating infrastructure, and optimizing secure cloud-native solutions.
Cloud Architect – Networking
Cloud Architect leading secure Azure networking architecture and technical delivery for 3Cloud client engagements. Coaching infrastructure teams and unblocking complex cloud implementations.
Senior Salesforce Health Cloud Developer
Senior Salesforce Health Cloud Developer building and integrating government healthcare systems. Developing Apex, LWC, APIs, and CI/CD solutions in Salesforce Government Cloud.
Technical Engineer – Cloud Engineer
Cloud Engineer optimizing cloud platforms, enterprise architecture, and automation for Highmark Health’s national insurance and healthcare network. Supporting reliable technical solutions and next-generation technologies.