Big Data Platform Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Big Data Platform Engineer managing AKS-based infrastructure, administering Azure services, and ensuring platform compliance. Optimizing performance through infrastructure automation and collaborative support with Data Engineering teams.
Responsibilities:
- Manage and support AKS-based Big Data platform infrastructure.
- Administer Apache NiFi, Azure Event Hub, and Azure Functions.
- Implement and maintain CI/CD pipelines using Azure DevOps.
- Monitor platform health using Azure Monitor and Log Analytics.
- Troubleshoot platform, networking, and infrastructure issues.
- Implement infrastructure automation through Terraform and Bicep.
- Ensure platform security, governance, and compliance.
- Collaborate with Data Engineering teams for platform enablement.
- Perform upgrades, patching, capacity planning, and optimization.
- Support production incidents and platform reliability initiatives.
Requirements:
- 8+ years of overall IT experience
- 5+ years of experience in Cloud Infrastructure Engineering
- 4+ years of experience working with Kubernetes platforms
- 3+ years of experience supporting enterprise-scale Data Platforms
- 3+ years of experience in DevOps/SRE practices
- 2+ years of experience working with Azure Data Services
- Strong hands-on experience in Apache NiFi
- Strong hands-on experience in Azure Event Hub
- Strong hands-on experience in Azure Functions
- Strong hands-on experience in Enterprise-scale Data Platform Administration
- Strong Expertise in Azure Kubernetes Service – AKS
- Strong Expertise in Kubernetes and Azure Networking
- Strong Expertise in Docker and Containerization
- Strong Expertise in Azure DevOps and CI/CD
- Strong Expertise in Monitoring and Observability
- Strong hands-on experience in Azure Event Hub
- Strong hands-on experience in Azure Functions
- Strong hands-on experience in Apache NiFi
- Must-Have Skills in Infrastructure as Code
- Good-to-Have Skills in Platform Engineering, Observability, Security, and Data Platform
Similar Jobs
AI Engineer II, AI Platform
AI Engineer II designing and maintaining AI platform components enabling AI-powered features at MeridianLink. Collaborating with teams to understand integration needs and establish standards.
Data Platform Engineer
Data Platform Engineer responsible for designing and operating cloud platform for secure data solutions. Focus on AWS platform engineering, automation, and operational excellence.
AI Engineer, Agent/Platform Tracks
AI Engineer developing specialized pharmacovigilance agents to enhance clinical research safety at Parexel. Collaborating with cross-functional teams to implement AI solutions using large language models.
Staff Engineer, AI Platform – Architecture
Staff Engineer responsible for guiding AI platform architecture and reusable component strategy at Shield AI. Leading engineering efforts for AI enablement and governance.
Senior Platform Engineer
Sr. Staff Platform Engineer developing internal developer platforms at eHealth. Leading AWS Cloud infrastructure optimization and AI/ML integration efforts to empower engineering teams.
Engineer II – Platform
Platform Engineer designing and building solutions for technology integration at OSC. Responsible for multi-cloud infrastructure, AI tooling, system architecture, and collaboration across teams.
Director – Platform & Infrastructure
Director of Platform & Infrastructure leading modern infrastructure initiatives for ShiftKey's SaaS and Marketplace engineering teams. Collaborating with distributed teams across various regions and time zones.
Senior Platform Engineer, Data Platform
Senior Platform Engineer joining the Data Platform team to manage identity and security services. Role involves backend work, Dgraph internals, and cross-team collaboration.
Principal Engineer I – Prepurchase Platform
Principal Engineer I specializing in building backend services for Ticketmaster's Prepurchase Platform. Writing code, solving problems, and modernizing systems for high-demand ticket sales.
Staff Platform Engineer
Staff Platform Engineer responsible for Moxie's infrastructure and incident response, utilizing DevOps and SRE practices to ensure platform reliability. Focus on cloud systems and CI/CD pipelines.
Senior Software Engineer, Identity Platform
Senior Software Engineer at Prove developing identity verification solutions. Collaborating with teams to build robust and high-performance applications.
Senior AIOps Platform Engineer
Senior AIOps Strategy & Implementation Engineer at EY optimizing observability platforms and building automation workflows in cloud and on-prem environments.
Principal Software Engineer, MDR Platform
Senior Software Engineer developing and maintaining core Golang applications for cybersecurity operations. Collaborating across teams and shaping architectural direction in a growing MDR platform.
Data Engineer – Platform
Data Engineer, Platform at DraftKings building and scaling data ingestion pipelines. Enhance customer experiences and streamline operations through innovative data solutions.
Platform Engineer
Platform Engineer supporting design and operation of Marathon’s cloud infrastructure. Collaborating with teams to implement automation, IaC, and maintain operational standards.
Senior Data Engineer – Data Platform
Senior Data Engineer building and operating foundational data infrastructure for analytics at SunStrong Management. Collaborates with teams to deliver production-grade solutions with quality controls and security.
Platform Engineer
Platform Engineer developing backend systems and infrastructure for financial platforms at Shift Markets. Building scalable systems with TypeScript and Python in a flexible remote environment.
Senior Mobile Engineer – Cross-Platform
Senior Mobile Engineer responsible for developing the Grove patient-facing mobile application. Collaborating with offshore teams and ensuring high-quality mobile experiences across iOS and Android.
Senior Software Engineer, Platform
Senior Software Engineer responsible for developing data-intensive APIs and maintaining infrastructure systems. Collaborate within a cross-functional team in a remote environment across Canada.
Platform Engineer – Karriereambitionen
Platform Engineer developing cloud projects at Auralis Group. Leading teams and managing cloud infrastructures with flexible remote working conditions.