Senior Data Platform Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Data Platform Engineer building scalable data lakes, ingestion pipelines, and Kubernetes infrastructure. Supporting analytics, product applications, AI workloads, and large-scale model execution.
Responsibilities:
- Build complex, high-volume batch and near-real-time ingestion services
- Help evolve the platform’s streaming capabilities
- Design and implement data solutions for application requirements in a distributed microservices environment
- Develop and optimize large-scale data pipelines for batch and streaming use cases
- Ensure data quality, compliance, and governance
- Manage and optimize Kubernetes infrastructure using KEDA to dynamically scale resources for large-scale data processing workloads
- Own the full lifecycle of the data stack using Terraform and Kubernetes, from provisioning to application-level frameworks
- Provide technical guidance and drive best practices around infrastructure, CI/CD, testing, and system design
- Perform memory management and performance tuning for large-scale distributed systems
Requirements:
- 5+ years of experience as a Data Platform/Infra Engineer
- Proven track record of building and operating scalable data infrastructure specifically within Data Lake architectures
- Proven experience designing and operating large-scale data processing workloads on Kubernetes, including workload autoscaling with KEDA
- Strong proficiency in Python and SQL
- Hands-on experience with Infrastructure as Code, preferably Terraform
- Experience with managing a data orchestration platform such as Airflow or Prefect
- Experience with Snowflake
Similar Jobs
Principal Engineer – Data Retrieval
Principal Engineer architecting web-scale crawling, ingestion, and retrieval infrastructure for NVIDIA’s foundation AI models. Leading distributed systems design, production scaling, and high-quality pretraining data delivery.
IT Data Science Principal, Data Warehouse
Leading enterprise data warehouse, analytics, and AI capabilities for Cornerstone Building Brands, a North American building-products manufacturer. Driving roadmaps, governance, modernization, and multidisciplinary delivery.
Data Engineer
Data Engineer building reliable pipelines for Branch’s fintech platform, which accelerates payments and provides financial services. Using Python, SQL, Airflow, Snowflake, DBT, and GCP.
Data Engineer
Data Engineer building reliable pipelines and transformations for Definitive Healthcare’s healthcare data products. Supporting Claims and Consumer domains with Python, SQL, Spark, and cloud platforms.
Medical Receptionist, Data Migration Specialist
Remote medical receptionist and data migration specialist transferring patient records from Elation Health to Dakota Health. Supporting U.S. practice scheduling, patient calls, and administrative workflows.
Data Engineer
Data Engineer building Azure Databricks pipelines for credit-card data at CGI, a global IT and business consulting firm. Modernizing financial data platforms and controls.
Working Student – Data Engineering
Working Student building Python and SQL data pipelines for AgentixLake’s cloud data and AI engineering projects. Supporting data quality, lakehouse platforms, testing, and monitoring.
Data Engineer – Temporary
Data Engineer building reliable Zuora-to-GCP ETL pipelines for Applaudo’s AI-powered solutions. Maintaining integrations, data quality, monitoring, and warehouse validation in an async distributed team.
WEKA Data Platform Administrator
WEKA Data Platform Administrator operating NeuralMesh enterprise AI data infrastructure for GPU-cloud customers. Managing storage, Kubernetes, networking, automation, and customer workload performance.
Analytics Engineer – Data Platform
Analytics Engineer building Snowflake and dbt models for Aline’s senior-living resident lifecycle platform. Creating governed data foundations for BI reporting and AI-driven care outcomes.
Senior Data Engineer – Mid Shift
Data Engineer building AWS and Snowflake platforms for Shore360, an Australian-owned Philippine staffing and administrative support firm. Developing reliable pipelines, APIs, and data quality controls.
Senior Software Engineer, Data Infrastructure – RDBMS
Senior engineer operating petabyte-scale PostgreSQL infrastructure for TRM Labs’ AI-powered crime-intelligence platforms. Automating database operations, optimizing performance, and managing CDC pipelines remotely across LATAM.
Data Architect
Databricks Data Architect designing secure Lakehouse architectures and AI-ready data platforms for Devoteam clients. Modernizing legacy systems across cloud and multi-cloud environments.
Senior Data Engineer
Senior Data Engineer building auditable GCP data pipelines for DysrupIT’s financial billing and invoicing platform. Ensuring accurate reconciliation, lineage, and trusted financial data.
Senior Software Engineer, Data Acquisition
Senior software engineer building distributed crawling and ingestion systems for People Data Labs, a provider of people and company data. Improving scalable backend services and data products.
IT Architect, Data Scientist
IT Architect/Data Scientist supporting DLH’s federal health and national-security programs. Leading cloud architecture, data science, interoperability, security engineering, and compliance for SCHARE.
Senior Business Data Engineer
Senior Business Data Engineer building AWS data infrastructure and automated reporting for Nex’s family motion-gaming platform. Improving pipeline reliability and supporting cross-functional business teams.
Senior Data Engineer
Senior Data Engineer building Python, Spark, and Databricks pipelines for Quantiphi’s data and AI solutions. Leading cloud migration, CI/CD automation, and scalable analytics engineering.
Senior Data Engineer – IBM DB2
Senior Data Engineer modernizing IBM DB2 data environments. Building reliable ETL/ELT pipelines and AWS integrations for enterprise applications and payroll systems.
Lead Data Architect
Lead Data Architect owning enterprise data architecture for C5MI Insight’s federal SAP S/4HANA modernization program. Leading data governance, migration, analytics architecture, and master data teams.