Senior Data Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Data Engineer building cloud-native Databricks and AWS data platforms for a commercial laser fusion company. Designing real-time pipelines, observability, and AI-enabled engineering workflows.
Responsibilities:
- Design and build scalable cloud-native data platforms from greenfield to production
- Design and implement near-real-time ingestion pipelines using event-driven patterns
- Define and enforce data platform standards including Data Lake/Lakehouse principles, medallion architecture, and data contracts
- Refactor, optimize, and modernize Spark and PySpark scripts
- Introduce best practices for code quality, testing, and CI/CD across data pipelines
- Drive adoption of AI tooling and agentic workflows within the data engineering team
- Ensure data quality, observability, and reliability across pipelines and platforms
- Design and deliver self-service tooling and microservices that abstract platform complexity
- Collaborate directly with the Customer’s scientific and engineering teams
- Monitor platform performance, reliability, and cloud infrastructure costs
Requirements:
- 5+ years of experience in data engineering
- 3+ years of production experience with Databricks and Delta Lake
- Strong expertise in medallion architecture and event-driven asynchronous pipeline design
- Advanced proficiency in Python, PySpark, and SQL
- Deep hands-on experience with Databricks, including Unity Catalog, Lakeflow Jobs/Workflows, and job optimization
- Strong AWS expertise including S3, IAM, KMS, AWS Batch, ECR, EventBridge, Lambda, or API Gateway
- Experience with infrastructure as code and CI/CD using Terraform and Databricks Asset Bundles
- Practical experience with Git workflows, pull request reviews, and automated testing
- Experience configuring secure CI pipelines using OIDC or service principals
- Practical MLflow experience including experiment tracking, model registration, and reproducibility practices
- Strong understanding of data quality, lineage, provenance, schema contracts, and versioning
- Experience with data quality frameworks such as Great Expectations, Soda, pyDeequ, or Databricks Labs DQX
- Experience with monitoring, observability, and cloud cost optimization
- Upper-Intermediate English level or higher
- Strong analytical mindset
- Ownership and accountability
- Ability to communicate complex technical concepts clearly
- Proactive approach to problem-solving
- Attention to scalability and maintainability
- Collaborative and team-oriented attitude
Benefits:
- Remote work option
- Opportunity to work on innovative global projects
- Opportunity to contribute to breakthrough fusion research and clean-energy sustainability
- Exposure to engineering excellence, automation, and AI adoption within the data platform ecosystem
Similar Jobs
Associate Data Engineer
Associate Data Engineer building Azure, Synapse, and Microsoft Fabric data solutions for Baker Tilly’s mid-market and enterprise clients. Developing pipelines, lakehouse models, governance, and analytics-ready assets.
Data Engineer
Data Engineer building Snowflake and Azure claims data infrastructure for ProgenyHealth. Supporting healthcare analytics, reporting, security, and reliable enterprise data pipelines.
Data Engineer
Data Engineer building scalable pipelines and analytics warehouses for Troveo’s licensed AI training data platform. Supporting data modeling, quality, reporting, and reliable delivery to AI teams.
Data Engineering Lead
Data Engineering Lead guiding engineers and consolidating Oxford Global Resources’ Azure and Microsoft Fabric data platform. Owning architecture, ETL reliability, governance, and delivery.
Principal Engineer – Data Retrieval
Principal Engineer architecting web-scale crawling, ingestion, and retrieval infrastructure for NVIDIA’s foundation AI models. Leading distributed systems design, production scaling, and high-quality pretraining data delivery.
IT Data Science Principal, Data Warehouse
Leading enterprise data warehouse, analytics, and AI capabilities for Cornerstone Building Brands, a North American building-products manufacturer. Driving roadmaps, governance, modernization, and multidisciplinary delivery.
Data Engineer
Data Engineer building reliable pipelines for Branch’s fintech platform, which accelerates payments and provides financial services. Using Python, SQL, Airflow, Snowflake, DBT, and GCP.
Data Engineer
Data Engineer building reliable pipelines and transformations for Definitive Healthcare’s healthcare data products. Supporting Claims and Consumer domains with Python, SQL, Spark, and cloud platforms.
Senior Data Platform Engineer
Senior Data Platform Engineer building scalable data lakes, ingestion pipelines, and Kubernetes infrastructure. Supporting analytics, product applications, AI workloads, and large-scale model execution.
Medical Receptionist, Data Migration Specialist
Remote medical receptionist and data migration specialist transferring patient records from Elation Health to Dakota Health. Supporting U.S. practice scheduling, patient calls, and administrative workflows.
Data Engineer
Data Engineer building Azure Databricks pipelines for credit-card data at CGI, a global IT and business consulting firm. Modernizing financial data platforms and controls.
Working Student – Data Engineering
Working Student building Python and SQL data pipelines for AgentixLake’s cloud data and AI engineering projects. Supporting data quality, lakehouse platforms, testing, and monitoring.
Data Engineer – Temporary
Data Engineer building reliable Zuora-to-GCP ETL pipelines for Applaudo’s AI-powered solutions. Maintaining integrations, data quality, monitoring, and warehouse validation in an async distributed team.
WEKA Data Platform Administrator
WEKA Data Platform Administrator operating NeuralMesh enterprise AI data infrastructure for GPU-cloud customers. Managing storage, Kubernetes, networking, automation, and customer workload performance.
Analytics Engineer – Data Platform
Analytics Engineer building Snowflake and dbt models for Aline’s senior-living resident lifecycle platform. Creating governed data foundations for BI reporting and AI-driven care outcomes.
Senior Data Engineer – Mid Shift
Data Engineer building AWS and Snowflake platforms for Shore360, an Australian-owned Philippine staffing and administrative support firm. Developing reliable pipelines, APIs, and data quality controls.
Senior Software Engineer, Data Infrastructure – RDBMS
Senior engineer operating petabyte-scale PostgreSQL infrastructure for TRM Labs’ AI-powered crime-intelligence platforms. Automating database operations, optimizing performance, and managing CDC pipelines remotely across LATAM.
Data Architect
Databricks Data Architect designing secure Lakehouse architectures and AI-ready data platforms for Devoteam clients. Modernizing legacy systems across cloud and multi-cloud environments.
Senior Data Engineer
Senior Data Engineer building auditable GCP data pipelines for DysrupIT’s financial billing and invoicing platform. Ensuring accurate reconciliation, lineage, and trusted financial data.
Senior Software Engineer, Data Acquisition
Senior software engineer building distributed crawling and ingestion systems for People Data Labs, a provider of people and company data. Improving scalable backend services and data products.