Senior Data Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Data Engineer building advanced, cloud-native data platforms and migrating legacy systems to the cloud. Collaborating with teams on AI architectures and data pipelines.
Responsibilities:
- Design and build scalable, cloud-native data platforms from greenfield to production
- Implement near-real-time ingestion pipelines using event-driven patterns
- Define and enforce platform standards, including Data Lake / Lakehouse principles, medallion architecture, and data contracts
- Refactor and optimise existing Spark and PySpark scripts for performance and maintainability
- Introduce best practices for code quality, testing, and CI/CD across data pipelines
- Drive adoption of AI tooling and agentic workflows within the data engineering team
- Ensure data quality, observability, and reliability across all pipelines and platforms
- Develop self-service tooling and microservices to simplify platform usage for other teams
Requirements:
- 5+ years of professional experience in Data Engineering
- Strong Python and SQL development skills for pipeline development and optimisation
- Proficiency in Apache Spark / PySpark, including query optimisation and performance tuning
- Hands-on experience with Databricks (preferred) or Snowflake
- Experience with at least one major cloud provider: Azure (preferred), AWS, or GCP
- Experience with stream processing technologies (Kafka, Spark Structured Streaming)
- Solid understanding of ETL/ELT patterns, data modelling (dimensional, Data Vault), and data warehousing
- Experience with orchestration tools (Apache Airflow, Azure Data Factory, or equivalent)
- Knowledge of Infrastructure as Code (Terraform or equivalent)
- Understanding of production-grade system requirements: reliability, scalability, observability, and performance
- Upper-Intermediate English level WILL BE A PLUS
- Familiarity with RAG pipeline design and LLM integration patterns
- Knowledge of data governance frameworks and tools (Unity Catalog, Apache Atlas, or similar)
- Experience with dbt for data transformation and modelling
- Familiarity with MLflow, Feature Stores, or ML platform integration
Benefits:
- Employees can work remotely
Similar Jobs
Senior Data Engineer – PowerBI, Data Integration
Senior Data Engineer at 3Pillar, leveraging data analytics to provide competitive advantages for clients. Work with AI-native products in a global team environment.
Senior Data Engineer, PowerBI, Data Integration
Senior Data Engineer contributing to AI-driven data analytics projects for global clients. Involves collaboration, coding, data processing, and pipeline management.
Senior Data Engineer, PowerBI, Data Integration
Senior Data Engineer shaping data analytics and solutions at 3Pillar Global. Collaborating with teams and clients to drive impactful data engineering projects.
Engenheiro de Dados Sênior
Data Engineer developing and maintaining scalable data pipelines in cloud environments for projects at FCamara. Collaborating with engineering and analytics teams to ensure data quality and performance.
Lead Data Engineer
Lead Data Engineer developing PySpark data solutions for U.S. clients in a 100% remote role from Latin America. Collaborate with cross-functional teams to meet business requirements.
Senior Data Architect – Hands on
Senior Data Architect at Stratus, managing canonical data architecture and supporting AI/ML workloads. Driving modernization and technical leadership in data engineering.
Engenheiro de Dados Sr
Senior Data Engineer developing and maintaining data pipelines for innovative agricultural solutions at Solinftec. Collaborating with teams to ensure data integrity and efficiency.
Senior Data Engineer
Senior Data Engineer responsible for designing and optimizing scalable data infrastructure. Collaborating with teams to enhance patient-centered healthcare through advanced analytics.
Data Architect
Data Architect role involves leading data architecture initiatives and collaborating with cross-functional teams. Positions focused on enterprise data solutions using PostgreSQL and SQL Server in a remote setting.
Principal Data Architect
Principal Data Architect leading data architecture and integration efforts at Lifeway. Overseeing data governance and technical mentorship while aligning with business priorities.
Senior Data Engineer
Senior Data Engineer building and maintaining customer data infrastructure for Jane's marketing platform. Overseeing data systems and driving tooling strategies while collaborating across teams.
Data Engineer
Data Engineer role focused on Databricks and SAP integration for international projects. Collaborating with teams from North America for data transformation, reporting, and analytics.
Senior Data Engineer, GCP
Senior Data Engineer responsible for data collection and processing in a digital streaming environment. Join a platform-focused engineering team building scalable data solutions on GCP.
Architect, Data Engineer
Lead Architect Data Engineer at Quantiphi, overseeing next-gen data layer for Agentic AI. Collaborating with clients and engineering teams to optimize data architecture.
Data Migration Specialist
Data Migration Specialist responsible for exporting and organizing data for FundCount's database. Collaborating with clients and internal teams for efficient onboarding and data validation.
SAP Data Migration Consultant – Senior
SAP Data Migration Consultant focusing on data migration projects in SAP FI and MM modules. Remote opportunity with a requirement for advanced knowledge and experience in SAP data migration.
Data Engineer – Healthcare Analytics
Data Engineer involved in building a Contract Performance Analytics platform for healthcare. Focusing on data architecture, pipeline development, and integration of healthcare data.
Senior Data Engineer, Oracle
Senior Data Engineer developing and maintaining OLAP ETL projects for Erbis. Collaborating with Java and Data Science teams to support Oracle database needs.
Principal Data Architect
Principal Data Architect managing the strategy, design, and evolution of enterprise data ecosystem. Collaborating with teams to define and execute scalable data architecture at Mercury Insurance.
Senior Data Engineer
Senior Data Engineer at Reply managing data pipelines and collaborating on data-driven solutions. Working with emerging technologies like AI, Cloud, and Big Data.