Senior Data Engineer – Databricks, Azure Data Platform
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Data Engineer responsible for building and maintaining a scalable data platform on Databricks. Supporting key processes in the insurance and reinsurance environment with multiple data sources integration.
Responsibilities:
- Support the build-out and further development of a modern data platform in the insurance and reinsurance environment
- Help design, build and maintain a scalable data platform based on Databricks on Azure
- Centralize fragmented data sources and establish data quality checks
- Organize data into meaningful business domains and provide reliable data layers for reporting, applications, analytics and future AI use cases
- Build, maintain and optimize data pipelines in Databricks
- Design and implement scalable Lakehouse and Medallion Architecture concepts
- Structure data into meaningful business domains
- Develop and maintain Bronze, Silver and Gold data layers
- Create Gold layers to support business use cases, especially applications, reports and analytics
- Integrate and process data from multiple source systems, primarily SAP
- Implement data quality checks, monitoring and validation mechanisms
- Support data modelling, database design, storage concepts and performance optimization
- Contribute to governance, security and access management concepts
- Support or build CI/CD pipelines for data engineering workflows
- Support reporting requirements, most likely with Power BI and/or Databricks
- Prepare the data platform for future AI and advanced analytics use cases
- Enable easy onboarding of business users onto the data platform
- Work closely with infrastructure, architecture, business, reporting and data teams
Requirements:
- Several years of experience as a Senior Data Engineer or in a comparable role
- Strong experience in building and maintaining data pipelines with Databricks
- Solid knowledge of Lakehouse Architecture and Medallion Architecture
- Good experience with cloud-based data platforms, ideally on Azure
- Strong understanding of data quality, data governance, security and access management
- Experience with data modelling, databases, storage technologies and performance optimization
- Experience with CI/CD concepts in a data engineering environment
- Good knowledge of the Databricks ecosystem and related components
- Experience with SAP data sources is a strong advantage
- Experience in insurance, reinsurance, finance or actuarial environments is beneficial
- Knowledge of Power BI and/or Databricks reporting is an advantage
- Ability to structure complex data landscapes and translate them into scalable solutions
- Senior, independent and structured working style
- Strong communication skills with both technical and business stakeholders.
Benefits:
- Flexible work arrangements
- Professional development opportunities
Similar Jobs
Consultant – Cloud Infrastructure Services
Consultant for Cloud Infrastructure Services at Red Hat delivering customer cloud projects and improving customer experiences. Engaging in design, implementation, and support of Red Hat cloud solutions.
Senior SAP Business One Consultant – Cloud Migration, Future Architecture
SAP Business One Consultant at Cloudiax focusing on cloud migration from SQL Server to SAP HANA. Collaborating with partners and clients to ensure their migration strategies succeed.
Senior Data Engineer – Azure Cloud
Engenheiro de Dados SR atuando na construção e sustentação de pipelines de dados em Azure. Contribuindo para soluções escaláveis e eficientes em projetos de Data & AI.
Senior Cloud Platform Engineer – Tech Cloud Development
Senior Python Cloud Platform & Full-Stack Engineer at EY building scalable cloud-native solutions and automation frameworks. Focus on improving operational efficiency through engineering tools and dashboards.
Principal Cloud Architect
Principal Cloud Architect leading multi-cloud infrastructure design and optimization at Banner Health. Collaborating with cross-functional teams to ensure scalability, security, and innovation across cloud systems.
Back-End Python Engineer, AWS – Desktop
Back-End Python Engineer designing scalable RESTful APIs and microservices at Miratech. Focused on cloud technologies and backend development in a global IT services context.
Back-End Python Engineer – AWS
Back-End Python Engineer developing scalable AWS applications for Miratech. Collaborating across teams to design and maintain secure backend systems using cloud technologies and Python.
Lead Java Developer – Azure
Lead Java Developer for Satalia creating enterprise quality optimisation and ML solutions. Collaborate with data science teams, ensuring robust and high-quality cloud solutions for customers.
Data Engineer, Snowflake, dbt, Azure
Data Engineer developing data pipelines and transformations using Snowflake and Azure Data Factory. Working remotely with an expert team in Data & AI at Logicalis Spain.
Cloud Architect
Cloud Architect at Techmahindra responsible for designing scalable enterprise solutions. Collaborating cross-functionally and ensuring architecture alignment with standards and practices.
Cloud Security Engineer
Join Periferia IT Group as a Cloud Security Engineer managing security for applications and microservices. Work fully remote with opportunities for professional growth in a tech-driven environment.
Cloud Network Engineer – III
Cloud Network Engineer III supporting multi-cloud infrastructure as part of a healthcare technology team. Responsible for documentation, development, and maintenance of cloud implementations.
Cloud Support Engineer
Cloud Support Engineer at Net at Work providing advanced remote technical support across multiple client environments. Focusing on swift service restoration and detailed documentation of technical issues.
Senior Tech Cloud Developer – ITSM, Service Integration Management
Senior Manager at EY responsible for ITIL/ITSM implementation and service management solutions. Leading strategic plans, project delivery, and service model framework creation with corporate stakeholders.
Software Engineer – API, Microservices, AWS
API/Microservices/AWS Software Engineer at CSAA Insurance Group designing scalable cloud-based microservices using modern tech. Collaborating in an Agile environment to deliver high-performing APIs and services.
Cloud Data Architect
Cloud Data Architect designing cloud-native data platforms for business insights at Data Ideology. Leading modernization efforts and supporting enterprise data systems with scalable solutions.
Azure Engineer
Azure Engineer designing, implementing, and managing cloud solutions for enterprise clients. Collaborating with cross-functional teams and ensuring optimal performance and security in Azure.
Cloud Ops Engineer
CloudOps Engineer managing AWS cloud infrastructure for Bloom Growth's platform. Automating operations, enhancing system reliability, and supporting production environments for optimal customer experience.
Senior Azure Cloud Engineer
Senior Azure Cloud Engineer designing, automating, and operating secure Azure cloud platforms at Applaudo. Collaborating across teams to implement cloud best practices and improve platform reliability.
Cloud Engineer
Cloud Engineer supporting and operating Microsoft Azure and Microsoft 365 platforms. Responsible for designing, implementing, and maintaining cloud services and infrastructure with a focus on identity solutions.