Lead Data Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Lead Data Engineer directing Microsoft Fabric, Python, and PySpark data engagements for CreateFuture, an AI-native consulting partner. Leading teams, architecture, pipelines, and client delivery.
Responsibilities:
- Own the technical direction of a data engagement
- Set standards for pipeline architecture, data modelling, testing, and operational readiness
- Lead and technically direct a team of data engineers, allocating work, reviewing designs and code, and unblocking technical challenges
- Design and evolve data platform architecture on Microsoft Fabric, including data pipelines, Azure OneLake, and the broader Azure ecosystem
- Set and enforce CI/CD, testing, and operational practices using Azure DevOps
- Build professional relationships with key client stakeholders and translate business and data requirements into a technical roadmap
- Troubleshoot and resolve complex issues in live data pipelines
- Lead incident response when issues occur
- Collaborate with CreateFuture’s wider engineering capability to ensure technically excellent and commercially sound delivery
Requirements:
- Extensive hands-on experience with a modern cloud data platform such as Microsoft Fabric, Databricks, Snowflake, or Synapse, including designing production data platforms
- Deep SQL expertise and strong understanding of data modelling and architecture, including Kimball, data vault, medallion/lakehouse patterns, or domain modelling
- Strong proficiency in Python and PySpark for batch/ELT data processing at scale
- Experience leading or technically directing a team of data engineers
- Experience setting standards for pipeline design, testing, and delivery
- Experience owning CI/CD and operational practices for data engineering workflows, ideally using Azure DevOps
- Experience working directly with client stakeholders
- Familiarity with the broader Azure ecosystem, including Azure OneLake and Azure ML, particularly for model industrialisation
Benefits:
- Flexible working, including hybrid and remote options
- Supportive and friendly culture
- Investment in professional and personal growth
- Occasional travel to client sites or CreateFuture offices when needed
- Clear, fair, and respectful hiring process
- Application adjustments or support available
- Great Place to Work-Certified™ company
- Best Workplaces UK recognition
Similar Jobs
Senior Data Operations Engineer
Senior Data Operations Engineer managing Snowflake health, performance, and costs for Anvilogic’s AI cybersecurity platform. Onboarding customers and supporting Snowflake and Databricks production environments.
Senior Data Engineer
Senior Data Engineer building Azure cloud data pipelines for Nimble Gravity, a growth-focused data solutions company. Leading ETL, transformation, warehousing, and data engineering teams.
Senior Data Engineer
Senior Data Engineer leading scalable data architecture, warehousing, and pipeline projects. Velir supports nonprofits and Fortune 500 brands through data and analytics consulting.
Principal Data Architect
Principal Data Architect modernizing a major US mortgage lender’s Finance data platform with Azure, Microsoft Fabric, and Snowflake. Designing CDC pipelines, governed semantic models, and dimensional data architecture at CI&T.
Principal Data Architect
Principal Data Architect modernizing a major US mortgage lender’s finance data platform with Azure, Microsoft Fabric, and Snowflake. Owning architecture, CDC pipelines, governed BI semantics, and hands-on delivery.
Geospatial Data Engineer
Geospatial Data Engineer building scalable GIS pipelines and data products for Arden University. Supporting evidence-based decisions across its expanding digital higher-education ecosystem.
Data Engineer
Senior Data Engineer building data ingestion, Snowflake, and CI/CD infrastructure. Supporting analytics across the business for a client in Gdynia, Poland.
Data Engineer
Lead Data Engineer designing scalable data-processing systems for Kainos, a UK digital technology services company. Leading engineers, ensuring software quality, and delivering secure ETL/ELT solutions.
Junior Data Engineer
Junior Data Engineer building Microsoft Fabric pipelines and data quality checks for Kreate’s plastic manufacturing operations. Supporting dashboards, planning, and daily reporting across ERP, HR, and retail data.
Data Engineer – Product, Pricing & Inventory Data
Data Engineer building scalable product, pricing, and inventory data products for INTERSPORT’s e-commerce platform. Ensuring data quality, integration, governance, and availability for digital products and analytics.
Data Engineer – Tracking & Behavioral Data
Data Engineer für Tracking und Verhaltensdaten bei INTERSPORTs E-Commerce-Digitalunit. Aufbau von Event-Infrastruktur, Datenpipelines und Datengrundlagen für Analytics, Personalisierung und A/B-Tests.
Data Engineer – Commerce and Customer Data
Data Engineer developing scalable commerce and customer data products for INTERSPORT’s e-commerce unit. Building ETL/ELT pipelines supporting analytics, product development, and AI applications.
Data Engineer – Fulfillment & Logistics Data
Data Engineer building fulfillment and logistics data products for INTERSPORT’s e-commerce platform. Developing pipelines, data models, and tracking integrations for orders, deliveries, returns, and partner performance.
Senior Data Engineer – Fluent English
Engenheiro de Dados Sênior desenvolvendo soluções analíticas com Azure, Snowflake e DBT. Atuando em pipelines, modelagem, APIs e governança de dados para clientes corporativos.
Senior Director of Data Engineering
Founding data leader scaling CuraLinc Healthcare’s mental-health analytics platform. Owning governed pipelines, executive dashboards, client reporting, and data team growth.
Junior Data Engineer – Snowflake
Analista de Dados Júnior na Keyrus, empresa internacional de dados e IA, atuando remotamente com Snowflake, SQL e soluções de Cloud Data Platform.
Data Engineer
Data Engineer building scalable data pipelines and integrations for VSP Vision, a vision-care company. Supporting analytics, data quality, and strategic business data use.
Principal Commercial Data Engineer
Principal Data Engineer architecting MGM Resorts’ commercial Lakehouse and analytics ecosystem. Modernizing Gaming, Hospitality, Loyalty, and Marketing data platforms for analytics and AI/ML.
Senior Denodo Data Engineer
Senior Denodo Data Engineer building secure, scalable data virtualization solutions for WLG’s enterprise clients. Supporting analytics, reporting, operational use cases and AI-enabled data services.
Senior Data Engineer
Senior Data Engineer building Teleport's internal data platform for AI infrastructure identity security. Designing warehouses, pipelines, and trusted business metrics across the company.