Senior Data Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Data Engineer enhancing Dotdigital's data platform and capabilities. Designing data pipelines and collaborating with teams to unlock the value of data.
Responsibilities:
- Design and build scalable, reliable, and secure data pipelines for streaming, batch, and real-time processing.
- Work in partnership with the Data Science teams to build scalable Pyspark workloads that can be leveraged to generate advanced models.
- Implement and optimise data models and storage solutions using Python and SQL with orchestration tools in a cloud environment.
- Leverage AI to automate both data processing and engineering processes.
- Advocate and uphold best practices for data governance, security, and monitoring.
- Collaborate cross-functionally with engineers, analysts, and data scientists to deliver impactful data solutions.
- Mentor and support junior engineers in data engineering principles and practices.
- Evaluate and recommend new tools and technologies to strengthen data services.
Requirements:
- Significant experience delivering Python-based projects for data engineering.
- Experience building and tuning spark pipelines that run at scale across large quantities of data.
- Strong hands-on experience with SQL and NoSQL databases (e.g. SQL Server, MongoDB, Cassandra).
- Proven experience with modern data warehousing and large-scale processing (e.g. Snowflake, DBT, BigQuery, Clickhouse).
- Proficient with data orchestration tools such as Airflow, Dagster, or Prefect.
- Experience with cloud platforms (Azure, AWS, or GCP) for data processing and storage.
- Practical experience with Kafka or equivalent event-driven architectures (e.g. AWS SQS, Azure EventHubs, AWS Kinesis).
- Good understanding of data modelling for OLAP and OLTP workloads.
- Familiar with agile methodologies and CI/CD processes in the context of data solutions.
- Experienced as a senior team member on complex data engineering projects.
- Able to design and optimise data structures for high-volume systems.
- Experienced in assisting with data platform modernisation or migration to the cloud.
- Takes initiative to solve challenging data issues and drive projects forward.
Benefits:
- Parental leave
- Medical benefits
- Paid sick leave
- Dotdigital day
- Share reward
- Wellbeing reward
- Wellbeing Days
- Loyalty reward
- DEI commitment
Similar Jobs
Senior Lead Software Engineer, Data Platform
Senior Software Engineer building scalable data and analytics platforms at Coupa. Focusing on advanced data ingestion and processing in a multi-cloud environment.
Workday/Fabric Data Engineer
Workday/Fabric Data Engineer supporting USNH's business intelligence initiatives by designing and maintaining data solutions. Collaborating with teams for effective reporting and analytics.
Director of Data Engineering – Platforms
Director of Data Engineering at Cotality leading data transformation initiatives and data architecture frameworks. Collaborating with product teams to drive actionable business intelligence.
Data Architect
Design and manage enterprise data architecture with a focus on Air Force Logistics Systems. Collaborate with engineers and government stakeholders for scalable and secure data solutions.
Data Engineer
Data Engineer building data pipelines for analytics and machine learning at Illumination Works. Collaborating across teams to ensure reliable data for insights and experimentation.
Data Architect – Short-Term Freelance Project, Pharma
Data Architect role focused on implementing modern data governance practices in pharmaceutical projects. Requires strong expertise in Governance as Code and data architecture.
Data Engineer – Medicine & Analytics
Data Engineer responsible for building reliable data solutions for analytics in Medicine. Collaborate with diverse teams to improve patient outcomes through data-driven insights.
Senior Application Data Architect
Senior Application Data Architect leading enterprise data architecture and engineering initiatives. Collaborating with cross-functional teams to design and build scalable data solutions.
Unit Lead, Data Engineering
Lead the Data Engineering team at evoila, driving innovative data projects and developing advanced technological solutions in Germany. Collaborate on Greenfield projects and enhance existing platforms.
Data Engineer – AI Platform
Data Engineer responsible for building data ingestion pipelines for the AI platform at TaskUs. Engaging in scalable data management with strong experience in Python, SQL, and AWS.
Data Engineer II
Data Engineer II contributing to data governance engineering programs at CareSource. Focused on solution design architecture and implementation of standards for data management.
Data Architect
Consultant for Taxwell assisting Americans in finding tax credits and deductions. Advocating for customers and providing guidance in a supportive environment.
Account Executive – AI Data Platform Sales
Account Executive specializing in AI Data Platform Sales, managing enterprise client relationships. Focus on new customer acquisition and account growth driving analytics and AI initiatives.
Lead Data Engineer
Data Engineer Lead designing scalable data platforms for advanced analytics at Blend. Collaborating with cross-functional teams to build robust data solutions.
Data Architect – Strong Azure Services, Finance Experience (GL, AR, AP)
Data Architect specializing in financial systems (GL, AR, AP) and using Azure. Fully remote position requiring 12+ years of relevant experience.
Senior Data Architect – 2 months project
Seeking a Senior Data Architect for a two-month project focused on building a data warehouse in the car rental industry. Must have experience in data engineering and warehouse architecture.
BI Data Engineer
Lead Data Engineering initiatives in IT Analytics for transforming raw data into strategic assets. Collaborate with business areas to develop data solutions with a strong analytical vision.
Senior Data Engineer
Senior Data Engineer designing and maintaining scalable data pipelines for Pax8's cloud marketplace. Collaborating on architecture and mentoring data engineers while driving data quality and reliability.
Data Engineer IV
Data Engineer IV developing scalable data pipelines and orchestration frameworks at Rackspace Technology. Leading teams in data governance and managing analytics workloads across Azure and Databricks.
Information/Data Architect
Information/Data Architect providing technical expertise and support for data architecture requirements at Concept Plus. Developing data strategies and maintaining information structures for effective governance.