AWS Data Engineer – Mid
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
AWS Data Engineer implementing data migration projects in Azure and AWS cloud environments. Involved in pipeline orchestration and governance while supporting CI/CD processes.
Responsibilities:
- Contribute to a data platform migration and modernization project;
- Map, analyze, and understand existing notebooks and jobs in Databricks (Azure);
- Define and execute migration waves for workloads to AWS EMR (EC2 and/or Serverless);
- Implement and maintain orchestration of pipelines and jobs using Apache Airflow (MWAA);
- Build and maintain CI/CD pipelines for deployment, versioning, and automation;
- Structure and organize the data catalog, metadata, and access governance using AWS Lake Formation;
- Ensure pipeline observability by integrating logs and metrics with Grafana;
- Structure data in Landing, Silver, and Gold layers using Hudi and Iceberg, ensuring adherence to the current model in ADLS;
- Participate in acceptance testing, change management processes, and production deployments;
- Promote best practices and conduct knowledge transfer to the involved teams.
Requirements:
- Experience with Databricks (Azure), including mapping, analyzing, and refactoring notebooks and jobs;
- Experience with AWS cloud services (EMR, Glue, Lambda);
- Experience with data pipeline orchestration using Apache Airflow (MWAA);
- Experience structuring data in layered architectures (Landing, Silver, and Gold);
- Experience with data governance, including catalog, metadata, and access control via AWS Lake Formation;
- Experience with pipeline observability, using logs, metrics, and integration with Grafana;
- Experience with Python.
Similar Jobs
Senior Data Engineer
Senior Data Engineer responsible for building robust data pipelines and aggregates for Zeta's AdTech platform. Collaborating with backend, ML, and product teams to deliver reliable data.
Principal Data Architect – Streaming, Data Platforms
Data Architect responsible for designing and implementing scalable streaming data platforms at Egen. Collaborating on data governance and architecture standards with a focus on real-time analytics.
Data Architect
Data Architect responsible for building data architecture and analytics infrastructure at Cobalt Service Partners. Leading technical authority on data architecture and visualization within a high-growth environment.
Data Architect
Data Architect at Valtech designing complex data architectures across cloud platforms. Leading strategic direction in data engineering with a focus on AI and modern data stacks.
Data Migration Engineer
Data Manager responsible for leading data migration and quality for complex enterprise customers at MoeGo. Collaborating across teams and ensuring successful implementation of data strategies.
Senior Data Engineer
Data Engineer building impactful data infrastructure at Murmuration, focusing on complex datasets for civic engagement efforts. Collaborating with teams to achieve research and product development goals.
Senior Data Engineer
Senior Data Engineer developing data solutions on Beam’s data platform for social impact. Collaborate in a highly motivated team delivering complex analytics for nonprofits.
Senior Data Architect
Senior Data Architect providing IT Services for cloud-based SaaS solutions at Capio Group. Ensure enterprise-level data systems meet complex public sector needs.
Data Engineer
Data Engineer owning and evolving Enode’s data platform while collaborating on Flex product analytics needs. Enhancing data quality and enabling self-serve analytics for a climate tech startup.
Principal Data Engineer – Experimentation
Principal Data Engineer managing analytics systems and data pipelines at Sysco. Leading technical teams and providing guidance on data engineering best practices for large-scale environments.
Senior Data Architect – Azure
Senior Data Architect designing enterprise-grade solutions leveraging Microsoft technology stack. Leading client engagements and delivering scalable technical solutions in a collaborative environment.
Software/Data Engineer
Software/Data Engineer contributing to the progressive political mission at Movement Labs. Gain hands-on experience while working on innovative data and technology initiatives during the Fellowship.
Senior Data Engineer
Sr Data Engineer at LegitScript developing AI solutions focused on risk detection algorithms and machine learning. Join a team using big data to enhance payment ecosystems' safety and transparency.
Lead Data Architect
Lead Data Architect at Henry Schein, driving the strategic vision and design of enterprise data architecture. Collaborating across teams to optimize data-driven decision-making and governance.
Staff Data Engineer – Science
Staff Data Engineer creating software applications that support cancer detection and treatment guidance at Exact Sciences. Leading technical design and collaborating with cross-functional teams for successful outcomes.
Senior Data Engineer
Senior Data Engineer developing scalable data pipelines and supporting AI/ML initiatives in digital healthcare space. Collaborating with cross-functional teams to deliver reliable data solutions.
Senior Data Engineer
Senior Data Engineer at COVU building an AI-first insurance platform. Focused on core data infrastructure including Policy Journal and data pipelines.
Data Architect – MS Fabric
Lead operational analytics strategy across the organization, guiding decision-making and ensuring improvements in operational efficiency.
Snowflake Data Architect
Snowflake Architect designing healthcare data systems for Datavail. Focus on EHRs, claims data, and cloud migration in compliance with HIPAA/HITRUST.
Response Data Engineer, 6mo Contract
Response Data Engineer providing hands-on engineering support for WCK's frontline data tools. Maintaining and improving data infrastructure for disaster response operations with collaborative team efforts.