Senior Data Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Data Engineer for Knowledge Services managing data extraction and integration into Snowflake. Leading data warehousing and ensuring data quality across diverse sources.
Responsibilities:
- Develop efficient and scalable data extraction methodologies to retrieve data from diverse sources, such as databases, APIs, web scraping, flat files, and streaming platforms.
- Design and implement robust data loading processes to efficiently ingest and integrate data into the latest data warehousing technology, ensuring data quality and consistency.
- Develop and maintain staging processes to facilitate the organization and transformation of raw data into structured formats, preparing it for downstream analysis and reporting.
- Implement data quality checks and validation processes to identify and address data anomalies, inconsistencies, and integrity issues.
- Identify and resolve performance bottlenecks in data extraction and loading processes, optimizing overall system performance and data availability.
- Ensure adherence to data security and privacy standards throughout the data extraction and warehousing processes, implementing appropriate access controls and encryption mechanisms.
- Create and maintain comprehensive documentation of data extraction and warehousing processes, including data flow diagrams, data dictionaries, and process workflows.
- Mentor and support junior data engineers, providing guidance on best practices, technical design, and professional development to elevate overall team capability and performance.
- Collaborate with cross-functional teams, including data scientists, data analysts, software engineers, and business stakeholders, to understand their data requirements and provide efficient data engineering solutions.
- Stay updated with the latest advancements in data engineering, data warehousing, and cloud technologies, and proactively propose innovative solutions to enhance data extraction and warehousing capabilities.
Requirements:
- Minimum of 5 years’ experience in data engineering, with a strong focus on data extraction and cloud-based warehousing; a combination of years of experience and relevant advanced technology proficiency will also be considered.
- Proficiency with Snowflake and data integration tools like Fivetran.
- Advanced SQL skills and experience with ETL/ELT frameworks.
- Experience with scripting languages such as Python for data processing and automation.
- Solid understanding of data modeling and relational database design.
- Strong communication skills and the ability to collaborate with technical and non-technical stakeholders.
- Strong analytical and problem-solving skills, with the ability to identify and resolve complex data engineering challenges.
Similar Jobs
Principal Data Engineer
Data Engineer leading teams to innovate technology in federal government. Working on scalable data pipelines using modern tech in an agile environment.
Staff Data Engineer
Staff Data Engineer at Zeta Global leading the design and implementation of a unified semantic data layer. Focused on integrating diverse data sources for enhanced AI and analytics capabilities.
Senior Data Engineer
Senior Data Engineer at Sightline Payments, overseeing data engineering and cloud-based solutions. Collaborating across teams to prepare and deliver complex data sets for improving customer-centric omnichannel experience.
Data Engineer
Júnior Data Engineer supporting SQL Server Data Warehouse, developing ETL routines and data models. Collaborating with Product, Engineering, and Analytics teams in a growing environment.
AVP, Data Engineering
AVP of Data Engineering owning Ensemble’s enterprise data platform strategy. Leading cloud-native data platform design on Azure and Databricks for scalable analytics and AI capabilities.
Data Engineer
Data Engineer developing and maintaining data pipelines and transformation models for analytics at Industrial Electric Manufacturing. Collaborating with stakeholders and ensuring data quality across business decisions.
Senior Data Engineering Manager
Senior Manager of Data Engineering leading a team that manages data at ReUp Education. Empowering adult learners and institutions by developing data solutions and pipelines.
Director, Data Engineering, ML – AI
Director of Data Engineering leading ML and AI initiatives for Measurabl. Responsible for data platform strategy and sustainability analytics in commercial real estate.
Data Engineer
Data Engineer responsible for building foundational infrastructure for AI deployments and product launches. Collaborating with top researchers and engineers in cutting-edge AI systems.
Data Engineer
Data Engineer at OrderlyMeds connecting multiple systems to BigQuery for strategic insights. Collaborating with a cross-functional team to enhance data utility and drive business decisions.
Data Engineer/Software Engineer
Data Engineer/Software Engineer responsible for implementing data-driven solutions and managing data pipelines. Join ALTEN Technology USA to work on challenging engineering projects.
Data Engineering Manager
Engineering leader managing Data Engineering team at Northrop Grumman. Overseeing data systems and leading a collaborative team to ensure engineering excellence.
Data Engineer
Senior Data Engineer serving in Prima's engineering team, building scalable data products and maintaining pipelines for optimal growth. Engaging in innovative solutions to transform motor insurance experience.
Senior Data Engineer
Senior Data Engineer leading design discussions and driving technical vision for healthcare data solutions. Collaborating cross-functionally to ensure high standards and timely project delivery.
Senior Analytics Data Engineer
Full Stack Engineer developing scalable applications for the largest real-world healthcare datasets provider. Collaborating with an international team under technical leadership for critical platform enhancements.
Director – Data Engineering
Director leading and scaling data engineering function for a Weekday's client. Building efficient data platforms powering analytics and business intelligence across the organization.
Data Engineer
Data Engineer building and maintaining data pipelines for marketing and analytics at an EdTech app with 70 million users. Collaborating with analysts and business teams for data quality and reporting.
Data Engineer
Data Engineer responsible for managing data infrastructure at Bemobi with innovative solutions for data access and analysis. Join the Data Platform team to enhance data lifecycle processes.
Senior Lead Software Engineer, Data Platform
Senior Software Engineer building scalable data and analytics platforms at Coupa. Focusing on advanced data ingestion and processing in a multi-cloud environment.
Workday/Fabric Data Engineer
Workday/Fabric Data Engineer supporting USNH's business intelligence initiatives by designing and maintaining data solutions. Collaborating with teams for effective reporting and analytics.