Senior Data Engineer – Data Science, AI
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Data Engineer architecting data supply chain and pipelines for AI applications in multi-cloud environments. Leading the engineering efforts to support Data Science initiatives at Sedgwick.
Responsibilities:
- Design and implement robust ETL/ELT pipelines to ingest data from legacy on-prem sources, AWS (S3/RDS), and Azure (Blob/SQL), centralizing it for consumption in Snowflake and AI services.
- Build and maintain Feature Stores and specialized datasets optimized for machine learning, ensuring Data Scientists have immediate access to clean, versioned, and statistically valid data.
- Develop the data pipelines required for Generative AI, including the automated extraction, chunking, and loading of unstructured data into vector stores across AWS and Azure.
- Act as the technical lead for our Snowflake data warehouse, implementing sophisticated data modeling, Snowpipe automation, and compute optimization to support high-concurrency AI workloads.
- Execute non-invasive data extraction patterns to unlock mission-critical data from decades-old on-premise systems without disrupting core business operations.
- Manage complex, cross-platform data workflows using Airflow, Step Functions, or Azure Data Factory, ensuring the synchronization of data across our multi-cloud AI posture.
- Partner directly with central IT, Database Administrators, and Security teams to solve connectivity hurdles (PrivateLink, IAM, firewalls) and secure 'license to operate' for new data flows.
- Implement automated validation and observability layers to detect data drift and quality issues that could compromise the accuracy of production AI and Data Science models.
- Drive the efficiency of our data stack by optimizing storage and query performance in Snowflake, AWS, and Azure to manage the ROI of the Transformation Office.
- Work as a dedicated engineering partner to MLOps and Data Science teams to rapidly iterate on data requirements for evolving AI use cases.
Requirements:
- Bachelor’s degree in Computer Science, Data Engineering, or a related field is required.
- A Master’s degree is highly desirable.
- 6+ years of hands-on data engineering experience, with a track record of building production-grade pipelines for Data Science and AI in multi-cloud environments.
- Expert-level proficiency in Snowflake architecture, including data sharing, performance tuning, and the integration of Snowflake with external cloud AI services.
- Advanced, hands-on knowledge of AWS (S3, Glue, Lambda) and Azure (Data Factory, Synapse) data services.
- Mastery of Python, SQL, and PySpark. Deep experience with data orchestration and containerization (Docker).
- Proven ability to interface with 'old world' tech (on-premise SQL, Mainframe extracts, flat files) and transform it for modern cloud consumption.
- A strong understanding of the specific data needs for Machine Learning (feature engineering) and Generative AI (vectorization and embedding pipelines).
- A 'get-it-done' attitude, capable of navigating enterprise bureaucracy and technical debt to ship code at the speed required by a Transformation Office.
Benefits:
- Health insurance
- 401(k) matching
- Flexible work hours
- Paid time off
- Professional development opportunities
Similar Jobs
Lead Data Engineer
Lead Data Engineer developing cloud-based data services for command and control systems. Work with agile teams on complex DoD projects requiring advanced technical skills and security clearance.
Data Migration Specialist
Data Migration Specialist managing client data migrations to gaiia's platform. Collaborating with teams to ensure accurate and timely data transitions.
Data Engineer
Data Engineer enhancing data operations for Rhythm’s renewable energy business by building scalable data pipelines. Partnering with stakeholders to drive business decisions through data solutions.
Senior Data Engineer
Senior Data Engineer responsible for developing data pipelines and collaborating on metrics at Mytra. Take ownership of data infrastructure and work with cross-functional teams.
Senior Data Engineer – Equity
Senior Data Engineer enhancing backend systems and optimizing recommendation algorithms for a social media app. Contributing to the architecture and performance of the recommendation infrastructure.
Data Engineering – Tech Lead / Manager
Leading the Data Engineering team for DIGITIZEI INFORMATICA LTDA, focusing on technical guidance and individual growth. Overseeing analytics projects with significant impact on business decisions and product development.
Senior Data Engineer – Engenheiro de Dados Sênior
Senior Data Engineer at Dadosfera leading data-focused projects for startups and corporations. Involves building data pipelines, cloud solutions, and quality assurance in data management.
Senior Product Manager, Data Platform
Senior Product Manager responsible for adopting AI in healthcare data solutions. Collaborating across teams to enhance product development and drive measurable impact.
Data Engineer
Data Engineer building and managing scalable data infrastructure at Alpha Lion. Transforming fragmented data into reliable and analytics-ready datasets for decision-making.
Senior Software Developer, Data Platform
Senior Software Developer modernizing Data Transfer Platform for Intrahealth, a healthcare EMR provider. Focusing on scalable and configurable backend systems in a complex environment.
Data Architect
Data Architect evaluating and evolving data architecture for partners at EVT. Focus on ingestion, processing, and structuring of data with a collaborative team environment.
Data Engineer
Data Engineer responsible for designing, building, and maintaining data pipelines at Loopback Health. Collaborating with cross-functional teams to drive system and process improvements.
Data Engineer
Data Engineer optimizing and maintaining data pipelines in Blackline Safety's IoT-enabled safety ecosystem. Collaborating with product, engineering, and analytics teams on impactful data-driven initiatives.
Senior Ingeniero de Datos
Senior Data Engineer developing data solutions for transforming lives at Sofka. Collaborating in a close-knit team focused on continuous improvement and technical excellence.
Senior Data Engineer
Senior Data Engineer at Arsipa developing scalable data platforms and ETL/ELT pipelines. Collaboration with teams to ensure data quality, performance, and innovation in data architecture.
Data Engineer
Data Engineer shaping cloud-based data solutions for top U.S. companies. Engage with diverse projects remotely from Latin America.
Senior Data Engineer
Senior Technical Consultant delivering end-to-end data solutions and managing client relationships. Collaborating with technical team to drive project success with Microsoft technologies.
Senior Data Engineer – Data for Products
Lead the design and implementation of scalable batch and streaming data pipelines on Azure. Collaborate with product teams to ensure data quality, performance, and governance across the data lifecycle.
Data Engineer
Data Engineer maintaining scalable data pipelines for an international company in Hamburg. Requires experience with Snowflake, ETL processes and Azure Data Factory.
IS Data Warehouse Architect – III
Data Warehouse Architect designing and developing data models and ETL specifications. Collaborating on data architecture and vendor relations for a nonprofit health care organization.