Data Engineer – DataOps, Infrastructure Focus
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Data Engineer responsible for building and maintaining AWS data infrastructure for Guidewire. Collaborating with teams to support AI-driven analytics and operational excellence.
Responsibilities:
- Architect, automate, and maintain production-grade data infrastructure on AWS (e.g., S3, EMR, Glue, Lambda, Redshift) using Terraform or CDK, with a focus on high availability, security, and consistent environments across the SDLC.
- Integrate Claude Code and other LLM-based agents into the engineering workflow to accelerate infrastructure provisioning, refactoring, and generation of technical documentation, embedding AI into daily development practices.
- Design, build, and optimize CI/CD pipelines that test, deploy, and monitor dbt models and AWS Glue/Spark jobs, ensuring reliable, repeatable delivery of governed data assets.
- Implement agentic operations for DataOps—configuring AI agents to triage and perform root-cause analysis of pipeline failures, surface cost-optimization signals, and proactively detect schema drift or data quality regressions.
- Engineer scalable, well-governed data pipelines and tables using Apache Iceberg, Airflow (MWAA), and Redshift, emphasizing simplicity, reusability, and clear ownership of data products.
- Operationalize security and compliance best practices in a regulated insurance environment, including IAM automation, encryption, audit-ready logging, and alignment with enterprise RBAC/MFA standards.
- Partner with Product Strategy, PDO, and data science teams to ensure data platforms and features can support AI-heavy products like the Agentic AI Platform, Claim Summary, and Underwriting Assistant at scale.
Requirements:
- 5+ years of experience in Data Engineering, Data Operations, or Platform Engineering building and operating cloud data infrastructure
- Deep proficiency with AWS (e.g., S3, EMR, Glue, Lambda, Redshift) and infrastructure-as-code (Terraform strongly preferred; CDK a plus)
- Strong experience with dbt in production (modeling, testing, documentation, deployment)
- Advanced SQL skills (performance tuning, complex joins and window functions)
- Solid Python experience for automation, orchestration, and data engineering tasks
- Hands-on experience with Apache Spark for large-scale batch or streaming workloads, ideally on AWS EMR or Glue
- Proven track record of building or maintaining CI/CD pipelines (Git-based workflows, automated testing, deployment, and monitoring) for data and analytics workloads
- Strong systems thinking and data modeling skills (e.g., Kimball, Data Vault)
- Clear, collaborative communication style with the ability to work across product, security, and business stakeholders in a distributed environment
Benefits:
- Flexible work environment
- Health and Wellness benefits
- Paid time off programs including volunteer time off
- Market-competitive pay and incentive programs
- Continual development and internal career growth opportunities
Similar Jobs
Principal Data Architect
Principal Data Architect at Committee for Children, defining and evolving data architecture for a social enterprise. Collaborate with teams to ensure data governance and accessibility for child welfare programs.
Senior Data Engineer, Snowflake
Sr. Data Engineer designing and implementing scalable data solutions in Snowflake for clients. Collaborating with teams to translate requirements into actionable technical solutions and optimizations.
Data Architect III
Data Architect III managing data strategy and overseeing data collection and organization in a remote setting for Veterans Affairs. Leading data architecture and governance initiatives to improve Veteran services.
Senior Manager, Data Architecture
Senior Manager of Data Architecture ensuring enterprise data is standardized, secure, and well modeled. Overseeing data architecture across platforms and applications and translating business outcomes into data models.
Senior Data Engineer
Senior Data Engineer at Peek transforming data sources into actionable insights for analytics. Collaborating across teams to support business intelligence and enhance data infrastructure.
Senior Data Engineer
Senior Data Engineer building and scaling data infrastructure for Wagmo’s pet health solutions. Collaborate with multiple teams to ensure clean data flows across the business.
Senior Clinical Data Engineer
Senior Clinical Data Engineer overseeing clinical data lifecycle for Oura's research studies. Collaborating with teams to maintain data integrity and regulatory compliance in clinical trials.
Lead Data Engineer
Lead Data Engineer overseeing data platform and analytics engineering teams at OneOncology. Responsible for operational excellence and technical direction in data processing and management.
Senior Data Engineer
Senior Data Engineer at Kainos designing and developing data processing software for large-scale data solutions. Collaborating with agile teams to ensure high code quality and solution effectiveness.
Lead Data Engineer
Lead Data Engineer at Mercator.ai building scalable data infrastructure and distributed data pipelines. Ensuring performance and reliability while mentoring junior engineers.
Data Engineer
Data Engineer at Egen designing and building data processing pipelines using advanced technologies. Collaborating with stakeholders to ensure data accessibility, security, and accuracy.
Data Architect
Data Architect providing data management and analytics support for HRSA's Organ Procurement program. Collaborating on patient safety performance, technical assistance, and data visualization.
AI Data Engineering Lead
AI Data Engineering Lead building data foundations for AI models at Ideagen. Leading a team to ensure data quality, compliance, and operational efficiency.
Cloud Data Architect
Cloud Data Architect at Hunter Strategy responsible for telemetry and data normalization in cybersecurity. Designs log collection and routing architectures for AWS and Azure environments.
Senior Data Engineering Analyst
Data Engineer managing data governance and modernization projects using Databricks at Keeggo. Collaborating with teams to enhance data pipeline architecture and performance.
Senior Data Engineer – AWS
Senior Data Analyst migrating data from SQL Server to AWS SaaS architecture, ensuring data integrity and developing reusable ETL processes.
Senior Data Engineer
Senior Data Engineer at Hyatt developing robust data pipelines and supporting data-driven decisions. Collaborating with cross-functional teams in a dynamic and fast-paced environment.
Senior Data Engineer
Senior Data Engineer designing, building, and maintaining data pipelines for global social discovery products. Collaborating with teams to turn data into insights and optimize data quality.
Staff Data Engineer
Staff Data Engineer at firsthand Health designing and optimizing data infrastructure. Engaging in impactful work supporting individuals living with serious mental illness with a modern technology stack.
Data Engineer, Databricks
Data Engineer overseeing data operations and solutions within Azure cloud environments. Collaborating on Databricks and ensuring reliable data processes and governance practices.