Senior Data Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Data Engineer building auditable GCP data pipelines for DysrupIT’s financial billing and invoicing platform. Ensuring accurate reconciliation, lineage, and trusted financial data.
Responsibilities:
- Design, build, and maintain data ingestion pipelines for financial and billing data on GCP
- Design and evolve BigQuery data models, including nested ARRAY/STRUCT tables, partitioning, and clustering strategies
- Implement reconciliation checks across source, staging, and output layers
- Build file validation and quarantine processes for invalid or incomplete source data
- Design idempotent pipelines that can be safely rerun without duplicating or corrupting data
- Implement append-only table patterns and use views as data checks to support auditability
- Handle monetary values accurately using BigQuery NUMERIC types and appropriate rounding rules
- Manage effective-dated reference data such as rates, mappings, and customer attributes
- Establish and maintain data lineage from outputs back to their sources
- Write clean, typed, and well-tested Python code using pydantic and pytest
- Write and optimize advanced SQL for transformation, reconciliation, and reporting
- Collaborate with engineers, analysts, and finance stakeholders to translate business rules into reliable data logic
- Mentor team members and contribute to data engineering standards and best practices
Requirements:
- 6+ years of data engineering experience
- Proven experience building financial or billing pipelines where every amount must reconcile
- Strong Python skills, including pydantic, typed code, and pytest
- Advanced SQL skills, including complex joins, window functions, and performance tuning
- In-depth Google BigQuery experience, including nested ARRAY/STRUCT tables, partitioning, and clustering
- Experience handling money accurately using NUMERIC data types
- Experience designing append-only tables and using views as data checks
- Strong understanding of data quality practices, including reconciliation checks, file validation, and quarantine
- Experience building idempotent, rerunnable pipelines
- Experience implementing data lineage and managing effective-dated reference data
- Invoicing domain experience, including credit notes, rebills, FX conversion, and month-end close (preferred)
- Experience working with finance or billing teams in a regulated or audited environment (preferred)
- Experience with other GCP services such as Cloud Storage, Cloud Run, or Firestore (preferred)
Benefits:
- Training and employee development opportunities
- Career growth and advancement opportunities
- Employees valued and respected
- Positive and supportive company culture
Similar Jobs
Data Architect
Databricks Data Architect designing secure Lakehouse architectures and AI-ready data platforms for Devoteam clients. Modernizing legacy systems across cloud and multi-cloud environments.
Senior Software Engineer, Data Acquisition
Senior software engineer building distributed crawling and ingestion systems for People Data Labs, a provider of people and company data. Improving scalable backend services and data products.
IT Architect, Data Scientist
IT Architect/Data Scientist supporting DLH’s federal health and national-security programs. Leading cloud architecture, data science, interoperability, security engineering, and compliance for SCHARE.
Senior Business Data Engineer
Senior Business Data Engineer building AWS data infrastructure and automated reporting for Nex’s family motion-gaming platform. Improving pipeline reliability and supporting cross-functional business teams.
Senior Data Engineer
Senior Data Engineer building Python, Spark, and Databricks pipelines for Quantiphi’s data and AI solutions. Leading cloud migration, CI/CD automation, and scalable analytics engineering.
Senior Data Engineer – IBM DB2
Senior Data Engineer modernizing IBM DB2 data environments. Building reliable ETL/ELT pipelines and AWS integrations for enterprise applications and payroll systems.
Lead Data Architect
Lead Data Architect owning enterprise data architecture for C5MI Insight’s federal SAP S/4HANA modernization program. Leading data governance, migration, analytics architecture, and master data teams.
FBS Data Engineer, SQL, Power BI
Data Engineer supporting Farmers’ P&C insurance analytics and financial loss reporting solutions. Translating business data needs into requirements, integrations, dashboards, and validated reporting.
Data Engineer – Insurance
Data Engineer building analytics solutions, data pipelines, and enterprise integrations for Farmers Insurance. Improving data quality, reporting, dashboards, and regulatory-compliant business outcomes.
Power BI Data Engineer
Power BI Data Engineer building Azure data pipelines, dimensional models, and executive dashboards for Legence. Transforming complex data into actionable business intelligence across the full BI lifecycle.
Azure Data Engineer
Azure Data Engineer building Azure and Microsoft Fabric data solutions for Quisitive, a global Microsoft cloud partner. Delivering pipelines, integrations, and analytics projects for clients.
Senior Data Engineer
Senior Data Engineer building multi-region data infrastructure for Horizon3’s NodeZero autonomous cybersecurity platform. Supporting Federal workloads, warehouses, pipelines, orchestration, and BI reporting.
Data Engineer, Snowflake
Data Engineer Snowflake diseñando pipelines cloud y transformaciones ELT/ETL para Logicalis Spain. Participación en proyectos de ingeniería de datos, IA y analítica avanzada.
Data Engineer
Data Engineer building Python and AWS data pipelines for 540’s U.S. Army financial-system modernization initiative. Improving secure, scalable data transformation and governance.
Senior Manager, Data Engineering
Data engineering leader building Databricks and Azure platforms for Cushman & Wakefield, a global commercial real estate services firm. Managing engineers, platform delivery, governance, and stakeholder outcomes.
Data Architect
Data Architect designing Azure-integrated data platforms for Tieto’s agricultural analytics projects. Building scalable datasets, ETL pipelines, warehouses, and lakes.
Data Engineer
Data Engineer building scalable cloud pipelines, models, and enterprise data products for a major U.S. insurer. Supporting analytics, reporting, governance, and operational data capabilities.
Senior Data Engineer
Engenheiro de Dados SR desenvolvendo pipelines, Data Lakes e governança para o Grupo SysMap. Empresa brasileira de tecnologia acelerando a transformação digital de organizações.
Data Engineer
Data Platform Engineer owning cloud infrastructure, Terraform, CI/CD, and observability for Seer Interactive’s digital marketing consultancy. Automating analytics and AI platform operations.
Data Engineering Lead
Data Engineering Lead architecting Seer Interactive’s GCP data platform and agentic engineering workflows. Enabling governed analytics and AI products for digital marketing clients.