Senior Data Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Data Engineer building secure Databricks data products for Oportun, a mission-driven financial services company. Leading pipelines, governance, integrations, and member communications infrastructure.
Responsibilities:
- Design and operate secure, scalable data products powering audience selection, eligibility, preferences, consent, campaign inputs, delivery data, and operational reporting
- Lead the design and implementation of scalable, secure data architectures, defining data models, contracts, lineage, and ownership
- Lead complex data engineering initiatives from technical requirements and design through production delivery
- Coordinate work across engineers and stakeholders while managing dependencies, risks, and technical trade-offs
- Design, develop, and optimize production-grade pipelines and integrations using Databricks, Apache Spark, PySpark, Spark SQL, Delta Lake, and SQL
- Modernize appropriate legacy workloads
- Build data-loading, reconciliation, retry, idempotency, recovery, and deployment patterns
- Operate data lakehouse and operational data assets for performance, integrity, availability, and security
- Establish data-quality and governance practices including validation, monitoring, alerts, dashboards, runbooks, and lineage
- Partner with Privacy, Compliance, Risk, and Security on PII, preferences, consent, suppression, retention, and access controls
- Provide technical leadership through design and code reviews, mentor junior engineers, and lead complex production investigations
- Identify root causes and implement durable fixes
- Partner with Product, Communications, Analytics, Engineering, Risk, Privacy, Compliance, and other stakeholders
- Drive release readiness through automated testing, CI/CD, rollout planning, and post-release measurement
- Monitor pipeline performance, quality, timeliness, cost, and reliability
- Use AI tools thoughtfully to reduce manual work and improve decision-making
Requirements:
- 6+ years of experience in data engineering, focused on data architecture, data pipelines, data platforms, and database or lakehouse management
- Bachelor's or master's degree in Computer Science, Data Science, Engineering, or a related field, or equivalent practical experience
- Strong proficiency in Python/PySpark and SQL
- Strong leadership, problem-solving, decision-making, and communication skills
- Hands-on experience with Databricks, Apache Spark, Spark SQL, Delta Lake, workflow orchestration, CI/CD, Git-based workflows, automated testing, and end-to-end data integrations or products in production environments
- Experience with data quality, observability, alerting, lineage, cloud platforms and services, especially AWS
- Experience with secure access controls, secrets management, encryption, auditability, sensitive member data, and Agile ways of working
- Experience in financial services, lending, marketing technology, customer communications, or another regulated environment
- Familiarity with Unity Catalog or comparable data-governance tooling, and Java or Scala
- Experience using AI tools such as ChatGPT or Copilot to improve engineering productivity, testing, documentation, troubleshooting, data quality, or operational effectiveness
- Familiarity with AI agents or automations and sound judgment for security, privacy, accuracy, and human review
Benefits:
- Medical, Dental & Vision Insurance
- Provident Fund
- Life & Accident Insurance
- Internet allowance
- Fuel allowance
- Meal allowance
- Telephone allowance
- Competitive compensation and benefits package supporting physical, financial, and professional well-being
Similar Jobs
Senior Data Engineer
Senior Data Engineer building OCR and ingestion pipelines for an AI consultancy serving insurers. Converting unstructured insurance documents into structured data for RAG and anti-financial-crime models.
Senior Data Engineer
Senior Data Engineer building cloud data platforms for Valtech, an experience innovation company. Enabling analytics, AI, GenAI, and data products through scalable pipelines and governance.
Senior Data Engineer
Senior Data Engineer building cloud data platforms for Valtech, an experience innovation company transforming brands through data, AI, creativity, and technology. Enabling analytics, GenAI, and production-grade data products.
Senior Data Engineer
Senior Data Engineer building cloud data platforms for Valtech, an experience innovation company transforming brands through data, AI, creativity, and technology. Enabling analytics, ML, GenAI, and production-grade data products.
Senior Software Engineer, Data Platform
Senior Software Engineer building distributed data infrastructure for Human Interest’s retirement fintech platform. Owning pipelines, AWS compute, security, and AI-ready architecture.
AI Data Engineer
AI Data Engineer improving internal annotation tools, dashboards, and workflows. Supporting RWS’s AI data collection, labeling, and validation services with Python and Azure.
Senior Data Architect
Senior Data Architect leading scalable data architecture, pipelines, and governance for CI&T’s enterprise EdTech transformation engagement. Hands-on technical leadership with US client stakeholders.
Data Platform Engineer – Mid-level
Data Platform Engineer building self-serve CI/CD, IaC, and governance tooling for ShippyPro’s global shipping platform. Owning deployment paths, pipeline modules, and data access workflows.
Senior AI / ML / Data Engineer
Senior AI/ML/Data Engineer developing ML, NLP, LLM, and RAG solutions for QAVION GROUP’s integrated consulting and technology business. Building scalable data pipelines and GenAI services.
Senior Data Engineer
Engenheiro de Dados Sênior remoto desenvolvendo pipelines e arquitetura lakehouse para a Monte Bravo. Construção de plataforma financeira inovadora com Python, PySpark, AWS e DBT.
Data Engineer
Data Engineer building secure Snowflake data architecture and pipelines for Blood Cancer United. Enabling enterprise analytics and business intelligence to improve blood cancer patient outcomes.
Senior - Principal Data Engineer
Senior Data Engineer shaping scalable Databricks data platforms for RevoData, a European Data & AI consultancy. Leading architecture, pipelines, cloud solutions and client-facing technical decisions.
Senior / Principal Data Engineer
Senior/Principal Data Engineer building scalable Databricks data platforms for RevoData, a boutique Data & AI consultancy. Shaping architecture, pipelines and client solutions across Europe.
Software Engineer, Data
Software Engineer building Python-based cloud data systems for Kunai’s banking and fintech clients. Designing scalable pipelines, real-time streams, and data platforms using AWS, Snowflake, and Databricks.
Founding Engineer – Data Platform
Founding Engineer building a lakehouse and FHIR data platform for a healthcare AI scale-up. Processing millions of clinical events and integrating hospital systems across Latin America.
Academic Content Developer – Financial Data Engineering, Credit Systems
Academic content developer creating fintech engineering curriculum for Jala University. Designing modules, labs, reference implementations, evaluation harnesses, and instructor guides.
Lead Geospatial Data Engineer
Lead geospatial data engineer guiding cloud-based geospatial, imagery, and enterprise data solutions at Logic20/20. Leading architecture, delivery, code quality, and engineering teams.
Lead Data Engineer – PySpark, Palantir Foundry
Lead Data Engineer building governed PySpark and Palantir Foundry pipelines for Logic20/20’s AI and analytics consulting practice. Improving releases, data infrastructure, and audit readiness for client projects.
Senior Manager, Data Engineering – Medicare Revenue
Senior Manager owning CVS Health’s Medicare revenue data engineering foundation. Building auditable healthcare data pipelines for revenue forecasting, risk adjustment, Finance, and Actuarial teams.
Senior Data Engineer
Senior Staff Data Engineer leading Capital One’s enterprise data pipelines and cloud architecture. Driving scalable data platforms, governance, and innovation across banking technology.