Senior Data Engineer – Full Stack
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Data Engineer building end-to-end Databricks data products for DigiCert, which secures digital interactions worldwide. Developing pipelines, APIs, applications, governance, and production operations.
Responsibilities:
- Partner directly with business and technical stakeholders to understand workflows, define desired outcomes, and translate ambiguous needs into technical requirements and delivery plans.
- Design, build, and maintain end-to-end data products on Databricks, spanning ingestion, Delta Lake storage, transformation, serving, APIs, and user-facing experiences.
- Develop and operate reliable batch, incremental, and streaming pipelines using SQL, Python, PySpark, Kafka, and Databricks-native capabilities.
- Design event-driven and near-real-time data solutions integrating operational systems and downstream consumers.
- Build backend services, APIs, and integrations that make governed data available to applications and operational workflows.
- Develop lightweight applications, dashboards, and interfaces with product, analytics, BI, and user-experience teams.
- Rapidly prototype solutions, validate them through stakeholder feedback, and prepare successful concepts for production use.
- Create scalable data models and curated datasets supporting analytics, reporting, AI/ML, and operational decision-making.
- Implement data-quality, security, lineage, and governance controls using Databricks and Unity Catalog.
- Establish automated testing, CI/CD, monitoring, alerting, and documentation across the data-product lifecycle.
- Optimize pipelines, streaming workloads, queries, services, and applications for reliability, performance, scalability, and cost.
- Diagnose and resolve issues across source systems, streaming platforms, pipelines, data models, APIs, applications, and downstream consumers.
- Collaborate with platform and product engineering teams to turn recurring stakeholder needs into reusable capabilities.
- Lead technical design and code reviews, mentor engineers, and elevate full-stack data-engineering practices.
Requirements:
- 5+ years of experience in data engineering, software engineering, or a related role, including ownership of production data solutions.
- Strong proficiency in SQL, Python, and PySpark, with experience building reliable, production-grade pipelines and data products.
- Hands-on experience with Databricks, Apache Spark, Delta Lake, and Unity Catalog or comparable data-governance capabilities.
- Experience designing and supporting streaming or near-real-time data pipelines using Kafka, Kinesis, Event Hubs, or similar event-streaming technologies.
- Strong understanding of event-driven architecture, message processing, schema evolution, data consistency, and streaming reliability.
- Experience delivering full-stack solutions that include data pipelines, backend services or APIs, and lightweight user-facing applications.
- Experience building REST APIs, services, and integrations using Python frameworks such as FastAPI, Flask, or comparable technologies.
- Experience with AWS, Azure, or GCP and cloud-native architecture patterns.
- Strong understanding of data modeling, data warehousing, distributed processing, and analytics-friendly data design.
- Experience with Git, automated testing, CI/CD, monitoring, and production-deployment practices.
- Demonstrated ability to work directly with stakeholders, navigate ambiguity, and translate business problems into practical technical solutions.
- Strong communication, technical leadership, problem-solving, and end-to-end ownership skills.
- Ability to balance rapid delivery with maintainability, security, governance, and operational reliability.
- Nice-to-have qualifications include React or another modern frontend framework; infrastructure as code, containerization, and automated cloud deployment; AI/ML pipelines, feature engineering, retrieval systems, or generative AI; data observability, platform engineering, or data-product management; JavaScript or TypeScript; forward-deployed engineering, solutions engineering, technical consulting, or stakeholder-embedded delivery; reusable data platforms; mentoring engineers; and Agile or Scrum experience.
Benefits:
- Generous time off policies
- Top shelf benefits
- Education, wellness and lifestyle support
Similar Jobs
Forward Deployed AI Engineer – Data Intelligence
Forward Deployed AI Engineer building Claude-powered enterprise data and RAG solutions for NewRocket’s ServiceNow clients. Integrating governed data, AI workflows, and enterprise systems for secure operational outcomes.
Domain Data Architect
Domain Data Architect designing governed data models, integrations, and analytics solutions. Supporting US LBM, a leading U.S. specialty building-materials distributor.
Senior Data Engineer – Operational Reporting, Data Platforms
Senior Data Engineer building Oracle-based reporting data layers and pipelines for CVS Health’s healthcare operations. Supporting near real-time business intelligence, production reliability, and data quality.
D365 Data Migration Lead
D365 Data Migration Lead owning Dynamics 365 data migrations for HSO, a global Microsoft business transformation partner. Leading governance, ETL, cutovers, and data quality across international client projects.
Senior Data Engineer, BizTech
Senior Data Engineer leading scalable batch and real-time data systems for Airbnb’s global hospitality platform. Shaping data strategy, governance, integrations, and engineering best practices.
Senior Data Engineer
Engenheiro de Dados Sênior construindo pipelines medalhão no Databricks e Azure para cliente automotivo global. Garantindo governança, qualidade e CI/CD para habilitar BI e Data Science.
Senior Data Engineer
Senior Data Engineer building data contracts and Delta Sharing for JLL’s enterprise real estate data platform. Enabling governed, self-service access for analytics, reporting, and product teams.
Senior AI Engineer – Data, MLOps
Senior AI Engineer building production ML, GenAI, and MLOps solutions for Teradyne’s test automation business. Operationalizing enterprise AI across Azure, Google Cloud, Snowflake, and Microsoft Fabric.
Principal Cloud Data Engineer
Cloud data engineer building governed, AI-ready data pipelines for Teradyne’s global test and automation solutions. Enabling enterprise AI, analytics, and operational insights across multicloud platforms.
Data Architect, Informatica
Senior Data Architect designing Informatica, data governance, and hybrid-cloud solutions. Delivering enterprise architectures for Talan’s international technology consulting clients.
Data Architect, Informatica
Senior Data & Informatica Architect designing secure hybrid enterprise data architectures for Talan, an international technology consulting group. Leading Informatica, cloud, integration, governance, and data security initiatives.
Data Architect, Informatica
Senior Data & Informatica Architect designing secure hybrid enterprise architectures for Talan’s technology consulting clients. Leading data integration, governance, cloud, and Informatica architecture initiatives.
Data Platform Engineer
Data Platform Engineer building data pipelines, APIs, and risk knowledge graphs for Worth AI’s AI-powered decision-making products. Supporting real-time onboarding and trusted data services.
Data Engineer
Data Engineer maintaining ETL pipelines and data quality for Cotiviti’s healthcare data operations. Supporting scalable ingestion, warehousing, governance, and production data delivery.
Principal Data Architect
Principal Data Architect designing scalable GCP Data Lakehouse platforms for Egen’s data and insights consultancy. Leading client architecture, pipelines, governance, and technical delivery.
Lead Data Engineer
Senior Data Engineer building GCP data pipelines for Egen’s healthcare data innovation. Driving machine learning initiatives and data solutions that improve patient outcomes.
Vice President, Data Engineering
Vice President leading data strategy, architecture, and engineering for PICTON Investments, a Canadian alternative investment manager. Building financial data platforms supporting portfolios, analytics, risk, and AI.
Director of Data Engineering
Director of Data Engineering leading scalable, governed data platforms for OnTrac’s same-day and next-day U.S. delivery services. Overseeing pipelines, AI-ready data products, budgets, and engineering teams.
Engagement Lead – Enterprise Data Warehouse
Engagement Lead owning CIO-level client relationships and prioritizing enterprise data warehouse delivery. Shaping Snowflake remediation and Power BI dashboards for Bounteous, a global AI services firm.
Senior Data Engineer, Databricks
Engenheiro de Dados Sênior atuando em migração e engenharia de dados no Azure Databricks. Automatizando pipelines e assegurando governança, desempenho e integridade em ambientes críticos.