Lead Data Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Lead Data Engineer at Greenbox Capital designing scalable data solutions using Azure for small businesses. Collaborate with teams to ensure high-quality, governed data pipelines that support critical business decisions.
Responsibilities:
- Design, develop, and maintain scalable data pipelines and ETL processes using Azure Data Factory, Azure Databricks, and other Azure services.
- Own the data engineering framework, including pipeline patterns, orchestration standards, and reusable components.
- Collaborate with data scientists, Software engineers, analysts, and other stakeholders to understand data requirements and deliver high-quality data solutions.
- Define, document, and enforce best practices for ADF, Databricks, Spark, and data modeling.
- Implement and maintain data storage solutions using Azure SQL Database, Azure Data Lake Storage, and Azure Cosmos DB.
- Ensure data quality and integrity by implementing data validation, cleansing, and transformation processes.
- Implement data quality checks, validation frameworks, and monitoring for critical data assets.
- Design and support governance patterns leveraging Databricks Unity Catalog and Azure-native controls.
- Develop and maintain documentation for data engineering processes and solutions.
Requirements:
- Bachelor’s degree in computer science, Information Technology, or a related field.
- 5+ years of experience in data engineering, with demonstrated ownership of production systems.
- 3+ years of experience in the Azure ecosystem.
- Proficiency in Azure Data Factory, Azure Databricks, Azure SQL Database, Azure Data Lake Storage, Azure Cosmos DB, Databricks Unity Catalog.
- Material experience interacting with relational and NoSQL, JSON, XML, and interacting with REST APIs.
- Deep hands-one experience with: Azure Data Factory (orchestration, patterns, parameterization), Azure Databricks / Apache Spark (PySpark, performance tuning, cluster design), Azure Data Lake Storage and Azure SQL.
- Advanced experience in programming in Python & SQL.
- Solid understanding of data modeling, ETL/ELT design, and analytical data platforms.
- Experience with Azure DevOps or GitHub and CI/CD pipelines.
- Experience designing and deploying data governance frameworks from the ground up.
- Proven experience owning data engineering frameworks, not individual pipelines.
Benefits:
- Competitive Pay - We know your worth and we pay accordingly.
- Flexible PTO - Work hard, rest well. Take the time you need to recharge.
- Full Benefits Package - Health, dental, vision
- Smart, Supportive Teammates - Collaborate with sharp minds who are kind, driven and uphold our core values: Commitment, Communication, Teamwork, Service and Integrity!
Similar Jobs
Data Engineer
Júnior Data Engineer supporting SQL Server Data Warehouse, developing ETL routines and data models. Collaborating with Product, Engineering, and Analytics teams in a growing environment.
AVP, Data Engineering
AVP of Data Engineering owning Ensemble’s enterprise data platform strategy. Leading cloud-native data platform design on Azure and Databricks for scalable analytics and AI capabilities.
Data Engineer
Data Engineer developing and maintaining data pipelines and transformation models for analytics at Industrial Electric Manufacturing. Collaborating with stakeholders and ensuring data quality across business decisions.
Senior Data Engineering Manager
Senior Manager of Data Engineering leading a team that manages data at ReUp Education. Empowering adult learners and institutions by developing data solutions and pipelines.
Director, Data Engineering, ML – AI
Director of Data Engineering leading ML and AI initiatives for Measurabl. Responsible for data platform strategy and sustainability analytics in commercial real estate.
Data Engineer
Data Engineer responsible for building foundational infrastructure for AI deployments and product launches. Collaborating with top researchers and engineers in cutting-edge AI systems.
Data Engineer
Data Engineer at OrderlyMeds connecting multiple systems to BigQuery for strategic insights. Collaborating with a cross-functional team to enhance data utility and drive business decisions.
Data Engineer/Software Engineer
Data Engineer/Software Engineer responsible for implementing data-driven solutions and managing data pipelines. Join ALTEN Technology USA to work on challenging engineering projects.
Data Engineering Manager
Engineering leader managing Data Engineering team at Northrop Grumman. Overseeing data systems and leading a collaborative team to ensure engineering excellence.
Data Engineer
Senior Data Engineer serving in Prima's engineering team, building scalable data products and maintaining pipelines for optimal growth. Engaging in innovative solutions to transform motor insurance experience.
Senior Data Engineer
Senior Data Engineer leading design discussions and driving technical vision for healthcare data solutions. Collaborating cross-functionally to ensure high standards and timely project delivery.
Senior Analytics Data Engineer
Full Stack Engineer developing scalable applications for the largest real-world healthcare datasets provider. Collaborating with an international team under technical leadership for critical platform enhancements.
Director – Data Engineering
Director leading and scaling data engineering function for a Weekday's client. Building efficient data platforms powering analytics and business intelligence across the organization.
Data Engineer
Data Engineer building and maintaining data pipelines for marketing and analytics at an EdTech app with 70 million users. Collaborating with analysts and business teams for data quality and reporting.
Data Engineer
Data Engineer responsible for managing data infrastructure at Bemobi with innovative solutions for data access and analysis. Join the Data Platform team to enhance data lifecycle processes.
Senior Lead Software Engineer, Data Platform
Senior Software Engineer building scalable data and analytics platforms at Coupa. Focusing on advanced data ingestion and processing in a multi-cloud environment.
Workday/Fabric Data Engineer
Workday/Fabric Data Engineer supporting USNH's business intelligence initiatives by designing and maintaining data solutions. Collaborating with teams for effective reporting and analytics.
Senior Data Engineer
Senior Data Engineer enhancing Dotdigital's data platform and capabilities. Designing data pipelines and collaborating with teams to unlock the value of data.
Director of Data Engineering – Platforms
Director of Data Engineering at Cotality leading data transformation initiatives and data architecture frameworks. Collaborating with product teams to drive actionable business intelligence.
Data Architect
Design and manage enterprise data architecture with a focus on Air Force Logistics Systems. Collaborate with engineers and government stakeholders for scalable and secure data solutions.