Senior Databricks Engineer – Lead
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Azure Databricks Engineer building scalable data pipelines and analytics solutions for Blend, an AI and data services provider. Optimizing Spark workloads on Azure.
Responsibilities:
- Design, develop, and maintain scalable data pipelines using Azure Databricks
- Implement ETL/ELT workflows using PySpark, Spark SQL, and Python
- Optimize Spark jobs for performance, cost, and scalability
- Work with structured and semi-structured data including Parquet, Delta, JSON, and CSV
- Build and manage Delta Lake tables with ACID, time travel, and schema evolution
- Integrate Databricks with Azure Data Lake Storage Gen2
- Develop complex queries and transformations using SQL
- Collaborate with data scientists, analysts, and stakeholders to support analytics and ML use cases
- Ensure data quality, validation, and monitoring
- Follow best practices for security, access control, and governance in Azure
Requirements:
- 6+ years of experience in Data Engineering
- Strong hands-on experience with Azure Databricks
- Proficiency in Python for data processing
- Strong knowledge of SQL, including joins, window functions, and performance tuning
- Hands-on experience with Apache Spark and PySpark
- Experience working with Delta Lake
- Knowledge of Azure Data Lake Storage Gen2
- Understanding of distributed computing concepts
- Experience with Git version control
- Experience with Azure Data Factory
- Exposure to CI/CD pipelines such as Azure DevOps and GitHub Actions
- Basic understanding of data modeling
- Familiarity with cloud security and RBAC in Azure
- Exposure to streaming data technologies including Spark Structured Streaming, Event Hub, and Kafka
Benefits:
- Remote work permitted
Similar Jobs
Analista Programador/a Senior Full-Stack
Senior Full-Stack Analyst Programmer building Java and Angular applications for The Whiteam Consulting. Supporting REST APIs, CI/CD pipelines, PostgreSQL, cloud environments and production operations.
Growth Engineer
Growth Engineer building e-commerce systems, AI-agent workflows, and experiments for The Cycle’s women’s health beverage brand. Owning Amazon, Shopify, analytics, acquisition, and retention.
Senior Software Engineer – Python Full Stack Developer
Senior Software Engineer building scalable Python and TypeScript cloud applications for First American India’s title insurance technology platforms. Developing AWS microservices, React frontends, workflows, and AI-assisted engineering solutions.
Software Engineer – Internal Tools, HR Automation
Software Engineer building Salmon’s internal HR database and automation workflows. Developing Retool applications, PostgreSQL structures, and integrations for a Southeast Asian fintech company.
Full Stack Developer
Full Stack Developer building secure, scalable AWS cloud applications for Guidehouse, a global AI-led professional services firm. Supporting modernization, prototypes, and client solutions.
AI Product Engineer
AI Product Engineer developing trusted, production-ready AI capabilities for Green Mountain Technology, a parcel logistics management and advisory firm. Leading generative AI products, analytics experiences, and decision-support solutions.
Senior Full Stack Engineer – F# / ASP.NET Core / ReactJS
Senior Full Stack Engineer building Boardwise’s board-management software with F#, ASP.NET Core, and React. Owning features, architecture, integrations, and scalable product development.
Senior Full Stack Software Engineer
Senior engineer building AI-driven full-stack cloud software for InfiniteChoice’s experience marketplace. Owning architecture, development, testing, deployment, monitoring, and continuous improvement.
Senior Software Engineer
Senior Software Engineer leading scalable software development for Pearson VUE's OnVUE remote proctoring platform. Designing cloud-native distributed systems, mentoring engineers, and driving technical excellence.
Senior Full Stack Engineer, .NET/React
Senior full-stack engineer building AI-enhanced .NET and React applications for a commercial real estate technology client. Developing scalable web products with Azure, Python, and modern DevOps practices.
Lead Engineer
Lead Engineer directing Unreal game development for Side, a global game services partner. Driving technical excellence, project delivery, mentoring, and cross-discipline collaboration across co-development engagements.
Senior Engineer
Senior Engineer optimising Unreal Engine 5 rendering and graphics for Side, a global game development and services provider. Driving performance solutions across PC, console, and other platforms.
Principal Full Stack Engineer – Travel & Ticketing Platform
Senior Full Stack Engineer building OleOle’s global football travel, ticketing, hospitality, and commerce platform. Scaling transactional web systems for fans attending matches and events worldwide.
Senior Full Stack Developer – React, FastAPI, Django
Full-stack developer improving React UX and building Python accounting features for mika’s AI-powered small-business finance platform. Owning complex features end-to-end in a senior, autonomous team.
Software Engineer, Medicaid
Backend Java engineer modernizing CMS Medicaid drug-pricing software for government services. Building services, data interfaces, testing, and drug-utilization review tools.
Senior Software Developer
Senior Software Developer owning Braze strategy, integrations, and data pipelines for Bally’s iGaming, sportsbook, casino, and CRM engagement systems.
Senior Software Engineer – Internal Efficiency
Senior Software Engineer embedding AI tools into Lyra Health’s mental healthcare workflows. Building agentic solutions that reduce repetitive work and improve measurable operational efficiency.
C++ Staff Software Engineer
Staff C++ Engineer developing NinjaOne’s endpoint management and autonomous patching platform. Building cross-platform agents that detect, install, and validate updates across Windows, macOS, and Linux.
Full Stack Developer – Java, React/Angular
Desarrollador Full Stack senior construyendo soluciones web Java y React/Angular para un proyecto tecnológico estadounidense. Integración de servicios, interfaces y control de versiones en un entorno internacional 100% remoto.
Desarrollador Full Stack – Java, React / Angular
Desarrollador Full Stack senior creando soluciones web para proyectos tecnológicos internacionales. Desarrollo end-to-end con Java/JEE y React o Angular, colaborando remotamente con equipos multidisciplinarios.