Data Engineer – Databricks, PySpark
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Data Engineer building Databricks and PySpark lakehouse solutions for Sigma Software’s AdTech analytics client. Developing scalable batch and near-real-time marketing data pipelines.
Responsibilities:
- Develop and maintain data pipelines for processing large volumes of data in batch and near real-time environments
- Work with Databricks, PySpark, and SQL to transform, validate, and optimize data
- Support implementation of Data Lakehouse solutions following engineering standards and best practices
- Build, test, and automate ETL processes and data workflows
- Improve data quality, organization, and reliability across platforms
- Collaborate with engineers, analysts, and stakeholders to deliver scalable solutions
- Troubleshoot, optimize, and monitor data pipelines
- Stay up to date with modern data engineering technologies
- Ensure deliverables meet quality standards and project requirements
Requirements:
- 2+ years of experience in Big Data development and database design
- Strong hands-on experience with SQL
- Strong Python development skills
- Hands-on experience with PySpark
- Experience developing and maintaining data pipelines in Databricks
- Experience with Delta Lake
- Experience with Medallion Architecture (Bronze, Silver, Gold layers)
- Strong analytical and problem-solving skills
- Intermediate or higher English level
- Experience with AWS services, including S3 is a plus
Benefits:
- Employees can work remotely
Similar Jobs
Data Engineer II – Data Platform
Data Engineer II building scalable AWS, Snowflake, and dbt data platforms for First Citizens Bank. Developing reliable pipelines, data quality controls, and analytics solutions.
Senior Data Engineer, Developer
Senior Data Engineer owning Azure data pipelines, integrations, and backend development. Supporting Smart Working’s global teams with financial analysis, automation, and business visibility.
Data Platform Engineer – Clearance Required
Data Platform Engineer building Palantir Foundry, Databricks, and Power Platform solutions for LMI’s Army and federal government clients. Leading data pipelines, AI/ML integration, and technical mentorship.
CRM/Data Platform Consultant
iGaming CRM consultant shaping an advanced player data platform for a MarTech company. Advising on strategy, architecture, vendor selection, retention, and multi-brand data.
Senior Data Engineer
Data Engineer building scalable data products and pipelines for SOFTSWISS’s enterprise analytics platform. Working with Spark, Kafka, Delta Lake, and modern data infrastructure.
Senior Data Engineer
Senior Data Engineer improving Snowflake, dbt, SQL, and Python data platforms. Enhancing reliable financial and healthcare technology solutions at Kodiak Solutions.
Staff Data Engineer
Staff Data Engineer improving Snowflake and dbt reliability, performance, and standards. Modernizing data platforms for Kodiak Solutions’ healthcare finance and revenue-cycle products.
Data Engineer, SAS
SAS/Data Engineer modernising SAS processes and migrating them to Databricks/Fabric. Bridge351 delivers cloud, cybersecurity, data, and advanced development solutions across Europe.
Senior Data Engineer
Senior Data Engineer modernizing data processes and pipelines for a cybersecurity solution protecting applications from automated attacks. Driving reliability, scalability, and architectural improvements.
Senior Data Scientist, Data Engineering Expert
Senior data scientists and engineers designing evaluation tasks and training frontier AI models. Remote contract work with Mercor and leading AI research labs.
Senior SAP S/4HANA Data Migration Analyst
Senior SAP S/4HANA migration analyst executing enterprise data migrations for TE Connectivity, a global industrial technology and connectivity solutions leader.
Principal Architect – Data Centers
Principal Architect leading data-center design operations, studio growth, and client programs for MG2, an affiliate of Colliers Engineering & Design. Overseeing strategic planning, staffing, financial performance, and business development.
Senior Data Engineer
Senior Data Engineer building semantic layers, pipelines, governance, and AI-ready data foundations. CodeRoad partners with North American companies on technology execution and modernization.
Senior Data Engineer
Senior Data Engineer architecting scalable pipelines and cloud data platforms for CodeRoad, a technology execution partner. Supporting analytics, modernization, and AI engineering projects for US-based clients.
Data Engineer
Data Engineer building secure analytical pipelines, semantic models, and data governance for CodeRoad’s US technology clients. Supporting AI querying, RAG architectures, and Agentic AI workflows.
Data Engineer
Data Engineer building scalable ETL/ELT pipelines and AI/RAG data workflows. Supporting CodeRoad’s US clients with SQL, Python, orchestration, and production data operations.
Data Engineer – Support Project L2
Data Engineer resolving SQL, pipeline, and reporting issues for CodeRoad, a technology execution partner for US companies. Automating support workflows with scripting and AI tools.
Product Owner, Data Migration
Product Owner leading Agile delivery, data migration, and legacy modernization initiatives for Global Alliant’s federal client. Coordinating technical and business teams through migration planning, testing, and cutover.
Senior Data Engineer
Senior Data Engineer building Finanzen.net’s data platform for investment information and trading services. Managing Databricks, Spark, dbt, Airbyte, and scalable ELT/ETL pipelines.
Data Engineer III, Finance
Data Engineer building scalable cloud data pipelines for Grainger’s MRO distribution business. Integrating enterprise sources to enable trusted Finance analytics and improve data quality.