Databricks Developer, Unity Catalog Migration
Posted 1ds ago
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Databricks Engineer specializing in Unity Catalog for enterprise data governance. Supporting large-scale hydrocarbon reporting initiatives and optimizing data engineering workflows.
Responsibilities:
- Design, develop, and maintain scalable data pipelines on the Databricks Lakehouse Platform.
- Implement and administer Unity Catalog for enterprise data governance, access control, lineage, and data sharing.
- Support hydrocarbon reporting workflows, including ingestion, transformation, validation, and reconciliation of production and operational data.
- Develop ETL/ELT pipelines using Spark, Delta Lake, and Delta Live Tables (Lakeflow experience is a plus).
- Optimize Databricks workloads for performance, reliability, and cost efficiency.
- Collaborate with business stakeholders to translate reporting requirements into scalable technical solutions.
- Implement data quality checks, monitoring, and automated validation frameworks.
- Work closely with solution architects and data engineers on platform enhancements and production support.
- Document technical designs, governance standards, and operational procedures.
- Participate in code reviews, CI/CD implementation, and DevOps best practices.
Requirements:
- Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or a related discipline.
- 5+ years of data engineering experience with 3+ years of hands-on Databricks experience.
- Strong experience implementing and managing Unity Catalog in enterprise environments.
- Experience supporting Oil & Gas, Energy, or Hydrocarbon Reporting projects.
- Expertise with: Databricks Lakehouse, Apache Spark (PySpark), Delta Lake, SQL, Python
- Experience building enterprise ETL/ELT pipelines.
- Experience with Azure Databricks (AWS Databricks also considered).
- Strong understanding of data governance, security, metadata management, and lineage.
Benefits:
- health insurance
- retirement plans
- paid time off
- flexible work arrangements
- professional development
















