Senior Data Engineer – Databricks, AWS
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Databricks Data Engineer specializing in AWS cloud-native data solutions. Supporting federal client data platform modernization initiatives with expertise in data pipelines and governance.
Responsibilities:
- Design, develop, and maintain scalable data ingestion, transformation, and publishing pipelines utilizing Databricks and AWS services.
- Implement and optimize Databricks Lakehouse capabilities including Unity Catalog, Delta Live Tables, Auto Loader, Databricks SQL, and Delta Sharing.
- Build and maintain governed data products supporting operational, analytical, reporting, and machine learning workloads.
- Develop and support medallion architecture data pipelines and enterprise data quality frameworks.
- Implement data governance controls, metadata management, lineage tracking, and data retention policies.
- Collaborate with cloud engineers, architects, cybersecurity specialists, and business stakeholders to deliver secure, production-ready solutions.
- Optimize platform performance through partitioning, clustering, caching, workload tuning, and query optimization techniques.
- Support analytics enablement through semantic layers, dashboards, reporting solutions, and self-service data access capabilities.
- Participate in architecture reviews, operational readiness activities, platform modernization initiatives, and continuous improvement efforts.
- Create and maintain technical documentation, design artifacts, operational procedures, and engineering standards.
Requirements:
- Bachelor's degree in Computer Science, Information Systems, Engineering, Mathematics, or a related discipline.
- Minimum eight (8) years of experience in cloud-based data engineering, platform development, or data architecture.
- Minimum three (3) years of hands-on Databricks experience in production environments.
- Experience designing, developing, and supporting large-scale ETL/ELT solutions.
- Strong experience with AWS services including S3, IAM, Glue, Athena, Lambda, Redshift, and CloudWatch.
- Experience implementing enterprise data governance and security controls.
- Strong SQL and Python development skills.
- Experience working with data lakes, data warehouses, and cloud-native analytics platforms.
- Strong troubleshooting, performance tuning, and optimization experience.
Benefits:
- Health insurance
- 401(k) matching
- Flexible work hours
- Paid time off
- Remote work options
Similar Jobs
Freelance Data Engineer
Data Engineer at Incubeta responsible for designing, building, and maintaining data processing systems to ensure data integrity and optimize pipelines. Mentors junior members and delivers data solutions effectively.
Senior Data Engineer
Senior Data Engineer delivering modern data solutions at Carrier, focused on data pipelines and lakehouse architectures. Collaborating with cross-functional teams to support analytics and reporting.
Lead Product Manager, Data Platform
Lead Product Manager at Alkami driving the data platform strategy for the fintech sector. Collaborate cross-functionally to enhance data products in a remote-first environment.
Data Warehouse Developer
Data Warehouse Developer assisting in the analytics solutions for the University of Wisconsin-Platteville. Collaborating in the design, build, testing, and deployment of data warehousing.
Senior Data Engineer
Senior Data Engineer building resilient data pipelines for Verantos's high-accuracy RWE generation platform. Leading data architecture and collaborating with cross-functional teams in a remote US-based role.
Senior Data Engineer
Senior Data Engineer designing and building streaming pipelines for Simple Life, an AI-powered health coaching app. Collaborate on ingesting and processing data to enhance user experience.
Senior Data Engineer – AI
Senior Data Engineer at Anaplan building AI capabilities and solutions for business decision-making. Working across the full stack of Anaplan AI applications and developing innovative features.
Associate Data Engineer – Artificial Intelligence, AI
Data Engineer at Cotiviti managing data processes and implementations for operational excellence. Involves data analysis, pipeline creation, and troubleshooting in collaboration with Engineering teams.
Data Engineer – AI, Agents, Context – Revenue Cycle (Associate)
Data Engineer developing and maintaining AI/context data capabilities for healthcare innovation. Building data pipelines and collaborating to enhance operational efficiency and clinical outcomes.
Data Engineer – AI, Agents, Context, Clinical – Senior Associate
Data Engineer contributing to healthcare AI context platform at Huron. Implementing data products and ensuring operational excellence in a remote role.
Data Operations Engineer
Data Operations Engineer Associate managing Power BI datasets for healthcare organizations. Designing data models and ensuring data quality to optimize analytics and reporting processes.
Senior Data Engineer, AI & Context Platform, Healthcare Insights, Rev Cycle
Senior Data Engineer focused on AI context platform for healthcare insights. Leading technical delivery in building scalable data capabilities leveraging unstructured data.
Data Architect – Data Governance
Data Architect at Smarthis, focusing on defining efficient data architectures and supporting engineering teams in implementations. Collaborating on data solutions across cloud environments.
Data Engineer, L3
Data Engineer (L3) designing scalable data architecture and robust data pipelines for social marketing. Collaborate in a fintech initiative providing insights on personal finance and growth marketing.
Data Engineer – L2
Data Engineer building and maintaining data pipelines for marketing analytics and support across business teams. Contributing to data ingestion and modelling from various platforms with a focus on Meta Ads.
Data Architect, Information Strategy
Data Architecture & Information Strategy Engineer applying engineering principles and analytical skills at Ford Motor Company. Collaborating on data strategies and improving business processes.
Director, Data Engineering – AI Native
Director of Data Engineering leading engineering in the Data and Analytics organization at Life360. Managing data from end to end, collaborating with technical and business teams.
Enterprise Data Warehouse Developer – Microsoft Fabric
Enterprise Data Warehouse Developer utilizing Microsoft Fabric to design and implement data solutions for the healthcare sector. Collaborating with teams to ensure optimal data-driven decisions.
Staff Data Engineer, Ads
Staff Data Engineer at Discord driving technical vision for advertising products with a focus on data infrastructure. Leading cross-functional initiatives while collaborating with data science and product teams.
Data Engineer – Specialist
Data Engineer specializing in AI for data modernization and system optimization. Join Compass UOL in shaping next-gen data solutions and AI-driven outputs.