Senior Data Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Data Engineer building production data pipelines for an Australia-based consulting team. Engaging in end-to-end data solutions and working directly with clients on efficient data processing.
Responsibilities:
- Design, build, and maintain scalable data pipelines in modern lakehouse architectures.
- Develop clean, efficient, and production-ready Python and SQL code.
- Implement ETL/ELT processes, transformations, and orchestration workflows.
- Model data using medallion architecture (Bronze/Silver/Gold), star schemas, and SCDs.
- Integrate multiple data sources (APIs, databases, SaaS platforms, flat files).
- Deploy pipelines using CI/CD tools and version control best practices.
- Leverage AI tools (e.g., agent-based workflows, automation scripts) to improve delivery speed and quality.
- Collaborate directly with clients on requirements, architecture, and delivery updates.
- Monitor, troubleshoot, and optimise pipelines for performance and reliability.
- Ensure data quality, integrity, and production readiness.
Requirements:
- 5+ years of professional experience in data engineering.
- Must have experience with Databricks or Fabric.
- Strong Python skills for production environments.
- Advanced SQL (CTEs, window functions, performance tuning, complex joins).
- Hands-on experience with modern data platforms (lakehouse, pipelines, distributed processing).
- Experience with dimensional modelling and analytics-ready data design.
- Solid understanding of CI/CD, Git workflows, and deployment practices.
- Strong communication skills with the ability to work directly with clients.
- Ability to translate business requirements into technical solutions.
- AI Proficiency (Required): This role requires strong, practical experience using AI in development workflows.
- Comfortable working in modern development environments with AI-assisted tooling.
- Proven experience building AI-powered tools, automations, or agents with real business impact.
- Ability to use AI across the development lifecycle (design, coding, debugging, testing, documentation).
- Strong judgment in validating and refining AI-generated outputs.
- Experience with prompt design, context handling, and tool integrations.
Similar Jobs
GCP API Data Engineer
Senior Staff Data Engineer designing and building API-driven data platforms on Google Cloud Platform. Collaborating with engineers and scientists to implement best practices and mentor team members.
Data Engineer, Data Quality & Automation
Data Engineer developing automated data quality pipelines supporting federal programs. Collaborating with data teams to ensure scalable and reliable data solutions within a DevSecOps framework.
Principal Data Architect
Principal Data Architect at Committee for Children, defining and evolving data architecture for a social enterprise. Collaborate with teams to ensure data governance and accessibility for child welfare programs.
Senior Data Engineer, Snowflake
Sr. Data Engineer designing and implementing scalable data solutions in Snowflake for clients. Collaborating with teams to translate requirements into actionable technical solutions and optimizations.
Data Engineer – DataOps, Infrastructure Focus
Data Engineer responsible for building and maintaining AWS data infrastructure for Guidewire. Collaborating with teams to support AI-driven analytics and operational excellence.
Data Architect III
Data Architect III managing data strategy and overseeing data collection and organization in a remote setting for Veterans Affairs. Leading data architecture and governance initiatives to improve Veteran services.
Senior Manager, Data Architecture
Senior Manager of Data Architecture ensuring enterprise data is standardized, secure, and well modeled. Overseeing data architecture across platforms and applications and translating business outcomes into data models.
Senior Data Engineer
Senior Data Engineer at Peek transforming data sources into actionable insights for analytics. Collaborating across teams to support business intelligence and enhance data infrastructure.
Senior Data Engineer
Senior Data Engineer building and scaling data infrastructure for Wagmo’s pet health solutions. Collaborate with multiple teams to ensure clean data flows across the business.
Senior Clinical Data Engineer
Senior Clinical Data Engineer overseeing clinical data lifecycle for Oura's research studies. Collaborating with teams to maintain data integrity and regulatory compliance in clinical trials.
Lead Data Engineer
Lead Data Engineer overseeing data platform and analytics engineering teams at OneOncology. Responsible for operational excellence and technical direction in data processing and management.
Senior Data Engineer
Senior Data Engineer at Kainos designing and developing data processing software for large-scale data solutions. Collaborating with agile teams to ensure high code quality and solution effectiveness.
Lead Data Engineer
Lead Data Engineer at Mercator.ai building scalable data infrastructure and distributed data pipelines. Ensuring performance and reliability while mentoring junior engineers.
Data Engineer
Data Engineer at Egen designing and building data processing pipelines using advanced technologies. Collaborating with stakeholders to ensure data accessibility, security, and accuracy.
Data Architect
Data Architect providing data management and analytics support for HRSA's Organ Procurement program. Collaborating on patient safety performance, technical assistance, and data visualization.
AI Data Engineering Lead
AI Data Engineering Lead building data foundations for AI models at Ideagen. Leading a team to ensure data quality, compliance, and operational efficiency.
Cloud Data Architect
Cloud Data Architect at Hunter Strategy responsible for telemetry and data normalization in cybersecurity. Designs log collection and routing architectures for AWS and Azure environments.
Senior Data Engineering Analyst
Data Engineer managing data governance and modernization projects using Databricks at Keeggo. Collaborating with teams to enhance data pipeline architecture and performance.
Senior Data Engineer – AWS
Senior Data Analyst migrating data from SQL Server to AWS SaaS architecture, ensuring data integrity and developing reusable ETL processes.
Senior Data Engineer
Senior Data Engineer at Hyatt developing robust data pipelines and supporting data-driven decisions. Collaborating with cross-functional teams in a dynamic and fast-paced environment.