Mid-Level Data Engineer – AWS, Databricks
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Engenheiro de Dados desenvolvendo pipelines escaláveis com Python, Spark e Databricks na Keyrus. Otimizando ingestão, qualidade, custos e performance para soluções de dados empresariais.
Responsibilities:
- Develop and maintain data ingestion pipelines using Databricks Autoloader
- Implement incremental processing and schema evolution
- Implement large-scale data processing solutions using Spark Structured Streaming and Spark SQL
- Apply partitioning, processing optimization, and Delta Lake
- Analyze and optimize the performance and costs of Databricks environments
- Implement data quality practices
- Work with Git and pull request processes
- Prepare technical documentation for developed solutions
- Collaborate with business and Analytics teams to ensure data availability, structure, and security
- Develop and enhance scalable, efficient, secure, and high-performance data solutions
Requirements:
- Experience in data engineering and processing large volumes of information
- Proficiency in Python and Apache Spark
- Experience with Databricks
- Knowledge of building and maintaining data ingestion pipelines
- Experience with Spark Structured Streaming and Spark SQL
- Knowledge of Delta Lake and processing optimization techniques
- Experience with Git and version control
- Knowledge of data quality practices
- Preferred: knowledge of modern data architectures and ETL/ELT processes
- Preferred: experience with agile methodologies such as Scrum and Kanban
- Preferred: experience optimizing costs and performance in scalable data environments
- Analytical, self-directed, and skilled at problem-solving
- Eagerness to learn new technologies and propose innovative solutions
- Excellent communication skills with technical and non-technical audiences
- Ability to translate business needs into practical, scalable solutions
- Ability to influence decisions and build trusted relationships with stakeholders
Benefits:
- Remote work environment
- Opportunities to develop skills and technical judgment
- Collaboration with international teams across multiple geographies
- Diversity and multiculturalism culture
- Knowledge sharing and continuous improvement
Similar Jobs
Cloud Infrastructure and Storage Architect, SME
Cloud infrastructure and storage architect modernizing secure Azure and hybrid-cloud environments. Leading storage, DevSecOps, compliance, and enterprise infrastructure initiatives for SEC federal operations.
Staff Cloud Engineer
Staff Cloud Engineer scaling AWS infrastructure and automation for Varicent’s SaaS revenue-performance platform. Improving Terraform, CI/CD, reliability, security, and cloud efficiency for multiple products.
Cloud Service Technician
Remote Cloud Service Technician providing frontline support for Rightworks’ managed cloud and applications. Troubleshooting customer technology issues, managing tickets, and escalating high-priority incidents.
Cloud Engineer I – Azure
L1 Azure infrastructure support engineer monitoring and troubleshooting Windows and Linux cloud systems. Supporting Rackspace Technology’s managed cloud operations for enterprise customers.
DevOps Cloud Engineer
Senior DevOps Engineer improving Azure cloud reliability, security, and automation for Evernorth Health Services. Building CI/CD, Kubernetes, Terraform, and AI-assisted engineering capabilities.
Cloud Engineer II
Cloud Engineer II supporting customer AWS infrastructure for Rackspace Technology, an enterprise cloud and AI operator. Troubleshooting incidents, managing changes, and resolving complex infrastructure issues.
AI Delivery Engineer – AWS Bedrock
AI delivery engineer building production generative-AI applications on AWS Bedrock for Kunai’s financial-services clients. Translating workflows into secure, tested solutions and enabling client engineering teams.
Senior Data Engineer, AWS, Databricks
Senior Data Engineer building AWS and Databricks data platforms for GDIT’s federal court modernization program. Delivering secure pipelines, governance, migration, analytics, and AI/ML capabilities.
Senior Engineer – Cloud
Cloud engineer building scalable systems and automation for CrowdStrike’s Falcon Complete cybersecurity platform. Improving reliability, observability, and analyst protection workflows.
Senior Software Engineer, Cloud Platform
Senior Software Engineer building NVIDIA accelerated compute cloud services. Developing scalable automation for hardware, networking, and multi-tenant infrastructure.
Java Developer, AWS, Terraform, Python
Java Developer desarrollando soluciones Java sobre AWS para knowmad mood, compañía de transformación digital. Trabajo con microservicios, arquitectura hexagonal, Terraform y Python.
Cloud Engineer
Ingénieur Cloud chez Synapsys, société de conseil en transformation des infrastructures IT. Conception d’environnements cloud sécurisés, automatisés et observables.
AWS Cloud Engineer
AWS Cloud Engineer designing secure IL6 infrastructure, backend services, and databases. Supporting North South Consulting Group’s mission-critical government and DoD programs.
Cloud Engineer, GCP, BigQuery, TypeScript
Cloud Engineer building TypeScript and Node.js data collection systems across GCP. Managing BigQuery pipelines, containers, Terraform infrastructure, and automated deployments.
Azure Cloud Engineer
Azure Cloud Engineer supporting VDOT tolling infrastructure through Azure administration, networking, security, and automation. Maintaining secure cloud environments with PowerShell, Python, Terraform, and Entra ID.
Salesforce Marketing Cloud Architect
Salesforce Marketing Cloud Architect designing Salesforce Marketing Cloud solutions for VXForward’s technology clients. Building integrations, data models, campaigns, and scalable omnichannel engagement platforms.
Senior Backend Engineer, Cloud Platform
Senior Backend Engineer building scalable AWS and EKS infrastructure for ClickUp’s AI-powered workspace. Improving reliability, security, and cost efficiency across multi-region systems.
Lead Cloud Architect
Lead cloud architecture for C5MI’s secure federal SAP S/4HANA modernization programs. Direct infrastructure, authorization, resilience, and platform engineering for mission-critical defense environments.
Service Cloud Developer
Salesforce Service Cloud Developer building scalable customer-service solutions for Spyrosoft, a software engineering company. Integrating Salesforce with enterprise platforms and automating global service processes.
Principal Cloud Computing Engineer
Principal Cloud Engineer building secure AWS and OCI infrastructure, data pipelines, and AI automation. Supporting SAIC’s regulated U.S. Government technology missions from design through production.