Senior Data Engineer – Data for Products
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Lead the design and implementation of scalable batch and streaming data pipelines on Azure. Collaborate with product teams to ensure data quality, performance, and governance across the data lifecycle.
Responsibilities:
- Lead the design and implementation of robust pipelines for large-scale data ingestion and processing, both batch and streaming, ensuring low latency, high availability, and resilience.
- Define and maintain data architecture standards, including partitioning, versioning, security, and governance, ensuring consistency across domains and consumer teams.
- Ensure data reliability, quality, and traceability through observability, monitoring, automated testing, and Data Quality practices across the entire data lifecycle.
- Act as the technical point of contact for product and business stakeholders, translating strategic needs into scalable solutions and clearly communicating trade-offs (time, cost, performance, and risk).
- Evaluate and recommend technologies and frameworks for the data stack, ensuring continuous evolution with a focus on operational efficiency and maintainability.
- Conduct performance and cost diagnostics and optimizations for queries, jobs, storage, and orchestration.
- Support the technical development of the team through code reviews, mentoring, and knowledge sharing, raising standards and accelerating engineering best practices.
- Collaborate with other disciplines (Analytics Engineers, Data Analysts, Software Engineers, etc.) to integrate end-to-end solutions and unblock dependencies between teams.
Requirements:
- Bachelor's degree in a quantitative field (Computer Science, Engineering, Mathematics, etc.).
- Strong experience in Data Engineering, with autonomy to lead projects and make high-impact technical decisions.
- Advanced knowledge of Microsoft Azure services and features.
- Experience with parallel processing and messaging technologies (Hadoop, Spark, Kafka, etc.).
- Proficiency in data ingestion, integration, and orchestration tools (Azure Data Factory, Airflow, etc.).
- Experience with large-scale file storage environments (HDFS, blob storage, Data Lake, etc.).
- Strong mastery of relational databases (queries, programming, modeling, and performance).
- Development experience in programming languages (Python, Spark (PySpark), and/or Scala).
- Experience building and maintaining REST APIs and cloud functions (Azure).
- Ability to communicate technical decisions clearly and influence stakeholder alignment.
- The following are a plus:
- Azure certification (Microsoft Certified: Azure Data Engineer Associate).
- Databricks certification (Data Engineer Associate or Professional).
- Knowledge of Data Mesh architecture focused on data domains and federation between platform and product teams.
- Experience with NoSQL databases (Cassandra, MongoDB, etc.).
- Knowledge of user event capture and tracking solutions (Mixpanel, Segment, etc.).
- Experience with data contracts, quality standards, and CI/CD automation for data pipelines.
- Experience with lakehouse architectures and open storage formats (Delta Lake, Apache Iceberg, etc.).
Benefits:
- 🕘 Flexible Hours: Greater autonomy to organize your schedule with balance and responsibility.
- 🏠 Flexible Work Arrangements: Remote, hybrid, or on-site, depending on role requirements.
- 👕 No Dress Code: Freedom to be yourself, without formalities.
- 🎉 Birthday Day Off: One day off in your birthday month to celebrate as you wish.
- 🔋 Blip Recharge: 5 paid days off per year for roles without time tracking, designed to help balance work.
- 🍽️ Meal Allowance or Food Voucher: R$ 1,144.00 per month, no deductions and credited during vacations and leaves.
- 🚍 Transportation Voucher: Available as needed for commuting.
- 🏋️ Wellhub (Gympass): Access to gyms, wellness apps, and fitness activities, also available for dependents.
- 🎭 SESC Partnership: Access to culture, leisure, sports, hotels, holiday camps, and more.
- 🩺 Health Insurance (SulAmérica): National coverage, private room for you and your dependents, with only co-payments applied.
- 🦷 Dental Plan: National coverage for you and your dependents, with three plan options and full company coverage of the chosen plan's cost.
- 🧠 Conexa Health: Online psychological counseling platform.
- 🔐 Life Insurance: Coverage equivalent to 24 times your monthly salary.
- 👶 Extended Maternity Leave: 180 days to support the start of this new phase with peace of mind.
- 👨👧 Extended Paternity Leave: 30 days to be present and strengthen family bonds.
- 🧸 Childcare Allowance: Reimbursement of R$ 676.81 for children up to 2 years old and R$ 592.21 for children up to 5 years old.
- 🧩 Support for Dependents with Disabilities: Assistance of R$ 846.02 for children with disabilities or neurodiversity.
- 📘 Educational Support: Partnerships with higher education institutions and language schools.
- 💰 ICP – Short-Term Incentive: Financial recognition tied to Blip's performance goals.
- 🏳️🌈 Your Name Matters: Reimbursement of up to R$250.00 for expenses related to first name and/or gender marker changes, supporting inclusion and respect for identity.
Similar Jobs
Data Architect
Data Architect evaluating and evolving data architecture for partners at EVT. Focus on ingestion, processing, and structuring of data with a collaborative team environment.
Data Engineer
Data Engineer responsible for designing, building, and maintaining data pipelines at Loopback Health. Collaborating with cross-functional teams to drive system and process improvements.
Data Engineer
Data Engineer optimizing and maintaining data pipelines in Blackline Safety's IoT-enabled safety ecosystem. Collaborating with product, engineering, and analytics teams on impactful data-driven initiatives.
Senior Ingeniero de Datos
Senior Data Engineer developing data solutions for transforming lives at Sofka. Collaborating in a close-knit team focused on continuous improvement and technical excellence.
Senior Data Engineer
Senior Data Engineer at Arsipa developing scalable data platforms and ETL/ELT pipelines. Collaboration with teams to ensure data quality, performance, and innovation in data architecture.
Data Engineer
Data Engineer shaping cloud-based data solutions for top U.S. companies. Engage with diverse projects remotely from Latin America.
Senior Data Engineer
Senior Technical Consultant delivering end-to-end data solutions and managing client relationships. Collaborating with technical team to drive project success with Microsoft technologies.
Data Engineer
Data Engineer maintaining scalable data pipelines for an international company in Hamburg. Requires experience with Snowflake, ETL processes and Azure Data Factory.
IS Data Warehouse Architect – III
Data Warehouse Architect designing and developing data models and ETL specifications. Collaborating on data architecture and vendor relations for a nonprofit health care organization.
Senior Director – Data Platform, Analytics
Leading the development of an integrated data platform at Harbor Compliance. Unifying fragmented data and establishing governance to empower decision-making.
Senior Data Engineer
Senior Data Engineer at D.A. Davidson optimizing data integrations and migrating legacy structures. Involves cloud architecture and coordination with product teams.
Data Architect
Professional Services Delivery Architect at Ciena guiding global delivery teams and ensuring high-quality outcomes for customer engagements. Leading complex transformations and influencing solution strategy.
Distinguished Architect, Data Platform
Distinguished Architect leading the design of a next-generation data platform at CloudZero. Driving technical decisions and architectural innovations for scalable cloud cost management solutions.
Senior Data Engineer
Data Engineer designing and maintaining data pipelines for Arrow Electronics. Collaborating with teams in engineering and architecture discussions across global projects.
Data Engineer – AI, Spark, Databricks, Healthcare
Data Engineer managing advanced Spark scripts for data validation and integration at Cotiviti. Collaborating with engineering teams to ensure data integrity and compliance in healthcare data processes.
Senior Software Engineer, Data Platform Team
Senior Software Engineer designing and developing features for data platforms at Sysdig. Leading technical strategy and mentoring teams in cloud security innovations.
Data Engineering Architect – Manager
Data Engineering Architect managing scalable AWS Data Lakehouse architectures at Naveera Technology. Overseeing data engineering projects with a focus on analytics and reporting solutions.
AI Data Engineer
AI Data Engineer bridging data warehousing and AI-driven analytics capabilities. Collaborating on automated solutions and operational workflows in eCommerce with global brands.
AI Data Engineer
AI Data Engineer at Netwrix designing and building enterprise-grade data and AI platforms. Focused on stability and repeatability while adhering to security and governance standards.
Mid-level Data Cloud Developer – Data Engineer
Data Cloud Developer building cloud data platforms to support business decisions. Collaborating with tech teams to implement data solutions with a focus on quality and performance.