Senior GenAI Data Engineer, Databricks
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Data Engineer specializing in AI applications for a fast-growing advertising tech company. Building data pipelines and enhancing data reliability while collaborating with cross-functional teams.
Responsibilities:
- Architect AI Agents: Build and deploy agents that can perform NLP-based data generation, automated data enrichment, and complex data reasoning within Databricks.
- Natural Language Interfaces: Develop "Chat with your Data" features, allowing stakeholders to query the data warehouse using natural language.
- Integrate LLMs into data workflows for automation and intelligence
- Develop scalable data models to support analytics and AI use cases
- Implement AI-driven enhancements such as anomaly detection and data enrichment
- Collaborate with data, analytics, and engineering teams to improve data reliability
- Optimize performance and scalability of data and AI workflows
- Support automation through CI/CD practices
- Ensure data quality, traceability, and maintainability across pipelines
Requirements:
- Databricks & AI Architecture (Must-Have)
- Strong experience working with Databricks Lakehouse architecture
- Nice to have expertise in Databricks Mosaic AI and Unity Catalog for governing AI assets
- Hands on experience Building RAG (Retrieval-Augmented Generation) pipelines using Vector Search
- SQL & Data Modeling (Must-Have)
- Advanced SQL development
- AI Engineering & Data Workflows (Must-Have)
- Experience integrating LLM APIs (OpenAI, Anthropic, etc.) into data workflows
- Hands-on experience using AI for:
- Data enrichment
- Anomaly detection
- Automated classification
- Experience with LangChain, LlamaIndex, or similar frameworks
- Exposure to Model Context Protocol (MCP) or similar approaches to connect AI models with external tools and data sources
- Strong understanding of Tool Calling / Function Calling: enabling LLMs to interact with SQL databases and external APIs securely.
- Experience in Prompt Engineering and Guardrailing: designing system prompts that maintain context and hierarchy (e.g., understanding team associations).
- Platform & Engineering Practices (Nice-to-Have / Medium)
- Experience with GitHub workflows
- Familiarity with CI/CD pipelines (Jenkins or similar)
- Experience working with YAML/YML configuration files
Benefits:
- Monetary compensation
- Year-end Bonus
- IMSS, AFORE, INFONAVIT
- Major Medical Expenses Insurance
- Life Insurance
- Funeral Expenses Coverage
- TDU Membership
- MediAccess
- Health Check-Up Subsidy
- Preferential rates for car insurance
- Vacations
- Official Mexican Holidays
- Life Happens Days
- Bereavement Leave
- Civil Marriage Leave
- English Classes
- Certifications
- Educational Agreements (Talisis, U-ERRE, UNID, TecMilenio, Tec de Monterrey, UDEM, SPIS)
- Corporate Agreements & Discounts (Sorteos Tec, Envia Flores, TopGolf)
- Taquitos Rewards
- Birthday Bonus
- Work-from-home Bonus
- Laptop Policy
Similar Jobs
Lead Data Engineer
Lead Data Engineer responsible for developing AI-driven data pipelines and infrastructure. Collaborate with the CTO and build a team within a fully remote environment.
Data Engineer
Data Engineer responsible for building data pipelines and ensuring data quality for analytics. Collaborating with cross-functional teams to support enterprise decision-making in a remote setup.
Senior Data Engineer, DataBricks, Data Privacy – LGPD
Senior Data Engineer supporting Databricks integration with focus on data privacy and governance. Working with cross-functional teams to ensure compliance with LGPD requirements.
Lead Architect – Master Data
Lead Architect driving global master data architecture strategy during SAP S/4HANA transformation at Scotts Miracle-Gro. Collaborate with VP, Data Intelligence & AI on enterprise data management initiatives.
Senior Data Science Engineer
Sr Data Science Engineer at LegitScript developing ML models for risk detection and prioritization. Collaborating on data pipelines and production-ready APIs in a flexible work environment.
Director, Data Engineering – Business Applications
Director of Data Engineering and Business Applications at FOCUS overseeing data pipelines and application management. Collaborating with C-level executives to align data infrastructure with strategic growth objectives.
Enterprise Architect – Data Management
Enterprise Architect tackling complex data challenges and shaping integration across the enterprise. Leading technical direction to improve outcomes and operational efficiency.
Director, Data Engineering – Business Applications
Director, Data Engineering responsible for enterprise data engineering and business applications at PEAK6. Collaborating with leaders to align data infrastructure with strategic objectives.
Senior Data Engineer
Senior Data Engineer responsible for building real-time data capabilities and infrastructure at DraftKings. Collaborating with various teams to optimize data solutions and drive innovative AI experiences.
Data Engineer III
Experienced Software Engineer role at Kimedics, LLC, focusing on healthcare staffing. Independently design complex features and mentor junior engineers within the team.
Data Engineer
Data Engineer responsible for developing Azure-based data pipelines and analytics solutions. Collaborating with global teams and delivering data-driven decision-making support at SCIEX.
Software Engineer – Data Platforms
Software Engineer designing and building data engines for TV and advertising data at smartclip. Ensuring systems are scalable and leading automation with cutting-edge tools.
Data Engineer
Data Engineer responsible for establishing data infrastructure and supporting analytics at fintech company Overflow. Collaborating with various teams to ensure reliable access to metrics and data governance.
Data Engineer
Data Engineer optimizing data estate and monitoring ETL processes for digital marketing sector. Collaborating on architecture migration and data analytics projects to enhance performance.
Senior GCP Data Engineer
Senior Data Engineer in healthcare focusing on Google Cloud Platform and scalable data solutions. Developing data pipelines and optimizations using GCP tools and best practices.
Senior Consultant II, Applied Machine Learning, Data Engineering
Applied Machine Learning Engineer Senior Consultant II developing AI-driven data pipelines and solutions. Support data automation initiatives to enhance operational efficiency and data quality.
Data Engineer – Fintech
Data Engineer developing scalable data solutions for the fintech sector at Hypoport. Responsible for end-to-end data product ownership and cross-functional collaboration.
Staff Data Engineer
Founding Data Engineer at Payabli developing data systems and pipelines for payments infrastructure. Owner of data reliability and architecture decisions critical to the platform's analytics and ML efforts.
Data & AI Engineer – Microsoft Fabric
Senior Data Engineer with deep expertise in Microsoft Fabric and AI integration at Reply. Delivering end-to-end data solutions and building large-scale data pipelines.
Data Engineer
Data Engineer designing, building, and optimizing scalable healthcare data solutions leveraging Microsoft Azure ecosystem. Focus on ingestion, transformation, modeling, and delivery of complex datasets.