Software Developer, Data Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Software Developer & Data Engineer building WISEcode’s food intelligence lakehouse. Designing ingestion, entity resolution, and data pipelines powering food decisions.
Responsibilities:
- Own end-to-end lakehouse implementation and collaborate on data/platform architecture
- Build source-ingestion patterns for APIs, public-source text, and external databases
- Merge product records across retailers and dates through identity and entity resolution
- Determine winning sources for conflicting values while preserving original source records for traceability
- Consolidate existing food-enrichment pipelines into a single pipeline
- Build connectors to deliver consolidated food data to WISE Intel and the consumer app
- Maintain fail-fast configuration discipline and early failures for missing settings
- Establish ingestion capabilities and quality reporting for Data Assembly readiness
- Write concise, dated technical decision records before code implementation
- Work as a senior individual contributor alongside the Data team and second engineer on The Pantry
Requirements:
- 3+ years of experience in a similar role
- Proven track record of designing and shipping a lakehouse or data warehouse from scratch, and operating it for at least one year in production
- Demonstrated experience treating data identity and entity resolution as first-class architectural questions
- Experience running batch data pipelines that invoke paid cloud/third-party APIs at scale with strict cost efficiency and budget controls
- Experience pairing daily with other engineers in a fast-paced environment
- Native proficiency in Python and SQL
- Experience with DuckDB, DuckLake, S3 Parquet, Postgres, DynamoDB, Prefect, Kubernetes/Fargate job runners, AWS infrastructure, Terraform, IAM, Quarto reporting, Medallion architecture, star schema, and GitHub Actions
- Familiarity with dbt, Dagster, or Airflow
- Comfort with ambiguity and pace
- Preferred: architectural trade-offs between DuckDB and distributed systems such as Spark, Databricks, or Snowflake
- Preferred: data acquisition methods such as web scraping and OCR API integration
- Food or nutrition domain knowledge, deep Spark/Databricks/Snowflake experience, and people management experience are explicitly not required
Benefits:
- Fully remote
- Unlimited PTO
- 10 paid holidays
- 401(k) with a 4% company match, immediately vested
- Comprehensive medical, dental, vision, life, and additional ancillary coverage
- 100% coverage for employees, with 80% covered for dependents
- 100% employer paid STD, LTD, Identity Theft, and Life insurance
- FSA and HSA plans available
- $200/month HSA contributions for Employee plans
- $400/month HSA contributions for Employee + Dependent plans
- Competitive bonus
- Meaningful equity
Similar Jobs
Data Engineer, AI & Analytics
Data Engineer building unified, AI-ready marketing data foundations for Power Digital, an AI-native growth firm. Owning ingestion, dbt modeling, semantic layers, and production pipelines.
Lead Data Engineer
Lead Data Engineer directing Microsoft Fabric, Python, and PySpark data engagements for CreateFuture, an AI-native consulting partner. Leading teams, architecture, pipelines, and client delivery.
Senior Data Operations Engineer
Senior Data Operations Engineer managing Snowflake health, performance, and costs for Anvilogic’s AI cybersecurity platform. Onboarding customers and supporting Snowflake and Databricks production environments.
Senior Data Engineer
Senior Data Engineer building Azure cloud data pipelines for Nimble Gravity, a growth-focused data solutions company. Leading ETL, transformation, warehousing, and data engineering teams.
Senior Data Engineer
Senior Data Engineer leading scalable data architecture, warehousing, and pipeline projects. Velir supports nonprofits and Fortune 500 brands through data and analytics consulting.
Principal Data Architect
Principal Data Architect modernizing a major US mortgage lender’s Finance data platform with Azure, Microsoft Fabric, and Snowflake. Designing CDC pipelines, governed semantic models, and dimensional data architecture at CI&T.
Principal Data Architect
Principal Data Architect modernizing a major US mortgage lender’s finance data platform with Azure, Microsoft Fabric, and Snowflake. Owning architecture, CDC pipelines, governed BI semantics, and hands-on delivery.
Geospatial Data Engineer
Geospatial Data Engineer building scalable GIS pipelines and data products for Arden University. Supporting evidence-based decisions across its expanding digital higher-education ecosystem.
Data Engineer
Senior Data Engineer building data ingestion, Snowflake, and CI/CD infrastructure. Supporting analytics across the business for a client in Gdynia, Poland.
Data Engineer
Lead Data Engineer designing scalable data-processing systems for Kainos, a UK digital technology services company. Leading engineers, ensuring software quality, and delivering secure ETL/ELT solutions.
Junior Data Engineer
Junior Data Engineer building Microsoft Fabric pipelines and data quality checks for Kreate’s plastic manufacturing operations. Supporting dashboards, planning, and daily reporting across ERP, HR, and retail data.
Data Engineer – Product, Pricing & Inventory Data
Data Engineer building scalable product, pricing, and inventory data products for INTERSPORT’s e-commerce platform. Ensuring data quality, integration, governance, and availability for digital products and analytics.
Data Engineer – Tracking & Behavioral Data
Data Engineer für Tracking und Verhaltensdaten bei INTERSPORTs E-Commerce-Digitalunit. Aufbau von Event-Infrastruktur, Datenpipelines und Datengrundlagen für Analytics, Personalisierung und A/B-Tests.
Data Engineer – Commerce and Customer Data
Data Engineer developing scalable commerce and customer data products for INTERSPORT’s e-commerce unit. Building ETL/ELT pipelines supporting analytics, product development, and AI applications.
Data Engineer – Fulfillment & Logistics Data
Data Engineer building fulfillment and logistics data products for INTERSPORT’s e-commerce platform. Developing pipelines, data models, and tracking integrations for orders, deliveries, returns, and partner performance.
Senior Data Engineer – Fluent English
Engenheiro de Dados Sênior desenvolvendo soluções analíticas com Azure, Snowflake e DBT. Atuando em pipelines, modelagem, APIs e governança de dados para clientes corporativos.
Senior Director of Data Engineering
Founding data leader scaling CuraLinc Healthcare’s mental-health analytics platform. Owning governed pipelines, executive dashboards, client reporting, and data team growth.
Junior Data Engineer – Snowflake
Analista de Dados Júnior na Keyrus, empresa internacional de dados e IA, atuando remotamente com Snowflake, SQL e soluções de Cloud Data Platform.
Data Engineer
Data Engineer building scalable data pipelines and integrations for VSP Vision, a vision-care company. Supporting analytics, data quality, and strategic business data use.
Principal Commercial Data Engineer
Principal Data Engineer architecting MGM Resorts’ commercial Lakehouse and analytics ecosystem. Modernizing Gaming, Hospitality, Loyalty, and Marketing data platforms for analytics and AI/ML.