Senior Data Engineer, C&I

Posted 38mins ago

Employment Information

Education
Salary
Experience
Job Type

Report this job

Job expired or something wrong with this job?

Job Description

Senior Data Engineer leading Snowflake and dbt pipelines for EnergyHub’s commercial demand-response programs. Improving measurement, verification, payment accuracy and grid flexibility for utilities and businesses.

Responsibilities:

  • Lead the design and delivery of large, loosely scoped data engineering projects on the C&I measurement and verification platform
  • Develop pipelines that process raw meter data through baselines, event performance and payments into operational and partner reports
  • Break complex, ambiguous problems into well-scoped work tracks and roadmap them with estimates of time, effort and risk
  • Drive technical direction for the Snowflake and dbt codebase, including layering, model design, incremental processing, Python in the warehouse, orchestration and CI
  • Own production pipeline health through data tests, monitoring, alerting, troubleshooting, incident resolution and on-call support
  • Improve pipeline performance, reduce technical debt and shorten build times
  • Ensure payment and performance outputs are accurate and explainable
  • Collaborate with product, operations and customer-facing teams to implement program rules
  • Improve team practices through code review, documentation and pairing
  • Teach teammates and contribute to ongoing learning
  • Work with Snowflake, dbt, Python, SQL, AWS S3, MongoDB and GitHub Actions
  • Perform other duties as assigned

Requirements:

  • Professional experience designing, building and operating production data pipelines
  • Track record of leading projects from ambiguous idea to successful release
  • Advanced SQL
  • Strong dbt experience in a layered staging, intermediate and marts project
  • Experience with incremental models, late-arriving, changed or deleted data, and confident backfills
  • Experience optimizing slow or expensive pipelines on Snowflake or a comparable cloud warehouse
  • Strong data modeling, including grain, historical data and configuration changes, reproducible and explainable results
  • Strong Python for data work, including pandas and numpy
  • Experience with time-series or event-based data and related data quality issues
  • Hands-on experience with automated testing, CI/CD, monitoring and alerting
  • Experience choosing among architectural approaches and tradeoffs
  • Clear written and verbal communication
  • Experience documenting initiatives and explaining technical concepts to technical and non-technical audiences
  • History of taking ownership of problems with no clear owner
  • Ability to collaborate, pair, teach and learn from teammates at every level
  • Warmth and welcome for collaborators and colleagues
  • Preferred: Snowflake Python UDTFs and stored procedures, streams and tasks, clustering, warehouse sizing and cost management
  • Preferred: Advanced dbt custom macros and materializations, cross-project refs and jobs-as-code
  • Preferred: MongoDB and CSV, JSON and Parquet
  • Preferred: Energy, demand response, utilities or billing/payment experience
  • Preferred: Migration leadership or long-running cross-team initiative experience

Benefits:

  • 100% paid medical for employees
  • 401(k) with employer match
  • Casual environment
  • Flexibility to set your own schedule
  • Fully stocked fridge and pantry
  • Free Citi Bike membership
  • Secure bike rack
  • Gym subsidy
  • Paid parental leave
  • Education assistance program
  • Real responsibility and immediate impact
  • Opportunities to work directly with executives
  • Exposure to IoT, SaaS and machine learning technologies