Senior Data Engineer

Posted 1ds ago

Employment Information

Education
Salary
Experience
Job Type

Report this job

Job expired or something wrong with this job?

Job Description

Senior Data Engineer building scalable ETL pipelines and data products for Spokeo’s people intelligence platform. Using AWS, Spark, Python, and Airflow to power large-scale public-record data systems.

Responsibilities:

  • Build infrastructure and data automation pipelines to ingest, process, and load data from various sources
  • Automate and integrate new components into the data pipeline
  • Collaborate with stakeholders and data science teams to develop data products, including entity resolution and best selection
  • Create unit and stress-test components to monitor technical performance and resolve identified issues
  • Develop data analysis tools to provide data insights and capture key metrics
  • Research solutions and maintain technical documentation
  • Follow best practices for data governance, quality, cleansing, and other ETL-related activities

Requirements:

  • 7+ years of development experience in data engineering within a production environment, excluding internships and academic settings
  • Experience working with large datasets exceeding 100M+ records or multiple terabytes
  • 5+ years of development experience in highly scalable, distributed systems and cluster architectures using AWS and EMR
  • 5+ years of hands-on programming experience with Python
  • 5+ years of professional experience in big data ecosystems
  • Spark required; PySpark preferable
  • 5+ years of experience with SQL, schema design, and dimensional data modeling
  • 5+ years of professional experience with dataflow orchestration tools such as Airflow
  • 2+ years of experience with non-relational databases such as DynamoDB or Elasticsearch
  • Bachelor’s degree in Computer Science, Information Systems, Mathematics, or a related field required

Benefits:

  • Bonus program
  • Equity plans
  • 401(k)
  • Discretionary, merit-based salary increase once a year
  • 100% medical, dental, and vision coverage
  • Unlimited employee PTO
  • Remote-first work environment