Lead Technology – Web Scraping, Data Platform
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Technical leader in technology for a startup specializing in market intelligence and data monitoring. Leading technical decisions in web scraping, data infrastructure, and back-end development.
Responsibilities:
- Provide technical leadership for the company's technology function
- Drive the evolution of the platform architecture, stability, and scalability
- Lead technical decisions related to scraping, infrastructure, data, and back-end
- Support the development of the technical team and the evolution of operations
- Actively participate in development, troubleshooting, and critical stack decisions
- Establish processes, best practices, and technical organization for the area
- Work closely with the founders/partners on technology-related decisions and business growth.
Requirements:
- Strong experience with Python applied to large-scale web scraping
- Experience with tools such as Scrapy, Playwright, Puppeteer, or similar
- Experience in back-end development and building APIs
- Knowledge of JavaScript and/or TypeScript
- Experience with cloud computing on AWS and/or GCP
- Experience with databases such as PostgreSQL, BigQuery, DynamoDB, and OpenSearch
- Ability to operate strategically without losing technical depth
- Experience providing technical leadership for people, projects, or operations.
Benefits:
- 100% remote
- Potential stock options/equity
- Compensation structured according to seniority and potential impact on the operation.
Similar Jobs
Data Engineer
Data Engineer architecting data flow for supply chain intelligence at Shipium. Designing ETL processes to optimize analytics, reporting, and drive AI products.
Data Engineer, Data Project Lead
Data Engineer leading data management and engineering projects supporting OHS program oversight. Ensuring compliance with federal policies while delivering high-quality data solutions.
Junior/Mid-level Data Engineer
Junior Data Engineer at Vitrio supporting data pipeline maintenance and data quality monitoring. Working with the Data Engineering team on Google Cloud Platform.
Senior Data Engineer
Senior Data Engineer focusing on data pipelines and collaboration with blockchain engineers to index blockchain data for the Solana ecosystem.
Senior Data Engineer
Data Engineer at Grupo SysMap focused on data modeling and pipeline development for multiple cloud environments. Analyzing business processes and ensuring data integrity with best practices.
Data Architect
Teamcenter Data Architect designing and configuring PLM data models at Accelint. Contributing to mission-ready PLM environments supporting Navy objectives with data consistency.
Data Engineer – Pipelines, APIs, AI Data Systems
Data Engineer designing and building scalable data pipelines for AI and analytics at Newfold Digital. Collaborating with analytics, BI, and AI teams for data accessibility and reuse.
Principal Data Engineer
Principal Data Engineer overseeing technical direction for AI data center engineering platform at NVIDIA-backed company. Responsible for architecture, design decisions, and mentoring the team in data infrastructure.
Senior Data Governance Analyst – Azure Purview, Data Engineering
Senior Data Governance Analyst leading Azure Purview implementations and shaping data governance strategy at Data Meaning. Working directly with clients and delivering enterprise data solutions.
Data Engineer – Contractor
Data Engineer designing and building reliable data pipelines for Fintech at myPOS. Creating scalable data architecture and ensuring data quality for business insights.
Data Engineer – Senior
Data Engineer responsible for designing and implementing data integration processes using Python and SQL. Focused on ETL automation, quality control, and data warehouse support.
Senior Data Engineer, Data Platform
Senior Data Engineer building core services for healthcare data platform. Empowering real-world patient and provider outcomes through automation and observability in a fully remote role.
AI Data Engineer
AI Data Engineer operating at the intersection of data engineering and applied AI. Building end-to-end solutions that power Power Digital’s internal AI platform with collaborative team efforts.
Data Engineer, Snowflake
Project-based Data Engineer with Snowflake for Evolved-Ideas helping organisations in Europe. Building, managing, and optimizing data pipelines while communicating with key data consumers.
Director, Interoperability Data Platform
Director leading strategy and execution for Humana’s interoperability data platform. Overseeing data ingestion and utilization across healthcare products and enterprise operations.
Data Engineer II
Data Engineer II managing data engineering functions including data modeling and ETL processes at TD. Requires 3+ years of relevant experience and a degree in a related field.
Fabric Data Engineer
Senior Fabric Data Engineer designing and optimizing enterprise data solutions using Microsoft Fabric. Collaborating with teams to modernize data platforms and providing analytics solutions.
Data Engineer – MS Fabric
Data Engineer creating and implementing modern data architectures for analytical solutions at Derevo. Working with technologies in Big Data and data management best practices.
Data Engineer, Databricks
Data Engineer focused on implementing high-quality data architectures and analytical solutions. Collaborating with internal teams and clients at Derevo to manage big data efficiently.
Senior Data Engineer – Databricks SME
Data Engineer supporting US Government client with data ingestion, deduplication, and tagging for Databricks migration. Requires extensive experience with Azure and data management technologies.