Senior Data Scraping Engineer, Python
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Python engineer building reliable scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Responsibilities:
- Handle end-to-end data extraction workflows across complex websites
- Deliver complete, accurate, and reliable structured datasets
- Use tools and custom workflows to accelerate data collection, validation, and task execution
- Extract data from dynamic and interactive web sources, including JavaScript-rendered content
- Adapt scraping approaches to changing site behavior
- Enforce data quality through validation checks, cross-source consistency controls, formatting specifications, and systematic verification
- Scale scraping operations for large datasets using efficient batching or parallelization
- Monitor failures and maintain stability against minor site structure changes
Requirements:
- At least 5+ years of relevant experience in data engineering, web scraping, automation, or software development
- Bachelor’s or Master’s Degree in Engineering, Applied Mathematics, Computer Science, or related technical fields is a plus
- Strong experience in Python web scraping, including BeautifulSoup, Selenium or similar, dynamic content, and APIs via proxies
- Proven ability to extract data from complex structures, including hierarchies, archived pages, and inconsistent HTML
- Solid background in data cleaning, normalization, and validation, delivering structured datasets in CSV, JSON, or Google Sheets
- Demonstrated experience handling anti-bot mechanisms and dynamic site structures at scale
- Experience with cloud infrastructure such as AWS or equivalent and containerization with Docker
- Hands-on experience with LLM frameworks such as LangChain, OpenRouter, or similar applied to automation tasks
- Strong attention to detail and commitment to data accuracy
- Self-directed work ethic with ability to troubleshoot independently
- English proficiency: Upper-intermediate (B2) or above
Benefits:
- Freelance project opportunity
- Part-time remote work
- Estimated 10–20 hours per week during active project phases
- Up to $30 per hour equivalent, depending on level and pace of contribution
Similar Jobs
Senior PHP Symfony Engineer
Senior PHP/Symfony Engineer owning backend development for Smart Working’s enterprise marketing and revenue data platform. Modernising Symfony, improving scalability, and integrating Angular, TypeScript, and AI services.
Flutter, Node.js Fullstack Engineer
Fullstack engineer evolving memodio’s prescribed digital health app with Flutter and Node.js. Building accessible features and regulated medical software for older users worldwide.
Senior Backend Engineer – Java / Go
Senior Backend Engineer building scalable Java/Go Social systems for Binance’s global blockchain ecosystem. Improving distributed-system performance, availability, governance, and AI-assisted engineering workflows.
Senior Python Data Scraping Engineer
Senior Python engineer building reliable scraping and structured-data workflows for Mindrift’s Tendem AI project. Handling dynamic websites, validation, scaling, and automation.
Senior Python Data Scraping Engineer, Freelance
Senior Python engineer building reliable scraping and data-processing workflows for Mindrift’s Tendem AI project. Extracting, validating, and scaling structured datasets from complex websites.
Senior Python Data Scraping Engineer, Freelance
Senior Python engineer building reliable scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer
Senior Python engineer building reliable scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer – Freelance
Senior Python engineer building reliable scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer
Senior Python engineer building reliable web-scraping and data-processing workflows for Mindrift’s Tendem AI project. Extracting, validating, and scaling structured datasets from complex websites.
Senior Python Data Scraping Engineer – Freelance
Senior Python engineer building scalable web-scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and processing structured data from complex, dynamic websites.
Senior Python Data Scraping Engineer
Senior Python engineer building reliable web-scraping and data-validation workflows for Mindrift’s AI technology projects. Handling dynamic sites, anti-bot mechanisms, cloud infrastructure, and LLM automation tools.
Senior Python Data Scraping Engineer
Senior Python engineer building reliable web-scraping workflows for Mindrift’s AI platform. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer
Senior Python engineer building reliable scraping workflows for Mindrift’s Tendem AI project. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Data Scraping Engineer, Python
Senior Python engineer building reliable web-scraping workflows for Mindrift’s Tendem AI project. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer – Freelance
Senior Python engineer building reliable web-scraping and data-processing workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer – Freelance
Senior Python engineer building reliable scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer – Freelance
Senior Python Data Scraping Engineer extracting and validating complex web data for Mindrift’s Tendem AI project. Scaling reliable scraping workflows with Python, cloud infrastructure, Docker, and LLM frameworks.
Senior Python Data Scraping Engineer – Freelance
Senior Python engineer building reliable web-scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer – Freelance
Senior Python engineer building reliable scraping and data extraction workflows for Mindrift’s AI technology projects. Processing dynamic websites into validated, structured datasets.
Senior Python Data Scraping Engineer, Freelance
Senior Python engineer building reliable scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.