Senior Data Scraping Engineer, Python
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Python engineer building reliable web-scraping workflows for Mindrift’s Tendem AI project. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Responsibilities:
- Own end-to-end data extraction workflows across complex websites
- Ensure complete coverage, accuracy, and reliable delivery of structured datasets
- Use available tools and custom workflows to accelerate data collection, validation, and task execution
- Extract data reliably from dynamic and interactive web sources, including JavaScript-rendered content
- Adapt scraping approaches to changing site behavior
- Enforce data quality through validation checks, cross-source consistency controls, formatting specifications, and systematic verification
- Scale scraping operations for large datasets using efficient batching or parallelization
- Monitor failures and maintain stability against minor site structure changes
- Use tools such as Apify, OpenRouter, and other technologies for web extraction and processing
Requirements:
- At least 5+ years of relevant experience in data engineering, web scraping, automation, or software development
- Bachelor’s or Master’s Degree in Engineering, Applied Mathematics, Computer Science, or related technical fields is a plus
- Strong experience in Python web scraping, including BeautifulSoup, Selenium or similar, dynamic content (JS, AJAX, infinite scroll), and APIs via proxies
- Proven ability to extract data from complex structures, including hierarchies, archived pages, and inconsistent HTML
- Solid background in data cleaning, normalization, and validation, delivering structured datasets in CSV, JSON, or Google Sheets
- Demonstrated experience handling anti-bot mechanisms and dynamic site structures at scale
- Experience with cloud infrastructure, such as AWS or equivalent
- Experience with containerization using Docker
- Hands-on experience with LLM frameworks such as LangChain, OpenRouter, or similar, applied to automation tasks
- Strong attention to detail and commitment to data accuracy
- Self-directed work ethic and ability to troubleshoot independently
- English proficiency at Upper-intermediate (B2) or above
- GitHub link is a plus
Benefits:
- Freelance opportunity
- Part-time schedule estimated at around 10–20 hours per week during active phases
- Remote work
- Opportunity to contribute to innovative technology and AI development projects
Similar Jobs
Senior PHP Symfony Engineer
Senior PHP/Symfony Engineer owning backend development for Smart Working’s enterprise marketing and revenue data platform. Modernising Symfony, improving scalability, and integrating Angular, TypeScript, and AI services.
Flutter, Node.js Fullstack Engineer
Fullstack engineer evolving memodio’s prescribed digital health app with Flutter and Node.js. Building accessible features and regulated medical software for older users worldwide.
Senior Backend Engineer – Java / Go
Senior Backend Engineer building scalable Java/Go Social systems for Binance’s global blockchain ecosystem. Improving distributed-system performance, availability, governance, and AI-assisted engineering workflows.
Senior Python Data Scraping Engineer
Senior Python engineer building reliable scraping and structured-data workflows for Mindrift’s Tendem AI project. Handling dynamic websites, validation, scaling, and automation.
Senior Python Data Scraping Engineer, Freelance
Senior Python engineer building reliable scraping and data-processing workflows for Mindrift’s Tendem AI project. Extracting, validating, and scaling structured datasets from complex websites.
Senior Python Data Scraping Engineer, Freelance
Senior Python engineer building reliable scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer
Senior Python engineer building reliable scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer – Freelance
Senior Python engineer building reliable scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer
Senior Python engineer building reliable web-scraping and data-processing workflows for Mindrift’s Tendem AI project. Extracting, validating, and scaling structured datasets from complex websites.
Senior Python Data Scraping Engineer – Freelance
Senior Python engineer building scalable web-scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and processing structured data from complex, dynamic websites.
Senior Python Data Scraping Engineer
Senior Python engineer building reliable web-scraping and data-validation workflows for Mindrift’s AI technology projects. Handling dynamic sites, anti-bot mechanisms, cloud infrastructure, and LLM automation tools.
Senior Python Data Scraping Engineer
Senior Python engineer building reliable web-scraping workflows for Mindrift’s AI platform. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer
Senior Python engineer building reliable scraping workflows for Mindrift’s Tendem AI project. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer – Freelance
Senior Python engineer building reliable web-scraping and data-processing workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer – Freelance
Senior Python engineer building reliable scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Data Scraping Engineer, Python
Senior Python engineer building reliable scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer – Freelance
Senior Python Data Scraping Engineer extracting and validating complex web data for Mindrift’s Tendem AI project. Scaling reliable scraping workflows with Python, cloud infrastructure, Docker, and LLM frameworks.
Senior Python Data Scraping Engineer – Freelance
Senior Python engineer building reliable web-scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer – Freelance
Senior Python engineer building reliable scraping and data extraction workflows for Mindrift’s AI technology projects. Processing dynamic websites into validated, structured datasets.
Senior Python Data Scraping Engineer, Freelance
Senior Python engineer building reliable scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.