Senior Python Data Scraping Engineer
Employment Information
Report this job
Job expired or something wrong with this job?
Job Description
Senior Python engineer building reliable web-scraping and data-validation workflows for Mindrift’s AI technology projects. Handling dynamic sites, anti-bot mechanisms, cloud infrastructure, and LLM automation tools.
Responsibilities:
- Own end-to-end data extraction workflows across complex websites
- Ensure complete coverage, accuracy, and reliable delivery of structured datasets
- Use available tools and custom workflows to accelerate data collection, validation, and task execution
- Extract data reliably from dynamic and interactive web sources, including JavaScript-rendered content
- Adapt scraping approaches to changing site behavior
- Enforce data quality through validation checks, cross-source consistency controls, formatting specifications, and systematic verification
- Scale scraping operations for large datasets using efficient batching or parallelization
- Monitor failures and maintain stability against minor site structure changes
- Apply tools such as Apify, OpenRouter, and other technologies to scraping and processing workflows
Requirements:
- At least 5+ years of relevant experience in data engineering, web scraping, automation, or software development
- Bachelor’s or Master’s Degree in Engineering, Applied Mathematics, Computer Science, or related technical fields is a plus
- Strong technical foundation and practical experience with scripting, automation, and data extraction workflows
- Strong experience in Python web scraping, including BeautifulSoup, Selenium or similar, dynamic content, and APIs via proxies
- Ability to extract data from complex structures such as hierarchies, archived pages, and inconsistent HTML
- Solid background in data cleaning, normalization, and validation
- Demonstrated experience handling anti-bot mechanisms and dynamic site structures at scale
- Experience with cloud infrastructure such as AWS or equivalent and containerization with Docker
- Hands-on experience with LLM frameworks such as LangChain, OpenRouter, or similar applied to automation tasks
- Strong attention to detail and commitment to data accuracy
- Self-directed work ethic and ability to troubleshoot independently
- English proficiency at Upper-intermediate (B2) or above
- A link to GitHub is a plus
Benefits:
- Freelance opportunity
- Part-time remote work
- Estimated 10–20 hours per week during active project phases
- Compensation up to $25 per hour equivalent, depending on level and pace of contribution
- Opportunity to contribute to innovative technology and AI development projects
Similar Jobs
Senior PHP Symfony Engineer
Senior PHP/Symfony Engineer owning backend development for Smart Working’s enterprise marketing and revenue data platform. Modernising Symfony, improving scalability, and integrating Angular, TypeScript, and AI services.
Flutter, Node.js Fullstack Engineer
Fullstack engineer evolving memodio’s prescribed digital health app with Flutter and Node.js. Building accessible features and regulated medical software for older users worldwide.
Senior Backend Engineer – Java / Go
Senior Backend Engineer building scalable Java/Go Social systems for Binance’s global blockchain ecosystem. Improving distributed-system performance, availability, governance, and AI-assisted engineering workflows.
Senior Python Data Scraping Engineer
Senior Python engineer building reliable scraping and structured-data workflows for Mindrift’s Tendem AI project. Handling dynamic websites, validation, scaling, and automation.
Senior Python Data Scraping Engineer, Freelance
Senior Python engineer building reliable scraping and data-processing workflows for Mindrift’s Tendem AI project. Extracting, validating, and scaling structured datasets from complex websites.
Senior Python Data Scraping Engineer, Freelance
Senior Python engineer building reliable scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer
Senior Python engineer building reliable scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer – Freelance
Senior Python engineer building reliable scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer
Senior Python engineer building reliable web-scraping and data-processing workflows for Mindrift’s Tendem AI project. Extracting, validating, and scaling structured datasets from complex websites.
Senior Python Data Scraping Engineer – Freelance
Senior Python engineer building scalable web-scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and processing structured data from complex, dynamic websites.
Senior Python Data Scraping Engineer
Senior Python engineer building reliable web-scraping workflows for Mindrift’s AI platform. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer
Senior Python engineer building reliable scraping workflows for Mindrift’s Tendem AI project. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Data Scraping Engineer, Python
Senior Python engineer building reliable web-scraping workflows for Mindrift’s Tendem AI project. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer – Freelance
Senior Python engineer building reliable web-scraping and data-processing workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer – Freelance
Senior Python engineer building reliable scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Data Scraping Engineer, Python
Senior Python engineer building reliable scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer – Freelance
Senior Python Data Scraping Engineer extracting and validating complex web data for Mindrift’s Tendem AI project. Scaling reliable scraping workflows with Python, cloud infrastructure, Docker, and LLM frameworks.
Senior Python Data Scraping Engineer – Freelance
Senior Python engineer building reliable web-scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.
Senior Python Data Scraping Engineer – Freelance
Senior Python engineer building reliable scraping and data extraction workflows for Mindrift’s AI technology projects. Processing dynamic websites into validated, structured datasets.
Senior Python Data Scraping Engineer, Freelance
Senior Python engineer building reliable scraping workflows for Mindrift’s AI technology projects. Extracting, validating, and scaling structured datasets from complex dynamic websites.