Python Web Scrapping Engineer - Competetive salary - Remote

We are looking for a Senior Python Scraping Engineer to design, build, and operate large-scale, resilient web scraping systems, with a strong focus on Google and Google-like search environments.

This role sits at the intersection of data engineering, reverse engineering, and systems reliability, and is critical to delivering accurate, timely, and trustworthy data at scale.

You will work on complex scraping problems where naïve approaches do not work-dynamic rendering, sophisticated anti-bot defences, frequent markup changes, and high-volume data collection.

We are leaders in adopting AI-assisted and agentic coding workflows. This role is ideal for someone who believes in leveraging AI tools to accelerate development and wants to deepen their skills in this area.

• Design, build, and maintain large-scale Python scraping systems that operate reliably against Google and other highly defended websites

• Develop scrapers for dynamic, JavaScript-heavy pages using browser automation and hybrid scraping approaches

• Continuously adapt to changes in page structure, request flows, and anti-automation mechanisms

• Engineer robust extraction pipelines that prioritise data accuracy, consistency, and observability

• Implement and optimise Proxy, fingerprinting, and request-routing strategies to maximise stability and throughput

• Monitor scrape health, detect failures early, and debug production issues quickly and methodically

• Optimise performance, cost, and latency across scraping infrastructure

• Collaborate closely with data engineers, data scientists, and product teams to ensure scraped data is usable and trusted

• Document scraping logic, architectural decisions, and operational runbooks for long-term maintainability

• Strong professional experience with Python, including production-grade scraping systems

• Deep, hands-on experience scraping Google or similarly protected platforms

• Solid understanding of HTTP, TLS, cookies, headers, redirects, and browser networking behaviour

• Experience with browser automation frameworks such as Playwright, Selenium, Puppeteer, or equivalents

• Strong knowledge of HTML parsing, DOM traversal, and data extraction techniques

• Proven experience handling rate limiting, CAPTCHAs, IP rotation, and bot detection systems

• Comfortable designing and operating asynchronous and concurrent scraping architectures

• Experience running scraping systems at scale in cloud environments

• Excellent debugging skills and the ability to reason about complex, non-obvious failure modes

• Hands-on experience with headless browser fingerprinting and anti-bot evasion techniques

To apply for this role please submit your CV or contact Dillon Blackburn (see below)

Tenth Revolution Group are the go-to recruiter for Data & AI roles in the UK offering more opportunities across the country than any other recruitment agency. We're the proud sponsor and supporter of SQLBits, Power Platform World Tour, and the London Fabric User Group. We are the global leaders in Data & AI recruitment.