apply is seeking a Scraping Tech Lead Engineer to serve as the technical leader for a high-load web scraping platform. You will drive architectural decisions and shape the future of the scraping infrastructure, which supports both cloud and on-premises deployments.

What You'll Do

Design architecture for the data-plane, including high-parallel crawlers, session management, anti-bot bypass, and CAPTCHA solving.
Design architecture for the control-plane, covering APIs, authorization, billing, quotas, and monitoring.
Design infrastructure using microservices, Docker/K8s, CI/CD, and observability tools.
Lead development: participate in hiring backend, browser, DevOps, and QA engineers; establish development processes; oversee code quality.
Conduct anti-bot research and bypass, handling browser and network-level fingerprinting and bypassing protections like Cloudflare and Akamai.
Manage DevOps/SRE: set up CI/CD and Helm charts, configure scaling, and organize monitoring with Prometheus, Grafana, and Loki.

What We're Looking For

5+ years of engineering experience in high-load backend or data platforms.
Extensive, hands-on web scraping experience.
Excellent understanding of HTTP, TLS, HTTP/2, proxies, and the browser stack.
Experience with async Python and Go, Redis, queues, and message brokers.
Advanced level expertise with DevOps tools: Kubernetes, Helm, Prometheus, and CI/CD.
Fluent English for documentation and team communication.
Strong decision-making skills, mentoring abilities, and accountability for results.