Apply is seeking a Tech Lead Engineer to lead the technical direction and development of our high-load web scraping platform. You will drive architectural decisions and build the infrastructure supporting both cloud and on-premises deployments.
What You'll Do
- Design architecture for the data-plane, including high-parallel crawlers, session management, anti-bot bypass, and CAPTCHA solvers.
- Design architecture for the control-plane, covering API, authorization, billing, quotas, and monitoring.
- Design infrastructure built on microservices, Docker/K8s, CI/CD, and observability.
- Lead development, including hiring backend, browser, DevOps, and QA engineers.
- Establish development and release processes and oversee code quality and technical decisions.
- Conduct anti-bot research and develop bypasses for browser and network-level fingerprinting.
- Work on bypassing protections from providers like Cloudflare and Akamai.
- Handle DevOps/SRE tasks, including CI/CD setup, Helm charts, and scaling configuration.
- Organize monitoring with Prometheus, Grafana, Loki and establish SLA processes.
What We're Looking For
- 5+ years of engineering experience building high-load backend or data platforms.
- Extensive, mandatory experience with web scraping.
- Excellent understanding of HTTP, TLS, HTTP/2, proxies, and the browser stack.
- Professional experience with async Python and Go, Redis, queues, and message brokers.
- Advanced experience with DevOps tools: Kubernetes, Helm, Prometheus, and CI/CD.
- Fluent English for documentation and team communication.
- Strong decision-making skills, mentoring abilities, and accountability for results.
Nice to Have
- Experience analyzing and bypassing bot protection systems.
Technical Stack
- Languages: Python, Go
- Data: Redis
- Infrastructure: Kubernetes, Helm, Docker
- Monitoring: Prometheus, Grafana, Loki
Team & Environment
You will lead the web scraping development team as its technical leader.
Benefits & Compensation
- Opportunity to develop the company's key product from scratch.
- High autonomy and influence on architecture and team decisions.
- Budget allocated for conferences, courses, and work tools.
- Competitive compensation package.
- Remote work flexibility.
Work Mode
This is a fully remote position.
Apply is an equal opportunity employer.
