Responsibilities
- Design and implement distributed web crawlers and spiders to collect data from diverse web platforms, including single-page and server-side rendered applications.
- Develop techniques to avoid detection by anti-bot systems, using methods such as proxy rotation, browser fingerprint manipulation, and CAPTCHA resolution.
- Build and refine SQL database schemas to support high-volume data storage and execute advanced data processing and validation routines.
- Monitor data extraction tools proactively and update codebases rapidly in response to changes in website structure or functionality.
- Improve the speed and efficiency of data pipelines using asynchronous programming in Python.
Benefits
- Comprehensive health coverage with employer contributions covering 85% to 100% of premiums for medical, dental, vision, life, and disability insurance for employees and their families.
- Retirement savings options including both Traditional and Roth 401(K) plans with a competitive company match.
- Unlimited flexible time-off policy, supporting employee autonomy in managing work-life balance.
- Twelve paid federal holidays provided annually.
- Financial support for continuing education through a tuition reimbursement program.
Work Arrangement
Hybrid
Equal Opportunity Employer
- The company is an equal opportunity and affirmative action employer.
- All applicants receive fair consideration for employment regardless of sex, gender identity, sexual orientation, race, color, religion, national origin, disability, Veteran status, age, or other legally protected characteristics.
- There is no discrimination based on inquiries, discussions, or disclosures about pay, in line with pay transparency regulations.
- Reasonable accommodations are provided to qualified employees with disabilities as required by applicable laws.
- Additional information about accommodations is available upon request.
