People Data Labs is hiring a Senior Data Acquisition Engineer to accelerate our efforts to build standalone data products and ensure customers have standardized, high-quality data. You will work on our web crawling technologies, data pipelines, and new data product initiatives.
What You'll Do
- Use and develop web crawling technologies to capture and catalog data on the internet
- Support and improve our web crawling infrastructure
- Structure, define, and model captured data, providing semantic data definition and automate data quality monitoring
- Develop new techniques to increase speed, efficiency, scalability, and reliability of web crawls
- Use big data processing platforms to build data pipelines, publish data, and ensure reliable availability of data
- Work with our data product and engineering team to design and implement new data products and enhance existing ones
What We're Looking For
- 7+ years industry experience with clear examples of strategic technical problem solving and implementation
- Strong software development architecture and fundamentals for backend applications
- Solid understanding of browser rendering pipeline and web application architecture
- Solid programming experience with a strong grasp of object-oriented design and asynchronous programming
- Experience building crawlers
- Proficient in Linux / Unix command line utilities, system administration, architecture, and resource management
- Experience evaluating data quality and maintaining consistently high data standards
- Must thrive in a fast paced environment and be able to work independently
- Can work effectively remotely, being proactive about managing blockers and communication
- Strong written communication skills on Slack/Chat and in documents
- Experienced in writing data design docs for pipelines, dataflow, and schema design
- Can scope and breakdown projects, communicating progress and blockers effectively with manager, team, and stakeholders
Nice to Have
- Degree in a quantitative discipline such as computer science, mathematics, statistics, or engineering
- Experience as a Red Teamer
- Experience working in data acquisition
- Experience in network architecture and how to debug and inspect network traffic
- Experience with Apache Spark
- Experience with SQL, including writing advanced queries
- Experience with streaming data platforms like Kafka or Spark streaming
- Experience with cloud computing services like AWS, GCP, or Azure
- Experience working in Databricks
- Knowledge of modern data design and storage patterns
- Experience with data warehousing platforms like Snowflake, Redshift, or BigQuery
- Understanding of modern data storage formats and tools like Parquet, ORC, Avro, or Delta Lake
Technical Stack
- Linux/Unix
- Apache Spark
- SQL
- Kafka
- AWS, GCP, Azure
- Databricks, Snowflake, Redshift, BigQuery
- Parquet, ORC, Avro, Delta Lake
Team & Environment
You will be part of the Data Engineering & Acquisition Team.
Benefits & Compensation
- Salary range: $160K - $200K
- Stock options
- Competitive Salaries
- Unlimited paid time off
- Medical, dental, & vision insurance
- Health, fitness, and office stipends
- The permanent ability to work wherever and however you want
Work Mode
This role is global and can be performed from anywhere.
People Data Labs does not discriminate on the basis of race, sex, color, religion, age, national origin, marital status, disability, veteran status, genetic information, sexual orientation, gender identity or any other reason prohibited by law in provision of employment opportunities and benefits.
