About the Role
The position involves designing and implementing pre-training approaches for large language models, with an emphasis on improving model capabilities, safety, and interpretability through empirical research and engineering.
Compensation
Competitive salary and benefits package
Work Arrangement
Hybrid or remote options available
Team
Part of a research-focused team advancing AI model development
Responsibilities
- Design and run experiments to improve pre-training methodologies
- Develop scalable training pipelines for large language models
- Analyze model behavior during pre-training phases
- Collaborate with researchers to test hypotheses about model scaling
- Implement and refine data filtering and curation techniques
- Optimize training efficiency and compute utilization
- Debug and resolve issues in distributed training setups
- Contribute to internal tools for monitoring model training
- Evaluate the impact of architectural choices on pre-training outcomes
- Document findings and share insights across teams
Qualifications
- Strong programming skills in Python and machine learning frameworks
- Experience with deep learning and transformer-based models
- Familiarity with large-scale training infrastructure
- Background in natural language processing or related field
- Proven ability to conduct independent research
- Experience with distributed training systems
- Knowledge of data handling at scale
- Understanding of model evaluation techniques
- Ability to analyze and interpret training dynamics
- Clear communication skills for technical collaboration
Preferred Qualifications
- PhD in computer science, machine learning, or related discipline
- Prior work on language model pre-training
- Experience with reinforcement learning from human feedback
- Contributions to open-source machine learning projects
- Publications in relevant research venues
- Familiarity with safety-focused AI research
- Knowledge of interpretability methods
- Experience optimizing training for robustness
What We Value
- Rigorous empirical approach to research
- Focus on long-term model safety
- Collaborative problem-solving
- Transparency in experimental design
- Commitment to reproducible results
- Curiosity about model internals
- Attention to ethical implications
- Willingness to iterate on failed hypotheses
Visa sponsorship may be available for qualified candidates
