Flower Labs seeks a Founding Research Engineer to join its new Frontier Model Team. You will push the boundaries of open-source AI by building category-defining models that integrate cutting-edge practices with Flower's decentralized learning methods. As a founding member, you will help invent the training paradigms that will define the next decade of AI.
What You'll Do
- Play a critical role in building state-of-the-art LLMs and foundation models within a small, high-impact team.
- Shape every part of the scientific foundation of our frontier models.
- Turn ideas into working systems and collaborate with the team to scale the most effective approaches.
- Design and implement techniques and run experiments across data curation, evaluations, pre-training, and post-training.
- Contribute ideas, be heard, and influence the company's direction across the board.
What We're Looking For
- Deep understanding of recent architectures and training methodology used for LLMs and foundation models.
- Experience with pre-training or post-training such as SFT, RLHF, DPO, or reward modeling.
- Strong grounding in optimization techniques: AdamW variants, LR scheduling, mixed precision, stabilization methods, and scaling behaviors.
- Strong experimental design skills: ablations, controlled comparisons, scaling experiments.
- Fluency in PyTorch or JAX for implementing research ideas efficiently.
- Ability to collaborate effectively with both research-oriented and engineering-oriented colleagues.
- Ability to turn conceptual research directions into runnable prototypes that integrate into the training system.
- Familiarity with common tools like the Linux command line, git, and Docker.
- Openness to adopting new tooling.
- Strong written English.
- Open, honest and transparent communication skills.
Nice to Have
- PhD or Masters degree in a relevant discipline.
- Knowledge in distributed training frameworks such as FSDP, ZeRO, or tensor/sequence/pipeline parallelism.
- Experience running large experiments on multi-GPU or multi-node clusters.
- Experience in RLHF at scale, advanced preference modeling techniques, or safety alignment.
- Prior experience in advanced distributed training frameworks and concepts.
- Track record of working in open-source projects.
- Strong publication record in ML, NLP, optimization, or related fields.
Technical Stack
- PyTorch
- JAX
- Linux
- git
- Docker
Team & Environment
You will be a founding member of the new Frontier Model Team, a small, high-impact group operating in a collaborative, fast-paced, and demanding start-up environment at Flower Labs. The team consists of experts in their respective fields, and everyone still learns something new every day.
Work Mode
This is a remote position, open to candidates working remotely from California or other locations within the United States.
