About

I am an ML Engineer, with experience ranging from pure research to early-stage companies:

• Senior ML Engineer @ Loka, starting soon.
• Co-founder @ uxo.ai, building agents that understand and navigate the web.
• ML Engineer @ Aimino, developing solutions robust against distribution shifts.
• Research Assistant @ EML, exploring catastrophic forgetting in foundation models.

Co-authored "Momentum-based Weight Interpolation of Strong Zero-Shot Models for Continual Learning", which won the "Best Paper Award" at INTERPOLATE @ NeurIPS 2022, and has since been cited by several publications affiliated with Google DeepMind [1][2][3], and Meta AI [4][5].

Created and open-sourced Word Game Bench, a fun way to evaluate language models on word puzzles.

Wrote and open-sourced "Answers to Chip Huyen's Machine Learning Interview Questions", a booklet answering interview questions covering Math, Computer Science, ML workflows and algorithms. Also, wrote parts of the course script for a grad-level class in "Probabilistic Machine Learning".

Obtained Master's degree in Machine Learning – University of Tübingen, Germany (GPA: 1.0/1.0).
(fun fact: Johannes Kepler, the father of modern astronomy, studied here in 1591!)

Open-source

I am a passionate proponent of open-source, and have contributed to several libraries:

• EleutherAI/lm-evaluation-harness (#1928, #1897, #1893, #1863, #1916, #1865)
• huggingface/transformers (#30722, #30678, #30395)
• ContinualAI/avalanche (#896)

Blog

• Speeding up decoder inference with a Key-Value (KV) cache
• Boosting transformer efficiency with Grouped-Query Attention (GQA)
• Stabilizing training and improving model convergence with RMSNorm
• Activating neurons with Gated Linear Units (GLU) and Friends
• Encoding positional information with Rotary Position Embeddings (RoPE)
• Decoupling weight decay with AdamW
• Overcoming catastrophic forgetting with Elastic Weight Consolidation (EWC)