
Experience
2023
Machine Learning Engineer
Aimino
Implemented computer vision solutions robust against distribution shifts.
2022
Research Assistant
Explainable Machine Learning
Explored catastrophic forgetting in foundation models.
Open-source
I am a passionate proponent of open-source, and have contributed to several libraries:
#47
#57
#62
#65
#73
#74
#75
#85
#86
#87
#91
#94
#100
#101
#102
#114
#115
#116
#117
#118
#119
#126
#130
#131
#132
#133
#134
#135
#136
#138
#142
#143
#146
#148
#149
#150
#165
#168
#177
#182
#186
#191
#192
#241
#247
#254
#262
#263
#266
#267
#270
#271
#274
#275
#278
#279
#286
#287
#288
#292
#293
#294
#296
#297
#298
#299
#300
#305
#306
#307
#308
#311
#312
#313
#319
#321
#322
#326
#327
#333
#341
#342
#343
#344
#347
#348
#349
#350
#355
#357
#363
#365
#368
#373
#388
#393
#396
#397
Projects
Sometimes, I build stuff:
- Word Game Bench – evaluating language models on puzzles. OpenRouter sponsored the project, and Mark Chen (SVP of Research at OpenAI) expressed interest in the results.
- Answers to Chip Huyen's ML Interview Questions – a booklet answering interview questions covering Math, Computer Science, ML workflows and algorithms.
- Laser Hockey – my winning entry to a Reinforcement Learning tournament with 70 participants, organized by the Max Planck Institute for Intelligent Systems.
- Morty – An "agent" based on IBM Watson intents and an LSTM pre-trained on Reddit data I built all the way back in 2018!
Blog
Other times, I write stuff:
- Speeding up decoder inference with a Key-Value (KV) cache
- Boosting transformer efficiency with Grouped-Query Attention (GQA)
- Stabilizing training and improving model convergence with RMSNorm
- Activating neurons with Gated Linear Units (GLU) and Friends
- Encoding positional information with Rotary Position Embeddings (RoPE)
- Decoupling weight decay with AdamW
- Overcoming catastrophic forgetting with Elastic Weight Consolidation (EWC)
Biomarkers
With the goal of keeping myself accountable and maintaining good physical health, I'll continually open-source my biomarkers.