Kartik's Blog

Hi — I’m Kartik.

I’m a grad student studying computers and math at CMU MLD. I enjoy working on practical problems – which means I enjoy nasty heuristics, wrangling with datasets, and debugging model behaviors.

I presently work on developing triton-kernels at Goomba Lab.

I have also interned on the planner team at Waymo where I worked with large autoregressive planning models.

I also work on research around safety and unlearning — you can find a few of my papers on the publications page.

If you want to know more

Mamba Notes

Note: This is a rough, work-in-progress post — things may change or be incomplete. This post describes the triton implementation of the backward pass of the Mamba-2 Chunking sequence layer. We use adjoint notation, i.e \(\bar{A} \) means \(\frac{\partial L} {\partial {A}} \).The only important result to keep in mind here is how adjoints flow over matmuls, and that broadcasting calls for sum over the partial gradients over the dimension that was broadcasted on in the first place ...