Writing

Essays on machine learning architecture, GPU efficiency, and neural network optimization.

Writing Custom Triton Kernels for Memory-Efficient Attention

An in-depth look into GPU memory bandwidth bottlenecks, tile scheduling, and writing custom block-level kernels using OpenAI's Triton compiler.

An exploration of how EfficientNet revolutionized neural network scaling by replacing arbitrary depth/width expansion with compound scaling (phi).

A deep mathematical look into gradient instability during backpropagation in deep neural networks and recurrent architectures.

Analyzing ResNet skip connections, residual formulation F(x) + x, and how identity mappings enable training deep architectures without degradation.