Focal Modulation Networks, a lightweight transformer alternative in computer vision that uses depthwise convolutions to model local and global context and does late aggregation with a gating mechanism.
Research into compiler optimizations for machine learning, focusing on XLA's fusion patterns, intermediate representations, and memory management.
A glossary of terms related to machine learning compilation and optimization.
Comparing regression and classification loss functions on a simple low dimensional example.
