██ ██ ██ ██ █████ ███ ███
██ ██ ██ ██ ██ ██ ████ ████
███████ ██ ██ ███████ ██ ████ ██
██ ██ ██ ██ ██ ██ ██ ██ ██
██ ██ ██ ▄█ ██ ██ ██ ██ ██
███████ ██ ██ ██████ ███████ ██ ██ █████ ███████ ██ ██ ██ ██ █████ ██████
██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██
███████ ███████ ██████ █████ ████ ███████ ███████ ███████ █████ ███████ ██████
██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██
███████ ██ ██ ██ ██ ███████ ██ ██ ██ ███████ ██ ██ ██ ██ ██ ██ ██ ██● session ready. type for commands.
selected projects
recent writing
all postsThe only Muon Optimizer guide you need
All neural networks use a form of gradient descent for updating their parameters. The fundamental intuition to all neural net’s parameter optimization seems obvious to us, i.e., to …
A deep-dive into RoPE, and why it matters?
What is Positional Encoding and why it matters? When training any large language model based on Transformers architecture, our input token sequences tend to form a $\text{seq\_len} …
Decoding Karpathy's min-char-rnn (character level Recurrent Neural Network)
Recurrent Neural Networks (RNN) have existed for long at this point, and RNNs without attention mechanism (plain-simple RNN architecture) are no longer the hottest thing either. …