Slowformer

Slowformer is a transformer framework designed to prioritize:

• Weight Transparency: Gain deep insights into the behavior of weights at every layer, making the training and inference processes more interpretable.
• Monitorability: Seamlessly track weight updates, gradients, and activations with robust logging and visualization tools.
• Deliberate Speed: Crafted for research and experimentation, Slowformer values clarity and explainability over raw computational speed.

ToDo List

• Implement Basic Transformer Model

Reading List

Transformer Architecture

• Attention Is All You Need