Project 14
Transformer From Scratch
A full encoder-decoder Transformer in PyTorch, trained for English to German.
A way to understand attention properly by building every part of 'Attention is All You Need' myself.
This is a full encoder-decoder Transformer implemented from scratch in PyTorch, following Vaswani et al. (2017). It includes scaled dot-product and multi-head attention, positional encoding, feed-forward layers, and six-layer encoder and decoder stacks.
Training uses the Noam learning-rate scheduler and label smoothing on the Multi30K English-German translation task. The model code lives in src/, with step-by-step Colab notebooks alongside.