Large Language Models
Give attention a place in the block
A journey through transformer blocks with Botie.
Large Language Models · A journey with Botie
First visit? Learn the workshop controlsBotie's notebook
Could Every Position Consult the Others Directly?
Part 1 of 3
A recurrent network carries information through successive steps. When two words are far apart, their relationship may have to survive many intermediate operations. The sequential processing also limits how much of a training sequence can be computed at once.