Why didn't we get GPT-2 in 2005? (dynomight.net via hn) 6w
event
GPT 2
-
Why didn't we get GPT-2 in 2005? The ancient Romans were never great at building ships and never tried to explore the Atlantic.
-
Should low-precision transformer inference use stochastic rounding (SR) or round-to-nearest (RN)? The answer depends on where in the network you look.
-
Attention-head ablation, zeroing a head and measuring the resulting change in task performance, is a common method for inferring which components of a language model are causally responsible for a behavior. We show using GPT-2 small that t…