Clarifying commonly confused NLP concepts: argmax vs. multinomial sampling, beam search vs. sampling, and Pre-LN vs. Post-LN in GPT and Transformer.

Survey of recent NLP trends covering QA, open-domain chatbots, unsupervised machine translation, text style transfer, and in-context learning.

Overview of GPT-2, GPT-3, and ALBERT covering zero/few-shot learning, model scaling, and efficient pre-training approaches.

Overview of self-supervised pre-training models including GPT-1 and BERT, covering masked language modeling and transfer learning.

Week 7 retrospective of Naver Boostcamp AI Tech covering Transformer and BERT studies, mentoring Q&A on attention mechanisms and positional encoding.

Introduction to the Transformer architecture: self-attention with Query/Key/Value, scaled dot-product attention, and how it overcomes RNN limitations.