Overview of GPT-2, GPT-3, and ALBERT covering zero/few-shot learning, model scaling, and efficient pre-training approaches.

Overview of self-supervised pre-training models including GPT-1 and BERT, covering masked language modeling and transfer learning.

Week 7 retrospective of Naver Boostcamp AI Tech covering Transformer and BERT studies, mentoring Q&A on attention mechanisms and positional encoding.

Introduction to the Transformer architecture: self-attention with Query/Key/Value, scaled dot-product attention, and how it overcomes RNN limitations.

Week 6 study summary of Naver Boostcamp AI Tech covering NLP lectures, bucketing for data batching, and peer session discussions.

BLEU score for machine translation evaluation: precision, recall, F1, n-gram overlap, and brevity penalty.