Latent variable models explained: variational inference, ELBO, VAE, and adversarial auto-encoders (AAE).

Introduction to generative models: probability distributions, independence assumptions, chain rule, and auto-regressive models.

Hands-on PyTorch implementation of Scaled Dot-Product Attention and Multi-Head Attention for the Transformer.

Core concepts of the Transformer model: encoder self-attention, Query/Key/Value embeddings, and multi-head attention mechanics.

Week 2 of Naver Boostcamp AI Tech covering deep learning basics, data visualization, and sample variance with degrees of freedom.

Discussion on how the relationship between model parameter count and generalization performance shifted with scaling law findings.