A review of the paper that enhances diffusion models using retrieval-based approaches from NLP, achieving high-fidelity image generation with fewer parameters.

A review of Med-BERT, a BERT-based model pretrained on structured EHR data for disease prediction.

Paper review of Condenser and coCondenser for dense retrieval pre-training.

Survey of NLP trends including reinforcement learning for summarization and QA, and multimodal NLP+CV approaches like Show and Tell and GLAC Net.

Overview of phrase retrieval for open-domain QA, using dense-sparse representations to bypass the retriever-reader pipeline.

An approach that enables large-batch contrastive learning under memory constraints by using in-batch negatives instead of explicit negative sampling.

Types of training bias in MRC and ODQA, and techniques to mitigate them including negative sampling and annotation bias handling.

Deep dive into Open Domain Question Answering, covering the Retriever-Reader approach, distant supervision, and passage granularity strategies.

Scaling passage retrieval with approximate similarity search techniques: scalar quantization, inverted file indexing, and FAISS.

Dense embedding for passage retrieval: training bi-encoders with contrastive learning and negative sampling strategies.

Fundamentals of passage retrieval for open-domain QA, covering sparse embedding, TF-IDF, and similarity search in vector space.

Generation-based MRC using seq2seq models like BART: comparison with extraction-based approaches, preprocessing, and decoding strategies.

Survey of post-BERT NLP models including XLNet, RoBERTa, BART, T5, Meena, and controllable language models like PPLM.

Evolution of GPT language models from GPT-1 to GPT-3: architecture changes, zero/few-shot learning, and scaling effects.

Basics of Unicode handling in Python and subword tokenization methods including BPE (Byte-Pair Encoding).

Introduction to Machine Reading Comprehension covering dataset types, key challenges like coreference resolution and multi-hop reasoning, and evaluation metrics.

Token-level classification tasks with BERT, covering Named Entity Recognition (NER) and POS tagging for Korean text.

Overview of Korean dependency parsing and single sentence classification tasks in KLUE, including BERT-based model architecture and training pipeline.

BERT-based sentence pair classification covering NLI, semantic text pair similarity, and information retrieval QA.

BERT training pipeline details: tokenizer creation, dataset construction with NSP, masking strategies, segment handling, and truncation.

Practical applications of BERT for sentiment analysis, relation extraction, semantic similarity, NER, and machine reading comprehension.

Hands-on regex practice dissecting a password validation pattern with lookaheads and character classes.

Summary of language model benchmarks from GLUE to multilingual evaluations, covering the evolution from RNN to ELMo, BERT, and beyond.

Clarifying commonly confused NLP concepts: argmax vs. multinomial sampling, beam search vs. sampling, and Pre-LN vs. Post-LN in GPT and Transformer.

Survey of recent NLP trends covering QA, open-domain chatbots, unsupervised machine translation, text style transfer, and in-context learning.

Overview of GPT-2, GPT-3, and ALBERT covering zero/few-shot learning, model scaling, and efficient pre-training approaches.

Overview of self-supervised pre-training models including GPT-1 and BERT, covering masked language modeling and transfer learning.

Introduction to the Transformer architecture: self-attention with Query/Key/Value, scaled dot-product attention, and how it overcomes RNN limitations.

BLEU score for machine translation evaluation: precision, recall, F1, n-gram overlap, and brevity penalty.

Explanation of beam search decoding as a trade-off between greedy decoding and exhaustive search in sequence generation.

Explanation of the attention mechanism in Seq2Seq models, covering attention vectors, context vectors, teacher forcing, and similarity functions.

Detailed explanation of LSTM and GRU architectures, covering gate mechanisms, cell state, and how they solve RNN's long-term dependency problem.

Deep dive into RNN types (one-to-one, one-to-many, many-to-many), character-level language models, and LSTM/GRU architectures.

NLP text preprocessing techniques: stopword removal, lemmatization, and punctuation handling with NLTK and spaCy.

Introduction to word embedding and Word2Vec: how words are mapped to vector spaces to capture semantic relationships.

Introduction to Bag-of-Words text representation with one-hot encoding and Naive Bayes classification.

Overview of NLP fundamentals including tokenization, NER, sentiment analysis, machine translation, and the impact of Transformer on the field.

Hands-on PyTorch implementation of Scaled Dot-Product Attention and Multi-Head Attention for the Transformer.

Core concepts of the Transformer model: encoder self-attention, Query/Key/Value embeddings, and multi-head attention mechanics.

Fundamentals of RNN including sequence data handling, latent autoregressive models, BPTT, and truncated backpropagation.