Welcome to this Map of Content.
Notes
Foundations
- Neural Networks - Stub.
- Batch Normalization - Batch Normalization (BN) is a technique applied between the hidden layers of a neural network.
- CNNs - Forward pass, backpropagation through convolutional and pooling layers.
- Keras vs. PyTorch vs. TensorFlow - Framework comparison: Keras, TensorFlow, and PyTorch.
Sequence Modeling & NLP
- RNNs & LSTMs - RNN/LSTM architecture, parameter counting, text generation, and Seq2Seq machine translation. Best practices: always use LSTM, prefer Bi-RNN, pretrain embeddings.
- Attention & Transformers - Attention mechanism, self-attention, multi-head attention, Transformer encoder/decoder architecture, and BERT pre-training.
Dimensionality Reduction & Representation Learning
- Autoencoders - Dense, convolutional, and denoising autoencoders; bottleneck architecture, reconstruction loss, t-SNE pipeline, and discriminative latent spaces.
- Variational Autoencoders - Probabilistic extension; reparameterization trick, KL divergence loss, and generative latent space arithmetic.
Generative Models
- Variational Autoencoders - VAE: structured latent space, KL regularization, image interpolation.
- Generative Adversarial Networks - GAN: Generator vs. Discriminator adversarial training, instability issues and stabilization tricks.