Type
Preprocessing, Dimensionality Reduction

Please see “Singular Value Decomposition” section under 2. Linear Algebra for SVD formulation.

Possible Limitations of SVD:

• Computational cost scales quadratically
• Different learning regime than other DL models
• Bad for millions of words or documents (if used for 3. Word Vectors in NLP - See section “Representing with a Co-occurrence Matrix”)
• Hard to incorporate new words or documents (if used for 3. Word Vectors in NLP - See section “Representing with a Co-occurrence Matrix”)