Characteristics of LLMs:

• Scale: They contain millions, billions, or even hundreds of billions of parameters
• General capabilities: They can perform multiple tasks without task-specific training
• In-context learning: They can learn from examples provided in the prompt
• Emergent abilities: They can demonstrate capabilities that weren’t explicitly programmed or anticipated

Limitations of LLMs:

• Hallucinations: They can generate incorrect information confidently
• Lack of true understanding: They lack true understanding of the world and operate purely on statistical patterns
• Bias: They may reproduce biases present in their training data or inputs.
• Context windows: They have limited context windows (though this is improving)
• Computational resources: They require significant computational resources