RAG (Retrieval-Augmented Generation)
Using LLMs on new data without fine-tuning.
RAG allows using LLMs on previously unseen data without the need for fine-tuning.
In the RAG setup, the data is stored in vector form within an external database. Using RAG, the LLM retrieves necessary information from it and bases its answer to the user query on retrieved facts.
Benefits:
- Conserves resources by avoiding continuous fine-tuning as data updates
- Enables easy modification of external databases for dynamic data control
Original paper: https://arxiv.org/abs/2005.11401v4