en.wikipedia.org/wiki/Chatbot
1 correction found
require a large amount of conversational data to train
Modern LLM chatbots are not primarily trained on large conversational datasets. Their base models are pre-trained on broad text corpora such as web pages, books, and other general text, then optionally fine-tuned for chat behavior.
Full reasoning
This sentence misstates how modern LLM chatbots are trained.
The core training of models such as GPT-3 and GPT-4 is pretraining on large corpora of text, not on a large amount of conversational data. OpenAI describes GPT-4's base model as being trained on publicly available internet data and licensed data, and says the chat behavior is added later through fine-tuning with reinforcement learning from human feedback (RLHF). Google Research likewise describes large language models as being trained on sufficiently large and diverse corpora of text.
So while conversational examples may be used during later alignment or fine-tuning, it is incorrect to say these chatbots require a large amount of conversational data to train as a general description of how LLM chatbots are built.
3 sources
- GPT-4 | OpenAI
Like previous GPT models, the GPT-4 base model was trained to predict the next word in a document, and was trained using publicly available data (such as internet data) as well as data we’ve licensed. ... To align it with the user’s intent ... we fine-tune the model’s behavior using reinforcement learning with human feedback (RLHF).
- Google Research, 2022 & beyond: Language, vision and generative models
When large models are trained on sufficiently large and diverse corpora of text, the models can generate coherent, contextual, natural-sounding responses.
- Language Models are Few-Shot Learners
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.