Training/Pre-training
The process by which an AI model learns by analyzing massive amounts of data.
Detailed Definition
Training is the process by which an AI model learns by analyzing massive amounts of data, such as large portions of the internet, books, and other media. For LLMs, the core training method is "next-token prediction," where the model learns to predict the next word in a sequence. As it trains, the model adjusts millions of internal settings called "weights." This process helps the model improve its understanding of facts, grammar, reasoning, and language. Training state-of-the-art models can take months and cost hundreds of millions of dollars.
Learning MethodsMore in this Category
Few-Shot Learning
The ability of AI models to learn new tasks with only a small number of training examples.
Fine-tuning
A post-training technique to specialize a trained model on specific data for a particular task.
Post-training
Additional steps taken after a model's initial training is complete to make it more useful.
RLHF (reinforcement learning from human feedback)
A post-training technique to align AI models with human preferences using feedback.
Supervised learning
A type of machine learning where a model is trained on labeled data with correct answers provided.
Unsupervised learning
A type of machine learning where a model is given unlabeled data to find patterns on its own.