AI Glossary
A comprehensive reference of 100 key terms in artificial intelligence, machine learning, and data science. Each definition explains what the concept is, how it works, and why it matters for your AI career.
Whether you are preparing for an interview, learning new concepts, or exploring career paths in AI, this glossary provides clear, professional definitions with links to related job categories and deeper explanations.
A
Activation Function
An activation function is a mathematical function applied to the output of each neuron in a neural network. It introduces non-linearity, enabling the network to learn complex patterns beyond simple linear relationships.
Adversarial Attack
An adversarial attack is a technique that deliberately manipulates input data to cause a machine learning model to make incorrect predictions. These attacks expose vulnerabilities in AI systems by exploiting how models process and interpret data.
Agentic RAG
Agentic RAG combines autonomous agent capabilities with retrieval-augmented generation, creating AI systems that can plan multi-step research, decide what to retrieve, evaluate retrieval quality, and iteratively refine their searches to produce comprehensive, grounded answers.
AI Agent Framework
An AI agent framework is a software library that provides tools and abstractions for building autonomous AI agents. Popular frameworks like LangChain, LlamaIndex, and CrewAI simplify the process of creating agents that can reason, use tools, and accomplish multi-step tasks.
AI Safety
AI safety is the field dedicated to ensuring that AI systems do not cause harm, either through unintended behavior, misuse, or as they become more capable. It encompasses alignment research, robustness testing, red-teaming, and governance frameworks.
Alignment
Alignment refers to the challenge of ensuring that AI systems behave in accordance with human intentions, values, and goals. It is a central concern in AI safety research, particularly as models become more capable and autonomous.
Attention Is All You Need
The landmark 2017 paper by Google researchers that introduced the Transformer architecture. By demonstrating that self-attention alone could replace recurrence and convolution for sequence modeling, it laid the foundation for virtually all modern AI systems.
Attention Mechanism
An attention mechanism allows a neural network to focus on specific parts of the input when producing each part of the output. It assigns different weights to different input elements, enabling the model to capture long-range dependencies and contextual relationships.
AutoML
AutoML (Automated Machine Learning) refers to tools and techniques that automate the process of building machine learning models, including feature engineering, model selection, and hyperparameter tuning. It aims to make ML accessible to non-experts and improve efficiency for practitioners.
Autonomous Agent
An autonomous agent is an AI system that can perceive its environment, make decisions, and take actions to achieve goals with minimal human intervention. Modern AI agents often use large language models as their reasoning core, combined with tools and memory systems.
B
Backpropagation
Backpropagation is the algorithm used to compute gradients of a loss function with respect to each weight in a neural network. It enables efficient training by propagating error signals backward through the network layers.
Batch Normalization
Batch normalization is a technique that normalizes the inputs to each layer of a neural network by adjusting and scaling activations using statistics computed over a mini-batch. It stabilizes and accelerates training while acting as a form of regularization.
BERT
BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained language model developed by Google that reads text in both directions simultaneously. It established new benchmarks across many NLP tasks and popularized the pre-train then fine-tune paradigm.
Bias (in ML)
Bias in machine learning refers to systematic errors that cause a model to consistently produce unfair or inaccurate results. It can arise from training data, algorithm design, or the way problems are framed, and it can lead to discrimination against certain groups.
Bias-Variance Tradeoff
The bias-variance tradeoff is a fundamental concept describing the tension between a model's ability to fit training data closely (low bias) and its ability to generalize to unseen data (low variance). Achieving the right balance is central to building effective ML models.
C
Chain-of-Thought Prompting
Chain-of-thought (CoT) prompting is a technique that encourages large language models to generate intermediate reasoning steps before arriving at a final answer. It significantly improves performance on tasks requiring multi-step reasoning, arithmetic, and logical deduction.
Classification
Classification is a supervised learning task where a model learns to assign input data to one of several predefined categories. It is one of the most common applications of machine learning, used in spam detection, medical diagnosis, sentiment analysis, and many other domains.
Clustering
Clustering is an unsupervised learning technique that groups similar data points together without predefined labels. It is used for customer segmentation, anomaly detection, data exploration, and discovering hidden structure in datasets.
Computer Vision
Computer vision is a field of AI that enables machines to interpret and understand visual information from images and videos. It powers applications from autonomous driving to medical imaging to augmented reality.
Conformal Prediction
Conformal prediction is a framework for generating prediction sets with guaranteed coverage probabilities. Unlike point predictions, it provides statistically valid uncertainty estimates that tell you how confident a model is in its predictions.
Constitutional AI
Constitutional AI (CAI) is an approach developed by Anthropic for training AI systems to be helpful, harmless, and honest using a set of explicit principles (a "constitution") rather than relying solely on human feedback for every decision.
Convolutional Neural Network
A convolutional neural network (CNN) is a type of deep learning architecture specifically designed to process grid-structured data like images. CNNs use learnable filters to automatically detect spatial patterns and hierarchical features.
Cross-Validation
Cross-validation is a statistical technique for evaluating how well a machine learning model generalizes to unseen data. It partitions the dataset into multiple folds, training and testing on different subsets to produce a more reliable performance estimate.
D
Data Augmentation
Data augmentation is a technique that artificially increases the size and diversity of a training dataset by applying transformations to existing data. It is widely used to improve model generalization, especially when labeled data is limited.
Data Drift
Data drift occurs when the statistical properties of production data change over time relative to the training data. It causes model performance to degrade and is one of the most common reasons deployed ML models fail silently.
Data Labeling
Data labeling is the process of annotating raw data with meaningful tags or labels that supervised ML models use for training. It is a critical and often resource-intensive step that directly impacts model quality.
Decision Tree
A decision tree is a supervised learning algorithm that makes predictions by learning a hierarchy of if-then rules from training data. It splits data at each node based on feature values, creating an interpretable tree structure that maps inputs to outputs.
Deep Learning
Deep learning is a subset of machine learning that uses neural networks with multiple layers to learn hierarchical representations of data. It has driven breakthroughs in computer vision, natural language processing, speech recognition, and generative AI.
Diffusion Model
A diffusion model is a type of generative AI model that creates data by learning to reverse a gradual noising process. Diffusion models power leading image generators like Stable Diffusion, DALL-E, and Midjourney, producing high-quality, diverse outputs.
Dimensionality Reduction
Dimensionality reduction is a set of techniques that reduce the number of features in a dataset while preserving important information. It is used for visualization, noise reduction, and improving model performance on high-dimensional data.
E
Embeddings
Embeddings are dense vector representations that capture the semantic meaning of data (words, sentences, images, or other objects) in a continuous vector space. Similar items are mapped to nearby points, enabling mathematical operations on meaning.
Encoder-Decoder
An encoder-decoder is a neural network architecture where an encoder processes input data into a compact representation, and a decoder generates output from that representation. It is the foundation for machine translation, summarization, and sequence-to-sequence tasks.
Ensemble Methods
Ensemble methods combine multiple machine learning models to produce better predictions than any single model. Techniques like random forests, gradient boosting, and stacking are among the most effective approaches for structured data.
Epoch
An epoch is one complete pass through the entire training dataset during model training. The number of epochs is a key hyperparameter that affects model convergence—too few leads to underfitting, too many can cause overfitting.
Ethical AI
Ethical AI encompasses principles, practices, and governance frameworks for developing and deploying AI systems that are fair, transparent, accountable, and beneficial to society. It addresses risks including bias, privacy violations, job displacement, and misuse.
Evaluation and Benchmarking
Evaluation and benchmarking in AI encompass the methods, metrics, and datasets used to measure model performance. Proper evaluation is essential for comparing models, detecting issues, and ensuring AI systems meet quality requirements before deployment.
Explainable AI
Explainable AI (XAI) encompasses methods and tools that make AI model predictions understandable to humans. It addresses the "black box" problem of complex models, enabling trust, debugging, and compliance with regulations that require decision explanations.
F
Feature Engineering
Feature engineering is the process of creating, selecting, and transforming input variables to improve ML model performance. It leverages domain knowledge to create representations that make patterns in data more accessible to learning algorithms.
Federated Learning
Federated learning is a privacy-preserving ML technique where models are trained across multiple decentralized devices or servers without sharing raw data. Each participant trains locally and only shares model updates, keeping sensitive data on-device.
Few-Shot Learning
Few-shot learning enables ML models to learn new tasks from only a handful of examples. It addresses scenarios where labeled data is scarce or expensive to obtain, making AI more practical for specialized and emerging applications.
Fine-Tuning
Fine-tuning is the process of taking a pre-trained model and adapting it to a specific task or domain by training on task-specific data. It is a cornerstone technique in modern AI that enables efficient specialization of foundation models.
Foundation Model
A foundation model is a large AI model trained on broad data that can be adapted to a wide range of downstream tasks. Examples include GPT-4, Claude, LLaMA, and DALL-E. They represent a paradigm shift toward general-purpose models that serve as a base for many applications.
G
Generative Adversarial Network
A generative adversarial network (GAN) is a framework where two neural networks compete: a generator creates synthetic data and a discriminator evaluates its authenticity. This adversarial training process produces remarkably realistic generated content.
Generative AI
Generative AI refers to AI systems that can create new content including text, images, audio, video, and code. Powered by large language models and diffusion models, it represents one of the most commercially impactful AI developments.
GPT
GPT (Generative Pre-trained Transformer) is a family of large language models developed by OpenAI that generate text by predicting the next token in a sequence. GPT models pioneered the scaling approach that led to modern AI assistants and have become synonymous with the AI revolution.
Gradient Descent
Gradient descent is the fundamental optimization algorithm used to train ML models. It iteratively adjusts model parameters in the direction that reduces the loss function, guided by the gradient (slope) of the loss with respect to each parameter.
Graph Neural Network
A graph neural network (GNN) is a type of neural network designed to operate on graph-structured data, where entities (nodes) are connected by relationships (edges). GNNs are used for social network analysis, molecular modeling, recommendation systems, and fraud detection.
Grounding
Grounding in AI refers to connecting model outputs to verifiable sources of truth, such as retrieved documents, databases, or real-world observations. It is a primary strategy for reducing hallucinations and improving the factual reliability of AI systems.
H
Hallucination
Hallucination in AI refers to when a model generates confident but factually incorrect or fabricated information. It is a significant challenge for language models and multimodal AI systems, affecting their reliability in high-stakes applications.
Hyperparameter Tuning
Hyperparameter tuning is the process of finding optimal configuration settings for ML models that are set before training begins. Unlike model parameters learned from data, hyperparameters like learning rate, batch size, and network depth must be chosen by the practitioner.
I
In-Context Learning
In-context learning (ICL) is the ability of large language models to perform new tasks by receiving examples directly in the prompt, without any parameter updates. It is one of the most powerful emergent capabilities of large-scale LLMs.
Inference
Inference is the process of using a trained ML model to make predictions on new data. Optimizing inference speed, cost, and quality is a critical engineering challenge as AI models are deployed in production at scale.
Instruction Tuning
Instruction tuning is the process of fine-tuning a pre-trained language model on a diverse dataset of instruction-response pairs, teaching it to follow human instructions across a wide range of tasks. It is what transforms a base language model into a helpful AI assistant.
K
Knowledge Distillation
Knowledge distillation is a model compression technique where a smaller "student" model is trained to replicate the behavior of a larger "teacher" model. It enables deploying powerful AI capabilities on resource-constrained devices and at lower cost.
Knowledge Graph
A knowledge graph is a structured representation of real-world entities and their relationships, stored as a network of nodes (entities) and edges (relationships). It provides a way to organize and query complex knowledge that complements neural network approaches.
L
Large Language Model
A large language model (LLM) is a neural network with billions of parameters trained on vast text corpora to understand and generate human language. LLMs like GPT-4, Claude, Gemini, and LLaMA power conversational AI, code generation, and a wide range of language tasks.
LoRA
LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique that adds small, trainable low-rank matrices to model layers while keeping original weights frozen. It enables fine-tuning large models at a fraction of the memory and compute cost.
Loss Function
A loss function (or cost function) measures how far a model's predictions are from the true values. It provides the signal that guides model training through gradient descent, making its design one of the most important decisions in ML.
LSTM
Long Short-Term Memory (LSTM) is a type of recurrent neural network with gated memory cells that can learn long-range dependencies in sequential data. While largely superseded by Transformers for NLP, LSTMs remain used for time-series and streaming applications.
M
Machine Learning
Machine learning is a field of AI where computer systems learn patterns from data to make predictions or decisions without being explicitly programmed for each task. It encompasses supervised, unsupervised, and reinforcement learning approaches.
Mixture of Agents
Mixture of Agents is a multi-agent AI pattern where multiple LLMs or specialized agents collaborate on tasks, with their outputs combined or orchestrated to produce better results than any single agent. It leverages diverse model strengths for improved accuracy and robustness.
Mixture of Experts
Mixture of Experts (MoE) is an architecture that routes each input to a subset of specialized "expert" sub-networks within a larger model. It enables training much larger models while keeping inference cost manageable, as only a fraction of parameters are active for each input.
MLOps
MLOps (Machine Learning Operations) is the practice of deploying, monitoring, and maintaining ML models in production. It combines ML engineering with DevOps principles to create reliable, scalable, and automated ML systems.
Model Card
A model card is a standardized documentation format that describes an ML model's intended use, performance characteristics, limitations, and ethical considerations. It promotes transparency and informed decision-making when selecting and deploying AI models.
Model Compression
Model compression refers to techniques that reduce the size and computational cost of ML models while preserving performance. It includes quantization, pruning, distillation, and architectural optimization, enabling deployment on resource-constrained devices.
Multimodal AI
Multimodal AI refers to systems that can process and reason across multiple types of data including text, images, audio, and video. Models like GPT-4V, Gemini, and Claude with vision represent the frontier of AI that understands the world through multiple senses.
N
Natural Language Processing
Natural language processing (NLP) is a field of AI focused on enabling computers to understand, interpret, and generate human language. It powers search engines, chatbots, translation services, and the language models that are transforming how humans interact with technology.
Neural Architecture Search
Neural Architecture Search (NAS) is the process of using automated methods to discover optimal neural network architectures. Instead of manually designing model structures, NAS explores a defined search space to find architectures that maximize performance for a given task and constraint.
Neural Network
A neural network is a computing system inspired by biological neurons that learns to perform tasks by adjusting connection weights based on data. Neural networks are the building blocks of deep learning and power virtually all modern AI applications.
O
Object Detection
Object detection is a computer vision task that identifies and localizes specific objects within images or video frames by predicting both class labels and bounding box coordinates. It powers autonomous driving, surveillance, medical imaging, and retail analytics.
Overfitting
Overfitting occurs when an ML model learns the training data too well, including its noise and peculiarities, causing poor performance on new unseen data. It is one of the most common and important challenges in machine learning.
P
PEFT
Parameter-Efficient Fine-Tuning (PEFT) is a family of methods that adapt large pre-trained models to new tasks by training only a small fraction of parameters. PEFT makes fine-tuning of billion-parameter models practical on consumer hardware.
Pre-training
Pre-training is the initial phase of training where a model learns general representations from large-scale data using self-supervised objectives. It provides the foundation of knowledge and capabilities that subsequent fine-tuning adapts for specific tasks.
Prompt Chaining
Prompt chaining is the technique of breaking a complex task into a sequence of simpler prompts, where each step's output feeds into the next step's input. It enables reliable completion of multi-step tasks that would be difficult to accomplish in a single prompt.
Prompt Engineering
Prompt engineering is the practice of designing and optimizing inputs to language models to elicit desired outputs. It encompasses techniques for structuring instructions, providing examples, and leveraging model capabilities to achieve specific tasks.
Prompt Injection
Prompt injection is a security vulnerability where malicious inputs manipulate a language model into ignoring its instructions or performing unintended actions. It is one of the most significant security challenges in deploying LLM-powered applications.
Q
R
Recurrent Neural Network
A recurrent neural network (RNN) is a neural network architecture designed for sequential data that maintains a hidden state across time steps, allowing it to capture temporal dependencies. While largely superseded by Transformers, RNNs remain relevant for specific applications.
Reinforcement Learning
Reinforcement learning (RL) is a paradigm where an agent learns to make decisions by interacting with an environment and receiving rewards or penalties. It powers game-playing AI, robotics, and is central to aligning language models through RLHF.
Responsible AI
Responsible AI is a governance framework that ensures AI systems are developed and deployed in ways that are ethical, safe, fair, transparent, and accountable. It encompasses organizational practices, technical methods, and policy considerations.
Retrieval-Augmented Generation
Retrieval-Augmented Generation (RAG) is a technique that enhances language model outputs by retrieving relevant information from external knowledge sources before generating a response. It reduces hallucinations and enables models to access up-to-date, domain-specific information.
RLHF
Reinforcement Learning from Human Feedback (RLHF) is a training technique that uses human preferences to align language model behavior. Human evaluators rank model outputs, training a reward model that guides reinforcement learning to make the model more helpful, honest, and safe.
S
Scaling Laws
Scaling laws describe the predictable relationship between model size, training data, compute, and performance in neural networks. They guide investment decisions in AI development by predicting how much resources are needed to achieve target performance levels.
Self-Supervised Learning
Self-supervised learning is a training paradigm where models learn representations from unlabeled data by solving pretext tasks that generate supervisory signals from the data itself. It powers the pre-training of foundation models and reduces dependence on expensive labeled data.
Semantic Search
Semantic search finds information based on meaning rather than keyword matching. By using embeddings to understand the intent and context of queries and documents, it retrieves results that are conceptually relevant even when they do not share exact words with the query.
Sentiment Analysis
Sentiment analysis is an NLP task that determines the emotional tone or opinion expressed in text, classifying it as positive, negative, or neutral. It is widely used for brand monitoring, customer feedback analysis, market research, and social media analytics.
Sparse Attention
Sparse attention is a family of efficient attention mechanisms that reduce the quadratic computational cost of standard self-attention by limiting which positions can attend to each other. It enables Transformers to process much longer sequences.
Stable Diffusion
Stable Diffusion is an open-source latent diffusion model for generating images from text descriptions. Released by Stability AI in 2022, it democratized AI image generation by providing a powerful, customizable model that can run on consumer hardware.
Supervised Learning
Supervised learning is the most common ML paradigm where a model learns from labeled training data to make predictions on new data. The "supervision" comes from known correct answers (labels) that guide the learning process.
Synthetic Data
Synthetic data is artificially generated data that mimics the statistical properties of real data. It is used to augment training datasets, protect privacy, address class imbalance, and enable model development when real data is scarce or restricted.
T
Tokenization
Tokenization is the process of splitting text into smaller units (tokens) that language models process. Tokens may be words, subwords, or characters. The tokenization strategy directly affects model vocabulary, efficiency, and ability to handle diverse languages and domains.
Tokenizer
A tokenizer is the specific software component that converts raw text into tokens and vice versa. Different LLMs use different tokenizers, and the choice of tokenizer affects model efficiency, multilingual performance, and how much text fits in the context window.
Transfer Learning
Transfer learning is a technique where knowledge gained from training on one task is applied to a different but related task. It is the foundation of the pre-train and fine-tune paradigm that makes modern AI practical for the vast majority of applications.
Transformer
The Transformer is a neural network architecture based on self-attention mechanisms that has become the foundation of modern AI. Introduced in 2017, it powers language models, vision systems, and multimodal AI, replacing earlier recurrent and convolutional approaches for most tasks.
U
V
Variational Autoencoder
A variational autoencoder (VAE) is a generative model that learns a compressed latent representation of data while enforcing a probabilistic structure. It enables data generation, interpolation, and smooth latent space exploration.
Vector Database
A vector database is a specialized database designed to store, index, and query high-dimensional vector embeddings efficiently. It is the backbone of semantic search, RAG systems, and recommendation engines, enabling fast similarity search over millions or billions of vectors.
Vision Transformer
The Vision Transformer (ViT) applies the Transformer architecture to image recognition by treating images as sequences of patches. It demonstrated that attention-based models can match or surpass CNNs for vision tasks, unifying the architecture used across modalities.
W
Z
Ready to Start Your AI Career?
Now that you know the terminology, explore open positions in machine learning, data science, NLP, and more.