What is Bias (in ML)?

Bias in machine learning is a multifaceted concept that encompasses both statistical and social dimensions. Statistical bias refers to a model's tendency to systematically deviate from the true values it is trying to predict, often due to simplifying assumptions or insufficient data. Social or fairness bias refers to systematic discrimination against particular groups based on attributes like race, gender, age, or socioeconomic status. Both forms of bias can severely undermine the reliability and trustworthiness of AI systems.

Data bias is the most common source of unfairness in ML systems. Training data often reflects historical inequalities, sampling biases, and labeling inconsistencies. A hiring model trained on historical decisions may learn to discriminate against women if past hiring was biased. A facial recognition system trained predominantly on light-skinned faces will perform worse on darker-skinned faces. A language model trained on internet text may reproduce stereotypes and toxic content present in that data. Recognizing and mitigating these data biases requires careful dataset curation, analysis of data distributions, and ongoing monitoring.

Algorithmic bias can arise even with balanced data if the model architecture or optimization process introduces systematic errors. Feature selection, proxy variables, and feedback loops can all amplify bias. For example, using zip code as a feature can serve as a proxy for race due to residential segregation patterns. Feedback loops occur when biased model predictions influence future data collection, reinforcing and amplifying the original bias over time.

Multiple approaches exist for detecting and mitigating bias. Pre-processing methods aim to transform training data to remove bias before model training. In-processing methods incorporate fairness constraints directly into the training objective. Post-processing methods adjust model outputs to satisfy fairness criteria. Metrics for measuring fairness include demographic parity, equalized odds, calibration, and individual fairness. The choice of fairness metric depends on the specific context and often involves trade-offs, as satisfying multiple fairness criteria simultaneously is mathematically impossible in many cases.

Industry and regulatory responses to ML bias have grown significantly. Regulations like the EU AI Act require bias assessments for high-risk AI systems. Organizations have established responsible AI teams, fairness review processes, and model documentation standards (model cards, datasheets for datasets). Understanding bias is no longer optional for ML practitioners; it is a professional and legal responsibility that affects every stage of the ML lifecycle from problem formulation to deployment and monitoring.