Machine learning core concepts: learning types, features, bias and variance
Machine learning finds rules from examples. This page explains the kinds of learning, what a feature is, why models underfit or overfit, and the workflow every project follows.
Overview
Instead of writing rules by hand, machine learning fits a model to examples. The model's parameters are adjusted until its predictions match the known answers, and it is then used on new cases. The ideas below apply to every algorithm, from linear regression to large foundation models.
Kinds of learning
| Kind | Data | Goal | Examples | Read more |
|---|---|---|---|---|
| Supervised | Inputs with known answers | Predict the answer | Price, churn, spam | Supervised learning |
| Unsupervised | Inputs only | Find structure | Customer segments, anomalies | Unsupervised learning |
| Semi-supervised | Few labels, many unlabelled | Use both | Labelling at scale | Semi-supervised |
| Self-supervised | Labels made from the data | Learn representations | Pre-training | Self-supervised |
| Reinforcement | Rewards after actions | Learn a policy | Games, control, LLM tuning | Reinforcement learning |
Features and labels
A feature is an input column; the label is the answer to predict. Most of the gain in classic projects comes from good features: ratios instead of raw totals, dates split into weekday and season, text turned into counts or embeddings. See feature engineering and data preprocessing.
Bias, variance and capacity
A model that is too simple misses real patterns (high bias); one that is too flexible learns noise (high variance). Regularisation, more data and cross-validation find the balance. The bias-variance page and the overfitting simulation show the trade-off.
The workflow
- Define the decision the model supports and the metric that reflects it.
- Collect and split data before looking closely, so the test set stays untouched.
- Build a simple baseline.
- Improve features and models, comparing on validation data.
- Check the final model once on the test set, then monitor it in production.
The mathematics behind it
Linear algebra, probability, statistics and optimisation are the four foundations; the site's pages explain the parts machine learning uses.
Frequently asked questions
How much data do I need?
It depends on the signal and the model. Tabular foundation models and gradient boosting work well with hundreds to thousands of rows; deep networks on raw data need far more.
Which algorithm should I start with?
A simple linear or tree model as a baseline, then gradient boosting or a tabular foundation model for tables.
Summary
Know your kind of learning, invest in features, balance bias and variance, and follow a disciplined workflow with an untouched test set. The machine learning foundations path teaches these step by step.
Back to the topic
This page is part of Machine learning, which maps every concept, tool, simulation and article on the subject.