Regression vs Classification: When to Use Each
Regression or classification? The difference is simple, but choosing the framing is not always obvious. This guide shows how the target and the decision should decide it, then tests both framings on wine quality scores that can be modelled either way, including a yes-or-no question where the framing clearly mattered.
GloryTecks Team
Training & Placement Faculty
Last updated: 25 September 2026
Every supervised machine learning problem starts with a framing decision. Predicting a number, such as a price, a demand or a delivery time, is regression. Predicting a category, such as spam or not, churn or stay, or which product line, is classification. Most algorithms come in both versions; scikit-learn, for example, provides a classifier and a regressor for most of its model families (scikit-learn: supervised learning). The definitions are simple. The judgement lies in the grey areas, where a target could be treated either way, and in matching the framing to the decision the model supports.
The quick rule
| The target is… | Framing | Examples | Typical metrics |
|---|---|---|---|
| A quantity where differences are meaningful | Regression | Price, revenue, temperature, minutes | MAE, RMSE |
| Categories with no order | Classification | Fraud or not, product type, language | Precision, recall, F1, average precision |
| Ordered grades | Either; test both | Ratings of 1 to 5, quality scores, severity levels | Both kinds, chosen by the decision |
| A count | Regression, often with a count-specific loss | Orders per day, visits per hour | MAE, Poisson deviance |
Classification models usually output a probability for each class, and a threshold turns that into a label (Google: classification). Regression models output a number directly.
A target that can go either way
Wine quality is a good test case. The UCI repository's description of the wine quality data says it can be viewed as either a classification or a regression task, and notes that the classes are ordered and unbalanced (UCI: wine quality). Here are the 1,599 red wines, each scored by tasters, with eleven chemical measurements (OpenML: wine-quality-red):
import numpy as np
from sklearn.datasets import fetch_openml
from sklearn.ensemble import HistGradientBoostingClassifier, HistGradientBoostingRegressor
from sklearn.metrics import accuracy_score, average_precision_score, mean_absolute_error, precision_score, recall_score
from sklearn.model_selection import StratifiedKFold, cross_val_predict
wine = fetch_openml(data_id=40691, as_frame=True)
X, quality = wine.data, wine.target.astype(int)
print(len(quality), "wines, quality counts:", quality.value_counts().sort_index().to_dict())1599 wines, quality counts: {3: 10, 4: 53, 5: 681, 6: 638, 7: 199, 8: 18}Most wines score 5 or 6; grades 3, 4 and 8 are rare. Now model it both ways with the same algorithm family, gradient-boosted trees, and score both on the same terms:
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=0)
as_number = cross_val_predict(HistGradientBoostingRegressor(random_state=0), X, quality, cv=cv)
as_rounded = np.clip(np.round(as_number), 3, 8)
as_class = cross_val_predict(HistGradientBoostingClassifier(random_state=0), X, quality, cv=cv)
print(f"always predict 5: MAE {mean_absolute_error(quality, np.full(len(quality), 5)):.3f}, accuracy {(quality == 5).mean():.3f}")
print(f"regression, rounded: MAE {mean_absolute_error(quality, as_rounded):.3f}, accuracy {accuracy_score(quality, as_rounded):.3f}")
print(f"classification (6 grades): MAE {mean_absolute_error(quality, as_class):.3f}, accuracy {accuracy_score(quality, as_class):.3f}")
for grade in (3, 4, 8):
mask = quality == grade
print(f"grade {grade} ({mask.sum()} wines): regression right {(as_rounded[mask] == grade).mean():.0%}, "
f"classification right {(as_class[mask] == grade).mean():.0%}")
print(f"off by two grades or more: regression {(np.abs(quality - as_rounded) >= 2).sum()}, "
f"classification {(np.abs(quality - as_class) >= 2).sum()}")always predict 5: MAE 0.727, accuracy 0.426
regression, rounded: MAE 0.347, accuracy 0.684
classification (6 grades): MAE 0.353, accuracy 0.679
grade 3 (10 wines): regression right 0%, classification right 0%
grade 4 (53 wines): regression right 6%, classification right 0%
grade 8 (18 wines): regression right 0%, classification right 11%
off by two grades or more: regression 48, classification 50On overall accuracy and error, the two framings are effectively tied, and both beat the do-nothing baseline by a wide margin. The differences are in the details. In principle, regression uses the order of the grades, while classification treats 3 and 8 as unrelated labels; here, though, both framings make large mistakes on almost the same number of wines, 48 and 50 off by two grades or more. Both almost never predict the rare extreme grades, the ones a wine buyer might care about most. The honest conclusion for ordered targets: try both, and judge them on the errors that matter to the user of the model.
When the decision is yes or no, model the decision
Suppose the real question is simpler: which wines are good, scoring 7 or more? You could reuse the regression and cut its prediction at 6.5, or train a classifier directly on the yes-or-no target:
good = (quality >= 7).astype(int)
p_good = cross_val_predict(HistGradientBoostingClassifier(random_state=0), X, good, cv=cv, method="predict_proba")[:, 1]
print(f"'good' wines (7 or more): {good.mean():.1%}")
print(f"classifier trained on the decision: average precision {average_precision_score(good, p_good):.3f}, "
f"precision {precision_score(good, p_good >= 0.5):.3f}, recall {recall_score(good, p_good >= 0.5):.3f}")
print(f"regression score, cut at 6.5: average precision {average_precision_score(good, as_number):.3f}, "
f"precision {precision_score(good, as_number >= 6.5):.3f}, recall {recall_score(good, as_number >= 6.5):.3f}")'good' wines (7 or more): 13.6%
classifier trained on the decision: average precision 0.749, precision 0.719, recall 0.590
regression score, cut at 6.5: average precision 0.644, precision 0.684, recall 0.558The dedicated classifier ranks good wines better, with average precision 0.749 against 0.644, and wins on precision and recall at its default threshold too. The regression spent its effort getting the whole scale right, including the difference between a 4 and a 5, which does not matter for this decision. When the business decision is binary, training directly for it usually pays off.
The reverse also holds. Turning a meaningful number into categories, such as splitting delivery time into "late" and "on time", throws away information you may later need. If different users need different cut-offs, predict the number and let each apply its own threshold.
Questions that settle the framing
- What action will the prediction drive? A yes-or-no action suggests classification; a quantity to plan for, such as stock to order, suggests regression.
- Are the differences between values meaningful? If a 4 is closer to a 5 than to a 1, the target has order that regression can use.
- Which errors are costly? Being off by a little on every case, or missing rare but important cases? The answer chooses the metric, and often the framing.
- Is a probability needed? Classifiers give probabilities of events, useful for expected-value decisions and risk scores.
- How much data is there per category? Classes with only a handful of examples, like grade 3 here, will rarely be predicted by either framing without more data.
Common mistakes
- Regressing on arbitrary codes. Categories stored as numbers, such as product type 1, 2 and 3, have no order. Treating them as a quantity invents one.
- Judging a classifier by accuracy alone when one class dominates. Evaluation metrics explained covers what to use instead.
- Rounding regression output without checking the ends. Predictions often cluster in the middle, as they did here, so extreme values are rarely predicted.
- Fixing the framing too early. For ordered targets, a quick test of both framings with cross-validation costs little and removes guesswork.
Is logistic regression a regression or a classification algorithm?
Can the same algorithm do both regression and classification?
What is ordinal regression?
Which metric should I use for each?
For the algorithms that implement both framings, see the top 10 machine learning algorithms. Supervised learning with regression and classification models is part of the machine learning module of the Data Science course.
Sources
- Wine Quality (UCI Machine Learning Repository) — UCI Machine Learning Repository
- wine-quality-red (OpenML dataset 40691) — www.openml.org
- Supervised learning (scikit-learn) — scikit-learn documentation
- Classification (Google Machine Learning Crash Course) — Google for Developers
GloryTecks Team
Training & Placement Faculty
The GloryTecks faculty team mentors learners across Data Science, AI and Data Engineering, with a focus on real-time projects and placement readiness in Hyderabad.
Want to master Data Science?
Learn Data Science at GloryTecks in Hyderabad, with trainers who have industry experience, real-time projects and placement assistance.
Related articles
Data Scientist vs AI Engineer in 2027: Which Role Fits You?
Data scientists turn data into answers and predictive models; AI engineers build products with foundation models, such as retrieval systems and agents. This comparison covers the daily work, skills, tools, interviews and career paths of both roles in 2027, how certifications reflect the shift, and how to choose.
Data Science Roadmap 2027: Skills, Order and a 12-Month Plan
A data science roadmap for 2027 in four phases over twelve months: Python, pandas 3 and SQL; statistics and experiments; machine learning with honest evaluation; and LLMs, deployment and a portfolio. It covers what changed for data scientists in 2026, the tool versions to install and a month-by-month plan.
Machine Learning Skills That Still Matter in 2027, and What LLMs Changed
Language models changed how we build text applications, but most business prediction problems still run on tables, where classical machine learning is strong. These are the machine learning skills that still matter in 2027, a runnable example of why the right metric matters, and where LLMs now fit into ML work.