Dimensionality Reduction: PCA and t-SNE Explained
Dimensionality reduction with PCA and t-SNE, measured on real 64-dimensional digit images: how much variance PCA keeps per component, what compressing features costs a model's accuracy, why t-SNE preserves neighbourhoods far better in a 2D plot, and the rules for reading t-SNE plots without fooling yourself.
GloryTecks Team
Training & Placement Faculty
Last updated: 25 September 2026
A dataset with dozens or hundreds of columns is hard to visualise and can be slow or noisy to model. Dimensionality reduction maps it into fewer dimensions while keeping as much of the important structure as possible. The two methods most people meet first, PCA and t-SNE, solve different problems: PCA compresses data for modelling and understanding, and t-SNE draws maps of it for looking. Using one for the other's job is a common mistake. Here both are measured on the same data: 1,797 handwritten digit images, each a 64-dimensional vector of pixel values.
PCA: find the directions that matter
Principal component analysis finds new axes, the principal components, along which the data varies most. The first component captures the most variance, the second the most of what is left, at right angles to the first, and so on (scikit-learn: PCA). Keeping only the first few components compresses the data with the least loss of variance.
import numpy as np
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
from sklearn.linear_model import LogisticRegression
from sklearn.manifold import TSNE, trustworthiness
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
X, y = load_digits(return_X_y=True)
X_scaled = StandardScaler().fit_transform(X)
cumulative = np.cumsum(PCA().fit(X_scaled).explained_variance_ratio_)
print({k: round(cumulative[k - 1], 3) for k in (2, 10, 20, 30, 40)})
print("components for 95% of variance:", int(np.argmax(cumulative >= 0.95)) + 1){2: 0.216, 10: 0.589, 20: 0.793, 30: 0.893, 40: 0.951}
components for 95% of variance: 40Two components keep only about a fifth of the variance: a 2D PCA plot of this data shows a lot of overlap. Forty of the 64 components keep 95%. Standardising first matters, because PCA chases variance, and unscaled features with large ranges would dominate the components.
What compression costs a model
Fewer features can mean faster training and less noise, but also less information. Putting PCA inside a pipeline and cross-validating shows the trade-off directly:
for k in (5, 10, 20, 40, None):
steps = [StandardScaler()] + ([PCA(n_components=k)] if k else []) + [LogisticRegression(max_iter=3000)]
accuracy = cross_val_score(make_pipeline(*steps), X, y, cv=5).mean()
print(f"{k if k else 'all 64'} components: accuracy {accuracy:.3f}")5 components: accuracy 0.771
10 components: accuracy 0.840
20 components: accuracy 0.899
40 components: accuracy 0.914
all 64 components: accuracy 0.920Forty components get within a percentage point of using all 64. Twenty cost two more points. Five lose a lot. Choose the number of components from a curve like this, for your model and metric, rather than from a variance threshold alone. Keeping PCA inside the pipeline also means it is fitted only on training folds, which avoids leaking information from the test data.
t-SNE: a map of neighbourhoods
t-SNE does something different. It places points in two or three dimensions so that points that are close in the original space stay close in the map, caring much less about large distances (scikit-learn: t-SNE). The trustworthiness score measures how well each method keeps each point's nearest neighbours as neighbours, from 0 to 1 (scikit-learn: trustworthiness):
pca_2d = PCA(n_components=2).fit_transform(X_scaled)
tsne_2d = TSNE(n_components=2, perplexity=30, init="pca", random_state=0).fit_transform(X_scaled)
print("PCA:", round(trustworthiness(X_scaled, pca_2d, n_neighbors=5), 3),
"| t-SNE:", round(trustworthiness(X_scaled, tsne_2d, n_neighbors=5), 3))
from sklearn.metrics import silhouette_score
print("digit separation (silhouette on true labels) PCA:", round(silhouette_score(pca_2d, y), 3),
"| t-SNE:", round(silhouette_score(tsne_2d, y), 3))PCA: 0.818 | t-SNE: 0.993
digit separation (silhouette on true labels) PCA: 0.055 | t-SNE: 0.485In two dimensions, t-SNE keeps neighbourhoods almost perfectly. The second line measures how well the ten digits separate in each 2D map, using the silhouette score of the true labels: close to zero for PCA, where the digit groups overlap heavily, and far higher for t-SNE, where they form distinct groups. For looking at structure, t-SNE wins decisively.
Reading t-SNE plots without fooling yourself
The scikit-learn documentation lists the caveats, and they matter (scikit-learn: t-SNE):
- Distances between clusters mean little. Two islands far apart are not necessarily more different than two close ones.
- Cluster sizes mean little. t-SNE expands dense regions and contracts sparse ones.
- Results depend on perplexity and randomness. Try several perplexity values and seeds; trust only patterns that persist.
- There is no transform for new data. t-SNE maps the points it was fitted on; it is not a preprocessing step for a model.
So use t-SNE to explore: to see whether classes separate, to spot mislabelled points sitting in the wrong island, or to check whether embeddings group sensibly. Do not measure distances on it, and do not feed its output to a model.
PCA in practice: reading the components
PCA is not only a compression tool; its components can explain your data. Each component is a weighted combination of the original features, and the weights, called loadings, show which features move together. In a customer dataset, a first component with large positive loadings on visits, spend and items per order might be read as overall engagement, and a second contrasting discount use with full-price purchases as price sensitivity. Interpretations like these are hypotheses to check, not facts, but they often reveal redundant features and suggest simpler, more meaningful ones to build.
A workflow for high-dimensional data
- Standardise numeric features.
- Run PCA and look at the cumulative explained variance to see how much redundancy there is.
- Visualise with t-SNE or UMAP to explore groups and outliers, with a few perplexity settings.
- If you compress for a model, put PCA inside the pipeline and choose the number of components by cross-validated performance.
- Report what was lost: the variance dropped and the metric change, as in the table above.
Choosing between them
| Goal | Use |
|---|---|
| Compress features before a model | PCA, inside the pipeline |
| Remove noise and correlated features | PCA |
| Understand which original features drive variation | PCA (inspect component loadings) |
| Visualise clusters or classes in 2D | t-SNE (or UMAP) |
| Apply the same reduction to new data | PCA |
UMAP is a popular alternative to t-SNE for visualisation that is often faster and preserves more of the global layout; it lives in a separate library.
Does PCA work with categorical data?
Should I always reduce dimensions before modelling?
Is dimensionality reduction the same as feature selection?
Is t-SNE a clustering algorithm?
Reduced data is often clustered next; clustering algorithms compared shows how. Dimensionality reduction with PCA is part of the preprocessing and feature engineering module of the Data Science course.
Sources
- Principal component analysis (scikit-learn) — scikit-learn documentation
- t-distributed Stochastic Neighbor Embedding (scikit-learn) — scikit-learn documentation
- sklearn.manifold.trustworthiness — scikit-learn documentation
GloryTecks Team
Training & Placement Faculty
The GloryTecks faculty team mentors learners across Data Science, AI and Data Engineering, with a focus on real-time projects and placement readiness in Hyderabad.
Want to master Data Science?
Learn Data Science at GloryTecks in Hyderabad, with trainers who have industry experience, real-time projects and placement assistance.
Related articles
Data Scientist vs AI Engineer in 2027: Which Role Fits You?
Data scientists turn data into answers and predictive models; AI engineers build products with foundation models, such as retrieval systems and agents. This comparison covers the daily work, skills, tools, interviews and career paths of both roles in 2027, how certifications reflect the shift, and how to choose.
Data Science Roadmap 2027: Skills, Order and a 12-Month Plan
A data science roadmap for 2027 in four phases over twelve months: Python, pandas 3 and SQL; statistics and experiments; machine learning with honest evaluation; and LLMs, deployment and a portfolio. It covers what changed for data scientists in 2026, the tool versions to install and a month-by-month plan.
Machine Learning Skills That Still Matter in 2027, and What LLMs Changed
Language models changed how we build text applications, but most business prediction problems still run on tables, where classical machine learning is strong. These are the machine learning skills that still matter in 2027, a runnable example of why the right metric matters, and where LLMs now fit into ML work.