Model Deployment: From Notebook to Production
The steps between a model that works in a notebook and one people can rely on: choosing batch or online serving, turning the notebook into a reproducible pipeline, a concrete case of training-serving skew and how a saved pipeline prevents it, a small validated prediction API, and what to monitor after launch.
GloryTecks Team
Training & Placement Faculty
Last updated: 25 September 2026
A model that scores well in a notebook has proved that a problem can be solved. It has not yet solved it for anyone. Between the two lie several steps that have little to do with accuracy: making training repeatable, making sure the model sees data in production exactly as it did in training, checking inputs, and noticing when the world changes. This guide walks through those steps with a small credit risk model built on the German credit dataset, 1,000 loan applicants labelled good or bad risks (OpenML: credit-g).
First, decide how predictions will be used
The right deployment depends on when predictions are needed:
| Pattern | How it works | Good for |
|---|---|---|
| Batch scoring | A scheduled job scores many records and writes results to a table | Daily churn scores, weekly demand forecasts, lead ranking |
| Online API | A service returns a prediction per request, in milliseconds | Checkout fraud checks, instant loan decisions, recommendations |
| Embedded | The model runs inside an app or device | Offline mobile features, edge sensors |
Start with batch if the decision can wait. It is simpler to build, test and monitor, and the scores can go straight into dashboards people already use. Batch vs real-time inference compares the two in depth.
Step 1: turn the notebook into a training script
Notebooks encourage running cells out of order and keeping state in memory, and a result that depends on that cannot be reproduced. Before anything else:
- Restart and run all. If the notebook fails or gives different numbers, fix that first.
- Move the steps into functions and a script: load data, build features, train, evaluate, save. The script should run from start to finish with one command.
- Fix random seeds, pin library versions in a requirements file, and record which data snapshot was used.
Step 2: package the whole pipeline, not just the model
The most common deployment bug is preprocessing that exists only in the notebook. Put every transformation inside one pipeline, so scaling and encoding are fitted with the model and saved with it (scikit-learn: ColumnTransformer):
import json
import joblib
import pandas as pd
import sklearn
from sklearn.compose import make_column_transformer
from sklearn.datasets import fetch_openml
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
credit = fetch_openml(data_id=31, as_frame=True).frame
features = ["duration", "credit_amount", "age", "purpose", "housing"]
X, y = credit[features], (credit["class"] == "bad").astype(int)
pipeline = make_pipeline(
make_column_transformer(
(StandardScaler(), ["duration", "credit_amount", "age"]),
(OneHotEncoder(handle_unknown="ignore"), ["purpose", "housing"])),
LogisticRegression(max_iter=1000))
cv_auc = cross_val_score(pipeline, X, y, cv=5, scoring="roc_auc").mean()
pipeline.fit(X, y)
joblib.dump(pipeline, "credit_risk.joblib")
with open("credit_risk.json", "w") as f:
json.dump({"features": features, "scikit_learn": sklearn.__version__, "cv_roc_auc": round(cv_auc, 3)}, f)
print("CV ROC AUC:", round(cv_auc, 3), "| saved with scikit-learn", sklearn.__version__)CV ROC AUC: 0.689 | saved with scikit-learn 1.8.0The model is deliberately simple, using five of the dataset's twenty attributes; the point here is the packaging. Two details matter. The metadata file records the features, the library version and the validation score, because a saved model should be loaded with the same scikit-learn version that saved it, and only from sources you trust, since loading such files can execute code (scikit-learn: model persistence). And handle_unknown="ignore" means a category never seen in training does not crash the service.
A caution specific to this example: it uses applicant age as a feature. In real credit decisions, using attributes such as age can be restricted by law and raises fairness questions, so any real model needs that review before deployment.
Step 3: avoid training-serving skew
Google's Rules of Machine Learning define training-serving skew as a difference between a model's performance in training and in serving, and one of its main causes is handling data differently in the two places. Their advice: re-use code between the training pipeline and the serving pipeline whenever possible (Google: rules of machine learning). Here is why, using the common notebook habit of one-hot encoding with pd.get_dummies:
train_columns = pd.get_dummies(X).columns
one_request = pd.DataFrame([{"duration": 24, "credit_amount": 4000, "age": 35, "purpose": "used car", "housing": "rent"}])
print("columns at training:", len(train_columns), "| columns for one request:", len(pd.get_dummies(one_request).columns))
print("pipeline prediction:", pipeline.predict_proba(one_request)[0, 1].round(3))columns at training: 16 | columns for one request: 5
pipeline prediction: 0.193Encoded in the notebook, the training data had 16 columns: the three numbers plus one column per purpose and housing category. A single incoming application, encoded the same way, has only 5, because it contains one purpose and one housing value. A model trained on 16 columns either rejects the request or, worse, if someone pads the columns by hand in the wrong order, returns a confident wrong answer. The saved pipeline carries its fitted encoder, so it produces the same 16 inputs for one row as for a thousand, and returns a risk of 0.193.
Step 4: serve it behind a validated API
For online use, wrap the saved pipeline in a small web service. FastAPI with Pydantic lets you declare what a valid request looks like, so bad inputs are rejected before they reach the model. FastAPI's TestClient sends requests to the app directly, which makes the service testable without starting a server (FastAPI: testing):
from typing import Literal
from fastapi import FastAPI
from fastapi.testclient import TestClient
from pydantic import BaseModel, Field
model = joblib.load("credit_risk.joblib")
app = FastAPI()
class Application(BaseModel):
duration: int = Field(gt=0, le=120)
credit_amount: float = Field(gt=0)
age: int = Field(ge=18, le=100)
purpose: str
housing: Literal["own", "rent", "for free"]
@app.post("/score")
def score(application: Application):
row = pd.DataFrame([application.model_dump()])
return {"risk": round(float(model.predict_proba(row)[0, 1]), 3), "model_version": "credit_risk-2026-09"}
client = TestClient(app)
ok = client.post("/score", json={"duration": 24, "credit_amount": 4000, "age": 35, "purpose": "used car", "housing": "rent"})
print(ok.status_code, ok.json())
bad = client.post("/score", json={"duration": 24, "credit_amount": 4000, "age": 15, "purpose": "used car", "housing": "castle"})
print(bad.status_code, [(e["loc"][-1], e["msg"]) for e in bad.json()["detail"]])200 {'risk': 0.193, 'model_version': 'credit_risk-2026-09'}
422 [('age', 'Input should be greater than or equal to 18'), ('housing', "Input should be 'own', 'rent' or 'for free'")]The valid request returns the same risk as the pipeline did directly, with the model version, so every prediction can be traced to the model that made it. The invalid one is refused with a 422 status and a message per problem field. Tests like these belong in the project's test suite and run on every change. Packaging this service in a container and running it is the subject of serving ML models with FastAPI and Docker.
Step 5: release carefully
- Version everything: model files, training code, data snapshot and metadata, so any prediction can be reproduced.
- Release gradually. Run the new model alongside the old one without acting on its outputs, or send it a small share of traffic first, and compare. Shadow deployment and canary releases covers both.
- Keep the previous model ready, so rolling back is one configuration change.
Step 6: monitor what the model sees and does
A deployed model degrades silently when the world changes. Watch four things:
- Inputs: distributions of incoming features compared with training data, plus missing values and unseen categories.
- Outputs: the distribution of predicted risks; a sudden shift is often the first sign of trouble.
- Outcomes: once real results arrive, such as whether loans were repaid, recompute the validation metric on recent predictions.
- Service health: latency, error rates and rejected requests.
Rule 29 in Google's guide suggests logging the features used at serving time so they can be used for training, which also makes skew easy to detect. Decide in advance which changes trigger an investigation or retraining.
A deployment checklist
| Question | Done when |
|---|---|
| Can training be reproduced? | One command rebuilds the model from a recorded data snapshot |
| Is preprocessing inside the saved artefact? | A single pipeline file goes from raw inputs to predictions |
| Are inputs validated? | Invalid requests are rejected with clear errors, covered by tests |
| Can predictions be traced? | Every response or score row carries a model version |
| Can you roll back? | The previous model can be restored by configuration |
| Will you notice decay? | Input, output and outcome monitoring with agreed thresholds |
Do I need Docker and Kubernetes to deploy a model?
Should I use pickle or joblib to save models?
How often should a model be retrained?
What is the difference between deployment and MLOps?
Model serialisation with joblib, REST APIs with FastAPI, batch versus real-time inference, model versioning, and monitoring for drift with retraining are all part of the MLOps course.
Sources
- Rules of Machine Learning (Google) — Google for Developers
- ColumnTransformer for heterogeneous data (scikit-learn) — scikit-learn documentation
- Model persistence (scikit-learn) — scikit-learn documentation
- Testing (FastAPI documentation) — FastAPI documentation
- credit-g (OpenML dataset 31) — www.openml.org
GloryTecks Team
Training & Placement Faculty
The GloryTecks faculty team mentors learners across Data Science, AI and Data Engineering, with a focus on real-time projects and placement readiness in Hyderabad.
Want to master Data Science?
Learn MLOps at GloryTecks in Hyderabad, with trainers who have industry experience, real-time projects and placement assistance.
Related articles
Data Scientist vs AI Engineer in 2027: Which Role Fits You?
Data scientists turn data into answers and predictive models; AI engineers build products with foundation models, such as retrieval systems and agents. This comparison covers the daily work, skills, tools, interviews and career paths of both roles in 2027, how certifications reflect the shift, and how to choose.
Data Science Roadmap 2027: Skills, Order and a 12-Month Plan
A data science roadmap for 2027 in four phases over twelve months: Python, pandas 3 and SQL; statistics and experiments; machine learning with honest evaluation; and LLMs, deployment and a portfolio. It covers what changed for data scientists in 2026, the tool versions to install and a month-by-month plan.
Machine Learning Skills That Still Matter in 2027, and What LLMs Changed
Language models changed how we build text applications, but most business prediction problems still run on tables, where classical machine learning is strong. These are the machine learning skills that still matter in 2027, a runnable example of why the right metric matters, and where LLMs now fit into ML work.