Teach Python to Spot Spam

Text Classification

You should already know
Notebook cells and the training/test split from Train Your First Model.
You'll build
A model that labels short messages as ordinary or spam.
You'll be able to
Turn text into numeric features, fit a pipeline and compare precision with recall.
You need
Your local setup, any plain-text editor, uv and internet access for installing packages. Python 3.14 is selected for these examples.

Our word counter gives the same answer whenever it sees the same text. Could a program learn to recognize a category of message from examples instead of a hand-written list of suspicious words?

We'll train a small classifier using the public SMS Spam Collection. This is a local text-classification experiment, not a system that reads your inbox or sends messages.

Open the Training Notebook

mkdir spam-model
cd spam-model
uv init --bare --vcs none --no-workspace --python 3.14
uv python pin 3.14
uv add jupyterlab pandas scikit-learn

Save spam.ipynb, sms.tsv, the source metadata and the original readme together. Run uv run jupyter lab and open the notebook.

The dataset is credited to Almeida and Hidalgo (2011), UCI Machine Learning Repository, under CC BY 4.0. Each row contains a label, a tab and a message. ham means an ordinary message; spam means an unwanted promotional or deceptive one according to this dataset's labels.

Count the Labels Before Modeling

The notebook reads the table, removes duplicate message text and prints the number of examples in each class. Removing duplicates before splitting prevents an identical message from appearing in both training and test sets.

The labels are unevenly distributed. An always-ham classifier can therefore achieve a surprisingly high accuracy. Predict why before looking at its baseline score.

As with Titanic, the split happens before fitting the preprocessing steps. Known test labels are used for evaluation, not passed to the model while it predicts.

Turn Words Into Inputs

Start with three invented messages in the notebook, separate from the SMS training data:

practice_messages = ['free pizza', 'meet for pizza', 'free prize']
practice_vectorizer = TfidfVectorizer()
practice_vectors = practice_vectorizer.fit_transform(practice_messages)
pd.DataFrame(practice_vectors.toarray(), columns=practice_vectorizer.get_feature_names_out())

Before running, list the words shared by the first and last messages. Each learned word becomes a column in the displayed table. Nonzero values indicate words present in that message. The values are weights rather than simple counts. .toarray() makes this tiny representation displayable as a regular table; the full model can keep a compact representation internally.

Next, the notebook transforms free telescope with this same practice vectorizer. Predict whether telescope gets a column. It doesn't: .transform() uses the vocabulary already learned by .fit_transform(). Only free has a nonzero feature.

Checkpoint: Explain which object remembers the vocabulary. Keep this practice object separate; the actual experiment must learn its word features from X_train alone.

Now connect a fresh vectorizer and classifier. The complete modeling pipeline is short:

model = make_pipeline(TfidfVectorizer(), MultinomialNB())
model.fit(X_train, y_train)
predicted = model.predict(X_test)

A vectorizer turns text into numeric features. This one learns a vocabulary from training messages and weights words using TF-IDF, which considers their frequency within messages and across the training collection.

MultinomialNB is a Naive Bayes classifier suited to this kind of nonnegative text feature. It learns associations between the features and the supplied labels. The pipeline fits both stages only on training data, then applies them to the held-out messages.

This model classifies messages. It doesn't generate replies or establish a sender's intention. Words missing from its training vocabulary can't become new learned features during prediction.

Accuracy Isn't the Whole Result

The notebook prints a classification report. Before interpreting its decimal scores, try a small example: suppose we call 10 messages spam, eight of those really are spam, and there are 20 actual spam messages in all. Precision is 8/10 (0.8); recall is 8/20 (0.4). Two ordinary messages were wrongly flagged, and 12 spam messages were missed.

For the spam class:

A false positive labels an ordinary message as spam. A false negative lets spam through. An inbox that hides important messages and one that misses junk have different costs, so a single accuracy number isn't enough.

Read several wrong predictions from the notebook's final table. The corpus contains historical messages; its test score doesn't establish how the classifier would perform on today's messages from another source.

Try Messages You Invent

The notebook includes two sample messages. Add five of your own, writing down expected labels before running the prediction cell. Try a legitimate message that contains a word such as “free,” then a spam-like message that avoids it.

These invented messages help you investigate behavior. They aren't a representative new evaluation set. Keep the held-out report alongside them rather than choosing only flattering examples.

Restart and run all cells before sharing the notebook. Use invented messages rather than pasting personal conversations into an artifact you plan to publish.

Your Turn

Find one false positive and one false negative, if the test results contain both. Explain which error would matter more for the product you imagine building.

An optional extension is comparing another classifier using validation data. Another direction is finding similar documents with a pretrained embedding model: that uses an existing model rather than training this small classifier. Model downloads and new dependencies make it a separate experiment.

In Your Own Words

Explain what the vectorizer learns, what the classifier learns and why accuracy alone can be misleading when labels are imbalanced.