Python for Humans · Draft v0.1 · Oct 29, 2026 · Draft position: chapter 15 · Download public text and earthquake data, save attributed weather responses, and analyze local copies without fresh requests.

The web in one chapter

Your weather report can get a live answer. Run it later, and the answer may change. Run it without a connection, and it can't fetch an answer at all.

What if you want to check yesterday's result, compare a change to your code, or analyze the same data on a train?

Keep a local copy of the source data. Then make refreshing that copy a separate action from analyzing it.

We'll download a text file and a public CSV dataset, then save the weather response from the previous chapter. These are local Python programs: use the same course folder and uv run commands, with no additional package installation.

Download a File With Python

You downloaded Shakespeare manually in Keeping Your Work. A program can do that job too.

Create download_data.py. This version downloads the complete works from Project Gutenberg into a new data folder, using a separate filename so your existing course text stays in place:

# download_data.py
import json
from datetime import datetime, timezone
from pathlib import Path

import requests


def download_file(url, filename):
    response = requests.get(url, timeout=30)
    response.raise_for_status()

    with open(filename, 'wb') as downloaded:
        downloaded.write(response.content)

    metadata = {
        'source_url': response.url,
        'retrieved_at': datetime.now(timezone.utc).isoformat()
    }
    with open(filename + '.source.json', 'w', encoding='utf-8') as saved:
        json.dump(metadata, saved, indent=2)


def main():
    Path('data').mkdir(exist_ok=True)
    url = 'https://www.gutenberg.org/cache/epub/100/pg100.txt'
    filename = 'data/shakespeare-download.txt'

    try:
        download_file(url, filename)
    except requests.RequestException as error:
        print('Could not download the data:', error)
        return
    print('Saved', filename)


if __name__ == '__main__':
    main()

Run:

uv run download_data.py

Open data/shakespeare-download.txt in your editor and confirm that it contains the book's text and Gutenberg's source and license information. This edition can differ from the course's shakespeare.txt, so don't expect every count to match.

Path('data').mkdir(exist_ok=True) creates the folder, allowing it to exist already. It doesn't create the input or output files itself.

response.content contains the downloaded bytes. Opening a file with 'wb' means write binary data, which lets us save those bytes without first interpreting them as text. We don't supply a text encoding for this binary write. Later, reading the saved UTF-8 text will turn its bytes into a string.

This simple downloader holds the response in memory. It's suitable for the modest files in this chapter; downloading very large datasets needs a different approach.

Keep the Source Beside the Data

The downloader also writes data/shakespeare-download.txt.source.json. Open it and look at the two fields.

source_url records the final response URL, including any redirect. retrieved_at records when we saved the download. The standard-library datetime tools supply the current time in UTC, and .isoformat() turns it into a timestamp string.

We use the standard-library json module to write this dictionary as JSON text. json.dump(metadata, saved, indent=2) writes it to the open file, with indentation for readability. This is different from Requests' response.json(), which decodes an incoming response into Python values.

The source URL and retrieval time help explain a later result. Keep the provider's documentation and reuse terms with your project notes too. For Gutenberg, retain the text's own license information rather than treating the downloaded words as anonymous input.

The downloader checks the HTTP status before opening the destination. A failed request therefore leaves an existing downloaded file in place. A successful refresh replaces the file and its metadata; copy or rename a snapshot first if you want to retain both versions.

Choose a Public Dataset

A text file is one kind of dataset. Another common form is CSV, where rows describe records and columns describe fields.

We'll use the U.S. Geological Survey's recent-earthquake CSV feed. Its past-day feed is a small, regularly updated file:

https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.csv

In download_data.py, change the two values inside main() to:

url = 'https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.csv'
filename = 'data/earthquakes.csv'

Keep them indented inside main(), save, and run the downloader once. You now have data/earthquakes.csv and data/earthquakes.csv.source.json.

A dataset download gives us a file of records. The weather API accepted parameters and assembled a response for our query. Both use HTTP; the useful distinction here is how we request and interpret the returned data.

Before counting anything, read the source's field definitions. Ask:

• What does one row represent, and what time window does the feed cover?
• Which columns answer our question?
• What do their units and missing values mean?
• Can records be updated after their first appearance?

For this feed, time describes an event's time, place supplies a location description, mag gives its magnitude, magType identifies the magnitude method, and type distinguishes earthquakes from other recorded event types. The feed also includes fields such as updated and status that help interpret revisions.

Our question will be narrow: Among the earthquake records in this saved file, how many have a reported magnitude of at least 2.5? This is a count from the saved feed, not a claim that the file captures every earthquake worldwide.

Read Rows as Dictionaries

Create analyze_earthquakes.py:

import csv

with open('data/earthquakes.csv', newline='', encoding='utf-8') as source:
    rows = csv.DictReader(source)
    for row in rows:
        print(row['time'], row['place'], row['mag'])
        break

Run uv run analyze_earthquakes.py. The loop prints the first record, if one exists, then stops.

csv.DictReader uses the header row as dictionary keys. Each subsequent row becomes a dictionary, so row['mag'] looks up that record's magnitude field. The values are strings at this stage.

Some place descriptions contain commas. The CSV reader handles quoted fields, keeping a place such as 28 km NNW of Willow, Alaska together. Splitting each line with .split(',') would break those fields apart. newline='' lets the CSV module handle the file's line endings.

If you skipped the optional spreadsheet extension in Modules, this is all the CSV machinery you need here: open the file, create a reader, then visit its row dictionaries.

Count With a Stated Rule

We need to distinguish three things: an earthquake record, a missing magnitude, and a magnitude that meets our threshold.

Try writing the loop before comparing the version below. Keep it in a function that accepts rows, so we can check it with tiny invented records too.

# analyze_earthquakes.py
import csv


def summarize_earthquakes(rows, minimum):
    total = 0
    missing = 0
    at_or_above = 0

    for row in rows:
        if row['type'] != 'earthquake':
            continue
        total = total + 1
        magnitude_text = row['mag'].strip()
        if magnitude_text == '':
            missing = missing + 1
            continue
        magnitude = float(magnitude_text)
        if magnitude >= minimum:
            at_or_above = at_or_above + 1

    return {
        'earthquakes': total,
        'missing_magnitudes': missing,
        'at_or_above': at_or_above
    }


def main():
    minimum = 2.5
    with open('data/earthquakes.csv', newline='', encoding='utf-8') as source:
        summary = summarize_earthquakes(csv.DictReader(source), minimum)

    print('Earthquake records:', summary['earthquakes'])
    print('Missing magnitudes:', summary['missing_magnitudes'])
    print('Magnitude at least', minimum, ':', summary['at_or_above'])


if __name__ == '__main__':
    main()

This helper has two parameters: rows and minimum. A call supplies the collection of row dictionaries first, then the numeric threshold.

float() converts a numeric string such as '2.67' into a number with a fractional part. We check for an empty field before conversion. An empty field isn't a zero-magnitude event; it means no magnitude was supplied in that field.

We keep missing magnitudes in their own count and exclude them from the threshold count. Negative magnitudes, when present, are valid numeric values and compare normally. Magnitude methods vary, so read magType before trying a scientific comparison beyond this modest report.

The summary function doesn't fetch data, open a file or print. The program's main() connects it to a saved CSV and presents the result.

Try it: Run the analysis twice without running the downloader. The counts should remain the same. Then explicitly refresh the download and analyze again. A changed result can come from changed source records, not necessarily a code change.

Check the Analysis Without the Internet

Create test_earthquakes.py beside the analyzer:

# test_earthquakes.py
from analyze_earthquakes import summarize_earthquakes


def test_threshold_and_missing_values():
    rows = [
        {'type': 'earthquake', 'mag': '3.0'},
        {'type': 'earthquake', 'mag': '1.2'},
        {'type': 'earthquake', 'mag': ''},
        {'type': 'quarry blast', 'mag': '4.0'}
    ]
    assert summarize_earthquakes(rows, 2.5) == {
        'earthquakes': 3,
        'missing_magnitudes': 1,
        'at_or_above': 1
    }


def test_empty_feed():
    assert summarize_earthquakes([], 2.5) == {
        'earthquakes': 0,
        'missing_magnitudes': 0,
        'at_or_above': 0
    }


if __name__ == '__main__':
    test_threshold_and_missing_values()
    test_empty_feed()
    print('Earthquake checks passed.')

Run:

uv run test_earthquakes.py

These invented rows check our calculation, including how we treat missing values and a different event type. They don't assert that today's feed contains a particular number of events.

For an offline CSV example, save the six-row USGS excerpt as data/earthquakes.csv, with its source metadata beside it as data/earthquakes.csv.source.json. The excerpt retains the original columns and values, with its selection described in the metadata. It contains six earthquake records, none with a missing magnitude, and two at or above 2.5.

You can use that example after installing the project dependencies, even if you can't make live requests. A future refresh replaces it with the full current feed.

Save the Weather Response Too

CSV isn't required for every dataset. Our weather API returns nested data naturally represented by JSON.

Keep the weather.py helpers from the previous chapter. Create save_weather.py:

# save_weather.py
import json
from datetime import datetime, timezone
from pathlib import Path

import requests
from weather import fetch_weather


def main():
    parameters = {
        'latitude': 37.7749,
        'longitude': -122.4194,
        'current': 'temperature_2m',
        'temperature_unit': 'celsius',
        'timezone': 'auto'
    }
    try:
        data = fetch_weather(parameters)
    except requests.RequestException as error:
        print('Could not refresh weather:', error)
        return

    snapshot = {
        'source_url': 'https://api.open-meteo.com/v1/forecast',
        'parameters': parameters,
        'retrieved_at': datetime.now(timezone.utc).isoformat(),
        'attribution': 'Weather data: Open-Meteo, CC BY 4.0.',
        'data': data
    }
    Path('data').mkdir(exist_ok=True)
    with open('data/weather.json', 'w', encoding='utf-8') as saved:
        json.dump(snapshot, saved, indent=2)
    print('Saved data/weather.json')


if __name__ == '__main__':
    main()

Run uv run save_weather.py once. The wrapper keeps the source endpoint, request parameters, retrieval time and attribution alongside the returned data. The retrieval time is when we fetched the response; the response's own weather timestamp describes the weather value. They answer different questions.

Now create read_weather.py:

# read_weather.py
import json
from weather import format_weather


def main():
    with open('data/weather.json', encoding='utf-8') as saved:
        snapshot = json.load(saved)

    print('Retrieved:', snapshot['retrieved_at'])
    print(format_weather(snapshot['data']))


if __name__ == '__main__':
    main()

Run uv run read_weather.py several times. json.load() reads the file and reconstructs its dictionaries and values. Importing format_weather doesn't call weather.main(), so it doesn't fetch anything.

For an offline example, save the archived weather snapshot as data/weather.json. It includes the wrapper expected by this reader. The previous chapter's raw example response doesn't include that wrapper, so use this snapshot for this exercise.

The reading program displays a saved answer. To request a newer one, explicitly run save_weather.py, then read it again. Keep the source attribution when sharing a report.

Recognize Different Failures

A connection failure or timeout means we couldn't complete the request. An unsuccessful HTTP status means the server answered but reported a problem. A data-format problem means the content doesn't match what our analysis expects.

For example, a server might return an HTML page where we expected CSV, an API might omit a key, or a magnitude field might contain an unexpected nonnumeric value. A successful HTTP status doesn't prove that our expected columns and values are present.

Inspect the saved file, its first row or relevant keys, and the source documentation. KeyError, ValueError and JSON decoding errors provide different clues. Don't hide them with a broad handler that silently produces zero counts.

Some APIs require an account or API key, and services publish usage limits. Our required examples are keyless. Keep credentials out of shared examples, and read a new source's access instructions before adapting the downloader. Request once per refresh rather than repeatedly downloading inside an analysis loop.

Your Turn

Predict Before Refreshing

You save a weather response, change the report's label, and run only read_weather.py. Which information can change, and which weather values should stay the same? What extra action would make newer weather data available?

Catch a Missing-Value Bug

Temporarily remove the empty-magnitude check from the analyzer. Run test_earthquakes.py and identify the failure. Explain why converting the missing value to zero would make a different claim about the data. Restore the check.

Ask Another Question of the Dataset

Adapt the report to a different magnitude threshold. Add a tiny test that includes an event exactly on that boundary, a missing magnitude and an event below it. Use your saved dataset to compare the reports before downloading anything new.

For a larger extension, count events by a documented field such as magType. State how you treat blank values and why the grouping helps answer your question.

Try Another Public Source

Find a small CSV from an official public-data source. Identify its documentation, column meanings, units, missing-value conventions, update schedule and reuse terms. Save its URL and retrieval time, then write one simple report with a tiny checkable example before using the full file.

Use different filenames for the new dataset. Changing the URL alone doesn't make an earthquake analyzer suitable for unrelated rows.

You can now fetch information from another program, keep its source and timing, and analyze a saved copy. In Classes, we'll build a deck that remembers its remaining cards and use it in a small card game.