feat: overhaul for relaunch

- Create `requirements.txt` with pinned dependencies for building site.
- Update .gitignore.
- Enhance Makefile.
- Rename existing lessons to be lesson/dd_specific.py (two digits, lower case).
- Notebooks with other names are not included in index page.
- Add SQL tutorial.
- Add scripts to regenerate SQLite databases.
- Add queueing theory tutorial.
- Rename `scripts` directory to `bin`.
- Replace old `build.py` with:
- `bin/extract.py` gets metadata from `*/index.md` lesson pages.
- `bin/build.py` builds root home page and lesson home pages.
- `bin/check_empty_cells.py`: look for empty cells in notebooks (enhanced).
- `bin/check_missing_titles.py`: look for notebooks without an H1 title.
- `bin/check_notebook_packages.py`: check consistency of package versions within lesson.
- Add `make check_packages NOTEBOOKS="*/??_*.py"` to check package consistency within lesson.
- If `NOTEBOOKS` not specified, all notebooks are checked.
- Add `make check_exec NOTEBOOKS="*/??_*.py"` to check notebook execution.
- If `NOTEBOOKS` not specified, all notebooks are executed (slow).
- Fix missing package imports in notebook headers.
- Pin package versions in notebook headers.
- Make content of lesson home pages uniform.
- Update GitHub workflows to launch commands from Makefile.
- Requires using `uv` in workflows.
- Extract and modify CSS.
- Putt SVG icons in includeable files in `templates/icons/*.svg`.
- Make titles of notebooks more uniform.
- Building `pages/*.md` using `templates/page.html`.
- Add link checker.
- Requires local server to be running, and takes 10 minutes or more to execute.
- Fix multiple bugs in individual lessons.
- Most introduced by package version pinning.
- See notes below for outstanding issues.

Note: build [`marimo_learn`](https://github.com/gvwilson/marimo_learn) package
with utilities to localize SQLite database files.

Add `disabled=True` to prevent execution of deliberately buggy cells in script mode (?).

The code at line 497–499 calls `lz.sink_csv(..., lazy=True)`. The
`lazy=True` argument was added to return a lazy sink that could be
passed to pl.collect_all() for parallel execution — rather than
immediately writing the file. However, in polars 1.24.0, the lazy
parameter was removed from sink_csv() (and likely sink_parquet(),
sink_ndjson() too). The API for collecting multiple sinks in parallel
has changed.

These notebook use the `hf://` protocol to stream a parquet file
directly from Hugging Face:

```
URL = f"hf://datasets/{repo_id}@{branch}/{file_path}"
```

Polars is URL-encoding the slash in the repo name when it calls the HF
API, which then rejects it as an invalid repo name. The fix is to
download the file and store it locally, or make it available in some
other location.

Kagglehub requires Kaggle API credentials not available in the
browser. Either remove the data-loading step or substitute a bundled
sample dataset.

Replace numba with a pure-Python alternative for the WASM version, or
gate the numba cells with a WASM check and change prose accordingly:

```
import sys
if "pyodide" not in sys.modules:
import numba
```

- Add Altair notebooks from https://uwdata.github.io/visualization-curriculum/
- Add formative assessment widgets
- Update Little's Law notebook

.github/workflows/check-empty-cells.yml

@@ -17,6 +17,9 @@ jobs:
       - name: 🔄 Checkout code
         uses: actions/checkout@v4
 
+      - name: 🚀 Install uv
+        uses: astral-sh/setup-uv@v4
+
       - name: 🐍 Set up Python
         uses: actions/setup-python@v5
         with:
@@ -24,7 +27,7 @@ jobs:
 
       - name: 🔍 Check for empty cells
         run: |
-          python scripts/check_empty_cells.py
+          make check_empty
 
       - name: 📊 Report results
         if: failure()

.github/workflows/deploy.yml

@@ -32,7 +32,7 @@ jobs:
 
       - name: 🛠️ Export notebooks
         run: |
-          python scripts/build.py
+          make build
 
       - name: 📤 Upload artifact
         uses: actions/upload-pages-artifact@v3

.gitignore

@@ -175,3 +175,12 @@ __marimo__
 
 # Generated site content
 _site/
+
+# Editors
+*~
+
+# Temporary build files
+tmp/
+example.db
+example.db.wal
+log_data_filtered*.*

.typos.toml

@@ -15,7 +15,10 @@ extend-ignore-re = [
 
 # Words to explicitly accept
 [default.extend-words]
+bimap = "bimap"
 pn = "pn"
+setp = "setp"
+Plas = "Plas"
 
 # You can also exclude specific files or directories if needed
 # [files]

Makefile

@@ -1,24 +1,116 @@
-# Default target.
-all : commands
+ROOT := .
+SITE := _site
+TMP := ./tmp
+LESSON_DATA := ${TMP}/lessons.json
+TEMPLATES := $(wildcard templates/*.html)
 
-## commands : show all commands.
-commands :
+NOTEBOOK_INDEX := $(wildcard */index.md)
+NOTEBOOK_DIR := $(patsubst %/index.md,%,${NOTEBOOK_INDEX})
+NOTEBOOK_SRC := $(foreach dir,$(NOTEBOOK_DIR),$(wildcard $(dir)/??_*.py))
+NOTEBOOK_OUT := $(patsubst %.py,${SITE}/%.html,$(NOTEBOOK_SRC))
+
+DATABASES := \
+sql/public/lab.db \
+sql/public/penguins.db \
+sql/public/survey.db
+
+MARIMO := uv run marimo
+PYTHON := uv run python
+
+# Default target
+all: commands
+
+## commands : show all commands
+commands:
 	@grep -h -E '^##' ${MAKEFILE_LIST} | sed -e 's/## //g' | column -t -s ':'
 
-## install: install minimal required packages into current environment.
+## install: install required packages
 install:
-	uv pip install marimo jinja2 markdown
+	uv pip install -r requirements.txt
+
+## check: run all simple checks
+check:
+	-@make check_empty
+	-@make check_titles
+	-@make check_typos
+	-@make check_packages
 
-## build: build entire site.
-build:
-	rm -rf _site
-	uv run scripts/build.py
+## check_exec: run notebooks to check for runtime errors
+check_exec:
+	@if [ -z "$(NOTEBOOKS)" ]; then \
+		bash bin/run_notebooks.sh $(NOTEBOOK_SRC); \
+	else \
+		bash bin/run_notebooks.sh $(NOTEBOOKS); \
+	fi
 
-## serve: run local web server without rebuilding.
+## build: build website
+build: ${LESSON_DATA} ${NOTEBOOK_OUT} ${TEMPLATES}
+	${PYTHON} bin/build.py --root ${ROOT} --output ${SITE} --data ${LESSON_DATA}
+
+## links: check links locally (while 'make serve')
+links:
+	linkchecker -F text http://localhost:8000
+
+## serve: run local web server without rebuilding
 serve:
-	uv run python -m http.server --directory _site
+	${PYTHON} -m http.server --directory ${SITE}
+
+## databases: rebuild datasets for SQL lessons
+databases: ${DATABASES}
 
-## clean: clean up stray files.
+## ---: ---
+
+## clean: clean up stray files
 clean:
 	@find . -name '*~' -exec rm {} +
 	@find . -name '.DS_Store' -exec rm {} +
+	@rm -rf ${TMP}
+	@rm -f log_data_filtered*.*
+
+## check_empty: check for empty cells
+check_empty:
+	@${PYTHON} bin/check_empty_cells.py
+
+## check_titles: check for missing titles in notebooks
+check_titles:
+	@${PYTHON} bin/check_missing_titles.py
+
+## check_packages: check for inconsistent package versions across notebooks
+check_packages:
+	@if [ -z "$(NOTEBOOKS)" ]; then \
+		${PYTHON} bin/check_notebook_packages.py $(NOTEBOOK_SRC); \
+	else \
+		${PYTHON} bin/check_notebook_packages.py $(NOTEBOOKS); \
+	fi
+
+## check_typos: check for typos
+check_typos:
+	@typos ${TEMPLATES} ${NOTEBOOK_INDEX} ${NOTEBOOK_SRC}
+
+## extract: extract lesson data
+extract: ${LESSON_DATA}
+
+#
+# subsidiary targets
+#
+
+tmp/lessons.json: $(NOTEBOOK_INDEX)
+	${PYTHON} bin/extract.py --root ${ROOT} --data ${LESSON_DATA}
+
+${SITE}/%.html: %.py
+	${MARIMO} export html-wasm --force --mode edit $< -o $@ --sandbox
+
+sql/public/lab.db: bin/create_sql_lab.sql
+	@rm -f $@
+	@mkdir -p sql/public
+	sqlite3 $@ < $<
+
+sql/public/penguins.db: bin/create_sql_penguins.py data/penguins.csv
+	@rm -f $@
+	@mkdir -p sql/public
+	${PYTHON} $< data/penguins.csv $@
+
+sql/public/survey.db: bin/create_sql_survey.py
+	@rm -f $@
+	@mkdir -p sql/public
+	${PYTHON} $< $@ 192837

_server/README.md

@@ -1,8 +1,3 @@
----
-title: Readme
-marimo-version: 0.18.4
----
-
 # marimo learn server
 
 This folder contains server code for hosting marimo apps.

_server/main.py

@@ -6,14 +6,14 @@
 #     "starlette",
 #     "python-dotenv",
 #     "pydantic",
-#     "duckdb==1.3.2",
-#     "altair==5.5.0",
+#     "duckdb==1.4.4",
+#     "altair==6.0.0",
 #     "beautifulsoup4==4.13.3",
 #     "httpx==0.28.1",
 #     "marimo",
 #     "nest-asyncio==1.6.0",
 #     "numba==0.61.0",
-#     "numpy==2.1.3",
+#     "numpy==2.4.3",
 #     "polars==1.24.0",
 # ]
 # ///

altair/01_introduction.py

@@ -0,0 +1,671 @@
+# /// script
+# requires-python = ">=3.11"
+# dependencies = [
+#     "altair==6.0.0",
+#     "marimo",
+#     "pandas==3.0.1",
+#     "vega_datasets==0.9.0",
+# ]
+# ///
+
+import marimo
+
+__generated_with = "0.20.4"
+app = marimo.App()
+
+
+@app.cell
+def _():
+    import marimo as mo
+
+    return (mo,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    # Introduction to Altair
+
+    [Altair](https://altair-viz.github.io/) is a declarative statistical visualization library for Python. Altair offers a powerful and concise visualization grammar for quickly building a wide range of statistical graphics.
+
+    By *declarative*, we mean that you can provide a high-level specification of *what* you want the visualization to include, in terms of *data*, *graphical marks*, and *encoding channels*, rather than having to specify *how* to implement the visualization in terms of for-loops, low-level drawing commands, *etc*. The key idea is that you declare links between data fields and visual encoding channels, such as the x-axis, y-axis, color, *etc*. The rest of the plot details are handled automatically. Building on this declarative plotting idea, a surprising range of simple to sophisticated visualizations can be created using a concise grammar.
+
+    Altair is based on [Vega-Lite](https://vega.github.io/vega-lite/), a high-level grammar of interactive graphics. Altair provides a friendly Python [API (Application Programming Interface)](https://en.wikipedia.org/wiki/Application_programming_interface) that generates Vega-Lite specifications in [JSON (JavaScript Object Notation)](https://en.wikipedia.org/wiki/JSON) format. Environments such as Jupyter Notebooks, JupyterLab, and Colab can then take this specification and render it directly in the web browser. To learn more about the motivation and basic concepts behind Altair and Vega-Lite, watch the [Vega-Lite presentation video from OpenVisConf 2017](https://www.youtube.com/watch?v=9uaHRWj04D4).
+
+    This notebook will guide you through the basic process of creating visualizations in Altair. First, you will need to make sure you have the Altair package and its dependencies installed (for more, see the [Altair installation documentation](https://altair-viz.github.io/getting_started/installation.html)), or you are using a notebook environment that includes the dependencies pre-installed.
+
+    _This notebook is part of the [data visualization curriculum](https://github.com/uwdata/visualization-curriculum)._
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Imports
+
+    To start, we must import the necessary libraries: Pandas for data frames and Altair for visualization.
+    """)
+    return
+
+
+@app.cell
+def _():
+    import pandas as pd
+    import altair as alt
+
+    return alt, pd
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Renderers
+
+    Depending on your environment, you may need to specify a [renderer](https://altair-viz.github.io/user_guide/display_frontends.html) for Altair. If you are using __JupyterLab__, __Jupyter Notebook__, or __Google Colab__ with a live Internet connection you should not need to do anything. Otherwise, please read the documentation for [Displaying Altair Charts](https://altair-viz.github.io/user_guide/display_frontends.html).
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Data
+
+    Data in Altair is built around the Pandas data frame, which consists of a set of named data *columns*. We will also regularly refer to data columns as data *fields*.
+
+    When using Altair, datasets are commonly provided as data frames. Alternatively, Altair can also accept a URL to load a network-accessible dataset. As we will see, the named columns of the data frame are an essential piece of plotting with Altair.
+
+    We will often use datasets from the [vega-datasets](https://github.com/vega/vega-datasets) repository. Some of these datasets are directly available as Pandas data frames:
+    """)
+    return
+
+
+@app.cell
+def _():
+    from vega_datasets import data  # import vega_datasets
+    cars = data.cars()              # load cars data as a Pandas data frame
+    cars.head()                     # display the first five rows
+    return cars, data
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Datasets in the vega-datasets collection can also be accessed via URLs:
+    """)
+    return
+
+
+@app.cell
+def _(data):
+    data.cars.url
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Dataset URLs can be passed directly to Altair (for supported formats like JSON and [CSV](https://en.wikipedia.org/wiki/Comma-separated_values)), or loaded into a Pandas data frame like so:
+    """)
+    return
+
+
+@app.cell
+def _(data, pd):
+    pd.read_json(data.cars.url).head() # load JSON data into a data frame
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    For more information about data frames - and some useful transformations to prepare Pandas data frames for plotting with Altair! - see the [Specifying Data with Altair documentation](https://altair-viz.github.io/user_guide/data.html).
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Weather Data
+
+    Statistical visualization in Altair begins with ["tidy"](http://vita.had.co.nz/papers/tidy-data.html) data frames. Here, we'll start by creating a simple data frame (`df`) containing the average precipitation (`precip`) for a given `city` and `month` :
+    """)
+    return
+
+
+@app.cell
+def _(pd):
+    df = pd.DataFrame({
+        'city': ['Seattle', 'Seattle', 'Seattle', 'New York', 'New York', 'New York', 'Chicago', 'Chicago', 'Chicago'],
+        'month': ['Apr', 'Aug', 'Dec', 'Apr', 'Aug', 'Dec', 'Apr', 'Aug', 'Dec'],
+        'precip': [2.68, 0.87, 5.31, 3.94, 4.13, 3.58, 3.62, 3.98, 2.56]
+    })
+
+    df
+    return (df,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## The Chart Object
+
+    The fundamental object in Altair is the `Chart`, which takes a data frame as a single argument:
+    """)
+    return
+
+
+@app.cell
+def _(alt, df):
+    _chart = alt.Chart(df)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    So far, we have defined the `Chart` object and passed it the simple data frame we generated above. We have not yet told the chart to *do* anything with the data.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Marks and Encodings
+
+    With a chart object in hand, we can now specify how we would like the data to be visualized. We first indicate what kind of graphical *mark* (geometric shape) we want to use to represent the data. We can set the `mark` attribute of the chart object using the the `Chart.mark_*` methods.
+
+    For example, we can show the data as a point using `Chart.mark_point()`:
+    """)
+    return
+
+
+@app.cell
+def _(alt, df):
+    alt.Chart(df).mark_point()
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Here the rendering consists of one point per row in the dataset, all plotted on top of each other, since we have not yet specified positions for these points.
+
+    To visually separate the points, we can map various *encoding channels*, or *channels* for short, to fields in the dataset. For example, we could *encode* the field `city` of the data using the `y` channel, which represents the y-axis position of the points. To specify this, use the `encode` method:
+    """)
+    return
+
+
+@app.cell
+def _(alt, df):
+    alt.Chart(df).mark_point().encode(
+      y='city',
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The `encode()` method builds a key-value mapping between encoding channels (such as `x`, `y`, `color`, `shape`, `size`, *etc.*) to fields in the dataset, accessed by field name. For Pandas data frames, Altair automatically determines an appropriate data type for the mapped column, which in this case is the *nominal* type, indicating unordered, categorical values.
+
+    Though we've now separated the data by one attribute, we still have multiple points overlapping within each category. Let's further separate these by adding an `x` encoding channel, mapped to the `'precip'` field:
+    """)
+    return
+
+
+@app.cell
+def _(alt, df):
+    alt.Chart(df).mark_point().encode(
+        x='precip',
+        y='city'
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Seattle exhibits both the least-rainiest and most-rainiest months!_
+
+    The data type of the `'precip'` field is again automatically inferred by Altair, and this time is treated as a *quantitative* type (that is, a real-valued number). We see that grid lines and appropriate axis titles are automatically added as well.
+
+    Above we have specified key-value pairs using keyword arguments (`x='precip'`). In addition, Altair provides construction methods for encoding definitions, using the syntax `alt.X('precip')`. This alternative is useful for providing more parameters to an encoding, as we will see later in this notebook.
+    """)
+    return
+
+
+@app.cell
+def _(alt, df):
+    alt.Chart(df).mark_point().encode(
+        alt.X('precip'),
+        alt.Y('city')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The two styles of specifying encodings can be interleaved: `x='precip', alt.Y('city')` is also a valid input to the `encode` function.
+
+    In the examples above, the data type for each field is inferred automatically based on its type within the Pandas data frame. We can also explicitly indicate the data type to Altair by annotating the field name:
+
+    - `'b:N'` indicates a *nominal* type (unordered, categorical data),
+    - `'b:O'` indicates an *ordinal* type (rank-ordered data),
+    - `'b:Q'` indicates a *quantitative* type (numerical data with meaningful magnitudes), and
+    - `'b:T'` indicates a *temporal* type (date/time data)
+
+    For example, `alt.X('precip:N')`.
+
+    Explicit annotation of data types is necessary when data is loaded from an external URL directly by Vega-Lite (skipping Pandas entirely), or when we wish to use a type that differs from the type that was automatically inferred.
+
+    What do you think will happen to our chart above if we treat `precip` as a nominal or ordinal variable, rather than a quantitative variable? _Modify the code above and find out!_
+
+    We will take a closer look at data types and encoding channels in the next notebook of the [data visualization curriculum](https://github.com/uwdata/visualization-curriculum#data-visualization-curriculum).
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Data Transformation: Aggregation
+
+    To allow for more flexibility in how data are visualized, Altair has a built-in syntax for *aggregation* of data. For example, we can compute the average of all values by specifying an aggregation function along with the field name:
+    """)
+    return
+
+
+@app.cell
+def _(alt, df):
+    alt.Chart(df).mark_point().encode(
+        x='average(precip)',
+        y='city'
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Now within each x-axis category, we see a single point reflecting the *average* of the values within that category.
+
+    _Does Seattle really have the lowest average precipitation of these cities? (It does!) Still, how might this plot mislead? Which months are included? What counts as precipitation?_
+
+    Altair supports a variety of aggregation functions, including `count`, `min` (minimum), `max` (maximum), `average`, `median`, and `stdev` (standard deviation). In a later notebook, we will take a tour of data transformations, including aggregation, sorting, filtering, and creation of new derived fields using calculation formulas.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Changing the Mark Type
+
+    Let's say we want to represent our aggregated values using rectangular bars rather than circular points. We can do this by replacing `Chart.mark_point` with `Chart.mark_bar`:
+    """)
+    return
+
+
+@app.cell
+def _(alt, df):
+    alt.Chart(df).mark_bar().encode(
+        x='average(precip)',
+        y='city'
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Because the nominal field `a` is mapped to the `y`-axis, the result is a horizontal bar chart. To get a vertical bar chart, we can simply swap the `x` and `y` keywords:
+    """)
+    return
+
+
+@app.cell
+def _(alt, df):
+    alt.Chart(df).mark_bar().encode(
+        x='city',
+        y='average(precip)'
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Customizing a Visualization
+
+    By default Altair / Vega-Lite make some choices about properties of the visualization, but these can be changed using methods to customize the look of the visualization. For example, we can specify the axis titles using the `axis` attribute of channel classes, we can modify scale properties using the `scale` attribute, and we can specify the color of the marking by setting the `color` keyword of the `Chart.mark_*` methods to any valid [CSS color string](https://developer.mozilla.org/en-US/docs/Web/CSS/color_value):
+    """)
+    return
+
+
+@app.cell
+def _(alt, df):
+    alt.Chart(df).mark_point(color='firebrick').encode(
+      alt.X('precip', scale=alt.Scale(type='log'), axis=alt.Axis(title='Log-Scaled Values')),
+      alt.Y('city', axis=alt.Axis(title='Category')),
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    A subsequent module will explore the various options available for scales, axes, and legends to create customized charts.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Multiple Views
+
+    As we've seen above, the Altair `Chart` object represents a plot with a single mark type. What about more complicated diagrams, involving multiple charts or layers? Using a set of *view composition* operators, Altair can take multiple chart definitions and combine them to create more complex views.
+
+    As a starting point, let's plot the cars dataset in a line chart showing the average mileage by the year of manufacture:
+    """)
+    return
+
+
+@app.cell
+def _(alt, cars):
+    alt.Chart(cars).mark_line().encode(
+        alt.X('Year'),
+        alt.Y('average(Miles_per_Gallon)')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    To augment this plot, we might like to add `circle` marks for each averaged data point. (The `circle` mark is just a convenient shorthand for `point` marks that used filled circles.)
+
+    We can start by defining each chart separately: first a line plot, then a scatter plot. We can then use the `layer` operator to combine the two into a layered chart. Here we use the shorthand `+` (plus) operator to invoke layering:
+    """)
+    return
+
+
+@app.cell
+def _(alt, cars):
+    line = alt.Chart(cars).mark_line().encode(
+        alt.X('Year'),
+        alt.Y('average(Miles_per_Gallon)')
+    )
+
+    point = alt.Chart(cars).mark_circle().encode(
+        alt.X('Year'),
+        alt.Y('average(Miles_per_Gallon)')
+    )
+
+    line + point
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    We can also create this chart by *reusing* and *modifying* a previous chart definition! Rather than completely re-write a chart, we can start with the line chart, then invoke the `mark_point` method to generate a new chart definition with a different mark type:
+    """)
+    return
+
+
+@app.cell
+def _(alt, cars):
+    mpg = alt.Chart(cars).mark_line().encode(
+        alt.X('Year'),
+        alt.Y('average(Miles_per_Gallon)')
+    )
+
+    mpg + mpg.mark_circle()
+    return (mpg,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    <em>(The need to place points on lines is so common, the `line` mark also includes a shorthand to generate a new layer for you. Trying adding the argument `point=True` to the `mark_line` method!)</em>
+
+    Now, what if we'd like to see this chart alongside other plots, such as the average horsepower over time?
+
+    We can use *concatenation* operators to place multiple charts side-by-side, either vertically or horizontally. Here, we'll use the `|` (pipe) operator to perform horizontal concatenation of two charts:
+    """)
+    return
+
+
+@app.cell
+def _(alt, cars, mpg):
+    hp = alt.Chart(cars).mark_line().encode(
+        alt.X('Year'),
+        alt.Y('average(Horsepower)')
+    )
+
+    (mpg + mpg.mark_circle()) | (hp + hp.mark_circle())
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _We can see that, in this dataset, over the 1970s and early '80s the average fuel efficiency improved while the average horsepower decreased._
+
+    A later notebook will focus on *view composition*, including not only layering and concatenation, but also the `facet` operator for splitting data into sub-plots and the `repeat` operator to concisely generate concatenated charts from a template.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Interactivity
+
+    In addition to basic plotting and view composition, one of Altair and Vega-Lite's most exciting features is its support for interaction.
+
+    To create a simple interactive plot that supports panning and zooming, we can invoke the `interactive()` method of the `Chart` object. In the chart below, click and drag to *pan* or use the scroll wheel to *zoom*:
+    """)
+    return
+
+
+@app.cell
+def _(alt, cars):
+    alt.Chart(cars).mark_point().encode(
+        x='Horsepower',
+        y='Miles_per_Gallon',
+        color='Origin',
+    ).interactive()
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    To provide more details upon mouse hover, we can use the `tooltip` encoding channel:
+    """)
+    return
+
+
+@app.cell
+def _(alt, cars):
+    alt.Chart(cars).mark_point().encode(
+        x='Horsepower',
+        y='Miles_per_Gallon',
+        color='Origin',
+        tooltip=['Name', 'Origin'] # show Name and Origin in a tooltip
+    ).interactive()
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    For more complex interactions, such as linked charts and cross-filtering, Altair provides a *selection* abstraction for defining interactive selections and then binding them to components of a chart. We will cover this is in detail in a later notebook.
+
+    Below is a more complex example. The upper histogram shows the count of cars per year and  uses an interactive selection to modify the opacity of points in the lower scatter plot, which shows horsepower versus mileage.
+
+    _Drag out an interval in the upper chart and see how it affects the points in the lower chart. As you examine the code, **don't worry if parts don't make sense yet!** This is an aspirational example, and we will fill in all the needed details over the course of the different notebooks._
+    """)
+    return
+
+
+@app.cell
+def _(alt, cars):
+    # create an interval selection over an x-axis encoding
+    brush = alt.selection_interval(encodings=['x'])
+
+    # determine opacity based on brush
+    opacity = alt.condition(brush, alt.value(0.9), alt.value(0.1))
+
+    # an overview histogram of cars per year
+    # add the interval brush to select cars over time
+    overview = alt.Chart(cars).mark_bar().encode(
+        alt.X('Year:O', timeUnit='year', # extract year unit, treat as ordinal
+          axis=alt.Axis(title=None, labelAngle=0) # no title, no label angle
+        ),
+        alt.Y('count()', title=None), # counts, no axis title
+        opacity=opacity
+    ).add_params(
+        brush      # add interval brush selection to the chart
+    ).properties(
+        width=400, # set the chart width to 400 pixels
+        height=50  # set the chart height to 50 pixels
+    )
+
+    # a detail scatterplot of horsepower vs. mileage
+    # modulate point opacity based on the brush selection
+    detail = alt.Chart(cars).mark_point().encode(
+        alt.X('Horsepower'),
+        alt.Y('Miles_per_Gallon'),
+        # set opacity based on brush selection
+        opacity=opacity
+    ).properties(width=400) # set chart width to match the first chart
+
+    # vertically concatenate (vconcat) charts using the '&' operator
+    overview & detail
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Aside: Examining the JSON Output
+
+    As a Python API to Vega-Lite, Altair's main purpose is to convert plot specifications to a JSON string that conforms to the Vega-Lite schema. Using the `Chart.to_json` method, we can inspect the JSON specification that Altair is exporting and sending to Vega-Lite:
+    """)
+    return
+
+
+@app.cell
+def _(alt, df):
+    _chart = alt.Chart(df).mark_bar().encode(x='average(precip)', y='city')
+    print(_chart.to_json())
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Notice here that `encode(x='average(precip)')` has been expanded to a JSON structure with a `field` name, a `type` for the data, and includes an `aggregate` field. The `encode(y='city')` statement has been expanded similarly.
+
+    As we saw earlier, Altair's shorthand syntax includes a way to specify the type of the field as well:
+    """)
+    return
+
+
+@app.cell
+def _(alt):
+    _x = alt.X('average(precip):Q')
+    print(_x.to_json())
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    This short-hand is equivalent to spelling-out the attributes by name:
+    """)
+    return
+
+
+@app.cell
+def _(alt):
+    _x = alt.X(aggregate='average', field='precip', type='quantitative')
+    print(_x.to_json())
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Publishing a Visualization
+
+    Once you have visualized your data, perhaps you would like to publish it somewhere on the web. This can be done straightforwardly using the [vega-embed JavaScript package](https://github.com/vega/vega-embed). A simple example of a stand-alone HTML document can be generated for any chart using the `Chart.save` method:
+
+    ```python
+    chart = alt.Chart(df).mark_bar().encode(
+        x='average(precip)',
+        y='city',
+    )
+    chart.save('chart.html')
+    ```
+
+
+    The basic HTML template produces output that looks like this, where the JSON specification for your plot produced by `Chart.to_json` should be stored in the `spec` JavaScript variable:
+
+    ```html
+    <!DOCTYPE html>
+    <html>
+      <head>
+        <script src="https://cdn.jsdelivr.net/npm/vega@5"></script>
+        <script src="https://cdn.jsdelivr.net/npm/vega-lite@4"></script>
+        <script src="https://cdn.jsdelivr.net/npm/vega-embed@6"></script>
+      </head>
+      <body>
+      <div id="vis"></div>
+      <script>
+        (function(vegaEmbed) {
+          var spec = {}; /* JSON output for your chart's specification */
+          var embedOpt = {"mode": "vega-lite"}; /* Options for the embedding */
+
+          function showError(el, error){
+              el.innerHTML = ('<div style="color:red;">'
+                              + '<p>JavaScript Error: ' + error.message + '</p>'
+                              + "<p>This usually means there's a typo in your chart specification. "
+                              + "See the javascript console for the full traceback.</p>"
+                              + '</div>');
+              throw error;
+          }
+          const el = document.getElementById('vis');
+          vegaEmbed("#vis", spec, embedOpt)
+            .catch(error => showError(el, error));
+        })(vegaEmbed);
+      </script>
+    </body>
+    </html>
+     ```
+
+    The `Chart.save` method provides a convenient way to save such HTML output to file. For more information on embedding Altair/Vega-Lite, see the [documentation of the vega-embed project](https://github.com/vega/vega-embed).
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Next Steps
+
+    🎉 Hooray, you've completed the introduction to Altair! In the next notebook, we will dive deeper into creating visualizations using Altair's model of data types, graphical marks, and visual encoding channels.
+    """)
+    return
+
+
+if __name__ == "__main__":
+    app.run()

altair/02_marks_encoding.py

@@ -0,0 +1,1126 @@
+# /// script
+# requires-python = ">=3.11"
+# dependencies = [
+#     "altair==6.0.0",
+#     "marimo",
+#     "pandas==3.0.1",
+#     "vega_datasets==0.9.0",
+# ]
+# ///
+
+import marimo
+
+__generated_with = "0.20.4"
+app = marimo.App()
+
+
+@app.cell
+def _():
+    import marimo as mo
+
+    return (mo,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    # Data Types, Graphical Marks, and Visual Encoding Channels
+
+    A visualization represents data using a collection of _graphical marks_ (bars, lines, points, etc.). The attributes of a mark — such as its position, shape, size, or color — serve as _channels_ through which we can encode underlying data values.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    With a basic framework of _data types_, _marks_, and _encoding channels_, we can concisely create a wide variety of visualizations. In this notebook, we explore each of these elements and show how to use them to create custom statistical graphics.
+
+    _This notebook is part of the [data visualization curriculum](https://github.com/uwdata/visualization-curriculum)._
+    """)
+    return
+
+
+@app.cell
+def _():
+    import pandas as pd
+    import altair as alt
+
+    return (alt,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Global Development Data
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    We will be visualizing global health and population data for a number of countries, over the time period of 1955 to 2005. The data was collected by the [Gapminder Foundation](https://www.gapminder.org/) and shared in [Hans Rosling's popular TED talk](https://www.youtube.com/watch?v=hVimVzgtD6w). If you haven't seen the talk, we encourage you to watch it first!
+
+    Let's first load the dataset from the [vega-datasets](https://github.com/vega/vega-datasets) collection into a Pandas data frame.
+    """)
+    return
+
+
+@app.cell
+def _():
+    from vega_datasets import data as vega_data
+    data = vega_data.gapminder()
+    return (data,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    How big is the data?
+    """)
+    return
+
+
+@app.cell
+def _(data):
+    data.shape
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    693 rows and 6 columns! Let's take a peek at the data content:
+    """)
+    return
+
+
+@app.cell
+def _(data):
+    data.head(5)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    For each `country` and `year` (in 5-year intervals), we have measures of fertility in terms of the number of children per woman (`fertility`), life expectancy in years (`life_expect`), and total population (`pop`).
+
+    We also see a `cluster` field with an integer code. What might this represent? We'll try and solve this mystery as we visualize the data!
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Let's also create a smaller data frame, filtered down to values for the year 2000 only:
+    """)
+    return
+
+
+@app.cell
+def _(data):
+    data2000 = data.loc[data['year'] == 2000]
+    return (data2000,)
+
+
+@app.cell
+def _(data2000):
+    data2000.head(5)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Data Types
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The first ingredient in effective visualization is the input data. Data values can represent different forms of measurement. What kinds of comparisons do those measurements support? And what kinds of visual encodings then support those comparisons?
+
+    We will start by looking at the basic data types that Altair uses to inform visual encoding choices. These data types determine the kinds of comparisons we can make, and thereby guide our visualization design decisions.
+
+    ### Nominal (N)
+
+    *Nominal* data (also called *categorical* data) consist of category names.
+
+    With nominal data we can compare the equality of values: *is value A the same or different than value B? (A = B)*, supporting statements like “A is equal to B” or “A is not equal to B”.
+    In the dataset above, the `country` field is nominal.
+
+    When visualizing nominal data we should readily be able to see if values are the same or different: position, color hue (blue, red, green, *etc.*), and shape can help. However, using a size channel to encode nominal data might mislead us, suggesting rank-order or magnitude differences among values that do not exist!
+
+    ### Ordinal (O)
+
+    *Ordinal* data consist of values that have a specific ordering.
+
+    With ordinal data we can compare the rank-ordering of values: *does value A come before or after value B? (A < B)*, supporting statements like “A is less than B” or “A is greater than B”.
+    In the dataset above, we can treat the `year` field as ordinal.
+
+    When visualizing ordinal data, we should perceive a sense of rank-order. Position, size, or color value (brightness) might be appropriate, where as color hue (which is not perceptually ordered) would be less appropriate.
+
+    ### Quantitative (Q)
+
+    With *quantitative* data we can measure numerical differences among values. There are multiple sub-types of quantitative data:
+
+    For *interval* data we can measure the distance (interval) between points: *what is the distance to value A from value B? (A - B)*, supporting statements such as “A is 12 units away from B”.
+
+    For *ratio* data the zero-point is meaningful and so we can also measure proportions or scale factors: *value A is what proportion of value B? (A / B)*, supporting statements such as “A is 10% of B” or “B is 7 times larger than A”.
+
+    In the dataset above, `year` is a quantitative interval field (the value of year "zero" is subjective), whereas `fertility` and `life_expect` are quantitative ratio fields (zero is meaningful for calculating proportions).
+    Vega-Lite represents quantitative data, but does not make a distinction between interval and ratio types.
+
+    Quantitative values can be visualized using position, size, or color value, among other channels. An axis with a zero baseline is essential for proportional comparisons of ratio values, but can be safely omitted for interval comparisons.
+
+    ### Temporal (T)
+
+    *Temporal* values measure time points or intervals. This type is a special case of quantitative values (timestamps) with rich semantics and conventions (i.e., the [Gregorian calendar](https://en.wikipedia.org/wiki/Gregorian_calendar)). The temporal type in Vega-Lite supports reasoning about time units (year, month, day, hour, etc.), and provides methods for requesting specific time intervals.
+
+    Example temporal values include date strings such as `“2019-01-04”` and `“Jan 04 2019”`, as well as standardized date-times such as the [ISO date-time format](https://en.wikipedia.org/wiki/ISO_8601): `“2019-01-04T17:50:35.643Z”`.
+
+    There are no temporal values in our global development dataset above, as the `year` field is simply encoded as an integer. For more details about using temporal data in Altair, see the [Times and Dates documentation](https://altair-viz.github.io/user_guide/times_and_dates.html).
+
+    ### Summary
+
+    These data types are not mutually exclusive, but rather form a hierarchy: ordinal data support nominal (equality) comparisons, while quantitative data support ordinal (rank-order) comparisons.
+
+    Moreover, these data types do _not_ provide a fixed categorization. Just because a data field is represented using a number doesn't mean we have to treat it as a quantitative type! For example, we might interpret a set of ages (10 years old, 20 years old, etc) as nominal (underage or overage), ordinal (grouped by year), or quantitative (calculate average age).
+
+    Now let's examine how to visually encode these data types!
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Encoding Channels
+
+    At the heart of Altair is the use of *encodings* that bind data fields (with a given data type) to available encoding *channels* of a chosen *mark* type. In this notebook we'll examine the following encoding channels:
+
+    - `x`: Horizontal (x-axis) position of the mark.
+    - `y`: Vertical (y-axis) position of the mark.
+    - `size`: Size of the mark. May correspond to area or length, depending on the mark type.
+    - `color`: Mark color, specified as a [legal CSS color](https://developer.mozilla.org/en-US/docs/Web/CSS/color_value).
+    - `opacity`: Mark opacity, ranging from 0 (fully transparent) to 1 (fully opaque).
+    - `shape`: Plotting symbol shape for `point` marks.
+    - `tooltip`: Tooltip text to display upon mouse hover over the mark.
+    - `order`: Mark ordering, determines line/area point order and drawing order.
+    - `column`: Facet the data into horizontally-aligned subplots.
+    - `row`: Facet the data into vertically-aligned subplots.
+
+    For a complete list of available channels, see the [Altair encoding documentation](https://altair-viz.github.io/user_guide/encodings/index.html).
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### X
+
+    The `x` encoding channel sets a mark's horizontal position (x-coordinate). In addition, default choices of axis and title are made automatically. In the chart below, the choice of a quantitative data type results in a continuous linear axis scale:
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_point().encode(
+        alt.X('fertility:Q')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Y
+
+    The `y` encoding channel sets a mark's vertical position (y-coordinate). Here we've added the `cluster` field using an ordinal (`O`) data type. The result is a discrete axis that includes a sized band, with a default step size, for each unique value:
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_point().encode(
+        alt.X('fertility:Q'),
+        alt.Y('cluster:O')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _What happens to the chart above if you swap the `O` and `Q` field types?_
+
+    If we instead add the `life_expect` field as a quantitative (`Q`) variable, the result is a scatter plot with linear scales for both axes:
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_point().encode(
+        alt.X('fertility:Q'),
+        alt.Y('life_expect:Q')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    By default, axes for linear quantitative scales include zero to ensure a proper baseline for comparing ratio-valued data. In some cases, however, a zero baseline may be meaningless or you may want to focus on interval comparisons. To disable automatic inclusion of zero, configure the scale mapping using the encoding `scale` attribute:
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_point().encode(
+        alt.X('fertility:Q', scale=alt.Scale(zero=False)),
+        alt.Y('life_expect:Q', scale=alt.Scale(zero=False))
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Now the axis scales no longer include zero by default. Some padding still remains, as the axis domain end points are automatically snapped to _nice_ numbers like multiples of 5 or 10.
+
+    _What happens if you also add `nice=False` to the scale attribute above?_
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Size
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The `size` encoding channel sets a mark's size or extent. The meaning of the channel can vary based on the mark type. For `point` marks, the `size` channel maps to the pixel area of the plotting symbol, such that the diameter of the point matches the square root of the size value.
+
+    Let's augment our scatter plot by encoding population (`pop`) on the `size` channel. As a result, the chart now also includes a legend for interpreting the size values.
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_point().encode(
+        alt.X('fertility:Q'),
+        alt.Y('life_expect:Q'),
+        alt.Size('pop:Q')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    In some cases we might be unsatisfied with the default size range. To provide a customized span of sizes, set the `range` parameter of the `scale` attribute to an array indicating the smallest and largest sizes. Here we update the size encoding to range from 0 pixels (for zero values) to 1,000 pixels (for the maximum value in the scale domain):
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_point().encode(
+        alt.X('fertility:Q'),
+        alt.Y('life_expect:Q'),
+        alt.Size('pop:Q', scale=alt.Scale(range=[0,1000]))
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Color and Opacity
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The `color` encoding channel sets a mark's color. The style of color encoding is highly dependent on the data type: nominal data will default to a multi-hued qualitative color scheme, whereas ordinal and quantitative data will use perceptually ordered color gradients.
+
+    Here, we encode the `cluster` field using the `color` channel and a nominal (`N`) data type, resulting in a distinct hue for each cluster value. Can you start to guess what the `cluster` field might indicate?
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_point().encode(
+        alt.X('fertility:Q'),
+        alt.Y('life_expect:Q'),
+        alt.Size('pop:Q', scale=alt.Scale(range=[0,1000])),
+        alt.Color('cluster:N')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    If we prefer filled shapes, we can can pass a `filled=True` parameter to the `mark_point` method:
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_point(filled=True).encode(
+        alt.X('fertility:Q'),
+        alt.Y('life_expect:Q'),
+        alt.Size('pop:Q', scale=alt.Scale(range=[0,1000])),
+        alt.Color('cluster:N')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    By default, Altair uses a bit of transparency to help combat over-plotting. We are free to further adjust the opacity, either by passing a default value to the `mark_*` method, or using a dedicated encoding channel.
+
+    Here we demonstrate how to provide a constant value to an encoding channel instead of binding a data field:
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_point(filled=True).encode(
+        alt.X('fertility:Q'),
+        alt.Y('life_expect:Q'),
+        alt.Size('pop:Q', scale=alt.Scale(range=[0,1000])),
+        alt.Color('cluster:N'),
+        alt.OpacityValue(0.5)
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Shape
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The `shape` encoding channel sets the geometric shape used by `point` marks. Unlike the other channels we have seen so far, the `shape` channel can not be used by other mark types. The shape encoding channel should only be used with nominal data, as perceptual rank-order and magnitude comparisons are not supported.
+
+    Let's encode the `cluster` field using `shape` as well as `color`. Using multiple channels for the same underlying data field is known as a *redundant encoding*. The resulting chart combines both color and shape information into a single symbol legend:
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_point(filled=True).encode(
+        alt.X('fertility:Q'),
+        alt.Y('life_expect:Q'),
+        alt.Size('pop:Q', scale=alt.Scale(range=[0,1000])),
+        alt.Color('cluster:N'),
+        alt.OpacityValue(0.5),
+        alt.Shape('cluster:N')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Tooltips & Ordering
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    By this point, you might feel a bit frustrated: we've built up a chart, but we still don't know what countries the visualized points correspond to! Let's add interactive tooltips to enable exploration.
+
+    The `tooltip` encoding channel determines tooltip text to show when a user moves the mouse cursor over a mark. Let's add a tooltip encoding for the `country` field, then investigate which countries are being represented.
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_point(filled=True).encode(
+        alt.X('fertility:Q'),
+        alt.Y('life_expect:Q'),
+        alt.Size('pop:Q', scale=alt.Scale(range=[0,1000])),
+        alt.Color('cluster:N'),
+        alt.OpacityValue(0.5),
+        alt.Tooltip('country')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    As you mouse around you may notice that you can not select some of the points. For example, the largest dark blue circle corresponds to India, which is drawn on top of a country with a smaller population, preventing the mouse from hovering over that country. To fix this problem, we can use the `order` encoding channel.
+
+    The `order` encoding channel determines the order of data points, affecting both the order in which they are drawn and, for `line` and `area` marks, the order in which they are connected to one another.
+
+    Let's order the values in descending rank order by the population (`pop`), ensuring that smaller circles are drawn later than larger circles:
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_point(filled=True).encode(
+        alt.X('fertility:Q'),
+        alt.Y('life_expect:Q'),
+        alt.Size('pop:Q', scale=alt.Scale(range=[0,1000])),
+        alt.Color('cluster:N'),
+        alt.OpacityValue(0.5),
+        alt.Tooltip('country:N'),
+        alt.Order('pop:Q', sort='descending')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Now we can identify the smaller country being obscured by India: it's Bangladesh!
+
+    We can also now figure out what the `cluster` field represents. Mouse over the various colored points to formulate your own explanation.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    At this point we've added tooltips that show only a single property of the underlying data record. To show multiple values, we can provide the `tooltip` channel an array of encodings, one for each field we want to include:
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_point(filled=True).encode(
+        alt.X('fertility:Q'),
+        alt.Y('life_expect:Q'),
+        alt.Size('pop:Q', scale=alt.Scale(range=[0,1000])),
+        alt.Color('cluster:N'),
+        alt.OpacityValue(0.5),
+        alt.Order('pop:Q', sort='descending'),
+        tooltip = [
+            alt.Tooltip('country:N'),
+            alt.Tooltip('fertility:Q'),
+            alt.Tooltip('life_expect:Q')
+        ]   
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Now we can see multiple data fields upon mouse over!
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Column and Row Facets
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Spatial position is one of the most powerful and flexible channels for visual encoding, but what can we do if we already have assigned fields to the `x` and `y` channels? One valuable technique is to create a *trellis plot*, consisting of sub-plots that show a subset of the data. A trellis plot is one example of the more general technique of presenting data using [small multiples](https://en.wikipedia.org/wiki/Small_multiple) of views.
+
+    The `column` and `row` encoding channels generate either a horizontal (columns) or vertical (rows) set of sub-plots, in which the data is partitioned according to the provided data field.
+
+    Here is a trellis plot that divides the data into one column per \`cluster\` value:
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_point(filled=True).encode(
+        alt.X('fertility:Q'),
+        alt.Y('life_expect:Q'),
+        alt.Size('pop:Q', scale=alt.Scale(range=[0,1000])),
+        alt.Color('cluster:N'),
+        alt.OpacityValue(0.5),
+        alt.Tooltip('country:N'),
+        alt.Order('pop:Q', sort='descending'),
+        alt.Column('cluster:N')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The plot above does not fit on screen, making it difficult to compare all the sub-plots to each other! We can set the default `width` and `height` properties to create a smaller set of multiples. Also, as the column headers already label the `cluster` values, let's remove our `color` legend by setting it to `None`. To make better use of space we can also orient our `size` legend to the `'bottom'` of the chart.
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_point(filled=True).encode(
+        alt.X('fertility:Q'),
+        alt.Y('life_expect:Q'),
+        alt.Size('pop:Q', scale=alt.Scale(range=[0,1000]),
+                 legend=alt.Legend(orient='bottom', titleOrient='left')),
+        alt.Color('cluster:N', legend=None),
+        alt.OpacityValue(0.5),
+        alt.Tooltip('country:N'),
+        alt.Order('pop:Q', sort='descending'),
+        alt.Column('cluster:N')
+    ).properties(width=135, height=135)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Underneath the hood, the `column` and `row` encodings are translated into a new specification that uses the `facet` view composition operator. We will re-visit faceting in greater depth later on!
+
+    In the meantime, _can you rewrite the chart above to facet into rows instead of columns?_
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### A Peek Ahead: Interactive Filtering
+
+    In later modules, we'll dive into interaction techniques for data exploration. Here is a sneak peak: binding a range slider to the `year` field to enable interactive scrubbing through each year of data. Don't worry if the code below is a bit confusing at this point, as we will cover interaction in detail later.
+
+    _Drag the slider back and forth to see how the data values change over time!_
+    """)
+    return
+
+
+@app.cell
+def _(alt, data):
+    select_year = alt.selection_point(
+        name='select', fields=['year'], value=[{'year': 1955}],
+        bind=alt.binding_range(min=1955, max=2005, step=5)
+    )
+
+    alt.Chart(data).mark_point(filled=True).encode(
+        alt.X('fertility:Q', scale=alt.Scale(domain=[0,9])),
+        alt.Y('life_expect:Q', scale=alt.Scale(domain=[0,90])),
+        alt.Size('pop:Q', scale=alt.Scale(domain=[0, 1200000000], range=[0,1000])),
+        alt.Color('cluster:N', legend=None),
+        alt.OpacityValue(0.5),
+        alt.Tooltip('country:N'),
+        alt.Order('pop:Q', sort='descending')
+    ).add_params(select_year).transform_filter(select_year)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Graphical Marks
+
+    Our exploration of encoding channels above exclusively uses `point` marks to visualize the data. However, the `point` mark type is only one of the many geometric shapes that can be used to visually represent data. Altair includes a number of built-in mark types, including:
+
+    - `mark_area()` - Filled areas defined by a top-line and a baseline.
+    - `mark_bar()` -	Rectangular bars.
+    - `mark_circle()`	- Scatter plot points as filled circles.
+    - `mark_line()` - Connected line segments.
+    - `mark_point()` - Scatter plot points with configurable shapes.
+    - `mark_rect()` - Filled rectangles, useful for heatmaps.
+    - `mark_rule()` - Vertical or horizontal lines spanning the axis.
+    - `mark_square()` - Scatter plot points as filled squares.
+    - `mark_text()` - Scatter plot points represented by text.
+    - `mark_tick()` - Vertical or horizontal tick marks.
+
+    For a complete list, and links to examples, see the [Altair marks documentation](https://altair-viz.github.io/user_guide/marks/index.html). Next, we will step through a number of the most commonly used mark types for statistical graphics.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Point Marks
+
+    The `point` mark type conveys specific points, as in *scatter plots* and *dot plots*. In addition to `x` and `y` encoding channels (to specify 2D point positions), point marks can use `color`, `size`, and `shape` encodings to convey additional data fields.
+
+    Below is a dot plot of `fertility`, with the `cluster` field redundantly encoded using both the `y` and `shape` channels.
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_point().encode(
+        alt.X('fertility:Q'),
+        alt.Y('cluster:N'),
+        alt.Shape('cluster:N')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    In addition to encoding channels, marks can be stylized by providing values to the `mark_*()` methods.
+
+    For example: point marks are drawn with stroked outlines by default, but can be specified to use `filled` shapes instead. Similarly, you can set a default `size` to set the total pixel area of the point mark.
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_point(filled=True, size=100).encode(
+        alt.X('fertility:Q'),
+        alt.Y('cluster:N'),
+        alt.Shape('cluster:N')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Circle Marks
+
+    The `circle` mark type is a convenient shorthand for `point` marks drawn as filled circles.
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_circle(size=100).encode(
+        alt.X('fertility:Q'),
+        alt.Y('cluster:N'),
+        alt.Shape('cluster:N')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Square Marks
+
+    The `square` mark type is a convenient shorthand for `point` marks drawn as filled squares.
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_square(size=100).encode(
+        alt.X('fertility:Q'),
+        alt.Y('cluster:N'),
+        alt.Shape('cluster:N')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Tick Marks
+
+    The `tick` mark type conveys a data point using a short line segment or "tick". These are particularly useful for comparing values along a single dimension with minimal overlap. A *dot plot* drawn with tick marks is sometimes referred to as a *strip plot*.
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_tick().encode(
+        alt.X('fertility:Q'),
+        alt.Y('cluster:N'),
+        alt.Shape('cluster:N')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Bar Marks
+
+    The \`bar\` mark type draws a rectangle with a position, width, and height.
+
+    The plot below is a simple bar chart of the population (\`pop\`) of each country.
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_bar().encode(
+        alt.X('country:N'),
+        alt.Y('pop:Q')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The bar width is set to a default size. We will discuss how to adjust the bar width later in this notebook. (A subsequent notebook will take a closer look at configuring axes, scales, and legends.)
+
+    Bars can also be stacked. Let's change the `x` encoding to use the `cluster` field, and encode `country` using the `color` channel. We'll also disable the legend (which would be very long with colors for all countries!) and use tooltips for the country name.
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_bar().encode(
+        alt.X('cluster:N'),
+        alt.Y('pop:Q'),
+        alt.Color('country:N', legend=None),
+        alt.Tooltip('country:N')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    In the chart above, the use of the `color` encoding channel causes Altair / Vega-Lite to automatically stack the bar marks. Otherwise, bars would be drawn on top of each other! Try adding the parameter `stack=None` to the `y` encoding channel to see what happens if we don't apply stacking...
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The examples above create bar charts from a zero-baseline, and the `y` channel only encodes the non-zero value (or height) of the bar. However, the bar mark also allows you to specify starting and ending points to convey ranges.
+
+    The chart below uses the `x` (starting point) and `x2` (ending point) channels to show the range of life expectancies within each regional cluster. Below we use the `min` and `max` aggregation functions to determine the end points of the range; we will discuss aggregation in greater detail in the next notebook!
+
+    Alternatively, you can use `x` and `width` to provide a starting point plus offset, such that `x2 = x + width`.
+    """)
+    return
+
+
+@app.cell
+def _(alt, data2000):
+    alt.Chart(data2000).mark_bar().encode(
+        alt.X('min(life_expect):Q'),
+        alt.X2('max(life_expect):Q'),
+        alt.Y('cluster:N')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Line Marks
+
+    The `line` mark type connects plotted points with line segments, for example so that a line's slope conveys information about the rate of change.
+
+    Let's plot a line chart of fertility per country over the years, using the full, unfiltered global development data frame. We'll again hide the legend and use tooltips instead.
+    """)
+    return
+
+
+@app.cell
+def _(alt, data):
+    alt.Chart(data).mark_line().encode(
+        alt.X('year:O'),
+        alt.Y('fertility:Q'),
+        alt.Color('country:N', legend=None),
+        alt.Tooltip('country:N')
+    ).properties(
+        width=400
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    We can see interesting variations per country, but overall trends for lower numbers of children per family over time. Also note that we set a custom width of 400 pixels. _Try changing (or removing) the widths and see what happens!_
+
+    Let's change some of the default mark parameters to customize the plot. We can set the `strokeWidth` to determine the thickness of the lines and the `opacity` to add some transparency. By default, the `line` mark uses straight line segments to connect data points. In some cases we might want to smooth the lines. We can adjust the interpolation used to connect data points by setting the `interpolate` mark parameter. Let's use `'monotone'` interpolation to provide smooth lines that are also guaranteed not to inadvertently generate "false" minimum or maximum values as a result of the interpolation.
+    """)
+    return
+
+
+@app.cell
+def _(alt, data):
+    alt.Chart(data).mark_line(
+        strokeWidth=3,
+        opacity=0.5,
+        interpolate='monotone'
+    ).encode(
+        alt.X('year:O'),
+        alt.Y('fertility:Q'),
+        alt.Color('country:N', legend=None),
+        alt.Tooltip('country:N')
+    ).properties(
+        width=400
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The `line` mark can also be used to create *slope graphs*, charts that highlight the change in value between two comparison points using line slopes.
+
+    Below let's create a slope graph comparing the populations of each country at minimum and maximum years in our full dataset: 1955 and 2005. We first create a new Pandas data frame filtered to those years, then use Altair to create the slope graph.
+
+    By default, Altair places the years close together. To better space out the years along the x-axis, we can indicate the size (in pixels) of discrete steps along the width of our chart as indicated by the comment below. Try adjusting the width `step` value below and see how the chart changes in response.
+    """)
+    return
+
+
+@app.cell
+def _(alt, data):
+    dataTime = data.loc[(data['year'] == 1955) | (data['year'] == 2005)]
+
+    alt.Chart(dataTime).mark_line(opacity=0.5).encode(
+        alt.X('year:O'),
+        alt.Y('pop:Q'),
+        alt.Color('country:N', legend=None),
+        alt.Tooltip('country:N')
+    ).properties(
+        width={"step": 50} # adjust the step parameter
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Area Marks
+
+    The `area` mark type combines aspects of `line` and `bar` marks: it visualizes connections (slopes) among data points, but also shows a filled region, with one edge defaulting to a zero-valued baseline.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The chart below is an area chart of population over time for just the United States:
+    """)
+    return
+
+
+@app.cell
+def _(alt, data):
+    dataUS = data.loc[data['country'] == 'United States']
+
+    alt.Chart(dataUS).mark_area().encode(
+        alt.X('year:O'),
+        alt.Y('fertility:Q')
+    )
+    return (dataUS,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Similar to `line` marks, `area` marks support an `interpolate` parameter.
+    """)
+    return
+
+
+@app.cell
+def _(alt, dataUS):
+    alt.Chart(dataUS).mark_area(interpolate='monotone').encode(
+        alt.X('year:O'),
+        alt.Y('fertility:Q')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Similar to `bar` marks, `area` marks also support stacking. Here we create a new data frame with data for the three North American countries, then plot them using an `area` mark and a `color` encoding channel to stack by country.
+    """)
+    return
+
+
+@app.cell
+def _(alt, data):
+    dataNA = data.loc[
+        (data['country'] == 'United States') |
+        (data['country'] == 'Canada') |
+        (data['country'] == 'Mexico')
+    ]
+
+    alt.Chart(dataNA).mark_area().encode(
+        alt.X('year:O'),
+        alt.Y('pop:Q'),
+        alt.Color('country:N')
+    )
+    return (dataNA,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    By default, stacking is performed relative to a zero baseline. However, other `stack` options are available:
+
+    * `center` - to stack relative to a baseline in the center of the chart, creating a *streamgraph* visualization, and
+    * `normalize` - to normalize the summed data at each stacking point to 100%, enabling percentage comparisons.
+
+    Below we adapt the chart by setting the `y` encoding `stack` attribute to `center`. What happens if you instead set it `normalize`?
+    """)
+    return
+
+
+@app.cell
+def _(alt, dataNA):
+    alt.Chart(dataNA).mark_area().encode(
+        alt.X('year:O'),
+        alt.Y('pop:Q', stack='center'),
+        alt.Color('country:N')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    To disable stacking altogether, set the  `stack` attribute to `None`. We can also add `opacity` as a default mark parameter to ensure we see the overlapping areas!
+    """)
+    return
+
+
+@app.cell
+def _(alt, dataNA):
+    alt.Chart(dataNA).mark_area(opacity=0.5).encode(
+        alt.X('year:O'),
+        alt.Y('pop:Q', stack=None),
+        alt.Color('country:N')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The `area` mark type also supports data-driven baselines, with both the upper and lower series determined by data fields. As with `bar` marks, we can use the `x` and `x2` (or `y` and `y2`) channels to provide end points for the area mark.
+
+    The chart below visualizes the range of minimum and maximum fertility, per year, for North American countries:
+    """)
+    return
+
+
+@app.cell
+def _(alt, dataNA):
+    alt.Chart(dataNA).mark_area().encode(
+        alt.X('year:O'),
+        alt.Y('min(fertility):Q'),
+        alt.Y2('max(fertility):Q')
+    ).properties(
+        width={"step": 40}
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    We can see a larger range of values in 1995, from just under 4 to just under 7. By 2005, both the overall fertility values and the variability have declined, centered around 2 children per familty.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    All the `area` mark examples above use a vertically oriented area. However, Altair and Vega-Lite support horizontal areas as well. Let's transpose the chart above, simply by swapping the `x` and `y` channels.
+    """)
+    return
+
+
+@app.cell
+def _(alt, dataNA):
+    alt.Chart(dataNA).mark_area().encode(
+        alt.Y('year:O'),
+        alt.X('min(fertility):Q'),
+        alt.X2('max(fertility):Q')
+    ).properties(
+        width={"step": 40}
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Summary
+
+    We've completed our tour of data types, encoding channels, and graphical marks! You should now be well-equipped to further explore the space of encodings, mark types, and mark parameters. For a comprehensive reference &ndash; including features we've skipped over here! &ndash; see the Altair [marks](https://altair-viz.github.io/user_guide/marks/index.html) and [encoding](https://altair-viz.github.io/user_guide/encodings/index.html) documentation.
+
+    In the next module, we will look at the use of data transformations to create charts that summarize data or visualize new derived fields. In a later module, we'll examine how to further customize your charts by modifying scales, axes, and legends.
+
+    Interested in learning more about visual encoding?
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    <img title="Bertin's Taxonomy of Visual Encoding Channels" src="https://cdn-images-1.medium.com/max/2000/1*jsb78Rr2cDy6zrE7j2IKig.png" style="max-width: 650px;"><br/>
+
+    <small>Bertin's taxonomy of visual encodings from <a href="https://books.google.com/books/about/Semiology_of_Graphics.html?id=X5caQwAACAAJ"><em>Sémiologie Graphique</em></a>, as adapted by <a href="https://bost.ocks.org/mike/">Mike Bostock</a>.</small>
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    - The systematic study of marks, visual encodings, and backing data types was initiated by [Jacques Bertin](https://en.wikipedia.org/wiki/Jacques_Bertin) in his pioneering 1967 work [_Sémiologie Graphique (The Semiology of Graphics)_](https://books.google.com/books/about/Semiology_of_Graphics.html?id=X5caQwAACAAJ). The image above illustrates position, size, value (brightness), texture, color (hue), orientation, and shape channels, alongside Bertin's recommendations for the data types they support.
+    - The framework of data types, marks, and channels also guides _automated_ visualization design tools, starting with [Mackinlay's APT (A Presentation Tool)](https://scholar.google.com/scholar?cluster=10191273548472217907) in 1986 and continuing in more recent systems such as [Voyager](http://idl.cs.washington.edu/papers/voyager/) and [Draco](http://idl.cs.washington.edu/papers/draco/).
+    - The identification of nominal, ordinal, interval, and ratio types dates at least as far back as S. S. Steven's 1947 article [_On the theory of scales of measurement_](https://scholar.google.com/scholar?cluster=14356809180080326415).
+    """)
+    return
+
+
+if __name__ == "__main__":
+    app.run()

altair/03_data_transformation.py

@@ -0,0 +1,641 @@
+# /// script
+# requires-python = ">=3.11"
+# dependencies = [
+#     "altair==6.0.0",
+#     "marimo",
+#     "pandas==3.0.1",
+# ]
+# ///
+
+import marimo
+
+__generated_with = "0.20.4"
+app = marimo.App()
+
+
+@app.cell
+def _():
+    import marimo as mo
+
+    return (mo,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    # Data Transformation
+
+    In previous notebooks we learned how to use marks and visual encodings to represent individual data records. Here we will explore methods for *transforming* data, including the use of aggregates to summarize multiple records. Data transformation is an integral part of visualization: choosing the  variables to show and their level of detail is just as important as choosing appropriate visual encodings. After all, it doesn't matter how well chosen your visual encodings are if you are showing the wrong information!
+
+    As you work through this module, we recommend that you open the [Altair Data Transformations documentation](https://altair-viz.github.io/user_guide/transform/) in another tab. It will be a useful resource if at any point you'd like more details or want to see what other transformations are available.
+
+    _This notebook is part of the [data visualization curriculum](https://github.com/uwdata/visualization-curriculum)._
+    """)
+    return
+
+
+@app.cell
+def _():
+    import pandas as pd
+    import altair as alt
+
+    return alt, pd
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## The Movies Dataset
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    We will be working with a table of data about motion pictures, taken from the [vega-datasets](https://vega.github.io/vega-datasets/) collection. The data includes variables such as the film name, director, genre, release date, ratings, and gross revenues. However, _be careful when working with this data_: the films are from unevenly sampled years, using data combined from multiple sources. If you dig in you will find issues with missing values and even some subtle errors! Nevertheless, the data should prove interesting to explore...
+
+    Let's retrieve the URL for the JSON data file from the vega_datasets package, and then read the data into a Pandas data frame so that we can inspect its contents.
+    """)
+    return
+
+
+@app.cell
+def _(pd):
+    movies_url = 'https://cdn.jsdelivr.net/npm/vega-datasets@1/data/movies.json'
+    movies = pd.read_json(movies_url)
+    return movies, movies_url
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    How many rows (records) and columns (fields) are in the movies dataset?
+    """)
+    return
+
+
+@app.cell
+def _(movies):
+    movies.shape
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Now let's peek at the first 5 rows of the table to get a sense of the fields and data types...
+    """)
+    return
+
+
+@app.cell
+def _(movies):
+    movies.head(5)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Histograms
+
+    We'll start our transformation tour by _binning_ data into discrete groups and _counting_ records to summarize those groups. The resulting plots are known as [_histograms_](https://en.wikipedia.org/wiki/Histogram).
+
+    Let's first look at unaggregated data: a scatter plot showing movie ratings from Rotten Tomatoes versus ratings from IMDB users. We'll provide data to Altair by passing the movies data URL to the `Chart` method. (We could also pass the Pandas data frame directly to get the same result.) We can then encode the Rotten Tomatoes and IMDB ratings fields using the `x` and `y` channels:
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_circle().encode(
+        alt.X('Rotten_Tomatoes_Rating:Q'),
+        alt.Y('IMDB_Rating:Q')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    To summarize this data, we can *bin* a data field to group numeric values into discrete groups. Here we bin along the x-axis by adding `bin=True` to the `x` encoding channel. The result is a set of ten bins of equal step size, each corresponding to a span of ten ratings points.
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_circle().encode(
+        alt.X('Rotten_Tomatoes_Rating:Q', bin=True),
+        alt.Y('IMDB_Rating:Q')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Setting `bin=True` uses default binning settings, but we can exercise more control if desired. Let's instead set the maximum bin count (`maxbins`) to 20, which has the effect of doubling the number of bins. Now each bin corresponds to a span of five ratings points.
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_circle().encode(
+        alt.X('Rotten_Tomatoes_Rating:Q', bin=alt.BinParams(maxbins=20)),
+        alt.Y('IMDB_Rating:Q')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    With the data binned, let's now summarize the distribution of Rotten Tomatoes ratings. We will drop the IMDB ratings for now and instead use the `y` encoding channel to show an aggregate `count` of records, so that the vertical position of each point indicates the number of movies per Rotten Tomatoes rating bin.
+
+    As the `count` aggregate counts the number of total records in each bin regardless of the field values, we do not need to include a field name in the `y` encoding.
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_circle().encode(
+        alt.X('Rotten_Tomatoes_Rating:Q', bin=alt.BinParams(maxbins=20)),
+        alt.Y('count()')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    To arrive at a standard histogram, let's change the mark type from `circle` to `bar`:
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_bar().encode(
+        alt.X('Rotten_Tomatoes_Rating:Q', bin=alt.BinParams(maxbins=20)),
+        alt.Y('count()')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _We can now examine the distribution of ratings more clearly: we can see fewer movies on the negative end, and a bit more movies on the high end, but a generally uniform distribution overall. Rotten Tomatoes ratings are determined by taking "thumbs up" and "thumbs down" judgments from film critics and calculating the percentage of positive reviews. It appears this approach does a good job of utilizing the full range of rating values._
+
+    Similarly, we can create a histogram for IMDB ratings by changing the field in the `x` encoding channel:
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_bar().encode(
+        alt.X('IMDB_Rating:Q', bin=alt.BinParams(maxbins=20)),
+        alt.Y('count()')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _In contrast to the more uniform distribution we saw before, IMDB ratings exhibit a bell-shaped (though [negatively skewed](https://en.wikipedia.org/wiki/Skewness)) distribution. IMDB ratings are formed by averaging scores (ranging from 1 to 10) provided by the site's users. We can see that this form of measurement leads to a different shape than the Rotten Tomatoes ratings. We can also see that the mode of the distribution is between 6.5 and 7: people generally enjoy watching movies, potentially explaining the positive bias!_
+
+    Now let's turn back to our scatter plot of Rotten Tomatoes and IMDB ratings. Here's what happens if we bin *both* axes of our original plot.
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_circle().encode(
+        alt.X('Rotten_Tomatoes_Rating:Q', bin=alt.BinParams(maxbins=20)),
+        alt.Y('IMDB_Rating:Q', bin=alt.BinParams(maxbins=20)),
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Detail is lost due to *overplotting*, with many points drawn directly on top of each other.
+
+    To form a two-dimensional histogram we can add a `count` aggregate as before. As both the `x` and `y` encoding channels are already taken, we must use a different encoding channel to convey the counts. Here is the result of using circular area by adding a *size* encoding channel.
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_circle().encode(
+        alt.X('Rotten_Tomatoes_Rating:Q', bin=alt.BinParams(maxbins=20)),
+        alt.Y('IMDB_Rating:Q', bin=alt.BinParams(maxbins=20)),
+        alt.Size('count()')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Alternatively, we can encode counts using the `color` channel and change the mark type to `bar`. The result is a two-dimensional histogram in the form of a [*heatmap*](https://en.wikipedia.org/wiki/Heat_map).
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_bar().encode(
+        alt.X('Rotten_Tomatoes_Rating:Q', bin=alt.BinParams(maxbins=20)),
+        alt.Y('IMDB_Rating:Q', bin=alt.BinParams(maxbins=20)),
+        alt.Color('count()')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Compare the *size* and *color*-based 2D histograms above. Which encoding do you think should be preferred? Why? In which plot can you more precisely compare the magnitude of individual values? In which plot can you more accurately see the overall density of ratings?
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Aggregation
+
+    Counts are just one type of aggregate. We might also calculate summaries using measures such as the `average`, `median`, `min`, or `max`. The Altair documentation includes the [full set of available aggregation functions](https://altair-viz.github.io/user_guide/transform/aggregate.html#user-guide-aggregate-transform).
+
+    Let's look at some examples!
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Averages and Sorting
+
+    _Do different genres of films receive consistently different ratings from critics?_ As a first step towards answering this question, we might examine the [*average* (a.k.a. the *arithmetic mean*)](https://en.wikipedia.org/wiki/Arithmetic_mean) rating for each genre of movie.
+
+    Let's visualize genre along the `y` axis and plot `average` Rotten Tomatoes ratings along the `x` axis.
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_bar().encode(
+        alt.X('average(Rotten_Tomatoes_Rating):Q'),
+        alt.Y('Major_Genre:N')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _There does appear to be some interesting variation, but looking at the data as an alphabetical list is not very helpful for ranking critical reactions to the genres._
+
+    For a tidier picture, let's sort the genres in descending order of average rating. To do so, we will add a `sort` parameter to the `y` encoding channel, stating that we wish to sort by the *average* (`op`, the aggregate operation) Rotten Tomatoes rating (the `field`) in descending `order`.
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_bar().encode(
+        alt.X('average(Rotten_Tomatoes_Rating):Q'),
+        alt.Y('Major_Genre:N', sort=alt.EncodingSortField(
+            op='average', field='Rotten_Tomatoes_Rating', order='descending')
+        )
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _The sorted plot suggests that critics think highly of documentaries, musicals, westerns, and dramas, but look down upon romantic comedies and horror films... and who doesn't love `null` movies!?_
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Medians and the Inter-Quartile Range
+
+    While averages are a common way to summarize data, they can sometimes mislead. For example, very large or very small values ([*outliers*](https://en.wikipedia.org/wiki/Outlier)) might skew the average. To be safe, we can compare the genres according to the [*median*](https://en.wikipedia.org/wiki/Median) ratings as well.
+
+    The median is a point that splits the data evenly, such that half of the values are less than the median and the other half are greater. The median is less sensitive to outliers and so is referred to as a [*robust* statistic](https://en.wikipedia.org/wiki/Robust_statistics). For example, arbitrarily increasing the largest rating value will not cause the median to change.
+
+    Let's update our plot to use a `median` aggregate and sort by those values:
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_bar().encode(
+        alt.X('median(Rotten_Tomatoes_Rating):Q'),
+        alt.Y('Major_Genre:N', sort=alt.EncodingSortField(
+            op='median', field='Rotten_Tomatoes_Rating', order='descending')
+        )
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _We can see that some of the genres with similar averages have swapped places (films of unknown genre, or `null`, are now rated highest!), but the overall groups have stayed stable. Horror films continue to get little love from professional film critics._
+
+    It's a good idea to stay skeptical when viewing aggregate statistics. So far we've only looked at *point estimates*. We have not examined how ratings vary within a genre.
+
+    Let's visualize the variation among the ratings to add some nuance to our rankings. Here we will encode the [*inter-quartile range* (IQR)](https://en.wikipedia.org/wiki/Interquartile_range) for each genre. The IQR is the range in which the middle half of data values reside. A [*quartile*](https://en.wikipedia.org/wiki/Quartile) contains 25% of the data values. The inter-quartile range consists of the two middle quartiles, and so contains the middle 50%.
+
+    To visualize ranges, we can use the `x` and `x2` encoding channels to indicate the starting and ending points. We use the aggregate functions `q1` (the lower quartile boundary) and `q3` (the upper quartile boundary) to provide the inter-quartile range. (In case you are wondering, *q2* would be the median.)
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_bar().encode(
+        alt.X('q1(Rotten_Tomatoes_Rating):Q'),
+        alt.X2('q3(Rotten_Tomatoes_Rating):Q'),
+        alt.Y('Major_Genre:N', sort=alt.EncodingSortField(
+            op='median', field='Rotten_Tomatoes_Rating', order='descending')
+        )
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Time Units
+
+    _Now let's ask a completely different question: do box office returns vary by season?_
+
+    To get an initial answer, let's plot the median U.S. gross revenue by month.
+
+    To make this chart, use the `timeUnit` transform to map release dates to the `month` of the year. The result is similar to binning, but using meaningful time intervals. Other valid time units include `year`, `quarter`, `date` (numeric day in month), `day` (day of the week), and `hours`, as well as compound units such as `yearmonth` or `hoursminutes`. See the Altair documentation for a [complete list of time units](https://altair-viz.github.io/user_guide/transform/timeunit.html#user-guide-timeunit-transform).
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_area().encode(
+        alt.X('month(Release_Date):T'),
+        alt.Y('median(US_Gross):Q')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Looking at the resulting plot, median movie sales in the U.S. appear to spike around the summer blockbuster season and the end of year holiday period. Of course, people around the world (not just the U.S.) go out to the movies. Does a similar pattern arise for worldwide gross revenue?_
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_area().encode(
+        alt.X('month(Release_Date):T'),
+        alt.Y('median(Worldwide_Gross):Q')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Yes!_
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Advanced Data Transformation
+
+    The examples above all use transformations (*bin*, *timeUnit*, *aggregate*, *sort*) that are defined relative to an encoding channel. However, at times you may want to apply a chain of multiple transformations prior to visualization, or use transformations that don't integrate into encoding definitions. For such cases, Altair and Vega-Lite support data transformations defined separately from encodings. These transformations are applied to the data *before* any encodings are considered.
+
+    We *could* also perform transformations using Pandas directly, and then visualize the result. However, using the built-in transforms allows our visualizations to be published more easily in other contexts; for example, exporting the Vega-Lite JSON to use in a stand-alone web interface. Let's look at the built-in transforms supported by Altair, such as `calculate`, `filter`, `aggregate`, and `window`.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Calculate
+
+    _Think back to our comparison of U.S. gross and worldwide gross. Doesn't worldwide revenue include the U.S.? (Indeed it does.) How might we get a better sense of trends outside the U.S.?_
+
+    With the `calculate` transform we can derive new fields. Here we want to subtract U.S. gross from worldwide gross. The `calculate` transform takes a [Vega expression string](https://vega.github.io/vega/docs/expressions/) to define a formula over a single record. Vega expressions use JavaScript syntax. The `datum.` prefix accesses a field value on the input record.
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies):
+    alt.Chart(movies).mark_area().transform_calculate(
+        NonUS_Gross='datum.Worldwide_Gross - datum.US_Gross'
+    ).encode(
+        alt.X('month(Release_Date):T'),
+        alt.Y('median(NonUS_Gross):Q')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _We can see that seasonal trends hold outside the U.S., but with a more pronounced decline in the non-peak months._
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Filter
+
+    The *filter* transform creates a new table with a subset of the original data, removing rows that fail to meet a provided [*predicate*](https://en.wikipedia.org/wiki/Predicate_%28mathematical_logic%29) test. Similar to the *calculate* transform, filter predicates are expressed using the [Vega expression language](https://vega.github.io/vega/docs/expressions/).
+
+    Below we add a filter to limit our initial scatter plot of IMDB vs. Rotten Tomatoes ratings to only films in the major genre of "Romantic Comedy".
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_circle().encode(
+        alt.X('Rotten_Tomatoes_Rating:Q'),
+        alt.Y('IMDB_Rating:Q')
+    ).transform_filter('datum.Major_Genre == "Romantic Comedy"')
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _How does the plot change if we filter to view other genres? Edit the filter expression to find out._
+
+    Now let's filter to look at films released before 1970.
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_circle().encode(
+        alt.X('Rotten_Tomatoes_Rating:Q'),
+        alt.Y('IMDB_Rating:Q')
+    ).transform_filter('year(datum.Release_Date) < 1970')
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _They seem to score unusually high! Are older films simply better, or is there a [selection bias](https://en.wikipedia.org/wiki/Selection%5Fbias) towards more highly-rated older films in this dataset?_
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Aggregate
+
+    We have already seen `aggregate` transforms such as `count` and `average`  in the context of encoding channels. We can also specify aggregates separately, as a pre-processing step for other transforms (as in the `window` transform examples below). The output of an `aggregate` transform is a new data table with records that contain both the `groupby` fields and the computed `aggregate` measures.
+
+    Let's recreate our plot of average ratings by genre, but this time using a separate `aggregate` transform. The output table from the aggregate transform contains 13 rows, one for each genre.
+
+    To order the `y` axis we must include a required aggregate operation in our sorting instructions. Here we use the `max` operator, which works fine because there is only one output record per genre. We could similarly use the `min` operator and end up with the same plot.
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_bar().transform_aggregate(
+        groupby=['Major_Genre'],
+        Average_Rating='average(Rotten_Tomatoes_Rating)'
+    ).encode(
+        alt.X('Average_Rating:Q'),
+        alt.Y('Major_Genre:N', sort=alt.EncodingSortField(
+            op='max', field='Average_Rating', order='descending'
+          )
+        )
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Window
+
+    The `window` transform performs calculations over sorted groups of data records. Window transforms are quite powerful, supporting tasks such as ranking, lead/lag analysis, cumulative totals, and running sums or averages. Values calculated by a `window` transform are written back to the input data table as new fields. Window operations include the aggregate operations we've seen earlier, as well as specialized operations such as `rank`, `row_number`, `lead`, and `lag`. The Vega-Lite documentation lists [all valid window operations](https://vega.github.io/vega-lite/docs/window.html#ops).
+
+    One use case for a `window` transform is to calculate top-k lists. Let's plot the top 20 directors in terms of total worldwide gross.
+
+    We first use a `filter` transform to remove records for which we don't know the director. Otherwise, the director `null` would dominate the list! We then apply an `aggregate` to sum up the worldwide gross for all films, grouped by director. At this point we could plot a sorted bar chart, but we'd end up with hundreds and hundreds of directors. How can we limit the display to the top 20?
+
+    The `window` transform allows us to determine the top directors by calculating their rank order. Within our `window` transform definition we can `sort` by gross and use the `rank` operation to calculate rank scores according to that sort order. We can then add a subsequent `filter` transform to limit the data to only records with a rank value less than or equal to 20.
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_bar().transform_filter(
+        'datum.Director != null'
+    ).transform_aggregate(
+        Gross='sum(Worldwide_Gross)',
+        groupby=['Director']
+    ).transform_window(
+        Rank='rank()',
+        sort=[alt.SortField('Gross', order='descending')]
+    ).transform_filter(
+        'datum.Rank < 20'
+    ).encode(
+        alt.X('Gross:Q'),
+        alt.Y('Director:N', sort=alt.EncodingSortField(
+            op='max', field='Gross', order='descending'
+        ))
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _We can see that Steven Spielberg has been quite successful in his career! However, showing sums might favor directors who have had longer careers, and so have made more movies and thus more money. What happens if we change the choice of aggregate operation? Who is the most successful director in terms of  `average` or `median` gross per film? Modify the aggregate transform above!_
+
+    Earlier in this notebook we looked at histograms, which approximate the [*probability density function*](https://en.wikipedia.org/wiki/Probability_density_function) of a set of values. A complementary approach is to look at the [*cumulative distribution*](https://en.wikipedia.org/wiki/Cumulative_distribution_function). For example, think of a histogram in which each bin includes not only its own count but also the counts from all previous bins &mdash; the result is a _running total_, with the last bin containing the total number of records. A cumulative chart directly shows us, for a given reference value, how many data values are less than or equal to that reference.
+
+    As a concrete example, let's look at the cumulative distribution of films by running time (in minutes). Only a subset of records actually include running time information, so we first `filter` down to the subset of films for which we have running times. Next, we apply an `aggregate` to count the number of films per duration (implicitly using "bins" of 1 minute each). We then use a `window` transform to compute a running total of counts across bins, sorted by increasing running time.
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies_url):
+    alt.Chart(movies_url).mark_line(interpolate='step-before').transform_filter(
+        'datum.Running_Time_min != null'
+    ).transform_aggregate(
+        groupby=['Running_Time_min'],
+        Count='count()',
+    ).transform_window(
+        Cumulative_Sum='sum(Count)',
+        sort=[alt.SortField('Running_Time_min', order='ascending')]
+    ).encode(
+        alt.X('Running_Time_min:Q', axis=alt.Axis(title='Duration (min)')),
+        alt.Y('Cumulative_Sum:Q', axis=alt.Axis(title='Cumulative Count of Films'))
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Let's examine the cumulative distribution of film lengths. We can see that films under 110 minutes make up about half of all the films for which we have running times. We see a steady accumulation of films between 90 minutes and 2 hours, after which the distribution begins to taper off. Though rare, the dataset does contain multiple films more than 3 hours long!_
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Summary
+
+    We've only scratched the surface of what data transformations can do! For more details, including all the available transformations and their parameters, see the [Altair data transformation documentation](https://altair-viz.github.io/user_guide/transform/index.html).
+
+    Sometimes you will need to perform significant data transformation to prepare your data _prior_ to using visualization tools. To engage in [_data wrangling_](https://en.wikipedia.org/wiki/Data_wrangling) right here in Python, you can use the [Pandas library](https://pandas.pydata.org/).
+    """)
+    return
+
+
+if __name__ == "__main__":
+    app.run()

altair/04_scales_axes_legends.py

@@ -0,0 +1,840 @@
+# /// script
+# requires-python = ">=3.11"
+# dependencies = [
+#     "altair==6.0.0",
+#     "marimo",
+#     "pandas==3.0.1",
+# ]
+# ///
+
+import marimo
+
+__generated_with = "0.20.4"
+app = marimo.App()
+
+
+@app.cell
+def _():
+    import marimo as mo
+
+    return (mo,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    # Scales, Axes, and Legends
+
+    Visual encoding – mapping data to visual variables such as position, size, shape, or color – is the beating heart of data visualization. The workhorse that actually performs this mapping is the *scale*: a function that takes a data value as input (the scale *domain*) and returns a visual value, such as a pixel position or RGB color, as output (the scale *range*).  Of course, a visualization is useless if no one can figure out what it conveys! In addition to graphical marks, a chart needs reference elements, or *guides*, that allow readers to decode the graphic. Guides such as *axes* (which visualize scales with spatial ranges) and *legends* (which visualize scales with color, size, or shape ranges), are the unsung heroes of effective data visualization!
+
+    In this notebook, we will explore the options Altair provides to support customized designs of scale mappings, axes, and legends, using a running example about the effectiveness of antibiotic drugs.
+
+    _This notebook is part of the [data visualization curriculum](https://github.com/uwdata/visualization-curriculum)._
+    """)
+    return
+
+
+@app.cell
+def _():
+    import pandas as pd
+    import altair as alt
+
+    return alt, pd
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Antibiotics Data
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    After World War II, antibiotics were considered "wonder drugs", as they were an easy remedy for what had been intractable ailments. To learn which drug worked most effectively for which bacterial infection, performance of the three most popular antibiotics on 16 bacteria were gathered.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    We will be using an antibiotics dataset from the [vega-datasets collection](https://github.com/vega/vega-datasets). In the examples below, we will pass the URL directly to Altair:
+    """)
+    return
+
+
+@app.cell
+def _():
+    antibiotics = 'https://cdn.jsdelivr.net/npm/vega-datasets@1/data/burtin.json'
+    return (antibiotics,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    We can first load the data with Pandas to view the dataset in its entirety and get acquainted with the available fields:
+    """)
+    return
+
+
+@app.cell
+def _(antibiotics, pd):
+    pd.read_json(antibiotics)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The numeric values in the table indicate the [minimum inhibitory concentration (MIC)](https://en.wikipedia.org/wiki/Minimum_inhibitory_concentration), a measure of the effectiveness of the antibiotic, which represents the concentration of antibiotic (in micrograms per milliliter) required to prevent growth in vitro. The reaction of the bacteria to a procedure called [Gram staining](https://en.wikipedia.org/wiki/Gram_stain) is described by the nominal field `Gram_Staining`. Bacteria that turn dark blue or violet are Gram-positive. Otherwise, they are Gram-negative.
+
+    As we examine different visualizations of this dataset, ask yourself: What might we learn about the relative effectiveness of the antibiotics? What might we learn about the bacterial species based on their antibiotic response?
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Configuring Scales and Axes
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Plotting Antibiotic Resistance: Adjusting the Scale Type
+
+    Let's start by looking at a simple dot plot of the MIC for Neomycin.
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle().encode(
+        alt.X('Neomycin:Q')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _We can see that the MIC values span orders of magnitude: most points to cluster on the left, with a few large outliers to the right._
+
+    By default Altair uses a `linear` mapping between the domain values (MIC) and the range values (pixels). To get a better overview of the data, we can apply a different scale transformation.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    To change the scale type, we'll set the `scale` attribute, using the `alt.Scale` method and `type` parameter.
+
+    Here's the result of using a square root (`sqrt`) scale type. Distances in the pixel range now correspond to the square root of distances in the data domain.
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle().encode(
+        alt.X('Neomycin:Q',
+              scale=alt.Scale(type='sqrt'))
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _The points on the left are now better differentiated, but we still see some heavy skew._
+
+    Let's try using a [logarithmic scale](https://en.wikipedia.org/wiki/Logarithmic_scale) (`log`) instead:
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle().encode(
+        alt.X('Neomycin:Q',
+              scale=alt.Scale(type='log'))
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Now the data is much more evenly distributed and we can see the very large differences in concentrations required for different bacteria._
+
+    In a standard linear scale, a visual (pixel) distance of 10 units might correspond to an *addition* of 10 units in the data domain. A logarithmic transform maps between multiplication and addition, such that `log(u) + log(v) = log(u*v)`. As a result, in a logarithmic scale, a visual distance of 10 units instead corresponds to *multiplication* by 10 units in the data domain, assuming a base 10 logarithm. The `log` scale above defaults to using the logarithm base 10, but we can adjust this by providing a `base` parameter to the scale.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Styling an Axis
+
+    Lower dosages indicate higher effectiveness. However, some people may expect values that are "better" to be "up and to the right" within a chart. If we want to cater to this convention, we can reverse the axis to encode "effectiveness" as a reversed MIC scale.
+
+    To do this, we can set the encoding `sort` property to `'descending'`:
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle().encode(
+        alt.X('Neomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log'))
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Unfortunately the axis is starting to get a bit confusing: we're plotting data on a logarithmic scale, in the reverse direction, and without a clear indication of what our units are!_
+
+    Let's add a more informative axis title: we'll use the `title` property of the encoding to provide the desired title text:
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle().encode(
+        alt.X('Neomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log'),
+              title='Neomycin MIC (μg/ml, reverse log scale)')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Much better!
+
+    By default, Altair places the x-axis along the bottom of the chart. To change these defaults, we can add an `axis` attribute with `orient='top'`:
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle().encode(
+        alt.X('Neomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log'),
+              axis=alt.Axis(orient='top'),
+              title='Neomycin MIC (μg/ml, reverse log scale)')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Similarly, the y-axis defaults to a `'left'` orientation, but can be set to `'right'`.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Comparing Antibiotics: Adjusting Grid Lines, Tick Counts, and Sizing
+
+    _How does neomycin compare to other antibiotics, such as streptomycin and penicillin?_
+
+    To start answering this question, we can create scatter plots, adding a y-axis encoding for another antibiotic that mirrors the design of our x-axis for neomycin.
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle().encode(
+        alt.X('Neomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log'),
+              title='Neomycin MIC (μg/ml, reverse log scale)'),
+        alt.Y('Streptomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log'),
+              title='Streptomycin MIC (μg/ml, reverse log scale)')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _We can see that neomycin and streptomycin appear highly correlated, as the bacterial strains respond similarly to both antibiotics._
+
+    Let's move on and compare neomycin with penicillin:
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle().encode(
+        alt.X('Neomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log'),
+              title='Neomycin MIC (μg/ml, reverse log scale)'),
+        alt.Y('Penicillin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log'),
+              title='Penicillin MIC (μg/ml, reverse log scale)')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Now we see a more differentiated response: some bacteria respond well to neomycin but not penicillin, and vice versa!_
+
+    While this plot is useful, we can make it better. The x and y axes use the same units, but have different extents (the chart width is larger than the height) and different domains (0.001 to 100 for the x-axis, and 0.001 to 1,000 for the y-axis).
+
+    Let's equalize the axes: we can add explicit `width` and `height` settings for the chart, and specify matching domains  using the scale `domain` property.
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle().encode(
+        alt.X('Neomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              title='Neomycin MIC (μg/ml, reverse log scale)'),
+        alt.Y('Penicillin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              title='Penicillin MIC (μg/ml, reverse log scale)')
+    ).properties(width=250, height=250)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _The resulting plot is more balanced, and less prone to subtle misinterpretations!_
+
+    However, the grid lines are now rather dense. If we want to remove grid lines altogether, we can add `grid=False` to the `axis` attribute. But what if we instead want to reduce the number of tick marks, for example only including grid lines for each order of magnitude?
+
+    To change the number of ticks, we can specify a target `tickCount` property for an `Axis` object. The `tickCount` is treated as a *suggestion* to Altair, to be considered alongside other aspects such as using nice, human-friendly intervals. We may not get *exactly* the number of tick marks we request, but we should get something close.
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle().encode(
+        alt.X('Neomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Neomycin MIC (μg/ml, reverse log scale)'),
+        alt.Y('Penicillin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Penicillin MIC (μg/ml, reverse log scale)')
+    ).properties(width=250, height=250)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    By setting the `tickCount` to 5, we have the desired effect.
+
+    Our scatter plot points feel a bit small. Let's change the default size by setting the `size` property of the circle mark. This size value is the *area* of the mark in pixels.
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle(size=80).encode(
+        alt.X('Neomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Neomycin MIC (μg/ml, reverse log scale)'),
+        alt.Y('Penicillin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Penicillin MIC (μg/ml, reverse log scale)'), 
+    ).properties(width=250, height=250)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Here we've set the circle mark area to 80 pixels. _Further adjust the value as you see fit!_
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Configuring Color Legends
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Color by Gram Staining
+
+    _Above we saw that neomycin is more effective for some bacteria, while penicillin is more effective for others. But how can we tell which antibiotic to use if we don't know the specific species of bacteria? Gram staining serves as a diagnostic for discriminating classes of bacteria!_
+
+    Let's encode `Gram_Staining` on the `color` channel as a nominal data type:
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle(size=80).encode(
+        alt.X('Neomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Neomycin MIC (μg/ml, reverse log scale)'),
+        alt.Y('Penicillin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Penicillin MIC (μg/ml, reverse log scale)'),
+        alt.Color('Gram_Staining:N')
+    ).properties(width=250, height=250)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _We can see that Gram-positive bacteria seem most susceptible to penicillin, whereas neomycin is more effective for Gram-negative bacteria!_
+
+    The color scheme above was automatically chosen to provide perceptually-distinguishable colors for nominal (equal or not equal) comparisons. However, we might wish to customize the colors used. In this case, Gram staining results in [distinctive physical colorings: pink for Gram-negative, purple for Gram-positive](https://en.wikipedia.org/wiki/Gram_stain#/media/File:Gram_stain_01.jpg).
+
+    Let's use those colors by specifying an explicit scale mapping from the data `domain` to the color `range`:
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle(size=80).encode(
+        alt.X('Neomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Neomycin MIC (μg/ml, reverse log scale)'),
+        alt.Y('Penicillin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Penicillin MIC (μg/ml, reverse log scale)'),
+        alt.Color('Gram_Staining:N',
+              scale=alt.Scale(domain=['negative', 'positive'], range=['hotpink', 'purple'])
+        )
+    ).properties(width=250, height=250)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    By default legends are placed on the right side of the chart. Similar to axes, we can change the legend orientation using the `orient` parameter:
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle(size=80).encode(
+        alt.X('Neomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Neomycin MIC (μg/ml, reverse log scale)'),
+        alt.Y('Penicillin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Penicillin MIC (μg/ml, reverse log scale)'),
+        alt.Color('Gram_Staining:N',
+              scale=alt.Scale(domain=['negative', 'positive'], range=['hotpink', 'purple']),
+              legend=alt.Legend(orient='left')
+        )
+    ).properties(width=250, height=250)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    We can also remove a legend entirely by specifying `legend=None`:
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle(size=80).encode(
+        alt.X('Neomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Neomycin MIC (μg/ml, reverse log scale)'),
+        alt.Y('Penicillin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Penicillin MIC (μg/ml, reverse log scale)'),
+        alt.Color('Gram_Staining:N',
+              scale=alt.Scale(domain=['negative', 'positive'], range=['hotpink', 'purple']),
+              legend=None
+        )
+    ).properties(width=250, height=250)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Color by Species
+
+    _So far we've considered the effectiveness of antibiotics. Let's turn around and ask a different question: what might antibiotic response teach us about the different species of bacteria?_
+
+    To start, let's encode `Bacteria` (a nominal data field) using the `color` channel:
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle(size=80).encode(
+        alt.X('Neomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Neomycin MIC (μg/ml, reverse log scale)'),
+        alt.Y('Penicillin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Penicillin MIC (μg/ml, reverse log scale)'),
+        alt.Color('Bacteria:N')
+    ).properties(width=250, height=250)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _The result is a bit of a mess!_ There are enough unique bacteria that Altair starts repeating colors from its default 10-color palette for nominal values.
+
+    To use custom colors, we can update the color encoding `scale` property. One option is to provide explicit scale `domain` and `range` values to indicate the precise color mappings per value, as we did above for Gram staining. Another option is to use an alternative color scheme. Altair includes a variety of built-in color schemes. For a complete list, see the [Vega color scheme documentation](https://vega.github.io/vega/docs/schemes/#reference).
+
+    Let's try switching to a built-in 20-color scheme, `tableau20`, and set that using the scale `scheme` property.
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle(size=80).encode(
+        alt.X('Neomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Neomycin MIC (μg/ml, reverse log scale)'),
+        alt.Y('Penicillin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Penicillin MIC (μg/ml, reverse log scale)'),
+        alt.Color('Bacteria:N',
+              scale=alt.Scale(scheme='tableau20'))
+    ).properties(width=250, height=250)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _We now have a unique color for each bacteria, but the chart is still a mess. Among other issues, the encoding takes no account of bacteria that belong to the same genus. In the chart above, the two different Salmonella strains have very different hues (teal and pink), despite being biological cousins._
+
+    To try a different scheme, we can also change the data type from nominal to ordinal. The default ordinal scheme uses blue shades, ramping from light to dark:
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle(size=80).encode(
+        alt.X('Neomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Neomycin MIC (μg/ml, reverse log scale)'),
+        alt.Y('Penicillin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Penicillin MIC (μg/ml, reverse log scale)'),
+        alt.Color('Bacteria:O')
+    ).properties(width=250, height=250)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Some of those blue shades may be hard to distinguish._
+
+    For more differentiated colors, we can experiment with alternatives to the default `blues` color scheme. The `viridis` scheme ramps through both hue and luminance:
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle(size=80).encode(
+        alt.X('Neomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Neomycin MIC (μg/ml, reverse log scale)'),
+        alt.Y('Penicillin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Penicillin MIC (μg/ml, reverse log scale)'),
+        alt.Color('Bacteria:O',
+              scale=alt.Scale(scheme='viridis'))
+    ).properties(width=250, height=250)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Bacteria from the same genus now have more similar colors than before, but the chart still remains confusing. There are many colors, they are hard to look up in the legend accurately, and two bacteria may have similar colors but different genus._
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Color by Genus
+
+    Let's try to color by genus instead of bacteria. To do so, we will add a `calculate` transform that splits up the bacteria name on space characters and takes the first word in the resulting array. We can then encode the resulting `Genus` field using the `tableau20` color scheme.
+
+    (Note that the antibiotics dataset includes a pre-calculated `Genus` field, but we will ignore it here in order to further explore Altair's data transformations.)
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle(size=80).transform_calculate(
+        Genus='split(datum.Bacteria, " ")[0]'
+    ).encode(
+        alt.X('Neomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Neomycin MIC (μg/ml, reverse log scale)'),
+        alt.Y('Penicillin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Penicillin MIC (μg/ml, reverse log scale)'),
+        alt.Color('Genus:N',
+              scale=alt.Scale(scheme='tableau20'))
+    ).properties(width=250, height=250)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Hmm... While the data are better segregated by genus, this cacapohony of colors doesn't seem particularly useful._
+
+    _If we look at some of the previous charts carefully, we can see that only a handful of bacteria have a genus shared with another bacteria: Salmonella, Staphylococcus, and Streptococcus. To focus our comparison, we might add colors only for these repeated genus values._
+
+    Let's add another `calculate` transform that takes a genus name, keeps it if it is one of the repeated values, and otherwise uses the string `"Other"`.
+
+    In addition, we can add custom color encodings using explicit `domain` and `range` arrays for the color encoding `scale`.
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_circle(size=80).transform_calculate(
+      Split='split(datum.Bacteria, " ")[0]'
+    ).transform_calculate(
+      Genus='indexof(["Salmonella", "Staphylococcus", "Streptococcus"], datum.Split) >= 0 ? datum.Split : "Other"'
+    ).encode(
+        alt.X('Neomycin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Neomycin MIC (μg/ml, reverse log scale)'),
+        alt.Y('Penicillin:Q',
+              sort='descending',
+              scale=alt.Scale(type='log', domain=[0.001, 1000]),
+              axis=alt.Axis(tickCount=5),
+              title='Penicillin MIC (μg/ml, reverse log scale)'),
+        alt.Color('Genus:N',
+              scale=alt.Scale(
+                domain=['Salmonella', 'Staphylococcus', 'Streptococcus', 'Other'],
+                range=['rgb(76,120,168)', 'rgb(84,162,75)', 'rgb(228,87,86)', 'rgb(121,112,110)']
+              ))
+    ).properties(width=250, height=250)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _We now have a much more revealing plot, made possible by customizations to the axes and legend. Take a moment to examine the plot above. Notice any surprising groupings?_
+
+    _The upper-left region has a cluster of red Streptococcus bacteria, but with a grey Other bacteria alongside them. Meanwhile, towards the middle-right we see another red Streptococcus placed far away from its "cousins". Might we expect bacteria from the same genus (and thus presumably more genetically similar) to be grouped closer together?_
+
+    As it so happens, the underlying dataset actually contains errors. The dataset reflects the species designations used in the early 1950s. However, the scientific consensus has since been overturned. That gray point in the upper-left? It's now considered a Streptococcus! That red point towards the middle-right? It's no longer considered a Streptococcus!
+
+    Of course, on its own, this dataset doesn't fully justify these reclassifications. Nevertheless, the data contain valuable biological clues that went overlooked for decades! Visualization, when used by an appropriately skilled and inquisitive viewer, can be a powerful tool for discovery.
+
+    This example also reinforces an important lesson: **_always be skeptical of your data!_**
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Color by Antibiotic Response
+
+    We might also use the `color` channel to encode quantitative values. Though keep in mind that typically color is not as effective for conveying quantities as position or size encodings!
+
+    Here is a basic heatmap of penicillin MIC values for each bacteria. We'll use a `rect` mark and sort the bacteria by descending MIC values (from most to least resistant):
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_rect().encode(
+        alt.Y('Bacteria:N',
+          sort=alt.EncodingSortField(field='Penicillin', op='max', order='descending')
+        ),
+        alt.Color('Penicillin:Q')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    We can further improve this chart by combining features we've seen thus far: a log-transformed scale, a change of axis orientation, a custom color scheme (`plasma`), tick count adjustment, and custom title text. We'll also exercise configuration options to adjust the axis title placement and legend title alignment.
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics).mark_rect().encode(
+        alt.Y('Bacteria:N',
+          sort=alt.EncodingSortField(field='Penicillin', op='max', order='descending'),
+          axis=alt.Axis(
+            orient='right',     # orient axis on right side of chart
+            titleX=7,           # set x-position to 7 pixels right of chart
+            titleY=-2,          # set y-position to 2 pixels above chart
+            titleAlign='left',  # use left-aligned text
+            titleAngle=0        # undo default title rotation
+          )
+        ),
+        alt.Color('Penicillin:Q',
+          scale=alt.Scale(type='log', scheme='plasma', nice=True),
+          legend=alt.Legend(titleOrient='right', tickCount=5),
+          title='Penicillin MIC (μg/ml)'
+        )
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Alternatively, we can remove the axis title altogether, and use the top-level `title` property to add a title for the entire chart:
+    """)
+    return
+
+
+@app.cell
+def _(alt, antibiotics):
+    alt.Chart(antibiotics, title='Penicillin Resistance of Bacterial Strains').mark_rect().encode(
+        alt.Y('Bacteria:N',
+          sort=alt.EncodingSortField(field='Penicillin', op='max', order='descending'),
+          axis=alt.Axis(orient='right', title=None)
+        ),
+        alt.Color('Penicillin:Q',
+          scale=alt.Scale(type='log', scheme='plasma', nice=True),
+          legend=alt.Legend(titleOrient='right', tickCount=5),
+          title='Penicillin MIC (μg/ml)'
+        )
+    ).configure_title(
+      anchor='start', # anchor and left-align title
+      offset=5        # set title offset from chart
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Summary
+
+    Integrating what we've learned across the notebooks so far about encodings, data transforms, and customization, you should now be prepared to make a wide variety of statistical graphics. Now you can put Altair into everyday use for exploring and communicating data!
+
+    Interested in learning more about this topic?
+
+    - Start with the [Altair Customizing Visualizations documentation](https://altair-viz.github.io/user_guide/customization.html).
+    - For a complementary discussion of scale mappings, see ["Introducing d3-scale"](https://medium.com/@mbostock/introducing-d3-scale-61980c51545f).
+    - For a more in-depth exploration of all the ways axes and legends can be styled by the underlying Vega library (which powers Altair and Vega-Lite), see ["A Guide to Guides: Axes & Legends in Vega"](https://beta.observablehq.com/@jheer/a-guide-to-guides-axes-legends-in-vega).
+    - For a fascinating history of the antibiotics dataset, see [Wainer & Lysen's "That's Funny..."](https://www.americanscientist.org/article/thats-funny) in the _American Scientist_.
+    """)
+    return
+
+
+if __name__ == "__main__":
+    app.run()

altair/05_view_composition.py

@@ -0,0 +1,818 @@
+# /// script
+# requires-python = ">=3.11"
+# dependencies = [
+#     "altair==6.0.0",
+#     "marimo",
+#     "pandas==3.0.1",
+# ]
+# ///
+
+import marimo
+
+__generated_with = "0.20.4"
+app = marimo.App()
+
+
+@app.cell
+def _():
+    import marimo as mo
+
+    return (mo,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    # Multi-View Composition
+
+    When visualizing a number of different data fields, we might be tempted to use as many visual encoding channels as we can: `x`, `y`, `color`, `size`, `shape`, and so on. However, as the number of encoding channels increases, a chart can rapidly become cluttered and difficult to read. An alternative to "over-loading" a single chart is to instead _compose multiple charts_ in a way that facilitates rapid comparisons.
+
+    In this notebook, we will examine a variety of operations for _multi-view composition_:
+
+    - _layer_: place compatible charts directly on top of each other,
+    - _facet_: partition data into multiple charts, organized in rows or columns,
+    - _concatenate_: position arbitrary charts within a shared layout, and
+    - _repeat_: take a base chart specification and apply it to multiple data fields.
+
+    We'll then look at how these operations form a _view composition algebra_, in which the operations can be combined to build a variety of complex multi-view displays.
+
+    _This notebook is part of the [data visualization curriculum](https://github.com/uwdata/visualization-curriculum)._
+    """)
+    return
+
+
+@app.cell
+def _():
+    import pandas as pd
+    import altair as alt
+
+    return alt, pd
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Weather Data
+
+    We will be visualizing weather statistics for the U.S. cities of Seattle and New York. Let's load the dataset and peek at the first and last 10 rows:
+    """)
+    return
+
+
+@app.cell
+def _():
+    weather = 'https://cdn.jsdelivr.net/npm/vega-datasets@1/data/weather.csv'
+    return (weather,)
+
+
+@app.cell
+def _(pd, weather):
+    df = pd.read_csv(weather)
+    df.head(10)
+    return (df,)
+
+
+@app.cell
+def _(df):
+    df.tail(10)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    We will create multi-view displays to examine weather within and across the cities.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Layer
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    One of the most common ways of combining multiple charts is to *layer* marks on top of each other. If the underlying scale domains are compatible, we can merge them to form _shared axes_. If either of the `x` or `y` encodings is not compatible, we might instead create a _dual-axis chart_, which overlays marks using separate scales and axes.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Shared Axes
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Let's start by plotting the minimum and maximum average temperatures per month:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    alt.Chart(weather).mark_area().encode(
+      alt.X('month(date):T'),
+      alt.Y('average(temp_max):Q'),
+      alt.Y2('average(temp_min):Q')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _The plot shows us temperature ranges for each month over the entirety of our data. However, this is pretty misleading as it aggregates the measurements for both Seattle and New York!_
+
+    Let's subdivide the data by location using a color encoding, while also adjusting the mark opacity to accommodate overlapping areas:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    alt.Chart(weather).mark_area(opacity=0.3).encode(
+      alt.X('month(date):T'),
+      alt.Y('average(temp_max):Q'),
+      alt.Y2('average(temp_min):Q'),
+      alt.Color('location:N')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _We can see that Seattle is more temperate: warmer in the winter, and cooler in the summer._
+
+    In this case we've created a layered chart without any special features by simply subdividing the area marks by color. While the chart above shows us the temperature ranges, we might also want to emphasize the middle of the range.
+
+    Let's create a line chart showing the average temperature midpoint. We'll use a `calculate` transform to compute the midpoints between the minimum and maximum daily temperatures:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    alt.Chart(weather).mark_line().transform_calculate(
+      temp_mid='(+datum.temp_min + +datum.temp_max) / 2'
+    ).encode(
+      alt.X('month(date):T'),
+      alt.Y('average(temp_mid):Q'),
+      alt.Color('location:N')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Aside_: note the use of `+datum.temp_min` within the calculate transform. As we are loading the data directly from a CSV file without any special parsing instructions, the temperature values may be internally represented as string values. Adding the  `+` in front of the value forces it to be treated as a number.
+
+    We'd now like to combine these charts by layering the midpoint lines over the range areas. Using the syntax `chart1 + chart2`, we can specify that we want a new layered chart in which `chart1` is the first layer and `chart2` is a second layer drawn on top:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    tempMinMax = alt.Chart(weather).mark_area(opacity=0.3).encode(
+      alt.X('month(date):T'),
+      alt.Y('average(temp_max):Q'),
+      alt.Y2('average(temp_min):Q'),
+      alt.Color('location:N')
+    )
+
+    tempMid = alt.Chart(weather).mark_line().transform_calculate(
+      temp_mid='(+datum.temp_min + +datum.temp_max) / 2'
+    ).encode(
+      alt.X('month(date):T'),
+      alt.Y('average(temp_mid):Q'),
+      alt.Color('location:N')
+    )
+
+    tempMinMax + tempMid
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Now we have a multi-layer plot! However, the y-axis title (though informative) has become a bit long and unruly..._
+
+    Let's customize our axes to clean up the plot. If we set a custom axis title within one of the layers, it will automatically be used as a shared axis title for all the layers:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    tempMinMax_1 = alt.Chart(weather).mark_area(opacity=0.3).encode(alt.X('month(date):T', title=None, axis=alt.Axis(format='%b')), alt.Y('average(temp_max):Q', title='Avg. Temperature °C'), alt.Y2('average(temp_min):Q'), alt.Color('location:N'))
+    tempMid_1 = alt.Chart(weather).mark_line().transform_calculate(temp_mid='(+datum.temp_min + +datum.temp_max) / 2').encode(alt.X('month(date):T'), alt.Y('average(temp_mid):Q'), alt.Color('location:N'))
+    tempMinMax_1 + tempMid_1
+    return tempMid_1, tempMinMax_1
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _What happens if both layers have custom axis titles? Modify the code above to find out..._
+
+    Above used the `+` operator, a convenient shorthand for Altair's `layer` method. We can generate an identical layered chart using the `layer` method directly:
+    """)
+    return
+
+
+@app.cell
+def _(alt, tempMid_1, tempMinMax_1):
+    alt.layer(tempMinMax_1, tempMid_1)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Note that the order of inputs to a layer matters, as subsequent layers will be drawn on top of earlier layers. _Try swapping the order of the charts in the cells above. What happens? (Hint: look closely at the color of the `line` marks.)_
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Dual-Axis Charts
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Seattle has a reputation as a rainy city. Is that deserved?_
+
+    Let's look at precipitation alongside temperature to learn more. First let's create a base plot the shows average monthly precipitation in Seattle:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    alt.Chart(weather).transform_filter(
+      'datum.location == "Seattle"'
+    ).mark_line(
+      interpolate='monotone',
+      stroke='grey'
+    ).encode(
+      alt.X('month(date):T', title=None),
+      alt.Y('average(precipitation):Q', title='Precipitation')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    To facilitate comparison with the temperature data, let's create a new layered chart. Here's what happens if we try to layer the charts as we did earlier:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    tempMinMax_2 = alt.Chart(weather).transform_filter('datum.location == "Seattle"').mark_area(opacity=0.3).encode(alt.X('month(date):T', title=None, axis=alt.Axis(format='%b')), alt.Y('average(temp_max):Q', title='Avg. Temperature °C'), alt.Y2('average(temp_min):Q'))
+    _precip = alt.Chart(weather).transform_filter('datum.location == "Seattle"').mark_line(interpolate='monotone', stroke='grey').encode(alt.X('month(date):T'), alt.Y('average(precipitation):Q', title='Precipitation'))
+    alt.layer(tempMinMax_2, _precip)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _The precipitation values use a much smaller range of the y-axis then the temperatures!_
+
+    By default, layered charts use a *shared domain*: the values for the x-axis or y-axis are combined across all the layers to determine a shared extent. This default behavior assumes that the layered values have the same units. However, this doesn't hold up for this example, as we are combining temperature values (degrees Celsius) with precipitation values (inches)!
+
+    If we want to use different y-axis scales, we need to specify how we want Altair to *resolve* the data across layers. In this case, we want to resolve the y-axis `scale` domains to be `independent` rather than use a `shared` domain. The `Chart` object produced by a layer operator includes a `resolve_scale` method with which we can specify the desired resolution:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    tempMinMax_3 = alt.Chart(weather).transform_filter('datum.location == "Seattle"').mark_area(opacity=0.3).encode(alt.X('month(date):T', title=None, axis=alt.Axis(format='%b')), alt.Y('average(temp_max):Q', title='Avg. Temperature °C'), alt.Y2('average(temp_min):Q'))
+    _precip = alt.Chart(weather).transform_filter('datum.location == "Seattle"').mark_line(interpolate='monotone', stroke='grey').encode(alt.X('month(date):T'), alt.Y('average(precipitation):Q', title='Precipitation'))
+    alt.layer(tempMinMax_3, _precip).resolve_scale(y='independent')
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _We can now see that autumn is the rainiest season in Seattle (peaking in November), complemented by dry summers._
+
+    You may have noticed some redundancy in our plot specifications above: both use the same dataset and the same filter to look at Seattle only. If you want, you can streamline the code a bit by providing the data and filter transform to the top-level layered chart. The individual layers will then inherit the data if they don't have their own data definitions:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    tempMinMax_4 = alt.Chart().mark_area(opacity=0.3).encode(alt.X('month(date):T', title=None, axis=alt.Axis(format='%b')), alt.Y('average(temp_max):Q', title='Avg. Temperature °C'), alt.Y2('average(temp_min):Q'))
+    _precip = alt.Chart().mark_line(interpolate='monotone', stroke='grey').encode(alt.X('month(date):T'), alt.Y('average(precipitation):Q', title='Precipitation'))
+    alt.layer(tempMinMax_4, _precip, data=weather).transform_filter('datum.location == "Seattle"').resolve_scale(y='independent')
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    While dual-axis charts can be useful, _they are often prone to misinterpretation_, as the different units and axis scales may be incommensurate. As is feasible, you might consider transformations that map different data fields to shared units, for example showing [quantiles](https://en.wikipedia.org/wiki/Quantile) or relative percentage change.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Facet
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    *Faceting* involves subdividing a dataset into groups and creating a separate plot for each group. In earlier notebooks, we learned how to create faceted charts using the `row` and `column` encoding channels. We'll first review those channels and then show how they are instances of the more general `facet` operator.
+
+    Let's start with a basic histogram of maximum temperature values in Seattle:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    alt.Chart(weather).mark_bar().transform_filter(
+      'datum.location == "Seattle"'
+    ).encode(
+      alt.X('temp_max:Q', bin=True, title='Temperature (°C)'),
+      alt.Y('count():Q')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _How does this temperature profile change based on the weather of a given day – that is, whether there was drizzle, fog, rain, snow, or sun?_
+
+    Let's use the `column` encoding channel to facet the data by weather type. We can also use `color` as a redundant encoding, using a customized color range:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    _colors = alt.Scale(domain=['drizzle', 'fog', 'rain', 'snow', 'sun'], range=['#aec7e8', '#c7c7c7', '#1f77b4', '#9467bd', '#e7ba52'])
+    alt.Chart(weather).mark_bar().transform_filter('datum.location == "Seattle"').encode(alt.X('temp_max:Q', bin=True, title='Temperature (°C)'), alt.Y('count():Q'), alt.Color('weather:N', scale=_colors), alt.Column('weather:N')).properties(width=150, height=150)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Unsurprisingly, those rare snow days center on the coldest temperatures, followed by rainy and foggy days. Sunny days are warmer and, despite Seattle stereotypes, are the most plentiful. Though as any Seattleite can tell you, the drizzle occasionally comes, no matter the temperature!_
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    In addition to `row` and `column` encoding channels *within* a chart definition, we can take a basic chart definition and apply faceting using an explicit `facet` operator.
+
+    Let's recreate the chart above, but this time using `facet`. We start with the same basic histogram definition, but remove the data source, filter transform, and column channel. We can then invoke the `facet` method, passing in the data and specifying that we should facet into columns according to the `weather` field. The `facet` method accepts both `row` and `column` arguments. The two can be used together to create a 2D grid of faceted plots.
+
+    Finally we include our filter transform, applying it to the top-level faceted chart. While we could apply the filter transform to the histogram definition as before, that is slightly less efficient. Rather than filter out "New York" values within each facet cell, applying the filter to the faceted chart lets Vega-Lite know that we can filter out those values up front, prior to the facet subdivision.
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    _colors = alt.Scale(domain=['drizzle', 'fog', 'rain', 'snow', 'sun'], range=['#aec7e8', '#c7c7c7', '#1f77b4', '#9467bd', '#e7ba52'])
+    alt.Chart().mark_bar().encode(alt.X('temp_max:Q', bin=True, title='Temperature (°C)'), alt.Y('count():Q'), alt.Color('weather:N', scale=_colors)).properties(width=150, height=150).facet(data=weather, column='weather:N').transform_filter('datum.location == "Seattle"')
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Given all the extra code above, why would we want to use an explicit `facet` operator? For basic charts, we should certainly use the `column` or `row` encoding channels if we can. However, using the `facet` operator explicitly is useful if we want to facet composed views, such as layered charts.
+
+    Let's revisit our layered temperature plots from earlier. Instead of plotting data for New York and Seattle in the same plot, let's break them up into separate facets. The individual chart definitions are nearly the same as before: one area chart and one line chart. The only difference is that this time we won't pass the data directly to the chart constructors; we'll wait and pass it to the facet operator later. We can layer the charts much as before, then invoke `facet` on the layered chart object, passing in the data and specifying `column` facets based on the `location` field:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    tempMinMax_5 = alt.Chart().mark_area(opacity=0.3).encode(alt.X('month(date):T', title=None, axis=alt.Axis(format='%b')), alt.Y('average(temp_max):Q', title='Avg. Temperature (°C)'), alt.Y2('average(temp_min):Q'), alt.Color('location:N'))
+    tempMid_2 = alt.Chart().mark_line().transform_calculate(temp_mid='(+datum.temp_min + +datum.temp_max) / 2').encode(alt.X('month(date):T'), alt.Y('average(temp_mid):Q'), alt.Color('location:N'))
+    alt.layer(tempMinMax_5, tempMid_2).facet(data=weather, column='location:N')
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The faceted charts we have seen so far use the same axis scale domains across the facet cells. This default of using *shared* scales and axes helps aid accurate comparison of values. However, in some cases you may wish to scale each chart independently, for example if the range of values in the cells differs significantly.
+
+    Similar to layered charts, faceted charts also support _resolving_ to independent scales or axes across plots. Let's see what happens if we call the `resolve_axis` method to request `independent` y-axes:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    tempMinMax_6 = alt.Chart().mark_area(opacity=0.3).encode(alt.X('month(date):T', title=None, axis=alt.Axis(format='%b')), alt.Y('average(temp_max):Q', title='Avg. Temperature (°C)'), alt.Y2('average(temp_min):Q'), alt.Color('location:N'))
+    tempMid_3 = alt.Chart().mark_line().transform_calculate(temp_mid='(+datum.temp_min + +datum.temp_max) / 2').encode(alt.X('month(date):T'), alt.Y('average(temp_mid):Q'), alt.Color('location:N'))
+    alt.layer(tempMinMax_6, tempMid_3).facet(data=weather, column='location:N').resolve_axis(y='independent')
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _The chart above looks largely unchanged, but the plot for Seattle now includes its own axis._
+
+    What if we instead call `resolve_scale` to resolve the underlying scale domains?
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    tempMinMax_7 = alt.Chart().mark_area(opacity=0.3).encode(alt.X('month(date):T', title=None, axis=alt.Axis(format='%b')), alt.Y('average(temp_max):Q', title='Avg. Temperature (°C)'), alt.Y2('average(temp_min):Q'), alt.Color('location:N'))
+    tempMid_4 = alt.Chart().mark_line().transform_calculate(temp_mid='(+datum.temp_min + +datum.temp_max) / 2').encode(alt.X('month(date):T'), alt.Y('average(temp_mid):Q'), alt.Color('location:N'))
+    alt.layer(tempMinMax_7, tempMid_4).facet(data=weather, column='location:N').resolve_scale(y='independent')
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Now we see facet cells with different axis scale domains. In this case, using independent scales seems like a bad idea! The domains aren't very different, and one might be fooled into thinking that New York and Seattle have similar maximum summer temperatures._
+
+    To borrow a cliché: just because you *can* do something, doesn't mean you *should*...
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Concatenate
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Faceting creates [small multiple](https://en.wikipedia.org/wiki/Small_multiple) plots that show separate subdivisions of the data. However, we might wish to create a multi-view display with different views of the *same* dataset (not subsets) or views involving *different* datasets.
+
+    Altair provides *concatenation* operators to combine arbitrary charts into a composed chart. The `hconcat` operator (shorthand `|` ) performs horizontal concatenation, while the `vconcat` operator (shorthand `&`) performs vertical concatenation.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Let's start with a basic line chart showing the average maximum temperature per month for both New York and Seattle, much like we've seen before:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    alt.Chart(weather).mark_line().encode(
+      alt.X('month(date):T', title=None),
+      alt.Y('average(temp_max):Q'),
+      color='location:N'
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _What if we want to compare not just temperature over time, but also precipitation and wind levels?_
+
+    Let's create a concatenated chart consisting of three plots. We'll start by defining a "base" chart definition that contains all the aspects that should be shared by our three plots. We can then modify this base chart to create customized variants, with different y-axis encodings for the `temp_max`, `precipitation`, and `wind` fields. We can then concatenate them using the pipe (`|`) shorthand operator:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    base = alt.Chart(weather).mark_line().encode(alt.X('month(date):T', title=None), color='location:N').properties(width=240, height=180)
+    temp = base.encode(alt.Y('average(temp_max):Q'))
+    _precip = base.encode(alt.Y('average(precipitation):Q'))
+    wind = base.encode(alt.Y('average(wind):Q'))
+    temp | _precip | wind
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Alternatively, we could use the more explicit `alt.hconcat()` method in lieu of the pipe `|` operator. _Try rewriting the code above to use `hconcat` instead._
+
+    Vertical concatenation works similarly to horizontal concatenation. _Using the `&` operator (or `alt.vconcat` method), modify the code to use a vertical ordering instead of a horizontal ordering._
+
+    Finally, note that horizontal and vertical concatenation can be combined. _What happens if you write something like `(temp | precip) & wind`?_
+
+    _Aside_: Note the importance of those parentheses... what happens if you remove them? Keep in mind that these overloaded operators are still subject to [Python's operator precendence rules](https://docs.python.org/3/reference/expressions.html#operator-precedence), and so vertical concatenation with `&` will take precedence over horizontal concatenation with `|`!
+
+    As we will revisit later, concatenation operators let you combine any and all charts into a multi-view dashboard!
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Repeat
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The concatenation operators above are quite general, allowing arbitrary charts to be composed. Nevertheless, the example above was still a bit verbose: we have three very similar charts, yet have to define them separately and then concatenate them.
+
+    For cases where only one or two variables are changing, the `repeat` operator provides a convenient shortcut for creating multiple charts. Given a *template* specification with some free variables, the repeat operator will then create a chart for each specified assignment to those variables.
+
+    Let's recreate our concatenation example above using the `repeat` operator. The only aspect that changes across charts is the choice of data field for the `y` encoding channel. To create a template specification, we can use the *repeater variable* `alt.repeat('column')` as our y-axis field. This code simply states that we want to use the variable assigned to the `column` repeater, which organizes repeated charts in a horizontal direction. (As the repeater provides the field name only, we have to specify the field data type separately as `type='quantitative'`.)
+
+    We then invoke the `repeat` method, passing in data field names for each column:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    alt.Chart(weather).mark_line().encode(
+      alt.X('month(date):T',title=None),
+      alt.Y(alt.repeat('column'), aggregate='average', type='quantitative'),
+      color='location:N'
+    ).properties(
+      width=240,
+      height=180
+    ).repeat(
+      column=['temp_max', 'precipitation', 'wind']
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Repetition is supported for both columns and rows. _What happens if you modify the code above to use `row` instead of `column`?_
+
+    We can also use `row` and `column` repetition together! One common visualization for exploratory data analysis is the [scatter plot matrix (or SPLOM)](https://en.wikipedia.org/wiki/Scatter_plot#Scatterplot_matrices). Given a collection of variables to inspect, a SPLOM provides a grid of all pairwise plots of those variables, allowing us to assess potential associations.
+
+    Let's use the `repeat` operator to create a SPLOM for the `temp_max`, `precipitation`, and `wind` fields. We first create our template specification, with repeater variables for both the x- and y-axis data fields. We then invoke `repeat`, passing in arrays of field names to use for both `row` and `column`. Altair will then generate the [cross product (or, Cartesian product)](https://en.wikipedia.org/wiki/Cartesian_product) to create the full space of repeated charts:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    alt.Chart().mark_point(filled=True, size=15, opacity=0.5).encode(
+      alt.X(alt.repeat('column'), type='quantitative'),
+      alt.Y(alt.repeat('row'), type='quantitative')
+    ).properties(
+      width=150,
+      height=150
+    ).repeat(
+      data=weather,
+      row=['temp_max', 'precipitation', 'wind'],
+      column=['wind', 'precipitation', 'temp_max']
+    ).transform_filter(
+      'datum.location == "Seattle"'
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Looking at these plots, there does not appear to be a strong association between precipitation and wind, though we do see that extreme wind and precipitation events occur in similar temperature ranges (~5-15° C). However, this observation is not particularly surprising: if we revisit our histogram at the beginning of the facet section, we can plainly see that the days with maximum temperatures in the range of 5-15° C are the most commonly occurring._
+
+    *Modify the code above to get a better understanding of chart repetition. Try adding another variable (`temp_min`) to the SPLOM. What happens if you rearrange the order of the field names in either the `row` or `column` parameters for the `repeat` operator?*
+
+    _Finally, to really appreciate what the `repeat` operator provides, take a moment to imagine how you might recreate the SPLOM above using only `hconcat` and `vconcat`!_
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## A View Composition Algebra
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Together, the composition operators `layer`, `facet`, `concat`, and `repeat` form a *view composition algebra*: the various operators can be combined to construct a variety of multi-view visualizations.
+
+    As an example, let's start with two basic charts: a histogram and a simple line (a single `rule` mark) showing a global average.
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    basic1 = alt.Chart(weather).transform_filter(
+      'datum.location == "Seattle"'
+    ).mark_bar().encode(
+      alt.X('month(date):O'),
+      alt.Y('average(temp_max):Q')
+    )
+
+    basic2 = alt.Chart(weather).transform_filter(
+      'datum.location == "Seattle"'
+    ).mark_rule(stroke='firebrick').encode(
+      alt.Y('average(temp_max):Q')
+    )
+
+    basic1 | basic2
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    We can then combine the two charts using a `layer` operator, and then `repeat` that layered chart to show histograms with overlaid averages for multiple fields:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    alt.layer(
+      alt.Chart().mark_bar().encode(
+        alt.X('month(date):O', title='Month'),
+        alt.Y(alt.repeat('column'), aggregate='average', type='quantitative')
+      ),
+      alt.Chart().mark_rule(stroke='firebrick').encode(
+        alt.Y(alt.repeat('column'), aggregate='average', type='quantitative')
+      )
+    ).properties(
+      width=200,
+      height=150
+    ).repeat(
+      data=weather,
+      column=['temp_max', 'precipitation', 'wind']
+    ).transform_filter(
+      'datum.location == "Seattle"'
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Focusing only on the multi-view composition operators, the model for the visualization above is:
+
+    ```
+    repeat(column=[...])
+    |- layer
+       |- basic1
+       |- basic2
+    ```
+
+    Now let's explore how we can apply *all* the operators within a final [dashboard](https://en.wikipedia.org/wiki/Dashboard_%28business%29) that provides an overview of Seattle weather. We'll combine the SPLOM and faceted histogram displays from earlier sections with the repeated histograms above:
+    """)
+    return
+
+
+@app.cell
+def _(alt, weather):
+    splom = alt.Chart().mark_point(filled=True, size=15, opacity=0.5).encode(
+      alt.X(alt.repeat('column'), type='quantitative'),
+      alt.Y(alt.repeat('row'), type='quantitative')
+    ).properties(
+      width=125,
+      height=125
+    ).repeat(
+      row=['temp_max', 'precipitation', 'wind'],
+      column=['wind', 'precipitation', 'temp_max']
+    )
+
+    dateHist = alt.layer(
+      alt.Chart().mark_bar().encode(
+        alt.X('month(date):O', title='Month'),
+        alt.Y(alt.repeat('row'), aggregate='average', type='quantitative')
+      ),
+      alt.Chart().mark_rule(stroke='firebrick').encode(
+        alt.Y(alt.repeat('row'), aggregate='average', type='quantitative')
+      )
+    ).properties(
+      width=175,
+      height=125
+    ).repeat(
+      row=['temp_max', 'precipitation', 'wind']
+    )
+
+    tempHist = alt.Chart(weather).mark_bar().encode(
+      alt.X('temp_max:Q', bin=True, title='Temperature (°C)'),
+      alt.Y('count():Q'),
+      alt.Color('weather:N', scale=alt.Scale(
+        domain=['drizzle', 'fog', 'rain', 'snow', 'sun'],
+        range=['#aec7e8', '#c7c7c7', '#1f77b4', '#9467bd', '#e7ba52']
+      ))
+    ).properties(
+      width=115,
+      height=100
+    ).facet(
+      column='weather:N'
+    )
+
+    alt.vconcat(
+      alt.hconcat(splom, dateHist),
+      tempHist,
+      data=weather,
+      title='Seattle Weather Dashboard'
+    ).transform_filter(
+      'datum.location == "Seattle"'
+    ).resolve_legend(
+      color='independent'
+    ).configure_axis(
+      labelAngle=0
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The full composition model for this dashboard is:
+
+    ```
+    vconcat
+    |- hconcat
+    |  |- repeat(row=[...], column=[...])
+    |  |  |- splom base chart
+    |  |- repeat(row=[...])
+    |     |- layer
+    |        |- dateHist base chart 1
+    |        |- dateHist base chart 2
+    |- facet(column='weather')
+       |- tempHist base chart
+    ```
+
+    _Phew!_ The dashboard also includes a few customizations to improve the layout:
+
+    - We adjust chart `width` and `height` properties to assist alignment and ensure the full visualization fits on the screen.
+    - We add `resolve_legend(color='independent')` to ensure the color legend is associated directly with the colored histograms by temperature. Otherwise, the legend will resolve to the dashboard as a whole.
+    - We use `configure_axis(labelAngle=0)` to ensure that no axis labels are rotated. This helps to ensure proper alignment among the scatter plots in the SPLOM and the histograms by month on the right.
+
+    _Try removing or modifying any of these adjustments and see how the dashboard layout responds!_
+
+    This dashboard can be reused to show data for other locations or from other datasets. _Update the dashboard to show weather patterns for New York instead of Seattle._
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Summary
+
+    For more details on multi-view composition, including control over sub-plot spacing and header labels, see the [Altair Compound Charts documentation](https://altair-viz.github.io/user_guide/compound_charts.html).
+
+    Now that we've seen how to compose multiple views, we're ready to put them into action. In addition to statically presenting data, multiple views can enable interactive multi-dimensional exploration. For example, using _linked selections_ we can highlight points in one view to see corresponding values highlight in other views.
+
+    In the next notebook, we'll examine how to author *interactive selections* for both individual plots and multi-view compositions.
+    """)
+    return
+
+
+if __name__ == "__main__":
+    app.run()

altair/06_interaction.py

@@ -0,0 +1,671 @@
+# /// script
+# requires-python = ">=3.11"
+# dependencies = [
+#     "altair==6.0.0",
+#     "marimo",
+#     "pandas==3.0.1",
+# ]
+# ///
+
+import marimo
+
+__generated_with = "0.20.4"
+app = marimo.App()
+
+
+@app.cell
+def _():
+    import marimo as mo
+
+    return (mo,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    # Interaction
+
+    _“A graphic is not ‘drawn’ once and for all; it is ‘constructed’ and reconstructed until it reveals all the relationships constituted by the interplay of the data. The best graphic operations are those carried out by the decision-maker themself.”_ — [Jacques Bertin](https://books.google.com/books?id=csqX_xnm4tcC)
+
+    Visualization provides a powerful means of making sense of data. A single image, however, typically provides answers to, at best, a handful of questions. Through _interaction_ we can transform static images into tools for exploration: highlighting points of interest, zooming in to reveal finer-grained patterns, and linking across multiple views to reason about multi-dimensional relationships.
+
+    At the core of interaction is the notion of a _selection_: a means of indicating to the computer which elements or regions we are interested in. For example, we might hover the mouse over a point, click multiple marks, or draw a bounding box around a region to highlight subsets of the data for further scrutiny.
+
+    Alongside visual encodings and data transformations, Altair provides a _selection_ abstraction for authoring interactions. These selections encompass three aspects:
+
+    1. Input event handling to select points or regions of interest, such as mouse hover, click, drag, scroll, and touch events.
+    2. Generalizing from the input to form a selection rule (or [_predicate_](https://en.wikipedia.org/wiki/Predicate_%28mathematical_logic%29)) that determines whether or not a given data record lies within the selection.
+    3. Using the selection predicate to dynamically configure a visualization by driving _conditional encodings_, _filter transforms_, or _scale domains_.
+
+    This notebook introduces interactive selections and explores how to use them to author a variety of interaction techniques, such as dynamic queries, panning & zooming, details-on-demand, and brushing & linking.
+
+    _This notebook is part of the [data visualization curriculum](https://github.com/uwdata/visualization-curriculum)._
+    """)
+    return
+
+
+@app.cell
+def _():
+    import pandas as pd
+    import altair as alt
+
+    return alt, pd
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Datasets
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    We will visualize a variety of datasets from the [vega-datasets](https://github.com/vega/vega-datasets) collection:
+
+    - A dataset of `cars` from the 1970s and early 1980s,
+    - A dataset of `movies`, previously used in the [Data Transformation](https://github.com/uwdata/visualization-curriculum/blob/master/altair_data_transformation.ipynb) notebook,
+    - A dataset containing ten years of [S&P 500](https://en.wikipedia.org/wiki/S%26P_500_Index) (`sp500`) stock prices,
+    - A dataset of technology company `stocks`, and
+    - A dataset of `flights`, including departure time, distance, and arrival delay.
+    """)
+    return
+
+
+@app.cell
+def _():
+    cars = 'https://cdn.jsdelivr.net/npm/vega-datasets@1/data/cars.json'
+    movies = 'https://cdn.jsdelivr.net/npm/vega-datasets@1/data/movies.json'
+    sp500 = 'https://cdn.jsdelivr.net/npm/vega-datasets@1/data/sp500.csv'
+    stocks = 'https://cdn.jsdelivr.net/npm/vega-datasets@1/data/stocks.csv'
+    flights = 'https://cdn.jsdelivr.net/npm/vega-datasets@1/data/flights-5k.json'
+    return cars, flights, movies, sp500, stocks
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Introducing Selections
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Let's start with a basic selection: simply clicking a point to highlight it. Using the `cars` dataset, we'll start with a scatter plot of horsepower versus miles per gallon, with a color encoding for the number cylinders in the car engine.
+
+    In addition, we'll create a selection instance by calling `alt.selection_single()`, indicating we want a selection defined over a _single value_. By default, the selection uses a mouse click to determine the selected value. To register a selection with a chart, we must add it using the `.add_params()` method.
+
+    Once our selection has been defined, we can use it as a parameter for _conditional encodings_, which apply a different encoding depending on whether a data record lies in or out of the selection. For example, consider the following code:
+
+    ~~~ python
+    color=alt.condition(selection, 'Cylinders:O', alt.value('grey'))
+    ~~~
+
+    This encoding definition states that data points contained within the `selection` should be colored according to the `Cylinder` field, while non-selected data points should use a default `grey`. An empty selection includes _all_ data points, and so initially all points will be colored.
+
+    _Try clicking different points in the chart below. What happens? (Click the background to clear the selection state and return to an "empty" selection.)_
+    """)
+    return
+
+
+@app.cell
+def _(alt, cars):
+    _selection = alt.selection_point(toggle=False)
+    alt.Chart(cars).mark_circle().add_params(_selection).encode(x='Horsepower:Q', y='Miles_per_Gallon:Q', color=alt.condition(_selection, 'Cylinders:O', alt.value('grey')), opacity=alt.condition(_selection, alt.value(0.8), alt.value(0.1)))
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Of course, highlighting individual data points one-at-a-time is not particularly exciting! As we'll see, however, single value selections provide a useful building block for more powerful interactions. Moreover, single value selections are just one of the three selection types provided by Altair:
+
+    - `selection_single` - select a single discrete value, by default on click events.
+    - `selection_multi` - select multiple discrete values. The first value is selected on mouse click and additional values toggled using shift-click.
+    - `selection_interval` - select a continuous range of values, initiated by mouse drag.
+
+    Let's compare each of these selection types side-by-side. To keep our code tidy we'll first define a function (`plot`) that generates a scatter plot specification just like the one above. We can pass a selection to the `plot` function to have it applied to the chart:
+    """)
+    return
+
+
+@app.cell
+def _(alt, cars):
+    def plot(selection):
+        return alt.Chart(cars).mark_circle().add_params(selection).encode(x='Horsepower:Q', y='Miles_per_Gallon:Q', color=alt.condition(selection, 'Cylinders:O', alt.value('grey')), opacity=alt.condition(selection, alt.value(0.8), alt.value(0.1))).properties(width=240, height=180)
+
+    return (plot,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Let's use our `plot` function to create three chart variants, one per selection type.
+
+    The first (`single`) chart replicates our earlier example. The second (`multi`) chart supports shift-click interactions to toggle inclusion of multiple points within the selection. The third (`interval`) chart generates a selection region (or _brush_) upon mouse drag. Once created, you can drag the brush around to select different points, or scroll when the cursor is inside the brush to scale (zoom) the brush size.
+
+    _Try interacting with each of the charts below!_
+    """)
+    return
+
+
+@app.cell
+def _(alt, plot):
+    alt.hconcat(
+      plot(alt.selection_point(toggle=False)).properties(title='Single (Click)'),
+      plot(alt.selection_point()).properties(title='Multi (Shift-Click)'),
+      plot(alt.selection_interval()).properties(title='Interval (Drag)')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The examples above use default interactions (click, shift-click, drag) for each selection type. We can further customize the interactions by providing input event specifications using [Vega event selector syntax](https://vega.github.io/vega/docs/event-streams/). For example, we can modify our `single` and `multi` charts to trigger upon `mouseover` events instead of `click` events.
+
+    _Hold down the shift key in the second chart to "paint" with data!_
+    """)
+    return
+
+
+@app.cell
+def _(alt, plot):
+    alt.hconcat(
+      plot(alt.selection_point(toggle=False, on='mouseover')).properties(title='Single (Mouseover)'),
+      plot(alt.selection_point(on='mouseover')).properties(title='Multi (Shift-Mouseover)')
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Now that we've covered the basics of Altair selections, let's take a tour through the various interaction techniques they enable!
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Dynamic Queries
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Dynamic queries_ enables rapid, reversible exploration of data to isolate patterns of interest. As defined by [Ahlberg, Williamson, & Shneiderman](https://www.cs.umd.edu/~ben/papers/Ahlberg1992Dynamic.pdf), a dynamic query:
+
+    - represents a query graphically,
+    - provides visible limits on the query range,
+    - provides a graphical representation of the data and query result,
+    - gives immediate feedback of the result after every query adjustment,
+    - and allows novice users to begin working with little training.
+
+    A common approach is to manipulate query parameters using standard user interface widgets such as sliders, radio buttons, and drop-down menus. To generate dynamic query widgets, we can apply a selection's `bind` operation to one or more data fields we wish to query.
+
+    Let's build an interactive scatter plot that uses a dynamic query to filter the display. Given a scatter plot of movie ratings (from Rotten Tomates and IMDB), we can add a selection over the `Major_Genre` field to enable interactive filtering by film genre.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    To start, let's extract the unique (non-null) genres from the `movies` data:
+    """)
+    return
+
+
+@app.cell
+def _(movies, pd):
+    df = pd.read_json(movies) # load movies data
+    genres = df['Major_Genre'].unique() # get unique field values
+    genres = list(filter(pd.notna, genres)) # filter out None/NaN values
+    genres.sort() # sort alphabetically
+    return (genres,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    For later use, let's also define a list of unique `MPAA_Rating` values:
+    """)
+    return
+
+
+@app.cell
+def _():
+    mpaa = ['G', 'PG', 'PG-13', 'R', 'NC-17', 'Not Rated']
+    return (mpaa,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Now let's create a `single` selection bound to a drop-down menu.
+
+    *Use the dynamic query menu below to explore the data. How do ratings vary by genre? How would you revise the code to filter `MPAA_Rating` (G, PG, PG-13, etc.) instead of `Major_Genre`?*
+    """)
+    return
+
+
+@app.cell
+def _(alt, genres, movies):
+    selectGenre = alt.selection_point(
+        toggle=False,
+        name='Select', # name the selection 'Select'
+        fields=['Major_Genre'], # limit selection to the Major_Genre field
+        value=[{'Major_Genre': genres[0]}], # use first genre entry as initial value
+        bind=alt.binding_select(options=genres) # bind to a menu of unique genre values
+    )
+
+    alt.Chart(movies).mark_circle().add_params(
+        selectGenre
+    ).encode(
+        x='Rotten_Tomatoes_Rating:Q',
+        y='IMDB_Rating:Q',
+        tooltip='Title:N',
+        opacity=alt.condition(selectGenre, alt.value(0.75), alt.value(0.05))
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Our construction above leverages multiple aspects of selections:
+
+    - We give the selection a name (`'Select'`). This name is not required, but allows us to influence the label text of the generated dynamic query menu. (_What happens if you remove the name? Try it!_)
+    - We constrain the selection to a specific data field (`Major_Genre`). Earlier when we used a `single` selection, the selection mapped to individual data points. By limiting the selection to a specific field, we can select _all_ data points whose `Major_Genre` field value matches the single selected value.
+    - We initialize `init=...` the selection to a starting value.
+    - We `bind` the selection to an interface widget, in this case a drop-down menu via `binding_select`.
+    - As before, we then use a conditional encoding to control the opacity channel.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Binding Selections to Multiple Inputs
+
+    One selection instance can be bound to _multiple_ dynamic query widgets. Let's modify the example above to provide filters for _both_ `Major_Genre` and `MPAA_Rating`, using radio buttons instead of a menu. Our `single` selection is now defined over a single _pair_ of genre and MPAA rating values
+
+    _Look for surprising conjunctions of genre and rating. Are there any G or PG-rated horror films?_
+    """)
+    return
+
+
+@app.cell
+def _(alt, genres, movies, mpaa):
+    # single-value selection over [Major_Genre, MPAA_Rating] pairs
+    # use specific hard-wired values as the initial selected values
+    _selection = alt.selection_point(toggle=False, name='Select', fields=['Major_Genre', 'MPAA_Rating'], value=[{'Major_Genre': 'Drama', 'MPAA_Rating': 'R'}], bind={'Major_Genre': alt.binding_select(options=genres), 'MPAA_Rating': alt.binding_radio(options=mpaa)})
+    # scatter plot, modify opacity based on selection
+    alt.Chart(movies).mark_circle().add_params(_selection).encode(x='Rotten_Tomatoes_Rating:Q', y='IMDB_Rating:Q', tooltip='Title:N', opacity=alt.condition(_selection, alt.value(0.75), alt.value(0.05)))
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Fun facts: The PG-13 rating didn't exist when the movies [Jaws](https://www.imdb.com/title/tt0073195/) and [Jaws 2](https://www.imdb.com/title/tt0077766/) were released. The first film to receive a PG-13 rating was 1984's [Red Dawn](https://www.imdb.com/title/tt0087985/)._
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Using Visualizations as Dynamic Queries
+
+    Though standard interface widgets show the _possible_ query parameter values, they do not visualize the _distribution_ of those values. We might also wish to use richer interactions, such as multi-value or interval selections, rather than input widgets that select only a single value at a time.
+
+    To address these issues, we can author additional charts to both visualize data and support dynamic queries. Let's add a histogram of the count of films per year and use an interval selection to dynamically highlight films over selected time periods.
+
+    *Interact with the year histogram to explore films from different time periods. Do you seen any evidence of [sampling bias](https://en.wikipedia.org/wiki/Sampling_bias) across the years? (How do year and critics' ratings relate?)*
+
+    _The years range from 1930 to 2040! Are future films in pre-production, or are there "off-by-one century" errors? Also, depending on which time zone you're in, you may see a small bump in either 1969 or 1970. Why might that be? (See the end of the notebook for an explanation!)_
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies):
+    _brush = alt.selection_interval(encodings=['x'])
+    years = alt.Chart(movies).mark_bar().add_params(_brush).encode(alt.X('year(Release_Date):T', title='Films by Release Year'), alt.Y('count():Q', title=None)).properties(width=650, height=50)  # limit selection to x-axis (year) values
+    ratings = alt.Chart(movies).mark_circle().encode(x='Rotten_Tomatoes_Rating:Q', y='IMDB_Rating:Q', tooltip='Title:N', opacity=alt.condition(_brush, alt.value(0.75), alt.value(0.05))).properties(width=650, height=400)
+    # dynamic query histogram
+    # scatter plot, modify opacity based on selection
+    alt.vconcat(years, ratings).properties(spacing=5)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The example above provides dynamic queries using a _linked selection_ between charts:
+
+    - We create an `interval` selection (`brush`), and set `encodings=['x']` to limit the selection to the x-axis only, resulting in a one-dimensional selection interval.
+    - We register `brush` with our histogram of films per year via `.add_params(brush)`.
+    - We use `brush` in a conditional encoding to adjust the scatter plot `opacity`.
+
+    This interaction technique of selecting elements in one chart and seeing linked highlights in one or more other charts is known as [_brushing & linking_](https://en.wikipedia.org/wiki/Brushing_and_linking).
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Panning & Zooming
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The movie rating scatter plot is a bit cluttered in places, making it hard to examine points in denser regions. Using the interaction techniques of _panning_ and _zooming_, we can inspect dense regions more closely.
+
+    Let's start by thinking about how we might express panning and zooming using Altair selections. What defines the "viewport" of a chart? _Axis scale domains!_
+
+    We can change the scale domains to modify the visualized range of data values. To do so interactively, we can bind an `interval` selection to scale domains with the code `bind='scales'`. The result is that instead of an interval brush that we can drag and zoom, we instead can drag and zoom the entire plotting area!
+
+    _In the chart below, click and drag to pan (translate) the view, or scroll to zoom (scale) the view. What can you discover about the precision of the provided rating values?_
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies):
+    alt.Chart(movies).mark_circle().add_params(
+        alt.selection_interval(bind='scales')
+    ).encode(
+        x='Rotten_Tomatoes_Rating:Q',
+        y=alt.Y('IMDB_Rating:Q', axis=alt.Axis(minExtent=30)), # use min extent to stabilize axis title placement
+        tooltip=['Title:N', 'Release_Date:N', 'IMDB_Rating:Q', 'Rotten_Tomatoes_Rating:Q']
+    ).properties(
+        width=600,
+        height=400
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Zooming in, we can see that the rating values have limited precision! The Rotten Tomatoes ratings are integers, while the IMDB ratings are truncated to tenths. As a result, there is overplotting even when we zoom, with multiple movies sharing the same rating values._
+
+    Reading the code above, you may notice the code `alt.Axis(minExtent=30)` in the `y` encoding channel. The `minExtent` parameter ensures a minimum amount of space is reserved for axis ticks and labels. Why do this? When we pan and zoom, the axis labels may change and cause the axis title position to shift. By setting a minimum extent we can reduce distracting movements in the plot. _Try changing the `minExtent` value, for example setting it to zero, and then zoom out to see what happens when longer axis labels enter the view._
+
+    Altair also includes a shorthand for adding panning and zooming to a plot. Instead of directly creating a selection, you can call `.interactive()` to have Altair automatically generate an interval selection bound to the chart's scales:
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies):
+    alt.Chart(movies).mark_circle().encode(
+        x='Rotten_Tomatoes_Rating:Q',
+        y=alt.Y('IMDB_Rating:Q', axis=alt.Axis(minExtent=30)), # use min extent to stabilize axis title placement
+        tooltip=['Title:N', 'Release_Date:N', 'IMDB_Rating:Q', 'Rotten_Tomatoes_Rating:Q']
+    ).properties(
+        width=600,
+        height=400
+    ).interactive()
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    By default, scale bindings for selections include both the `x` and `y` encoding channels. What if we want to limit panning and zooming along a single dimension? We can invoke `encodings=['x']` to constrain the selection to the `x` channel only:
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies):
+    alt.Chart(movies).mark_circle().add_params(
+        alt.selection_interval(bind='scales', encodings=['x'])
+    ).encode(
+        x='Rotten_Tomatoes_Rating:Q',
+        y=alt.Y('IMDB_Rating:Q', axis=alt.Axis(minExtent=30)), # use min extent to stabilize axis title placement
+        tooltip=['Title:N', 'Release_Date:N', 'IMDB_Rating:Q', 'Rotten_Tomatoes_Rating:Q']
+    ).properties(
+        width=600,
+        height=400
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _When zooming along a single axis only, the shape of the visualized data can change, potentially affecting our perception of relationships in the data. [Choosing an appropriate aspect ratio](http://vis.stanford.edu/papers/arclength-banking) is an important visualization design concern!_
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Navigation: Overview + Detail
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    When panning and zooming, we directly adjust the "viewport" of a chart. The related navigation strategy of _overview + detail_ instead uses an overview display to show _all_ of the data, while supporting selections that pan and zoom a separate focus display.
+
+    Below we have two area charts showing a decade of price fluctuations for the S&P 500 stock index. Initially both charts show the same data range. _Click and drag in the bottom overview chart to update the focus display and examine specific time spans._
+    """)
+    return
+
+
+@app.cell
+def _(alt, sp500):
+    _brush = alt.selection_interval(encodings=['x'])
+    _base = alt.Chart().mark_area().encode(alt.X('date:T', title=None), alt.Y('price:Q')).properties(width=700)
+    alt.vconcat(_base.encode(alt.X('date:T', title=None, scale=alt.Scale(domain=_brush))), _base.add_params(_brush).properties(height=60), data=sp500)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Unlike our earlier panning & zooming case, here we don't want to bind a selection directly to the scales of a single interactive chart. Instead, we want to bind the selection to a scale domain in _another_ chart. To do so, we update the `x` encoding channel for our focus chart, setting the scale `domain` property to reference our `brush` selection. If no interval is defined (the selection is empty), Altair ignores the brush and uses the underlying data to determine the domain. When a brush interval is created, Altair instead uses that as the scale `domain` for the focus chart.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Details on Demand
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Once we spot points of interest within a visualization, we often want to know more about them. _Details-on-demand_ refers to interactively querying for more information about selected values. _Tooltips_ are one useful means of providing details on demand. However, tooltips typically only show information for one data point at a time. How might we show more?
+
+    The movie ratings scatterplot includes a number of potentially interesting outliers where the Rotten Tomatoes and IMDB ratings disagree. Let's create a plot that allows us to interactively select points and show their labels. To trigger the filter query on either the hover or click interaction, we will use the [Altair composition operator](https://altair-viz.github.io/user_guide/interactions.html#composing-multiple-selections) `|` ("or").
+
+    _Mouse over points in the scatter plot below to see a highlight and title label. Shift-click points to make annotations persistent and view multiple labels at once. Which movies are loved by Rotten Tomatoes critics, but not the general audience on IMDB (or vice versa)? See if you can find possible errors, where two different movies with the same name were accidentally combined!_
+    """)
+    return
+
+
+@app.cell
+def _(alt, movies):
+    hover = alt.selection_point(toggle=False, on='mouseover', nearest=True, empty=False)
+    click = alt.selection_point(empty=False)
+    plot_1 = alt.Chart().mark_circle().encode(x='Rotten_Tomatoes_Rating:Q', y='IMDB_Rating:Q')
+    _base = plot_1.transform_filter(hover | click)
+    alt.layer(plot_1.add_params(hover).add_params(click), _base.mark_point(size=100, stroke='firebrick', strokeWidth=1), _base.mark_text(dx=4, dy=-8, align='right', stroke='white', strokeWidth=2).encode(text='Title:N'), _base.mark_text(dx=4, dy=-8, align='right').encode(text='Title:N'), data=movies).properties(width=600, height=450)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The example above adds three new layers to the scatter plot: a circular annotation, white text to provide a legible background, and black text showing a film title. In addition, this example uses two selections in tandem:
+
+    1. A single selection (`hover`) that includes `nearest=True` to automatically select the nearest data point as the mouse moves.
+    2. A multi selection (`click`) to create persistent selections via shift-click.
+
+    Both selections include the set `empty='none'` to indicate that no points should be included if a selection is empty. These selections are then combined into a single filter predicate — the logical _or_ of `hover` and `click` — to include points that reside in _either_ selection. We use this predicate to filter the new layers to show annotations and labels for selected points only.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Using selections and layers, we can realize a number of different designs for details on demand! For example, here is a log-scaled time series of technology stock prices, annotated with a guideline and labels for the date nearest the mouse cursor:
+    """)
+    return
+
+
+@app.cell
+def _(alt, stocks):
+    # select a point for which to provide details-on-demand
+    label = alt.selection_point(toggle=False, encodings=['x'], on='mouseover', nearest=True, empty=False)
+    _base = alt.Chart().mark_line().encode(alt.X('date:T'), alt.Y('price:Q', scale=alt.Scale(type='log')), alt.Color('symbol:N'))  # limit selection to x-axis value
+    # define our base line chart of stock prices
+    alt.layer(_base, alt.Chart().mark_rule(color='#aaa').encode(x='date:T').transform_filter(label), _base.mark_circle().encode(opacity=alt.condition(label, alt.value(1), alt.value(0))).add_params(label), _base.mark_text(align='left', dx=5, dy=-5, stroke='white', strokeWidth=2).encode(text='price:Q').transform_filter(label), _base.mark_text(align='left', dx=5, dy=-5).encode(text='price:Q').transform_filter(label), data=stocks).properties(width=700, height=400)  # select on mouseover events  # select data point nearest the cursor  # empty selection includes no data points  # base line chart  # add a rule mark to serve as a guide line  # add circle marks for selected time points, hide unselected points  # add white stroked text to provide a legible background for labels  # add text labels for stock prices
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Putting into action what we've learned so far: can you modify the movie scatter plot above (the one with the dynamic query over years) to include a `rule` mark that shows the average IMDB (or Rotten Tomatoes) rating for the data contained within the year `interval` selection?_
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Brushing & Linking, Revisited
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Earlier in this notebook we saw an example of _brushing & linking_: using a dynamic query histogram to highlight points in a movie rating scatter plot. Here, we'll visit some additional examples involving linked selections.
+
+    Returning to the `cars` dataset, we can use the `repeat` operator to build a [scatter plot matrix (SPLOM)](https://en.wikipedia.org/wiki/Scatter_plot#Scatterplot_matrices) that shows associations between mileage, acceleration, and horsepower. We can define an `interval` selection and include it _within_ our repeated scatter plot specification to enable linked selections among all the plots.
+
+    _Click and drag in any of the plots below to perform brushing & linking!_
+    """)
+    return
+
+
+@app.cell
+def _(alt, cars):
+    _brush = alt.selection_interval(resolve='global')
+    alt.Chart(cars).mark_circle().add_params(_brush).encode(alt.X(alt.repeat('column'), type='quantitative'), alt.Y(alt.repeat('row'), type='quantitative'), color=alt.condition(_brush, 'Cylinders:O', alt.value('grey')), opacity=alt.condition(_brush, alt.value(0.8), alt.value(0.1))).properties(width=140, height=140).repeat(column=['Acceleration', 'Horsepower', 'Miles_per_Gallon'], row=['Miles_per_Gallon', 'Horsepower', 'Acceleration'])  # resolve all selections to a single global instance
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Note above the use of `resolve='global'` on the `interval` selection. The default setting of `'global'` indicates that across all plots only one brush can be active at a time. However, in some cases we might want to define brushes in multiple plots and combine the results. If we use `resolve='union'`, the selection will be the _union_ of all brushes: if a point resides within any brush it will be selected. Alternatively, if we use `resolve='intersect'`, the selection will consist of the _intersection_ of all brushes: only points that reside within all brushes will be selected.
+
+    _Try setting the `resolve` parameter to `'union'` and `'intersect'` and see how it changes the resulting selection logic._
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Cross-Filtering
+
+    The brushing & linking examples we've looked at all use conditional encodings, for example to change opacity values in response to a selection. Another option is to use a selection defined in one view to _filter_ the content of another view.
+
+    Let's build a collection of histograms for the `flights` dataset: arrival `delay` (how early or late a flight arrives, in minutes), `distance` flown (in miles), and `time` of departure (hour of the day). We'll use the `repeat` operator to create the histograms, and add an `interval` selection for the `x` axis with brushes resolved via intersection.
+
+    In particular, each histogram will consist of two layers: a gray background layer and a blue foreground layer, with the foreground layer filtered by our intersection of brush selections. The result is a _cross-filtering_ interaction across the three charts!
+
+    _Drag out brush intervals in the charts below. As you select flights with longer or shorter arrival delays, how do the distance and time distributions respond?_
+    """)
+    return
+
+
+@app.cell
+def _(alt, flights):
+    _brush = alt.selection_interval(encodings=['x'], resolve='intersect')
+    hist = alt.Chart().mark_bar().encode(alt.X(alt.repeat('row'), type='quantitative', bin=alt.Bin(maxbins=100, minstep=1), axis=alt.Axis(format='d', titleAnchor='start')), alt.Y('count():Q', title=None))
+    alt.layer(hist.add_params(_brush).encode(color=alt.value('lightgrey')), hist.transform_filter(_brush)).properties(width=900, height=100).repeat(row=['delay', 'distance', 'time'], data=flights).transform_calculate(delay='datum.delay < 180 ? datum.delay : 180', time='hours(datum.date) + minutes(datum.date) / 60').configure_view(stroke='transparent')  # up to 100 bins  # integer format, left-aligned title  # no y-axis title  # clamp delays > 3 hours  # fractional hours  # no outline
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _By cross-filtering you can observe that delayed flights are more likely to depart at later hours. This phenomenon is familiar to frequent fliers: a delay can propagate through the day, affecting subsequent travel by that plane. For the best odds of an on-time arrival, book an early flight!_
+
+    The combination of multiple views and interactive selections can enable valuable forms of multi-dimensional reasoning, turning even basic histograms into powerful input devices for asking questions of a dataset!
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Summary
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    For more information about the supported interaction options in Altair, please consult the [Altair interactive selection documentation](https://altair-viz.github.io/user_guide/interactions.html). For details about customizing event handlers, for example to compose multiple interaction techniques or support touch-based input on mobile devices, see the [Vega-Lite selection documentation](https://vega.github.io/vega-lite/docs/selection.html).
+
+    Interested in learning more?
+    - The _selection_ abstraction was introduced in the paper [Vega-Lite: A Grammar of Interactive Graphics](http://idl.cs.washington.edu/papers/vega-lite/), by Satyanarayan, Moritz, Wongsuphasawat, &amp; Heer.
+    - The PRIM-9 system (for projection, rotation, isolation, and masking in up to 9 dimensions) is one of the earliest interactive visualization tools, built in the early 1970s by Fisherkeller, Tukey, &amp; Friedman. [A retro demo video survives!](https://www.youtube.com/watch?v=B7XoW2qiFUA)
+    - The concept of brushing &amp; linking was crystallized by Becker, Cleveland, &amp; Wilks in their 1987 article [Dynamic Graphics for Data Analysis](https://scholar.google.com/scholar?cluster=14817303117298653693).
+    - For a comprehensive summary of interaction techniques for visualization, see [Interactive Dynamics for Visual Analysis](https://queue.acm.org/detail.cfm?id=2146416) by Heer &amp; Shneiderman.
+    - Finally, for a treatise on what makes interaction effective, read the classic [Direct Manipulation Interfaces](https://scholar.google.com/scholar?cluster=15702972136892195211) paper by Hutchins, Hollan, &amp; Norman.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    #### Appendix: On The Representation of Time
+
+    Earlier we observed a small bump in the number of movies in either 1969 and 1970. Where does that bump come from? And why 1969 _or_ 1970? The answer stems from a combination of missing data and how your computer represents time.
+
+    Internally, dates and times are represented relative to the [UNIX epoch](https://en.wikipedia.org/wiki/Unix_time), in which time "zero" corresponds to the stroke of midnight on January 1, 1970 in [UTC time](https://en.wikipedia.org/wiki/Coordinated_Universal_Time), which runs along the [prime meridian](https://en.wikipedia.org/wiki/Prime_meridian). It turns out there are a few movies with missing (`null`) release dates. Those `null` values get interpreted as time `0`, and thus map to January 1, 1970 in UTC time. If you live in the Americas &ndash; and thus in "earlier" time zones &ndash; this precise point in time corresponds to an earlier hour on December 31, 1969 in your local time zone. On the other hand, if you live near or east of the prime meridian, the date in your local time zone will be January 1, 1970.
+
+    The takeaway? Always be skeptical of your data, and be mindful that how data is represented (whether as date times, or floating point numbers, or latitudes and longitudes, _etc._) can sometimes lead to artifacts that impact analysis!
+    """)
+    return
+
+
+if __name__ == "__main__":
+    app.run()

altair/07_cartographic.py

@@ -0,0 +1,898 @@
+# /// script
+# requires-python = ">=3.11"
+# dependencies = [
+#     "altair==6.0.0",
+#     "marimo",
+#     "pandas==3.0.1",
+#     "vega_datasets==0.9.0",
+# ]
+# ///
+
+import marimo
+
+__generated_with = "0.20.4"
+app = marimo.App()
+
+
+@app.cell
+def _():
+    import marimo as mo
+
+    return (mo,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    # Cartographic Visualization
+
+    _“The making of maps is one of humanity's longest established intellectual endeavors and also one of its most complex, with scientific theory, graphical representation, geographical facts, and practical considerations blended together in an unending variety of ways.”_ — [H. J. Steward](https://books.google.com/books?id=cVy1Ms43fFYC)
+
+    Cartography – the study and practice of map-making – has a rich history spanning centuries of discovery and design. Cartographic visualization leverages mapping techniques to convey data containing spatial information, such as locations, routes, or trajectories on the surface of the Earth.
+
+    <div style="float: right; margin-left: 1em; margin-top: 1em;"><img width="300px" src="https://gist.githubusercontent.com/jheer/c90d582ef5322582cf4960ec7689f6f6/raw/8dc92382a837ccc34c076f4ce7dd864e7893324a/latlon.png" /></div>
+
+    Approximating the Earth as a sphere, we can denote positions using a spherical coordinate system of _latitude_ (angle in degrees north or south of the _equator_) and _longitude_ (angle in degrees specifying east-west position). In this system, a _parallel_ is a circle of constant latitude and a _meridian_ is a circle of constant longitude. The [_prime meridian_](https://en.wikipedia.org/wiki/Prime_meridian) lies at 0° longitude and by convention is defined to pass through the Royal Observatory in Greenwich, England.
+
+    To "flatten" a three-dimensional sphere on to a two-dimensional plane, we must apply a [projection](https://en.wikipedia.org/wiki/Map_projection) that maps (`longitude`, `latitude`) pairs to (`x`, `y`) coordinates. Similar to [scales](https://github.com/uwdata/visualization-curriculum/blob/master/altair_scales_axes_legends.ipynb), projections map from a data domain (spatial position) to a visual range (pixel position). However, the scale mappings we've seen thus far accept a one-dimensional domain, whereas map projections are inherently two-dimensional.
+
+    In this notebook, we will introduce the basics of creating maps and visualizing spatial data with Altair, including:
+
+    - Data formats for representing geographic features,
+    - Geo-visualization techniques such as point, symbol, and choropleth maps, and
+    - A review of common cartographic projections.
+
+    _This notebook is part of the [data visualization curriculum](https://github.com/uwdata/visualization-curriculum)._
+    """)
+    return
+
+
+@app.cell
+def _():
+    import pandas as pd
+    import altair as alt
+    from vega_datasets import data
+
+    return alt, data
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Geographic Data: GeoJSON and TopoJSON
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Up to this point, we have worked with JSON and CSV formatted datasets that correspond to data tables made up of rows (records) and columns (fields). In order to represent geographic regions (countries, states, _etc._) and trajectories (flight paths, subway lines, _etc._), we need to expand our repertoire with additional formats designed to support rich geometries.
+
+    [GeoJSON](https://en.wikipedia.org/wiki/GeoJSON) models geographic features within a specialized JSON format. A GeoJSON `feature` can include geometric data &ndash; such as `longitude`, `latitude` coordinates that make up a country boundary &ndash; as well as additional data attributes.
+
+    Here is a GeoJSON `feature` object for the boundary of the U.S. state of Colorado:
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ~~~ json
+    {
+      "type": "Feature",
+      "id": 8,
+      "properties": {"name": "Colorado"},
+      "geometry": {
+        "type": "Polygon",
+        "coordinates": [
+          [[-106.32056285448942,40.998675790862656],[-106.19134826714341,40.99813863734313],[-105.27607827344248,40.99813863734313],[-104.9422739227986,40.99813863734313],[-104.05212898774828,41.00136155846029],[-103.57475287338661,41.00189871197981],[-103.38093099236758,41.00189871197981],[-102.65589358559272,41.00189871197981],[-102.62000064466328,41.00189871197981],[-102.052892177978,41.00189871197981],[-102.052892177978,40.74889940428302],[-102.052892177978,40.69733266640851],[-102.052892177978,40.44003613055551],[-102.052892177978,40.3492571857556],[-102.052892177978,40.00333031918079],[-102.04930288388505,39.57414465707943],[-102.04930288388505,39.56823596836465],[-102.0457135897921,39.1331416175485],[-102.0457135897921,39.0466599009048],[-102.0457135897921,38.69751011321283],[-102.0457135897921,38.61478847120581],[-102.0457135897921,38.268861604631],[-102.0457135897921,38.262415762396685],[-102.04212429569915,37.738153927339205],[-102.04212429569915,37.64415206142214],[-102.04212429569915,37.38900413964724],[-102.04212429569915,36.99365914927603],[-103.00046581851544,37.00010499151034],[-103.08660887674611,37.00010499151034],[-104.00905745863294,36.99580776335414],[-105.15404227428235,36.995270609834606],[-105.2222388620483,36.995270609834606],[-105.7175614468747,36.99580776335414],[-106.00829426840322,36.995270609834606],[-106.47490250048605,36.99365914927603],[-107.4224761410235,37.00010499151034],[-107.48349414060355,37.00010499151034],[-108.38081766383978,36.99903068447129],[-109.04483707103458,36.99903068447129],[-109.04483707103458,37.484617466122884],[-109.04124777694163,37.88049961001363],[-109.04124777694163,38.15283644441336],[-109.05919424740635,38.49983761802722],[-109.05201565922046,39.36680339854235],[-109.05201565922046,39.49786885730673],[-109.05201565922046,39.66062637372313],[-109.05201565922046,40.22248895514744],[-109.05201565922046,40.653823231326896],[-109.05201565922046,41.000287251421234],[-107.91779872584989,41.00189871197981],[-107.3183866123281,41.00297301901887],[-106.85895696843116,41.00189871197981],[-106.32056285448942,40.998675790862656]]
+        ]
+      }
+    }
+    ~~~
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The `feature` includes a `properties` object, which can include any number of data fields, plus a `geometry` object, which in this case contains a single polygon that consists of `[longitude, latitude]` coordinates for the state boundary. The coordinates continue off to the right for a while should you care to scroll...
+
+    To learn more about the nitty-gritty details of GeoJSON, see the [official GeoJSON specification](http://geojson.org/) or read [Tom MacWright's helpful primer](https://macwright.org/2015/03/23/geojson-second-bite).
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    One drawback of GeoJSON as a storage format is that it can be redundant, resulting in larger file sizes. Consider: Colorado shares boundaries with six other states (seven if you include the corner touching Arizona). Instead of using separate, overlapping coordinate lists for each of those states, a more compact approach is to encode shared borders only once, representing the _topology_ of geographic regions. Fortunately, this is precisely what the [TopoJSON](https://github.com/topojson/topojson/blob/master/README.md) format does!
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Let's load a TopoJSON file of world countries (at 110 meter resolution):
+    """)
+    return
+
+
+@app.cell
+def _(data):
+    world = data.world_110m.url
+    world
+    return (world,)
+
+
+@app.cell
+def _(data):
+    world_topo = data.world_110m()
+    return (world_topo,)
+
+
+@app.cell
+def _(world_topo):
+    world_topo.keys()
+    return
+
+
+@app.cell
+def _(world_topo):
+    world_topo['type']
+    return
+
+
+@app.cell
+def _(world_topo):
+    world_topo['objects'].keys()
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Inspect the `world_topo` TopoJSON dictionary object above to see its contents._
+
+    In the data above, the `objects` property indicates the named elements we can extract from the data: geometries for all `countries`, or a single polygon representing all `land` on Earth. Either of these can be unpacked to GeoJSON data we can then visualize.
+
+    As TopoJSON is a specialized format, we need to instruct Altair to parse the TopoJSON format, indicating which named faeture object we wish to extract from the topology. The following code indicates that we want to extract GeoJSON features from the `world` dataset for the `countries` object:
+
+    ~~~ js
+    alt.topo_feature(world, 'countries')
+    ~~~
+
+    This `alt.topo_feature` method call expands to the following Vega-Lite JSON:
+
+    ~~~ json
+    {
+      "values": world,
+      "format": {"type": "topojson", "feature": "countries"}
+    }
+    ~~~
+
+    Now that we can load geographic data, we're ready to start making maps!
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Geoshape Marks
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    To visualize geographic data, Altair provides the `geoshape` mark type. To create a basic map, we can create a `geoshape` mark and pass it our TopoJSON data, which is then unpacked into GeoJSON features, one for each country of the world:
+    """)
+    return
+
+
+@app.cell
+def _(alt, world):
+    alt.Chart(alt.topo_feature(world, 'countries')).mark_geoshape()
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    In the example above, Altair applies a default blue color and uses a default map projection (`mercator`). We can customize the colors and boundary stroke widths using standard mark properties. Using the `project` method we can also add our own map projection:
+    """)
+    return
+
+
+@app.cell
+def _(alt, world):
+    alt.Chart(alt.topo_feature(world, 'countries')).mark_geoshape(
+        fill='#2a1d0c', stroke='#706545', strokeWidth=0.5
+    ).project(
+        type='mercator'
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    By default Altair automatically adjusts the projection so that all the data fits within the width and height of the chart. We can also specify projection parameters, such as `scale` (zoom level) and `translate` (panning), to customize the projection settings. Here we adjust the `scale` and `translate` parameters to focus on Europe:
+    """)
+    return
+
+
+@app.cell
+def _(alt, world):
+    alt.Chart(alt.topo_feature(world, 'countries')).mark_geoshape(
+        fill='#2a1d0c', stroke='#706545', strokeWidth=0.5
+    ).project(
+        type='mercator', scale=400, translate=[100, 550]
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Note how the 110m resolution of the data becomes apparent at this scale. To see more detailed coast lines and boundaries, we need an input file with more fine-grained geometries. Adjust the `scale` and `translate` parameters to focus the map on other regions!_
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    So far our map shows countries only. Using the `layer` operator, we can combine multiple map elements. Altair includes _data generators_ we can use to create data for additional map layers:
+
+    - The sphere generator (`{'sphere': True}`) provides a GeoJSON representation of the full sphere of the Earth. We can create an additional `geoshape` mark that fills in the shape of the Earth as a background layer.
+    - The graticule generator (`{'graticule': ...}`) creates a GeoJSON feature representing a _graticule_: a grid formed by lines of latitude and longitude. The default graticule has meridians and parallels every 10° between ±80° latitude. For the polar regions, there are meridians every 90°. These settings can be customized using the `stepMinor` and `stepMajor` properties.
+
+    Let's layer sphere, graticule, and country marks into a reusable map specification:
+    """)
+    return
+
+
+@app.cell
+def _(alt, world):
+    map = alt.layer(
+        # use the sphere of the Earth as the base layer
+        alt.Chart({'sphere': True}).mark_geoshape(
+            fill='#e6f3ff'
+        ),
+        # add a graticule for geographic reference lines
+        alt.Chart({'graticule': True}).mark_geoshape(
+            stroke='#ffffff', strokeWidth=1
+        ),
+        # and then the countries of the world
+        alt.Chart(alt.topo_feature(world, 'countries')).mark_geoshape(
+            fill='#2a1d0c', stroke='#706545', strokeWidth=0.5
+        )
+    ).properties(
+        width=600,
+        height=400
+    )
+    return (map,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    We can extend the map with a desired projection and draw the result. Here we apply a [Natural Earth projection](https://en.wikipedia.org/wiki/Natural_Earth_projection). The _sphere_ layer provides the light blue background; the _graticule_ layer provides the white geographic reference lines.
+    """)
+    return
+
+
+@app.cell
+def _(map):
+    map.project(
+        type='naturalEarth1', scale=110, translate=[300, 200]
+    ).configure_view(stroke=None)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Point Maps
+
+    In addition to the _geometric_ data provided by GeoJSON or TopoJSON files, many tabular datasets include geographic information in the form of fields for `longitude` and `latitude` coordinates, or references to geographic regions such as country names, state names, postal codes, _etc._, which can be mapped to coordinates using a [geocoding service](https://en.wikipedia.org/wiki/Geocoding). In some cases, location data is rich enough that we can see meaningful patterns by projecting the data points alone &mdash; no base map required!
+
+    Let's look at a dataset of 5-digit zip codes in the United States, including `longitude`, `latitude` coordinates for each post office in addition to a `zip_code` field.
+    """)
+    return
+
+
+@app.cell
+def _(data):
+    zipcodes = data.zipcodes.url
+    zipcodes
+    return (zipcodes,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    We can visualize each post office location using a small (1-pixel) `square` mark. However, to set the positions we do _not_ use `x` and `y` channels. _Why is that?_
+
+    While cartographic projections map (`longitude`, `latitude`) coordinates to (`x`, `y`) coordinates, they can do so in arbitrary ways. There is no guarantee, for example, that `longitude` → `x` and `latitude` → `y`! Instead, Altair includes special `longitude` and `latitude` encoding channels to handle geographic coordinates. These channels indicate which data fields should be mapped to `longitude` and `latitude` coordinates, and then applies a projection to map those coordinates to (`x`, `y`) positions.
+    """)
+    return
+
+
+@app.cell
+def _(alt, zipcodes):
+    alt.Chart(zipcodes).mark_square(
+        size=1, opacity=1
+    ).encode(
+        longitude='longitude:Q', # apply the field named 'longitude' to the longitude channel
+        latitude='latitude:Q'    # apply the field named 'latitude' to the latitude channel
+    ).project(
+        type='albersUsa'
+    ).properties(
+        width=900,
+        height=500
+    ).configure_view(
+        stroke=None
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Plotting zip codes only, we can see the outline of the United States and discern meaningful patterns in the density of post offices, without a base map or additional reference elements!_
+
+    We use the `albersUsa` projection, which takes some liberties with the actual geometry of the Earth, with scaled versions of Alaska and Hawaii in the bottom-left corner. As we did not specify projection `scale` or `translate` parameters, Altair sets them automatically to fit the visualized data.
+
+    We can now go on to ask more questions of our dataset. For example, is there any rhyme or reason to the allocation of zip codes? To assess this question we can add a color encoding based on the first digit of the zip code. We first add a `calculate` transform to extract the first digit, and encode the result using the color channel:
+    """)
+    return
+
+
+@app.cell
+def _(alt, zipcodes):
+    alt.Chart(zipcodes).transform_calculate(
+        digit='datum.zip_code[0]'
+    ).mark_square(
+        size=2, opacity=1
+    ).encode(
+        longitude='longitude:Q',
+        latitude='latitude:Q',
+        color='digit:N'
+    ).project(
+        type='albersUsa'
+    ).properties(
+        width=900,
+        height=500
+    ).configure_view(
+        stroke=None
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _To zoom in on a specific digit, add a filter transform to limit the data shown! Try adding an [interactive selection](https://github.com/uwdata/visualization-curriculum/blob/master/altair_interaction.ipynb) to filter to a single digit and dynamically update the map. And be sure to use strings (\`'1'\`) instead of numbers (\`1\`) when filtering digit values!_
+
+    (This example is inspired by Ben Fry's classic [zipdecode](https://benfry.com/zipdecode/) visualization!)
+
+    We might further wonder what the _sequence_ of zip codes might indicate. One way to explore this question is to connect each consecutive zip code using a `line` mark, as done in Robert Kosara's [ZipScribble](https://eagereyes.org/zipscribble-maps/united-states) visualization:
+    """)
+    return
+
+
+@app.cell
+def _(alt, zipcodes):
+    alt.Chart(zipcodes).transform_filter(
+        '-150 < datum.longitude && 22 < datum.latitude && datum.latitude < 55'
+    ).transform_calculate(
+        digit='datum.zip_code[0]'
+    ).mark_line(
+        strokeWidth=0.5
+    ).encode(
+        longitude='longitude:Q',
+        latitude='latitude:Q',
+        color='digit:N',
+        order='zip_code:O'
+    ).project(
+        type='albersUsa'
+    ).properties(
+        width=900,
+        height=500
+    ).configure_view(
+        stroke=None
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _We can now see how zip codes further cluster into smaller areas, indicating a hierarchical allocation of codes by location, but with some notable variability within local clusters._
+
+    If you were paying careful attention to our earlier maps, you may have noticed that there are zip codes being plotted in the upper-left corner! These correspond to locations such as Puerto Rico or American Samoa, which contain U.S. zip codes but are mapped to `null` coordinates (`0`, `0`) by the `albersUsa` projection. In addition, Alaska and Hawaii can complicate our view of the connecting line segments. In response, the code above includes an additional filter that removes points outside our chosen `longitude` and `latitude` spans.
+
+    _Remove the filter above to see what happens!_
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Symbol Maps
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Now let's combine a base map and plotted data as separate layers. We'll examine the U.S. commercial flight network, considering both airports and flight routes. To do so, we'll need three datasets.
+    For our base map, we'll use a TopoJSON file for the United States at 10m resolution, containing features for `states` or `counties`:
+    """)
+    return
+
+
+@app.cell
+def _(data):
+    usa = data.us_10m.url
+    usa
+    return (usa,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    For the airports, we will use a dataset with fields for the `longitude` and `latitude` coordinates of each airport as well as the `iata` airport code &mdash; for example, `'SEA'` for [Seattle-Tacoma International Airport](https://en.wikipedia.org/wiki/Seattle%E2%80%93Tacoma_International_Airport).
+    """)
+    return
+
+
+@app.cell
+def _(data):
+    airports = data.airports.url
+    airports
+    return (airports,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Finally, we will use a dataset of flight routes, which contains `origin` and `destination` fields with the IATA codes for the corresponding airports:
+    """)
+    return
+
+
+@app.cell
+def _(data):
+    flights = data.flights_airport.url
+    flights
+    return (flights,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Let's start by creating a base map using the `albersUsa` projection, and add a layer that plots `circle` marks for each airport:
+    """)
+    return
+
+
+@app.cell
+def _(airports, alt, usa):
+    alt.layer(
+        alt.Chart(alt.topo_feature(usa, 'states')).mark_geoshape(
+            fill='#ddd', stroke='#fff', strokeWidth=1
+        ),
+        alt.Chart(airports).mark_circle(size=9).encode(
+            latitude='latitude:Q',
+            longitude='longitude:Q',
+            tooltip='iata:N'
+        )
+    ).project(
+        type='albersUsa'
+    ).properties(
+        width=900,
+        height=500
+    ).configure_view(
+        stroke=None
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _That's a lot of airports! Obviously, not all of them are major hubs._
+
+    Similar to our zip codes dataset, our airport data includes points that lie outside the continental United States. So we again see points in the upper-left corner. We might want to filter these points, but to do so we first need to know more about them.
+
+    _Update the map projection above to `albers` &ndash; side-stepping the idiosyncratic behavior of `albersUsa` &ndash; so that the actual locations of these additional points is revealed!_
+
+    Now, instead of showing all airports in an undifferentiated fashion, let's identify major hubs by considering the total number of routes that originate at each airport. We'll use the `routes` dataset as our primary data source: it contains a list of flight routes that we can aggregate to count the number of routes for each `origin` airport.
+
+    However, the `routes` dataset does not include the _locations_ of the airports! To augment the `routes` data with locations, we need a new data transformation: `lookup`. The `lookup` transform takes a field value in a primary dataset and uses it as a _key_ to look up related information in another table. In this case, we want to match the `origin` airport code in our `routes` dataset against the `iata` field of the `airports` dataset, then extract the corresponding `latitude` and `longitude` fields.
+    """)
+    return
+
+
+@app.cell
+def _(airports, alt, flights, usa):
+    alt.layer(
+        alt.Chart(alt.topo_feature(usa, 'states')).mark_geoshape(
+            fill='#ddd', stroke='#fff', strokeWidth=1
+        ),
+        alt.Chart(flights).mark_circle().transform_aggregate(
+            groupby=['origin'],
+            routes='count()'
+        ).transform_lookup(
+            lookup='origin',
+            from_=alt.LookupData(data=airports, key='iata',
+                                 fields=['state', 'latitude', 'longitude'])
+        ).transform_filter(
+            'datum.state !== "PR" && datum.state !== "VI"'
+        ).encode(
+            latitude='latitude:Q',
+            longitude='longitude:Q',
+            tooltip=['origin:N', 'routes:Q'],
+            size=alt.Size('routes:Q', scale=alt.Scale(range=[0, 1000]), legend=None),
+            order=alt.Order('routes:Q', sort='descending')
+        )
+    ).project(
+        type='albersUsa'
+    ).properties(
+        width=900,
+        height=500
+    ).configure_view(
+        stroke=None
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Which U.S. airports have the highest number of outgoing routes?_
+
+    Now that we can see the airports, which may wish to interact with them to better understand the structure of the air traffic network. We can add a `rule` mark layer to represent paths from `origin` airports to `destination` airports, which requires two `lookup` transforms to retreive coordinates for each end point. In addition, we can use a `single` selection to filter these routes, such that only the routes originating at the currently selected airport are shown.
+
+    _Starting from the static map above, can you build an interactive version? Feel free to skip the code below to engage with the interactive map first, and think through how you might build it on your own!_
+    """)
+    return
+
+
+@app.cell
+def _(airports, alt, flights, usa):
+    # interactive selection for origin airport
+    # select nearest airport to mouse cursor
+    origin = alt.selection_point(
+        on='mouseover', nearest=True,
+        fields=['origin'], empty='none'
+    )
+
+    # shared data reference for lookup transforms
+    foreign = alt.LookupData(data=airports, key='iata',
+                             fields=['latitude', 'longitude'])
+    
+    alt.layer(
+        # base map of the United States
+        alt.Chart(alt.topo_feature(usa, 'states')).mark_geoshape(
+            fill='#ddd', stroke='#fff', strokeWidth=1
+        ),
+        # route lines from selected origin airport to destination airports
+        alt.Chart(flights).mark_rule(
+            color='#000', opacity=0.35
+        ).transform_filter(
+            origin # filter to selected origin only
+        ).transform_lookup(
+            lookup='origin', from_=foreign # origin lat/lon
+        ).transform_lookup(
+            lookup='destination', from_=foreign, as_=['lat2', 'lon2'] # dest lat/lon
+        ).encode(
+            latitude='latitude:Q',
+            longitude='longitude:Q',
+            latitude2='lat2',
+            longitude2='lon2',
+        ),
+        # size airports by number of outgoing routes
+        # 1. aggregate flights-airport data set
+        # 2. lookup location data from airports data set
+        # 3. remove Puerto Rico (PR) and Virgin Islands (VI)
+        alt.Chart(flights).mark_circle().transform_aggregate(
+            groupby=['origin'],
+            routes='count()'
+        ).transform_lookup(
+            lookup='origin',
+            from_=alt.LookupData(data=airports, key='iata',
+                                 fields=['state', 'latitude', 'longitude'])
+        ).transform_filter(
+            'datum.state !== "PR" && datum.state !== "VI"'
+        ).add_params(
+            origin
+        ).encode(
+            latitude='latitude:Q',
+            longitude='longitude:Q',
+            tooltip=['origin:N', 'routes:Q'],
+            size=alt.Size('routes:Q', scale=alt.Scale(range=[0, 1000]), legend=None),
+            order=alt.Order('routes:Q', sort='descending') # place smaller circles on top
+        )
+    ).project(
+        type='albersUsa'
+    ).properties(
+        width=900,
+        height=500
+    ).configure_view(
+        stroke=None
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Mouseover the map to probe the flight network!_
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Choropleth Maps
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    A [choropleth map](https://en.wikipedia.org/wiki/Choropleth_map) uses shaded or textured regions to visualize data values. Sized symbol maps are often more accurate to read, as people tend to be better at estimating proportional differences between the area of circles than between color shades. Nevertheless, choropleth maps are popular in practice and particularly useful when too many symbols become perceptually overwhelming.
+
+    For example, while the United States only has 50 states, it has thousands of counties within those states. Let's build a choropleth map of the unemployment rate per county, back in the recession year of 2008. In some cases, input GeoJSON or TopoJSON files might include statistical data that we can directly visualize. In this case, however, we have two files: our TopoJSON file that includes county boundary features (`usa`), and a separate text file that contains unemployment statistics:
+    """)
+    return
+
+
+@app.cell
+def _(data):
+    unemp = data.unemployment.url
+    unemp
+    return (unemp,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    To integrate our data sources, we will again need to use the `lookup` transform, augmenting our TopoJSON-based `geoshape` data with unemployment rates. We can then create a map that includes a `color` encoding for the looked-up `rate` field.
+    """)
+    return
+
+
+@app.cell
+def _(alt, unemp, usa):
+    alt.Chart(alt.topo_feature(usa, 'counties')).mark_geoshape(
+        stroke='#aaa', strokeWidth=0.25
+    ).transform_lookup(
+        lookup='id', from_=alt.LookupData(data=unemp, key='id', fields=['rate'])
+    ).encode(
+        alt.Color('rate:Q',
+                  scale=alt.Scale(domain=[0, 0.3], clamp=True), 
+                  legend=alt.Legend(format='%')),
+        alt.Tooltip('rate:Q', format='.0%')
+    ).project(
+        type='albersUsa'
+    ).properties(
+        width=900,
+        height=500
+    ).configure_view(
+        stroke=None
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    *Examine the unemployment rates by county. Higher values in Michigan may relate to the automotive industry. Counties in the [Great Plains](https://en.wikipedia.org/wiki/Great_Plains) and Mountain states exhibit both low **and** high rates. Is this variation meaningful, or is it possibly an [artifact of lower sample sizes](https://medium.com/@uwdata/surprise-maps-showing-the-unexpected-e92b67398865)? To explore further, try changing the upper scale domain (e.g., to `0.2`) to adjust the color mapping.*
+
+    A central concern for choropleth maps is the choice of colors. Above, we use Altair's default `'yellowgreenblue'` scheme for heatmaps. Below we compare other schemes, including a _single-hue sequential_ scheme (`teals`) that varies in luminance only, a _multi-hue sequential_ scheme (`viridis`) that ramps in both luminance and hue, and a _diverging_ scheme (`blueorange`) that uses a white mid-point:
+    """)
+    return
+
+
+@app.cell
+def _(alt, unemp, usa):
+    # utility function to generate a map specification for a provided color scheme
+    def map_(scheme):
+        return alt.Chart().mark_geoshape().project(type='albersUsa').encode(
+            alt.Color('rate:Q', scale=alt.Scale(scheme=scheme), legend=None)
+        ).properties(width=305, height=200)
+
+    alt.hconcat(
+        map_('tealblues'), map_('viridis'), map_('blueorange'),
+        data=alt.topo_feature(usa, 'counties')
+    ).transform_lookup(
+        lookup='id', from_=alt.LookupData(data=unemp, key='id', fields=['rate'])
+    ).configure_view(
+        stroke=None
+    ).resolve_scale(
+        color='independent'
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    _Which color schemes do you find to be more effective? Why might that be? Modify the maps above to use other available schemes, as described in the [Vega Color Schemes documentation](https://vega.github.io/vega/docs/schemes/)._
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Cartographic Projections
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Now that we have some experience creating maps, let's take a closer look at cartographic projections. As explained by [Wikipedia](https://en.wikipedia.org/wiki/Map_projection),
+
+    > _All map projections necessarily distort the surface in some fashion. Depending on the purpose of the map, some distortions are acceptable and others are not; therefore, different map projections exist in order to preserve some properties of the sphere-like body at the expense of other properties._
+
+    Some of the properties we might wish to consider include:
+
+    - _Area_: Does the projection distort region sizes?
+    - _Bearing_: Does a straight line correspond to a constant direction of travel?
+    - _Distance_: Do lines of equal length correspond to equal distances on the globe?
+    - _Shape_: Does the projection preserve spatial relations (angles) between points?
+
+    Selecting an appropriate projection thus depends on the use case for the map. For example, if we are assessing land use and the extent of land matters, we might choose an area-preserving projection. If we want to visualize shockwaves emanating from an earthquake, we might focus the map on the quake's epicenter and preserve distances outward from that point. Or, if we wish to aid navigation, the preservation of bearing and shape may be more important.
+
+    We can also characterize projections in terms of the _projection surface_. Cylindrical projections, for example, project surface points of the sphere onto a surrounding cylinder; the "unrolled" cylinder then provides our map. As we further describe below, we might alternatively project onto the surface of a cone (conic projections) or directly onto a flat plane (azimuthal projections).
+
+    *Let's first build up our intuition by interacting with a variety of projections! **[Open the online Vega-Lite Cartographic Projections notebook](https://observablehq.com/@vega/vega-lite-cartographic-projections).** Use the controls on that page to select a projection and explore projection parameters, such as the `scale` (zooming) and x/y translation (panning). The rotation ([yaw, pitch, roll](https://en.wikipedia.org/wiki/Aircraft_principal_axes)) controls determine the orientation of the globe relative to the surface being projected upon.*
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### A Tour of Specific Projection Types
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    [**Cylindrical projections**](https://en.wikipedia.org/wiki/Map_projection#Cylindrical) map the sphere onto a surrounding cylinder, then unroll the cylinder. If the major axis of the cylinder is oriented north-south, meridians are mapped to straight lines. [Pseudo-cylindrical](https://en.wikipedia.org/wiki/Map_projection#Pseudocylindrical) projections represent a central meridian as a straight line, with other meridians "bending" away from the center.
+    """)
+    return
+
+
+@app.cell
+def _(alt, map):
+    _minimap = map.properties(width=225, height=225)
+    alt.hconcat(_minimap.project(type='equirectangular').properties(title='equirectangular'), _minimap.project(type='mercator').properties(title='mercator'), _minimap.project(type='transverseMercator').properties(title='transverseMercator'), _minimap.project(type='naturalEarth1').properties(title='naturalEarth1')).properties(spacing=10).configure_view(stroke=None)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    - [Equirectangular](https://en.wikipedia.org/wiki/Equirectangular_projection) (`equirectangular`): Scale `lat`, `lon` coordinate values directly.
+    - [Mercator](https://en.wikipedia.org/wiki/Mercator_projection) (`mercator`): Project onto a cylinder, using `lon` directly, but subjecting `lat` to a non-linear transformation. Straight lines preserve constant compass bearings ([rhumb lines](https://en.wikipedia.org/wiki/Rhumb_line)), making this projection well-suited to navigation. However, areas in the far north or south can be greatly distorted.
+    - [Transverse Mercator](https://en.wikipedia.org/wiki/Transverse_Mercator_projection) (`transverseMercator`): A mercator projection, but with the bounding cylinder rotated to a transverse axis. Whereas the standard Mercator projection has highest accuracy along the equator, the Transverse Mercator projection is most accurate along the central meridian.
+    - [Natural Earth](https://en.wikipedia.org/wiki/Natural_Earth_projection) (`naturalEarth1`): A pseudo-cylindrical projection designed for showing the whole Earth in one view.
+    <br/><br/>
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    [**Conic projections**](https://en.wikipedia.org/wiki/Map_projection#Conic) map the sphere onto a cone, and then unroll the cone on to the plane. Conic projections are configured by two _standard parallels_, which determine where the cone intersects the globe.
+    """)
+    return
+
+
+@app.cell
+def _(alt, map):
+    _minimap = map.properties(width=180, height=130)
+    alt.hconcat(_minimap.project(type='conicEqualArea').properties(title='conicEqualArea'), _minimap.project(type='conicEquidistant').properties(title='conicEquidistant'), _minimap.project(type='conicConformal', scale=35, translate=[90, 65]).properties(title='conicConformal'), _minimap.project(type='albers').properties(title='albers'), _minimap.project(type='albersUsa').properties(title='albersUsa')).properties(spacing=10).configure_view(stroke=None)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    - [Conic Equal Area](https://en.wikipedia.org/wiki/Albers_projection) (`conicEqualArea`): Area-preserving conic projection. Shape and distance are not preserved, but roughly accurate within standard parallels.
+    - [Conic Equidistant](https://en.wikipedia.org/wiki/Equidistant_conic_projection) (`conicEquidistant`): Conic projection that preserves distance along the meridians and standard parallels.
+    - [Conic Conformal](https://en.wikipedia.org/wiki/Lambert_conformal_conic_projection) (`conicConformal`): Conic projection that preserves shape (local angles), but not area or distance.
+    - [Albers](https://en.wikipedia.org/wiki/Albers_projection) (`albers`): A variant of the conic equal area projection with standard parallels optimized for creating maps of the United States.
+    - [Albers USA](https://en.wikipedia.org/wiki/Albers_projection) (`albersUsa`): A hybrid projection for the 50 states of the United States of America. This projection stitches together three Albers projections with different parameters for the continental U.S., Alaska, and Hawaii.
+    <br/><br/>
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    [**Azimuthal projections**](https://en.wikipedia.org/wiki/Map_projection#Azimuthal_%28projections_onto_a_plane%29) map the sphere directly onto a plane.
+    """)
+    return
+
+
+@app.cell
+def _(alt, map):
+    _minimap = map.properties(width=180, height=180)
+    alt.hconcat(_minimap.project(type='azimuthalEqualArea').properties(title='azimuthalEqualArea'), _minimap.project(type='azimuthalEquidistant').properties(title='azimuthalEquidistant'), _minimap.project(type='orthographic').properties(title='orthographic'), _minimap.project(type='stereographic').properties(title='stereographic'), _minimap.project(type='gnomonic').properties(title='gnomonic')).properties(spacing=10).configure_view(stroke=None)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    - [Azimuthal Equal Area](https://en.wikipedia.org/wiki/Lambert_azimuthal_equal-area_projection) (`azimuthalEqualArea`): Accurately projects area in all parts of the globe, but does not preserve shape (local angles).
+    - [Azimuthal Equidistant](https://en.wikipedia.org/wiki/Azimuthal_equidistant_projection) (`azimuthalEquidistant`): Preserves proportional distance from the projection center to all other points on the globe.
+    - [Orthographic](https://en.wikipedia.org/wiki/Orthographic_projection_in_cartography) (`orthographic`): Projects a visible hemisphere onto a distant plane. Approximately matches a view of the Earth from outer space.
+    - [Stereographic](https://en.wikipedia.org/wiki/Stereographic_projection) (`stereographic`): Preserves shape, but not area or distance.
+    - [Gnomonic](https://en.wikipedia.org/wiki/Gnomonic_projection) (`gnomonic`): Projects the surface of the sphere directly onto a tangent plane. [Great circles](https://en.wikipedia.org/wiki/Great_circle) around the Earth are projected to straight lines, showing the shortest path between points.
+    <br/><br/>
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Coda: Wrangling Geographic Data
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    The examples above all draw from the vega-datasets collection, including geometric (TopoJSON) and tabular (airports, unemployment rates) data. A common challenge to getting starting with geographic visualization is collecting the necessary data for your task. A number of data providers abound, including services such as the [United States Geological Survey](https://www.usgs.gov/products/data/all-data) and [U.S. Census Bureau](https://www.census.gov/data/datasets.html).
+
+    In many cases you may have existing data with a geographic component, but require additional measures or geometry. To help you get started, here is one workflow:
+
+    1. Visit [Natural Earth Data](http://www.naturalearthdata.com/downloads/) and browse to select data for regions and resolutions of interest. Download the corresponding zip file(s).
+    2. Go to [MapShaper](https://mapshaper.org/) and drop your downloaded zip file onto the page. Revise the data as desired, and then "Export" generated TopoJSON or GeoJSON files.
+    3. Load the exported data from MapShaper for use with Altair!
+
+    Of course, many other tools &ndash; both open-source and proprietary &ndash; exist for working with geographic data. For more about geo-data wrangling and map creation, see Mike Bostock's tutorial series on [Command-Line Cartography](https://medium.com/@mbostock/command-line-cartography-part-1-897aa8f8ca2c).
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Summary
+
+    At this point, we've only dipped our toes into the waters of map-making. _(You didn't expect a single notebook to impart centuries of learning, did you?)_ For example, we left untouched topics such as [_cartograms_](https://en.wikipedia.org/wiki/Cartogram) and conveying [_topography_](https://en.wikipedia.org/wiki/Topography) &mdash; as in Imhof's illuminating book [_Cartographic Relief Presentation_](https://books.google.com/books?id=cVy1Ms43fFYC). Nevertheless, you should now be well-equipped to create a rich array of geo-visualizations. For more, MacEachren's book [_How Maps Work: Representation, Visualization, and Design_](https://books.google.com/books?id=xhAvN3B0CkUC) provides a valuable overview of map-making from the perspective of data visualization.
+    """)
+    return
+
+
+if __name__ == "__main__":
+    app.run()

altair/08_debugging.py

@@ -0,0 +1,370 @@
+# /// script
+# requires-python = ">=3.11"
+# dependencies = [
+#     "altair==6.0.0",
+#     "marimo",
+#     "pandas==3.0.1",
+#     "vega_datasets==0.9.0",
+# ]
+# ///
+
+import marimo
+
+__generated_with = "0.20.4"
+app = marimo.App()
+
+
+@app.cell
+def _():
+    import marimo as mo
+
+    return (mo,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    # Altair Debugging Guide
+
+    In this notebook we show you common debugging techniques that you can use if you run into issues with Altair.
+
+    You can jump to the following sections:
+
+    * [Installation and Setup](#Installation) when Altair is not installed correctly
+    * [Display Issues](#Display-Troubleshooting) when you don't see a chart
+    * [Invalid Specifications](#Invalid-Specifications) when you get an error
+    * [Properties are Being Ignored](#Properties-are-Being-Ignored) when you don't see any errors or warnings
+    * [Asking for Help](#Asking-for-Help) when you get stuck
+    * [Reporting Issues](#Reporting-Issues) when you find a bug
+
+    In addition to this notebook, you might find the [Frequently Asked Questions](https://altair-viz.github.io/user_guide/faq.html) and [Display Troubleshooting](https://altair-viz.github.io/user_guide/troubleshooting.html) guides helpful.
+
+    _This notebook is part of the [data visualization curriculum](https://github.com/uwdata/visualization-curriculum)._
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Installation
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    These instructions follow [the Altair documentation](https://altair-viz.github.io/getting_started/installation.html) but focus on some specifics for this series of notebooks.
+
+    In every notebook, we will import the [Altair](https://github.com/altair-viz/altair) and [Vega Datasets](https://github.com/altair-viz/vega_datasets) packages. If you are running this notebook on [Colab](https://colab.research.google.com), Altair and Vega Datasets should be preinstalled and ready to go. The notebooks in this series are designed for Colab but should also work in Jupyter Lab or the Jupyter Notebook (the notebook requires a bit more setup [described below](#Special-Setup-for-the-Jupyter-Notebook)) but additional packages are required.
+
+    If you are running in Jupyter Lab or Jupyter Notebooks, you have to install the necessary packages by running the following command in your terminal.
+
+    ```bash
+    pip install altair vega_datasets
+    ```
+
+    Or if you use [Conda](https://conda.io)
+
+    ```bash
+    conda install -c conda-forge altair vega_datasets
+    ```
+
+    You can run command line commands from a code cell by prefixing it with `!`. For example, to install Altair and Vega Datasets with [Pip](https://pip.pypa.io/), you can run the following cell.
+    """)
+    return
+
+
+@app.cell
+def _():
+    # packages added via marimo's package management: altair vega_datasets !pip install altair vega_datasets
+    return
+
+
+@app.cell
+def _():
+    import altair as alt
+    from vega_datasets import data
+
+    return alt, data
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Make sure you are Using the Latest Version of Altair
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    If you are running into issues with Altair, first make sure that you are running the latest version. To check the version of Altair that you have installed, run the cell below.
+    """)
+    return
+
+
+@app.cell
+def _(alt):
+    alt.__version__
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    To check what the latest version of altair is, go to [this page](https://pypi.org/project/altair/) or run the cell below (requires Python 3).
+    """)
+    return
+
+
+@app.cell
+def _():
+    import urllib.request, json 
+    with urllib.request.urlopen("https://pypi.org/pypi/altair/json") as url:
+        print(json.loads(url.read().decode())['info']['version'])
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    If you are not running the latest version, you can update it with `pip`. You can update Altair and Vega Datasets by running this command in your terminal.
+
+    ```
+    pip install -U altair vega_datasets
+    ```
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Try Making a Chart
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Now you can create an Altair chart.
+    """)
+    return
+
+
+@app.cell
+def _(alt, data):
+    cars = data.cars()
+
+    alt.Chart(cars).mark_point().encode(
+        x='Horsepower',
+        y='Displacement',
+        color='Origin'
+    )
+    return (cars,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Special Setup for the Jupyter Notebook
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    If you are running in Jupyter Lab, Jupyter Notebook, or Colab (and have a working Internet connection) you should be seeing a chart. If you are running in another environment (or offline), you will need to tell Altair to use a different renderer;
+
+    To activate a different renderer in a notebook cell:
+
+    ```python
+    # to run in nteract, VSCode, or offline in JupyterLab
+    alt.renderers.enable('mimebundle')
+
+    ```
+
+    To run offline in Jupyter Notebook you must install an additional dependency, the `vega` package. Run this command in your terminal:
+
+    ```bash
+    pip install vega
+    ```
+
+    Then activate the notebook renderer:
+
+    ```python
+    # to run offline in Jupyter Notebook
+    alt.renderers.enable('notebook')
+
+    ```
+
+
+    These instruction follow [the instructions on the Altair website](https://altair-viz.github.io/getting_started/installation.html#installation-notebook).
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Display Troubleshooting
+
+    If you are having issues with seeing a chart, make sure your setup is correct by following the [debugging instruction above](#Installation). If you are still having issues, follow the [instruction about debugging display issues in the Altair documentation](https://iliatimofeev.github.io/altair-viz.github.io/user_guide/troubleshooting.html).
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Non Existent Fields
+
+    A common error is [accidentally using a field that does not exist](https://iliatimofeev.github.io/altair-viz.github.io/user_guide/troubleshooting.html#plot-displays-but-the-content-is-empty).
+    """)
+    return
+
+
+@app.cell
+def _(alt):
+    import pandas as pd
+
+    df = pd.DataFrame({'x': [1, 2, 3],
+                         'y': [3, 1, 4]})
+
+    alt.Chart(df).mark_point().encode(
+        x='x:Q',
+        y='y:Q',
+        color='color:Q'  # <-- this field does not exist in the data!
+    )
+    return (df,)
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Check the spelling of your files and print the data source to confirm that the data and fields exist. For instance, here you see that `color` is not a vaid field.
+    """)
+    return
+
+
+@app.cell
+def _(df):
+    df.head()
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Invalid Specifications
+
+    Another common issue is creating an invalid specification and getting an error.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ### Invalid Properties
+
+    Altair might show an `SchemaValidationError` or `ValueError`. Read the error message carefully. Usually it will tell you what is going wrong.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    For example, if you forget the mark type, you will see this `SchemaValidationError`.
+    """)
+    return
+
+
+@app.cell
+def _(alt, cars):
+    alt.Chart(cars).encode(
+        y='Horsepower'
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    Or if you use a non-existent channel, you get a `TypeError`.
+    """)
+    return
+
+
+@app.cell
+def _(alt, cars):
+    try:
+        alt.Chart(cars).mark_point().encode(
+            z='Horsepower'
+        )
+    except TypeError as e:
+        print(f"TypeError: {e}")
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Properties are Being Ignored
+
+    Altair might ignore a property that you specified. In the chart below, we are using a `text` channel, which is only compatible with `mark_text`. You do not see an error or a warning about this in the notebook. However, the underlying Vega-Lite library will show a warning in the browser console.  Press <kbd>Alt</kbd>+<kbd>Cmd</kbd>+<kbd>I</kbd> on Mac or <kbd>Alt</kbd>+<kbd>Ctrl</kbd>+<kbd>I</kbd> on Windows and Linux to open the developer tools and click on the `Console` tab. When you run the example in the cell below, you will see a the following warning.
+
+    ```
+    WARN text dropped as it is incompatible with "bar".
+    ```
+    """)
+    return
+
+
+@app.cell
+def _(alt, cars):
+    alt.Chart(cars).mark_bar().encode(
+        y='mean(Horsepower)',
+        text='mean(Acceleration)'
+    )
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    If you find yourself debugging issues related to Vega-Lite, you can open the chart in the [Vega Editor](https://vega.github.io/editor/) either by clicking on the "Open in Vega Editor" link at the bottom of the chart or in the action menu (click to open) at the top right of a chart. The Vega Editor provides additional debugging but you will be writing Vega-Lite JSON instead of Altair in Python.
+
+    **Note**: The Vega Editor may be using a newer version of Vega-Lite and so the behavior may vary.
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Asking for Help
+
+    If you find a problem with Altair and get stuck, you can ask a question on Stack Overflow. Ask your question with the `altair` and `vega-lite` tags. You can find a list of questions people have asked before [here](https://stackoverflow.com/questions/tagged/altair).
+    """)
+    return
+
+
+@app.cell(hide_code=True)
+def _(mo):
+    mo.md(r"""
+    ## Reporting Issues
+
+    If you find a problem with Altair and believe it is a bug, please [create an issue in the Altair GitHub repo](https://github.com/altair-viz/altair/issues/new) with a description of your problem. If you believe the issue is related to the underlying Vega-Lite library, please [create an issue in the Vega-Lite GitHub repo](https://github.com/vega/vega-lite/issues/new).
+    """)
+    return
+
+
+if __name__ == "__main__":
+    app.run()

altair/index.md

@@ -0,0 +1,14 @@
+---
+title: Learn Altair
+description: >
+    Learn the basics of Altair, a high-performance visualization library,
+    using lessons developed at the University of Washington.
+---
+
+## Acknowledgments
+
+These notebooks were created by Jeffrey Heer, Dominik Moritz, Jake VanderPlas, and Brock Craft
+as part of the [Visualization Curriculum](https://uwdata.github.io/visualization-curriculum/intro.html)
+at the University of Washington.
+Our thanks to the authors for making their work available under an open license:
+if we all share a little, we all get a lot.

assets/styles.css

@@ -0,0 +1,51 @@
+:root {
+    --primary-green: #10B981;
+    --dark-green: #047857;
+    --light-green: #D1FAE5;
+}
+.bg-primary { background-color: var(--primary-green); }
+.text-primary { color: var(--primary-green); }
+.border-primary { border-color: var(--primary-green); }
+.bg-light { background-color: var(--light-green); }
+.hover-grow { transition: transform 0.2s ease; }
+.hover-grow:hover { transform: scale(1.02); }
+.card-shadow { box-shadow: 0 4px 6px rgba(0, 0, 0, 0.05), 0 1px 3px rgba(0, 0, 0, 0.1); }
+
+/* Prose styles for markdown-generated content */
+.prose h1 { font-size: 1.875rem; font-weight: 700; color: #1f2937; margin: 1.5rem 0 0.75rem; }
+.prose h2 { font-size: 1.5rem;   font-weight: 700; color: #1f2937; margin: 1.5rem 0 0.75rem; }
+.prose h3 { font-size: 1.25rem;  font-weight: 600; color: #1f2937; margin: 1.25rem 0 0.5rem; }
+.prose h4 { font-size: 1.125rem; font-weight: 600; color: #1f2937; margin: 1rem 0 0.5rem; }
+.prose p  { color: #4b5563; margin-bottom: 1rem; line-height: 1.75; }
+.prose ul { list-style-type: disc;    padding-left: 1.25rem; margin-bottom: 1rem; color: #4b5563; }
+.prose ol { list-style-type: decimal; padding-left: 1.25rem; margin-bottom: 1rem; color: #4b5563; }
+.prose li { margin-bottom: 0.25rem; line-height: 1.75; }
+.prose a  { color: var(--primary-green); }
+.prose a:hover { color: var(--dark-green); }
+.prose strong { font-weight: 600; }
+.prose code { font-family: ui-monospace, monospace; font-size: 0.875em;
+              background-color: #f3f4f6; padding: 0.1em 0.3em; border-radius: 0.25rem; }
+.prose pre  { background-color: #f3f4f6; color: #1f2937; padding: 1rem;
+              border-radius: 0.5rem; overflow-x: auto; margin-bottom: 1rem; }
+.prose pre code { background: none; padding: 0; font-size: 0.875rem; color: inherit; }
+
+/* Component classes */
+.logo-container { background-color: var(--light-green); padding: 0.25rem; border-radius: 0.5rem; }
+.card-accent    { height: 0.5rem; background-color: var(--primary-green); }
+.feature-card   { background-color: #ffffff; padding: 1.5rem; border-radius: 0.5rem;
+                  box-shadow: 0 4px 6px rgba(0, 0, 0, 0.05), 0 1px 3px rgba(0, 0, 0, 0.1); }
+.content-card   { background-color: #ffffff; border: 1px solid #e5e7eb; border-radius: 0.5rem;
+                  overflow: hidden; box-shadow: 0 4px 6px rgba(0, 0, 0, 0.05), 0 1px 3px rgba(0, 0, 0, 0.1); }
+.icon-container { width: 3rem; height: 3rem; background-color: var(--light-green);
+                  border-radius: 9999px; display: flex; align-items: center;
+                  justify-content: center; margin-bottom: 1rem; }
+
+.link-primary       { color: var(--primary-green); }
+.link-primary:hover { color: var(--dark-green); }
+
+.btn-primary       { background-color: var(--primary-green); color: #ffffff; font-weight: 500;
+                     border-radius: 0.375rem; transition: background-color 300ms ease-in-out; }
+.btn-primary:hover { background-color: var(--dark-green); }
+
+.footer-link       { color: #d1d5db; transition: color 300ms ease-in-out; }
+.footer-link:hover { color: #ffffff; }

bin/build.py

@@ -0,0 +1,93 @@
+#!/usr/bin/env python
+"""Generate a static site from Jinja2 templates and lesson data."""
+
+import argparse
+import datetime
+import json
+import re
+import shutil
+from pathlib import Path
+
+import frontmatter
+import markdown as md
+from jinja2 import Environment, FileSystemLoader
+
+from utils import get_notebook_title
+
+
+def transform_lessons(data: dict, root: Path) -> dict:
+    """Transform raw lesson data into template-ready form."""
+    for course_id, course in data.items():
+        desc = course.get("description", "").strip()
+        course["description_html"] = f"<p>{desc}</p>" if desc else ""
+        course["notebooks"] = [
+            {
+                "title": get_notebook_title(root / course_id / nb)
+                         or re.sub(r"^\d+_", "", nb.replace(".py", "")).replace("_", " ").title(),
+                "html_path": f"{course_id}/{nb.replace('.py', '.html')}",
+                "local_html_path": nb.replace(".py", ".html"),
+            }
+            for nb in course.get("notebooks", [])
+        ]
+        index_md = root / course_id / "index.md"
+        post = frontmatter.load(index_md)
+        course["body_html"] = md.markdown(post.content, extensions=["fenced_code", "tables"])
+    return data
+
+
+def render(template, path, **kwargs):
+    path.parent.mkdir(parents=True, exist_ok=True)
+    path.write_text(template.render(**kwargs))
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Generate static site from lesson data")
+    parser.add_argument("--root", required=True, help="Project root directory")
+    parser.add_argument("--output", required=True, help="Output directory")
+    parser.add_argument("--data", required=True, help="Path to lessons JSON file")
+    args = parser.parse_args()
+
+    root = Path(args.root)
+    output = Path(args.output)
+    output.mkdir(parents=True, exist_ok=True)
+
+    lessons = transform_lessons(json.loads(Path(args.data).read_text()), root)
+    env = Environment(loader=FileSystemLoader(root / "templates"))
+    current_year = datetime.date.today().year
+
+    render(
+        env.get_template("index.html"),
+        output / "index.html",
+        courses=lessons,
+        current_year=current_year,
+        root_path="",
+    )
+
+    assets_src = root / "assets"
+    if assets_src.exists():
+        shutil.copytree(assets_src, output / "assets", dirs_exist_ok=True)
+
+    for course_id, lesson in lessons.items():
+        render(
+            env.get_template("lesson.html"),
+            output / course_id / "index.html",
+            lesson=lesson,
+            current_year=current_year,
+            root_path="../",
+        )
+
+    page_template = env.get_template("page.html")
+    for page_src in sorted((root / "pages").glob("*.md")):
+        post = frontmatter.load(page_src)
+        render(
+            page_template,
+            output / page_src.stem / "index.html",
+            title=post.get("title", page_src.stem),
+            body_html=md.markdown(post.content, extensions=["fenced_code", "tables"]),
+            current_year=current_year,
+            root_path="../",
+        )
+
+
+if __name__ == "__main__":
+    main()

{scripts → bin}/check_empty_cells.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 """
 Script to detect empty cells in marimo notebooks.
 
@@ -15,7 +15,6 @@ This script will:
 """
 
 import os
-import re
 import sys
 from pathlib import Path
 from typing import List, Tuple
@@ -136,4 +135,4 @@ def main():
 
 
 if __name__ == "__main__":
-    main() 
+    main() 

bin/check_missing_titles.py

@@ -0,0 +1,21 @@
+#!/usr/bin/env python
+"""Report marimo notebooks that are missing an H1 title."""
+
+import sys
+from pathlib import Path
+
+from utils import get_notebook_title
+
+
+def main():
+    root = Path(__file__).parent.parent
+    notebooks = sorted(root.glob("*/[0-9]*.py"))
+    missing = [nb for nb in notebooks if get_notebook_title(nb) is None]
+    if missing:
+        for nb in missing:
+            print(nb.relative_to(root))
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    main()

bin/check_notebook_packages.py

@@ -0,0 +1,110 @@
+#!/usr/bin/env python
+"""Check that marimo notebooks in the same lesson directory agree on package versions.
+
+It is acceptable for different notebooks in a directory to specify different packages,
+but if two or more notebooks specify the same package, their version constraints must
+be identical.
+"""
+
+import argparse
+import re
+import sys
+from collections import defaultdict
+from pathlib import Path
+
+
+# Regex to extract the inline script metadata block (PEP 723)
+SCRIPT_BLOCK_RE = re.compile(r"^# /// script\s*\n((?:#[^\n]*\n)*?)# ///", re.MULTILINE)
+DEPENDENCY_LINE_RE = re.compile(r'^#\s+"([^"]+)",?\s*$')
+
+
+def parse_script_header(text: str) -> list[str]:
+    """Return the list of dependency strings from a PEP 723 script header, or []."""
+    match = SCRIPT_BLOCK_RE.search(text)
+    if not match:
+        return []
+    block = match.group(1)
+    deps: list[str] = []
+    in_deps = False
+    for raw_line in block.splitlines():
+        line = raw_line.lstrip("#").strip()
+        if line.startswith("dependencies"):
+            in_deps = True
+            continue
+        if in_deps:
+            if line.startswith("]"):
+                break
+            # strip surrounding quotes and comma: e.g. '    "polars==1.0",' -> 'polars==1.0'
+            stripped = line.strip().strip('"\'').rstrip(",").strip('"\'')
+            if stripped:
+                deps.append(stripped)
+    return deps
+
+
+def package_name(dep: str) -> str:
+    """Extract the bare package name from a PEP 508 dependency string.
+
+    Examples:
+        "polars==1.22.0"  -> "polars"
+        "pandas>=2.0,<3"  -> "pandas"
+        "marimo"          -> "marimo"
+    """
+    return re.split(r"[><=!;\s\[]", dep, maxsplit=1)[0].lower()
+
+
+def check_directory(lesson_dir: Path, only: set[str]) -> list[str]:
+    """Return a list of error messages for version inconsistencies among *only* in lesson_dir."""
+    # Map package name -> {version_spec: [notebook_path, ...]}
+    seen: dict[str, dict[str, list[str]]] = defaultdict(lambda: defaultdict(list))
+
+    for nb in sorted(lesson_dir.glob("*.py")):
+        if nb.name not in only:
+            continue
+        try:
+            text = nb.read_text(encoding="utf-8")
+        except IOError:
+            continue
+        if "marimo.App" not in text:
+            continue
+        for dep in parse_script_header(text):
+            name = package_name(dep)
+            seen[name][dep].append(nb.name)
+
+    errors: list[str] = []
+    for name, specs in sorted(seen.items()):
+        if len(specs) > 1:
+            errors.append(f"  Package '{name}' has conflicting specifications:")
+            for spec, files in sorted(specs.items()):
+                errors.append(f"    {spec!r} in: {', '.join(files)}")
+    return errors
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("notebooks", nargs="+", metavar="NOTEBOOK",
+                        help="notebook files to check (grouped by directory)")
+    args = parser.parse_args()
+
+    dir_filter: dict[Path, set[str]] = defaultdict(set)
+    for nb_path in (Path(p) for p in args.notebooks):
+        dir_filter[nb_path.parent].add(nb_path.name)
+
+    total_errors = 0
+    for lesson_dir, only in sorted(dir_filter.items()):
+        errors = check_directory(lesson_dir, only=only)
+        if errors:
+            print(f"\n{lesson_dir}/")
+            for msg in errors:
+                print(msg)
+            total_errors += len(errors)
+
+    if total_errors:
+        print(f"\nFound package version inconsistencies in {total_errors} package(s).")
+        sys.exit(1)
+    else:
+        print("All package version specifications are consistent.")
+        sys.exit(0)
+
+
+if __name__ == "__main__":
+    main()

bin/create_sql_lab.sql

@@ -0,0 +1,22 @@
+create table job (
+    name text not null,
+    credits real not null
+);
+
+create table work (
+    person text not null,
+    job text not null
+);
+
+insert into job values
+('calibrate', 1.5),
+('clean', 0.5);
+
+insert into work values
+('Amal', 'calibrate'),
+('Amal', 'clean'),
+('Amal', 'complain'),
+('Gita', 'clean'),
+('Gita', 'clean'),
+('Gita', 'complain'),
+('Madhi', 'complain');

bin/create_sql_penguins.py

@@ -0,0 +1,50 @@
+#!/usr/bin/env python
+
+import csv
+import sqlite3
+import sys
+
+
+SCHEMA = """
+CREATE TABLE penguins (
+    species text,
+    island text,
+    bill_length_mm real,
+    bill_depth_mm real,
+    flipper_length_mm real,
+    body_mass_g real,
+    sex text
+);
+"""
+
+def main():
+    infile = sys.argv[1]
+    outfile = sys.argv[2]
+
+    con = sqlite3.connect(outfile)
+    con.execute(SCHEMA)
+
+    with open(infile, newline="") as f:
+        reader = csv.DictReader(f)
+        rows = [
+            (
+                row["species"],
+                row["island"],
+                float(row["bill_length_mm"]) if row["bill_length_mm"] else None,
+                float(row["bill_depth_mm"]) if row["bill_depth_mm"] else None,
+                float(row["flipper_length_mm"]) if row["flipper_length_mm"] else None,
+                float(row["body_mass_g"]) if row["body_mass_g"] else None,
+                row["sex"] if row["sex"] else None,
+            )
+            for row in reader
+        ]
+
+    con.executemany(
+        "INSERT INTO penguins VALUES (?, ?, ?, ?, ?, ?, ?)", rows
+    )
+    con.commit()
+    con.close()
+
+
+if __name__ == "__main__":
+    main()

bin/create_sql_survey.py

@@ -0,0 +1,175 @@
+#!/usr/bin/env python
+
+import datetime
+import faker
+import itertools
+import random
+import sqlite3
+import sys
+
+
+LOCALE = "es"
+
+NUM_PERSONS = 6
+
+DATE_START = datetime.date(2025, 9, 1)
+DATE_END = datetime.date(2025, 12, 31)
+DATE_DURATION = 7
+
+NUM_MACHINES = 5
+
+CREATE_PERSONS = """\
+create table person(
+    person_id text not null primary key,
+    personal text not null,
+    family text not null,
+    supervisor_id text,
+    foreign key(supervisor_id) references person(person_id)
+);
+"""
+INSERT_PERSONS = """\
+insert into person values (:person_id, :personal, :family, :supervisor_id);
+"""
+
+CREATE_SURVEYS = """\
+create table survey(
+    survey_id text not null primary key,
+    person_id text not null,
+    start_date text,
+    end_date text,
+    foreign key(person_id) references person(person_id)
+);
+"""
+INSERT_SURVEYS = """\
+insert into survey values(:survey_id, :person_id, :start, :end);
+"""
+
+CREATE_MACHINES = """\
+create table machine(
+    machine_id text not null primary key,
+    machine_type text not null
+);
+"""
+INSERT_MACHINES = """\
+insert into machine values(:machine_id, :machine_type);
+"""
+
+CREATE_RATINGS = """\
+create table rating(
+    person_id text not null,
+    machine_id text not null,
+    level integer,
+    foreign key(person_id) references person(person_id),
+    foreign key(machine_id) references machine(machine_id)
+);
+"""
+INSERT_RATINGS = """\
+insert into rating values(:person_id, :machine_id, :level);
+"""
+
+def main():
+    db_name = sys.argv[1]
+    seed = int(sys.argv[2])
+    random.seed(seed)
+
+    persons_counter = itertools.count()
+    next(persons_counter)
+    persons = gen_persons(NUM_PERSONS, persons_counter)
+
+    supers = gen_persons(int(NUM_PERSONS / 2), persons_counter)
+    for p in persons:
+        p["supervisor_id"] = random.choice(supers)["person_id"]
+    if len(supers) > 1:
+        supers[0]["supervisor_id"] = supers[-1]["person_id"]
+
+    surveys = gen_surveys(persons + supers[0:int(len(supers)/2)])
+    surveys[int(len(surveys)/2)]["start"] = None
+
+    cnx = sqlite3.connect(db_name)
+    cur = cnx.cursor()
+
+    everyone = persons + supers
+    random.shuffle(everyone)
+    cur.execute(CREATE_PERSONS)
+    cur.executemany(INSERT_PERSONS, everyone)
+
+    cur.execute(CREATE_SURVEYS)
+    cur.executemany(INSERT_SURVEYS, surveys)
+
+    machines = gen_machines()
+    cur.execute(CREATE_MACHINES)
+    cur.executemany(INSERT_MACHINES, machines)
+
+    ratings = gen_ratings(everyone, machines)
+    cur.execute(CREATE_RATINGS)
+    cur.executemany(INSERT_RATINGS, ratings)
+
+    cnx.commit()
+    cnx.close()
+
+
+def gen_machines():
+    adjectives = "hydraulic rotary modular industrial automated".split()
+    nouns = "press conveyor generator actuator compressor".split()
+    machines = set()
+    while len(machines) < NUM_MACHINES:
+        candidate = f"{random.choice(adjectives)} {random.choice(nouns)}"
+        if candidate not in machines:
+            machines.add(candidate)
+    counter = itertools.count()
+    next(counter)
+    return [
+        {"machine_id": f"M{next(counter):04d}", "machine_type": m}
+        for m in machines
+    ]
+
+
+def gen_persons(num, counter):
+    fake = faker.Faker(LOCALE)
+    fake.seed_instance(random.randint(0, 1_000_000))
+    return [
+        {
+            "person_id": f"P{next(counter):03d}",
+            "personal": fake.first_name(),
+            "family": fake.last_name(),
+            "supervisor_id": None,
+        }
+        for _ in range(num)
+    ]
+
+
+def gen_ratings(persons, machines):
+    temp = {}
+    while len(temp) < int(len(persons) * len(machines) / 4):
+        p = random.choice(persons)["person_id"]
+        m = random.choice(machines)["machine_id"]
+        if (p, m) in temp:
+            continue
+        temp[(p, m)] = random.choice([None, 1, 2, 3])
+    return [
+        {"person_id": p, "machine_id": m, "level": v}
+        for ((p, m), v) in temp.items()
+    ]
+
+def gen_surveys(persons):
+    surveys = []
+    counter = itertools.count()
+    next(counter)
+    for person in persons:
+        person_id = person["person_id"]
+        start = DATE_START
+        while start <= DATE_END:
+            survey_id = f"S{next(counter):04d}"
+            end = start + datetime.timedelta(days=random.randint(1, DATE_DURATION))
+            surveys.append({
+                "survey_id": survey_id,
+                "person_id": person_id,
+                "start": start.isoformat(),
+                "end": end.isoformat() if end <= DATE_END else None
+            })
+            start = end + datetime.timedelta(days=random.randint(1, DATE_DURATION))
+    return surveys
+
+
+if __name__ == "__main__":
+    main()

bin/extract.py

@@ -0,0 +1,47 @@
+#!/usr/bin/env python
+"""Extract lesson metadata and notebook lists into a JSON file."""
+
+import argparse
+import json
+import re
+from pathlib import Path
+
+import frontmatter
+
+
+NOTEBOOK_PATTERN = re.compile(r"^\d{2}_.*\.py$")
+
+
+def extract_lessons(root: Path) -> dict:
+    lessons = {}
+    for index_file in sorted(root.glob("*/index.md")):
+        lesson_dir = index_file.parent
+        post = frontmatter.load(index_file)
+        notebooks = sorted(
+            p.name
+            for p in lesson_dir.glob("*.py")
+            if NOTEBOOK_PATTERN.match(p.name)
+        )
+        lessons[lesson_dir.name] = {
+            **post.metadata,
+            "notebooks": notebooks,
+        }
+    return lessons
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Extract lesson metadata to JSON")
+    parser.add_argument("--root", required=True, help="Project root directory")
+    parser.add_argument("--data", required=True, help="Output JSON file")
+    args = parser.parse_args()
+
+    root = Path(args.root)
+    data = Path(args.data)
+    data.parent.mkdir(parents=True, exist_ok=True)
+
+    lessons = extract_lessons(root)
+    data.write_text(json.dumps(lessons, indent=2))
+
+
+if __name__ == "__main__":
+    main()

{scripts → bin}/preview.py

@@ -1,10 +1,9 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 
 import os
 import subprocess
 import argparse
 import webbrowser
-import time
 import sys
 from pathlib import Path
 

bin/run_notebooks.sh

@@ -0,0 +1,11 @@
+#!/usr/bin/env bash
+for nb in $*
+do
+	cd $(dirname $nb)
+	if ! output=$(uv run $(basename $nb) 2>&1); then
+		echo "=== $nb ==="
+		echo "$output"
+		echo
+	fi
+	cd $OLDPWD
+done

bin/utils.py

@@ -0,0 +1,14 @@
+"""Utility functions for working with marimo notebooks."""
+
+import re
+from pathlib import Path
+
+
+def get_notebook_title(path: Path) -> str | None:
+    """Return the first H1 Markdown heading in a marimo notebook, or None."""
+    text = path.read_text(encoding="utf-8")
+    for match in re.finditer(r'mo\.md\(r?f?"""(.*?)"""', text, re.DOTALL):
+        for line in match.group(1).splitlines():
+            if line.strip().startswith("# "):
+                return line.strip()[2:].strip()
+    return None

daft/README.md

@@ -1,31 +0,0 @@
----
-title: Readme
-marimo-version: 0.18.4
----
-
-# Learn Daft
-
-_🚧 This collection is a work in progress. Please help us add notebooks!_
-
-This collection of marimo notebooks is designed to teach you the basics of
-Daft, a distributed dataframe engine that unifies data engineering, analytics & ML/AI workflows.
-
-**Help us build this course! ⚒️**
-
-We're seeking contributors to help us build these notebooks. Every contributor
-will be acknowledged as an author in this README and in their contributed
-notebooks. Head over to the [tracking
-issue](https://github.com/marimo-team/learn/issues/43) to sign up for a planned
-notebook or propose your own.
-
-**Running notebooks.** To run a notebook locally, use
-
-```bash
-uvx marimo edit <file_url>
-```
-
-You can also open notebooks in our online playground by appending marimo.app/ to a notebook's URL.
-
-**Thanks to all our notebook authors!**
-
-* [Péter Gyarmati](https://github.com/peter-gy)

daft/_index.md

@@ -0,0 +1,13 @@
+---
+title: Learn Daft
+description: >
+    These notebooks introduce Daft, a distributed dataframe engine
+    that unifies data engineering, analysis, and ML/AI workflows.
+tracking: 43
+---
+
+## Contributors
+
+Thanks to our notebook authors:
+
+* [Péter Gyarmati](https://github.com/peter-gy)

data/penguins.csv

@@ -0,0 +1,345 @@
+species,island,bill_length_mm,bill_depth_mm,flipper_length_mm,body_mass_g,sex
+Adelie,Torgersen,39.1,18.7,181,3750,MALE
+Adelie,Torgersen,39.5,17.4,186,3800,FEMALE
+Adelie,Torgersen,40.3,18,195,3250,FEMALE
+Adelie,Torgersen,,,,,
+Adelie,Torgersen,36.7,19.3,193,3450,FEMALE
+Adelie,Torgersen,39.3,20.6,190,3650,MALE
+Adelie,Torgersen,38.9,17.8,181,3625,FEMALE
+Adelie,Torgersen,39.2,19.6,195,4675,MALE
+Adelie,Torgersen,34.1,18.1,193,3475,
+Adelie,Torgersen,42,20.2,190,4250,
+Adelie,Torgersen,37.8,17.1,186,3300,
+Adelie,Torgersen,37.8,17.3,180,3700,
+Adelie,Torgersen,41.1,17.6,182,3200,FEMALE
+Adelie,Torgersen,38.6,21.2,191,3800,MALE
+Adelie,Torgersen,34.6,21.1,198,4400,MALE
+Adelie,Torgersen,36.6,17.8,185,3700,FEMALE
+Adelie,Torgersen,38.7,19,195,3450,FEMALE
+Adelie,Torgersen,42.5,20.7,197,4500,MALE
+Adelie,Torgersen,34.4,18.4,184,3325,FEMALE
+Adelie,Torgersen,46,21.5,194,4200,MALE
+Adelie,Biscoe,37.8,18.3,174,3400,FEMALE
+Adelie,Biscoe,37.7,18.7,180,3600,MALE
+Adelie,Biscoe,35.9,19.2,189,3800,FEMALE
+Adelie,Biscoe,38.2,18.1,185,3950,MALE
+Adelie,Biscoe,38.8,17.2,180,3800,MALE
+Adelie,Biscoe,35.3,18.9,187,3800,FEMALE
+Adelie,Biscoe,40.6,18.6,183,3550,MALE
+Adelie,Biscoe,40.5,17.9,187,3200,FEMALE
+Adelie,Biscoe,37.9,18.6,172,3150,FEMALE
+Adelie,Biscoe,40.5,18.9,180,3950,MALE
+Adelie,Dream,39.5,16.7,178,3250,FEMALE
+Adelie,Dream,37.2,18.1,178,3900,MALE
+Adelie,Dream,39.5,17.8,188,3300,FEMALE
+Adelie,Dream,40.9,18.9,184,3900,MALE
+Adelie,Dream,36.4,17,195,3325,FEMALE
+Adelie,Dream,39.2,21.1,196,4150,MALE
+Adelie,Dream,38.8,20,190,3950,MALE
+Adelie,Dream,42.2,18.5,180,3550,FEMALE
+Adelie,Dream,37.6,19.3,181,3300,FEMALE
+Adelie,Dream,39.8,19.1,184,4650,MALE
+Adelie,Dream,36.5,18,182,3150,FEMALE
+Adelie,Dream,40.8,18.4,195,3900,MALE
+Adelie,Dream,36,18.5,186,3100,FEMALE
+Adelie,Dream,44.1,19.7,196,4400,MALE
+Adelie,Dream,37,16.9,185,3000,FEMALE
+Adelie,Dream,39.6,18.8,190,4600,MALE
+Adelie,Dream,41.1,19,182,3425,MALE
+Adelie,Dream,37.5,18.9,179,2975,
+Adelie,Dream,36,17.9,190,3450,FEMALE
+Adelie,Dream,42.3,21.2,191,4150,MALE
+Adelie,Biscoe,39.6,17.7,186,3500,FEMALE
+Adelie,Biscoe,40.1,18.9,188,4300,MALE
+Adelie,Biscoe,35,17.9,190,3450,FEMALE
+Adelie,Biscoe,42,19.5,200,4050,MALE
+Adelie,Biscoe,34.5,18.1,187,2900,FEMALE
+Adelie,Biscoe,41.4,18.6,191,3700,MALE
+Adelie,Biscoe,39,17.5,186,3550,FEMALE
+Adelie,Biscoe,40.6,18.8,193,3800,MALE
+Adelie,Biscoe,36.5,16.6,181,2850,FEMALE
+Adelie,Biscoe,37.6,19.1,194,3750,MALE
+Adelie,Biscoe,35.7,16.9,185,3150,FEMALE
+Adelie,Biscoe,41.3,21.1,195,4400,MALE
+Adelie,Biscoe,37.6,17,185,3600,FEMALE
+Adelie,Biscoe,41.1,18.2,192,4050,MALE
+Adelie,Biscoe,36.4,17.1,184,2850,FEMALE
+Adelie,Biscoe,41.6,18,192,3950,MALE
+Adelie,Biscoe,35.5,16.2,195,3350,FEMALE
+Adelie,Biscoe,41.1,19.1,188,4100,MALE
+Adelie,Torgersen,35.9,16.6,190,3050,FEMALE
+Adelie,Torgersen,41.8,19.4,198,4450,MALE
+Adelie,Torgersen,33.5,19,190,3600,FEMALE
+Adelie,Torgersen,39.7,18.4,190,3900,MALE
+Adelie,Torgersen,39.6,17.2,196,3550,FEMALE
+Adelie,Torgersen,45.8,18.9,197,4150,MALE
+Adelie,Torgersen,35.5,17.5,190,3700,FEMALE
+Adelie,Torgersen,42.8,18.5,195,4250,MALE
+Adelie,Torgersen,40.9,16.8,191,3700,FEMALE
+Adelie,Torgersen,37.2,19.4,184,3900,MALE
+Adelie,Torgersen,36.2,16.1,187,3550,FEMALE
+Adelie,Torgersen,42.1,19.1,195,4000,MALE
+Adelie,Torgersen,34.6,17.2,189,3200,FEMALE
+Adelie,Torgersen,42.9,17.6,196,4700,MALE
+Adelie,Torgersen,36.7,18.8,187,3800,FEMALE
+Adelie,Torgersen,35.1,19.4,193,4200,MALE
+Adelie,Dream,37.3,17.8,191,3350,FEMALE
+Adelie,Dream,41.3,20.3,194,3550,MALE
+Adelie,Dream,36.3,19.5,190,3800,MALE
+Adelie,Dream,36.9,18.6,189,3500,FEMALE
+Adelie,Dream,38.3,19.2,189,3950,MALE
+Adelie,Dream,38.9,18.8,190,3600,FEMALE
+Adelie,Dream,35.7,18,202,3550,FEMALE
+Adelie,Dream,41.1,18.1,205,4300,MALE
+Adelie,Dream,34,17.1,185,3400,FEMALE
+Adelie,Dream,39.6,18.1,186,4450,MALE
+Adelie,Dream,36.2,17.3,187,3300,FEMALE
+Adelie,Dream,40.8,18.9,208,4300,MALE
+Adelie,Dream,38.1,18.6,190,3700,FEMALE
+Adelie,Dream,40.3,18.5,196,4350,MALE
+Adelie,Dream,33.1,16.1,178,2900,FEMALE
+Adelie,Dream,43.2,18.5,192,4100,MALE
+Adelie,Biscoe,35,17.9,192,3725,FEMALE
+Adelie,Biscoe,41,20,203,4725,MALE
+Adelie,Biscoe,37.7,16,183,3075,FEMALE
+Adelie,Biscoe,37.8,20,190,4250,MALE
+Adelie,Biscoe,37.9,18.6,193,2925,FEMALE
+Adelie,Biscoe,39.7,18.9,184,3550,MALE
+Adelie,Biscoe,38.6,17.2,199,3750,FEMALE
+Adelie,Biscoe,38.2,20,190,3900,MALE
+Adelie,Biscoe,38.1,17,181,3175,FEMALE
+Adelie,Biscoe,43.2,19,197,4775,MALE
+Adelie,Biscoe,38.1,16.5,198,3825,FEMALE
+Adelie,Biscoe,45.6,20.3,191,4600,MALE
+Adelie,Biscoe,39.7,17.7,193,3200,FEMALE
+Adelie,Biscoe,42.2,19.5,197,4275,MALE
+Adelie,Biscoe,39.6,20.7,191,3900,FEMALE
+Adelie,Biscoe,42.7,18.3,196,4075,MALE
+Adelie,Torgersen,38.6,17,188,2900,FEMALE
+Adelie,Torgersen,37.3,20.5,199,3775,MALE
+Adelie,Torgersen,35.7,17,189,3350,FEMALE
+Adelie,Torgersen,41.1,18.6,189,3325,MALE
+Adelie,Torgersen,36.2,17.2,187,3150,FEMALE
+Adelie,Torgersen,37.7,19.8,198,3500,MALE
+Adelie,Torgersen,40.2,17,176,3450,FEMALE
+Adelie,Torgersen,41.4,18.5,202,3875,MALE
+Adelie,Torgersen,35.2,15.9,186,3050,FEMALE
+Adelie,Torgersen,40.6,19,199,4000,MALE
+Adelie,Torgersen,38.8,17.6,191,3275,FEMALE
+Adelie,Torgersen,41.5,18.3,195,4300,MALE
+Adelie,Torgersen,39,17.1,191,3050,FEMALE
+Adelie,Torgersen,44.1,18,210,4000,MALE
+Adelie,Torgersen,38.5,17.9,190,3325,FEMALE
+Adelie,Torgersen,43.1,19.2,197,3500,MALE
+Adelie,Dream,36.8,18.5,193,3500,FEMALE
+Adelie,Dream,37.5,18.5,199,4475,MALE
+Adelie,Dream,38.1,17.6,187,3425,FEMALE
+Adelie,Dream,41.1,17.5,190,3900,MALE
+Adelie,Dream,35.6,17.5,191,3175,FEMALE
+Adelie,Dream,40.2,20.1,200,3975,MALE
+Adelie,Dream,37,16.5,185,3400,FEMALE
+Adelie,Dream,39.7,17.9,193,4250,MALE
+Adelie,Dream,40.2,17.1,193,3400,FEMALE
+Adelie,Dream,40.6,17.2,187,3475,MALE
+Adelie,Dream,32.1,15.5,188,3050,FEMALE
+Adelie,Dream,40.7,17,190,3725,MALE
+Adelie,Dream,37.3,16.8,192,3000,FEMALE
+Adelie,Dream,39,18.7,185,3650,MALE
+Adelie,Dream,39.2,18.6,190,4250,MALE
+Adelie,Dream,36.6,18.4,184,3475,FEMALE
+Adelie,Dream,36,17.8,195,3450,FEMALE
+Adelie,Dream,37.8,18.1,193,3750,MALE
+Adelie,Dream,36,17.1,187,3700,FEMALE
+Adelie,Dream,41.5,18.5,201,4000,MALE
+Chinstrap,Dream,46.5,17.9,192,3500,FEMALE
+Chinstrap,Dream,50,19.5,196,3900,MALE
+Chinstrap,Dream,51.3,19.2,193,3650,MALE
+Chinstrap,Dream,45.4,18.7,188,3525,FEMALE
+Chinstrap,Dream,52.7,19.8,197,3725,MALE
+Chinstrap,Dream,45.2,17.8,198,3950,FEMALE
+Chinstrap,Dream,46.1,18.2,178,3250,FEMALE
+Chinstrap,Dream,51.3,18.2,197,3750,MALE
+Chinstrap,Dream,46,18.9,195,4150,FEMALE
+Chinstrap,Dream,51.3,19.9,198,3700,MALE
+Chinstrap,Dream,46.6,17.8,193,3800,FEMALE
+Chinstrap,Dream,51.7,20.3,194,3775,MALE
+Chinstrap,Dream,47,17.3,185,3700,FEMALE
+Chinstrap,Dream,52,18.1,201,4050,MALE
+Chinstrap,Dream,45.9,17.1,190,3575,FEMALE
+Chinstrap,Dream,50.5,19.6,201,4050,MALE
+Chinstrap,Dream,50.3,20,197,3300,MALE
+Chinstrap,Dream,58,17.8,181,3700,FEMALE
+Chinstrap,Dream,46.4,18.6,190,3450,FEMALE
+Chinstrap,Dream,49.2,18.2,195,4400,MALE
+Chinstrap,Dream,42.4,17.3,181,3600,FEMALE
+Chinstrap,Dream,48.5,17.5,191,3400,MALE
+Chinstrap,Dream,43.2,16.6,187,2900,FEMALE
+Chinstrap,Dream,50.6,19.4,193,3800,MALE
+Chinstrap,Dream,46.7,17.9,195,3300,FEMALE
+Chinstrap,Dream,52,19,197,4150,MALE
+Chinstrap,Dream,50.5,18.4,200,3400,FEMALE
+Chinstrap,Dream,49.5,19,200,3800,MALE
+Chinstrap,Dream,46.4,17.8,191,3700,FEMALE
+Chinstrap,Dream,52.8,20,205,4550,MALE
+Chinstrap,Dream,40.9,16.6,187,3200,FEMALE
+Chinstrap,Dream,54.2,20.8,201,4300,MALE
+Chinstrap,Dream,42.5,16.7,187,3350,FEMALE
+Chinstrap,Dream,51,18.8,203,4100,MALE
+Chinstrap,Dream,49.7,18.6,195,3600,MALE
+Chinstrap,Dream,47.5,16.8,199,3900,FEMALE
+Chinstrap,Dream,47.6,18.3,195,3850,FEMALE
+Chinstrap,Dream,52,20.7,210,4800,MALE
+Chinstrap,Dream,46.9,16.6,192,2700,FEMALE
+Chinstrap,Dream,53.5,19.9,205,4500,MALE
+Chinstrap,Dream,49,19.5,210,3950,MALE
+Chinstrap,Dream,46.2,17.5,187,3650,FEMALE
+Chinstrap,Dream,50.9,19.1,196,3550,MALE
+Chinstrap,Dream,45.5,17,196,3500,FEMALE
+Chinstrap,Dream,50.9,17.9,196,3675,FEMALE
+Chinstrap,Dream,50.8,18.5,201,4450,MALE
+Chinstrap,Dream,50.1,17.9,190,3400,FEMALE
+Chinstrap,Dream,49,19.6,212,4300,MALE
+Chinstrap,Dream,51.5,18.7,187,3250,MALE
+Chinstrap,Dream,49.8,17.3,198,3675,FEMALE
+Chinstrap,Dream,48.1,16.4,199,3325,FEMALE
+Chinstrap,Dream,51.4,19,201,3950,MALE
+Chinstrap,Dream,45.7,17.3,193,3600,FEMALE
+Chinstrap,Dream,50.7,19.7,203,4050,MALE
+Chinstrap,Dream,42.5,17.3,187,3350,FEMALE
+Chinstrap,Dream,52.2,18.8,197,3450,MALE
+Chinstrap,Dream,45.2,16.6,191,3250,FEMALE
+Chinstrap,Dream,49.3,19.9,203,4050,MALE
+Chinstrap,Dream,50.2,18.8,202,3800,MALE
+Chinstrap,Dream,45.6,19.4,194,3525,FEMALE
+Chinstrap,Dream,51.9,19.5,206,3950,MALE
+Chinstrap,Dream,46.8,16.5,189,3650,FEMALE
+Chinstrap,Dream,45.7,17,195,3650,FEMALE
+Chinstrap,Dream,55.8,19.8,207,4000,MALE
+Chinstrap,Dream,43.5,18.1,202,3400,FEMALE
+Chinstrap,Dream,49.6,18.2,193,3775,MALE
+Chinstrap,Dream,50.8,19,210,4100,MALE
+Chinstrap,Dream,50.2,18.7,198,3775,FEMALE
+Gentoo,Biscoe,46.1,13.2,211,4500,FEMALE
+Gentoo,Biscoe,50,16.3,230,5700,MALE
+Gentoo,Biscoe,48.7,14.1,210,4450,FEMALE
+Gentoo,Biscoe,50,15.2,218,5700,MALE
+Gentoo,Biscoe,47.6,14.5,215,5400,MALE
+Gentoo,Biscoe,46.5,13.5,210,4550,FEMALE
+Gentoo,Biscoe,45.4,14.6,211,4800,FEMALE
+Gentoo,Biscoe,46.7,15.3,219,5200,MALE
+Gentoo,Biscoe,43.3,13.4,209,4400,FEMALE
+Gentoo,Biscoe,46.8,15.4,215,5150,MALE
+Gentoo,Biscoe,40.9,13.7,214,4650,FEMALE
+Gentoo,Biscoe,49,16.1,216,5550,MALE
+Gentoo,Biscoe,45.5,13.7,214,4650,FEMALE
+Gentoo,Biscoe,48.4,14.6,213,5850,MALE
+Gentoo,Biscoe,45.8,14.6,210,4200,FEMALE
+Gentoo,Biscoe,49.3,15.7,217,5850,MALE
+Gentoo,Biscoe,42,13.5,210,4150,FEMALE
+Gentoo,Biscoe,49.2,15.2,221,6300,MALE
+Gentoo,Biscoe,46.2,14.5,209,4800,FEMALE
+Gentoo,Biscoe,48.7,15.1,222,5350,MALE
+Gentoo,Biscoe,50.2,14.3,218,5700,MALE
+Gentoo,Biscoe,45.1,14.5,215,5000,FEMALE
+Gentoo,Biscoe,46.5,14.5,213,4400,FEMALE
+Gentoo,Biscoe,46.3,15.8,215,5050,MALE
+Gentoo,Biscoe,42.9,13.1,215,5000,FEMALE
+Gentoo,Biscoe,46.1,15.1,215,5100,MALE
+Gentoo,Biscoe,44.5,14.3,216,4100,
+Gentoo,Biscoe,47.8,15,215,5650,MALE
+Gentoo,Biscoe,48.2,14.3,210,4600,FEMALE
+Gentoo,Biscoe,50,15.3,220,5550,MALE
+Gentoo,Biscoe,47.3,15.3,222,5250,MALE
+Gentoo,Biscoe,42.8,14.2,209,4700,FEMALE
+Gentoo,Biscoe,45.1,14.5,207,5050,FEMALE
+Gentoo,Biscoe,59.6,17,230,6050,MALE
+Gentoo,Biscoe,49.1,14.8,220,5150,FEMALE
+Gentoo,Biscoe,48.4,16.3,220,5400,MALE
+Gentoo,Biscoe,42.6,13.7,213,4950,FEMALE
+Gentoo,Biscoe,44.4,17.3,219,5250,MALE
+Gentoo,Biscoe,44,13.6,208,4350,FEMALE
+Gentoo,Biscoe,48.7,15.7,208,5350,MALE
+Gentoo,Biscoe,42.7,13.7,208,3950,FEMALE
+Gentoo,Biscoe,49.6,16,225,5700,MALE
+Gentoo,Biscoe,45.3,13.7,210,4300,FEMALE
+Gentoo,Biscoe,49.6,15,216,4750,MALE
+Gentoo,Biscoe,50.5,15.9,222,5550,MALE
+Gentoo,Biscoe,43.6,13.9,217,4900,FEMALE
+Gentoo,Biscoe,45.5,13.9,210,4200,FEMALE
+Gentoo,Biscoe,50.5,15.9,225,5400,MALE
+Gentoo,Biscoe,44.9,13.3,213,5100,FEMALE
+Gentoo,Biscoe,45.2,15.8,215,5300,MALE
+Gentoo,Biscoe,46.6,14.2,210,4850,FEMALE
+Gentoo,Biscoe,48.5,14.1,220,5300,MALE
+Gentoo,Biscoe,45.1,14.4,210,4400,FEMALE
+Gentoo,Biscoe,50.1,15,225,5000,MALE
+Gentoo,Biscoe,46.5,14.4,217,4900,FEMALE
+Gentoo,Biscoe,45,15.4,220,5050,MALE
+Gentoo,Biscoe,43.8,13.9,208,4300,FEMALE
+Gentoo,Biscoe,45.5,15,220,5000,MALE
+Gentoo,Biscoe,43.2,14.5,208,4450,FEMALE
+Gentoo,Biscoe,50.4,15.3,224,5550,MALE
+Gentoo,Biscoe,45.3,13.8,208,4200,FEMALE
+Gentoo,Biscoe,46.2,14.9,221,5300,MALE
+Gentoo,Biscoe,45.7,13.9,214,4400,FEMALE
+Gentoo,Biscoe,54.3,15.7,231,5650,MALE
+Gentoo,Biscoe,45.8,14.2,219,4700,FEMALE
+Gentoo,Biscoe,49.8,16.8,230,5700,MALE
+Gentoo,Biscoe,46.2,14.4,214,4650,
+Gentoo,Biscoe,49.5,16.2,229,5800,MALE
+Gentoo,Biscoe,43.5,14.2,220,4700,FEMALE
+Gentoo,Biscoe,50.7,15,223,5550,MALE
+Gentoo,Biscoe,47.7,15,216,4750,FEMALE
+Gentoo,Biscoe,46.4,15.6,221,5000,MALE
+Gentoo,Biscoe,48.2,15.6,221,5100,MALE
+Gentoo,Biscoe,46.5,14.8,217,5200,FEMALE
+Gentoo,Biscoe,46.4,15,216,4700,FEMALE
+Gentoo,Biscoe,48.6,16,230,5800,MALE
+Gentoo,Biscoe,47.5,14.2,209,4600,FEMALE
+Gentoo,Biscoe,51.1,16.3,220,6000,MALE
+Gentoo,Biscoe,45.2,13.8,215,4750,FEMALE
+Gentoo,Biscoe,45.2,16.4,223,5950,MALE
+Gentoo,Biscoe,49.1,14.5,212,4625,FEMALE
+Gentoo,Biscoe,52.5,15.6,221,5450,MALE
+Gentoo,Biscoe,47.4,14.6,212,4725,FEMALE
+Gentoo,Biscoe,50,15.9,224,5350,MALE
+Gentoo,Biscoe,44.9,13.8,212,4750,FEMALE
+Gentoo,Biscoe,50.8,17.3,228,5600,MALE
+Gentoo,Biscoe,43.4,14.4,218,4600,FEMALE
+Gentoo,Biscoe,51.3,14.2,218,5300,MALE
+Gentoo,Biscoe,47.5,14,212,4875,FEMALE
+Gentoo,Biscoe,52.1,17,230,5550,MALE
+Gentoo,Biscoe,47.5,15,218,4950,FEMALE
+Gentoo,Biscoe,52.2,17.1,228,5400,MALE
+Gentoo,Biscoe,45.5,14.5,212,4750,FEMALE
+Gentoo,Biscoe,49.5,16.1,224,5650,MALE
+Gentoo,Biscoe,44.5,14.7,214,4850,FEMALE
+Gentoo,Biscoe,50.8,15.7,226,5200,MALE
+Gentoo,Biscoe,49.4,15.8,216,4925,MALE
+Gentoo,Biscoe,46.9,14.6,222,4875,FEMALE
+Gentoo,Biscoe,48.4,14.4,203,4625,FEMALE
+Gentoo,Biscoe,51.1,16.5,225,5250,MALE
+Gentoo,Biscoe,48.5,15,219,4850,FEMALE
+Gentoo,Biscoe,55.9,17,228,5600,MALE
+Gentoo,Biscoe,47.2,15.5,215,4975,FEMALE
+Gentoo,Biscoe,49.1,15,228,5500,MALE
+Gentoo,Biscoe,47.3,13.8,216,4725,
+Gentoo,Biscoe,46.8,16.1,215,5500,MALE
+Gentoo,Biscoe,41.7,14.7,210,4700,FEMALE
+Gentoo,Biscoe,53.4,15.8,219,5500,MALE
+Gentoo,Biscoe,43.3,14,208,4575,FEMALE
+Gentoo,Biscoe,48.1,15.1,209,5500,MALE
+Gentoo,Biscoe,50.5,15.2,216,5000,FEMALE
+Gentoo,Biscoe,49.8,15.9,229,5950,MALE
+Gentoo,Biscoe,43.5,15.2,213,4650,FEMALE
+Gentoo,Biscoe,51.5,16.3,230,5500,MALE
+Gentoo,Biscoe,46.2,14.1,217,4375,FEMALE
+Gentoo,Biscoe,55.1,16,230,5850,MALE
+Gentoo,Biscoe,44.5,15.7,217,4875,
+Gentoo,Biscoe,48.8,16.2,222,6000,MALE
+Gentoo,Biscoe,47.2,13.7,214,4925,FEMALE
+Gentoo,Biscoe,,,,,
+Gentoo,Biscoe,46.8,14.3,215,4850,FEMALE
+Gentoo,Biscoe,50.4,15.7,222,5750,MALE
+Gentoo,Biscoe,45.2,14.8,212,5200,FEMALE
+Gentoo,Biscoe,49.9,16.1,213,5400,MALE

duckdb/01_getting_started.py

@@ -2,14 +2,13 @@
 # requires-python = ">=3.11"
 # dependencies = [
 #     "marimo",
-#     "duckdb==1.3.2",
-#     "polars==1.17.1",
-#     "numpy==2.2.4",
-#     "pyarrow==19.0.1",
-#     "pandas==2.2.3",
-#     "sqlglot==26.12.1",
-#     "plotly==5.24.1",
-#     "statsmodels==0.14.4",
+#     "duckdb==1.4.4",
+#     "numpy==2.4.3",
+#     "pandas==2.3.2",
+#     "plotly[express]==6.3.0",
+#     "polars[pyarrow]==1.24.0",
+#     "sqlglot==27.0.0",
+#     "statsmodels==0.14.5",
 # ]
 # ///
 
@@ -32,9 +31,7 @@ def _(mo):
 @app.cell(hide_code=True)
 def _(mo):
     mo.md(rf"""
-    # 🦆 **DuckDB**: An Embeddable Analytical Database System
-
-    ## What is DuckDB?
+    # What is DuckDB?
 
     [DuckDB](https://duckdb.org/) is a _high-performance_, in-process, embeddable SQL OLAP (Online Analytical Processing) Database Management System (DBMS) designed for simplicity and speed. It's essentially a fully-featured database that runs directly within your application's process, without needing a separate server. This makes it excellent for complex analytical workloads, offering a robust SQL interface and efficient processing – perfect for learning about databases and data analysis concepts.  It's a great alternative to heavier database systems like PostgreSQL or MySQL when you don't need a full-blown server.
 

duckdb/{008_loading_parquet.py → 08_loading_parquet.py}

@@ -2,9 +2,9 @@
 # requires-python = ">=3.10"
 # dependencies = [
 #     "marimo",
-#     "duckdb==1.3.2",
-#     "pyarrow==19.0.1",
-#     "plotly.express",
+#     "duckdb==1.4.4",
+#     "polars[pyarrow]==1.24.0",
+#     "plotly[express]==6.3.0",
 #     "sqlglot==27.0.0",
 # ]
 # ///

duckdb/{009_loading_json.py → 09_loading_json.py}

@@ -2,9 +2,9 @@
 # requires-python = ">=3.11"
 # dependencies = [
 #     "marimo",
-#     "duckdb==1.3.2",
-#     "sqlglot==26.11.1",
-#     "polars[pyarrow]==1.25.2",
+#     "duckdb==1.4.4",
+#     "polars[pyarrow]==1.24.0",
+#     "sqlglot==27.0.0",
 # ]
 # ///
 

duckdb/{011_working_with_apache_arrow.py → 11_working_with_apache_arrow.py}

@@ -2,13 +2,11 @@
 # requires-python = ">=3.11"
 # dependencies = [
 #     "marimo",
-#     "duckdb==1.3.2",
-#     "pyarrow==19.0.1",
-#     "polars[pyarrow]==1.25.2",
-#     "pandas==2.2.3",
+#     "altair==6.0.0",
+#     "duckdb==1.4.4",
+#     "pandas==2.3.2",
+#     "polars[pyarrow]==1.24.0",
 #     "sqlglot==27.0.0",
-#     "psutil==7.0.0",
-#     "altair",
 # ]
 # ///
 
@@ -534,15 +532,8 @@ def _(mo):
 
 
 @app.cell
-def _(polars_data, psutil, time):
-    import os
-    import pyarrow.compute as pc  # Add this import
-
-    # Get current process
-    process = psutil.Process(os.getpid())
-
-    # Measure memory before operations
-    memory_before = process.memory_info().rss / 1024 / 1024  # MB
+def _(mo, polars_data, time):
+    import pyarrow.compute as pc
 
     # Perform multiple Arrow-based operations (zero-copy)
     latest_start_time = time.time()
@@ -550,11 +541,9 @@ def _(polars_data, psutil, time):
     # These operations use Arrow's zero-copy capabilities
     arrow_table = polars_data.to_arrow()
     arrow_sliced = arrow_table.slice(0, 100000)
-    # Use PyArrow compute functions for filtering
     arrow_filtered = arrow_table.filter(pc.greater(arrow_table['value'], 500000))
 
     arrow_ops_time = time.time() - latest_start_time
-    memory_after_arrow = process.memory_info().rss / 1024 / 1024  # MB
 
     # Compare with traditional copy-based operations
     latest_start_time = time.time()
@@ -565,16 +554,21 @@ def _(polars_data, psutil, time):
     pandas_filtered = pandas_copy[pandas_copy['value'] > 500000].copy()
 
     copy_ops_time = time.time() - latest_start_time
-    memory_after_copy = process.memory_info().rss / 1024 / 1024  # MB
 
-    print("Memory Usage Comparison:")
-    print(f"Initial memory: {memory_before:.2f} MB")
-    print(f"After Arrow operations: {memory_after_arrow:.2f} MB (diff: +{memory_after_arrow - memory_before:.2f} MB)")
-    print(f"After copy operations: {memory_after_copy:.2f} MB (diff: +{memory_after_copy - memory_before:.2f} MB)")
-    print(f"\nTime comparison:")
-    print(f"Arrow operations: {arrow_ops_time:.3f} seconds")
-    print(f"Copy operations: {copy_ops_time:.3f} seconds")
-    print(f"Speedup: {copy_ops_time/arrow_ops_time:.1f}x")
+    mo.vstack([
+        mo.md(f"""
+**Time comparison:**
+
+| Method | Time (s) |
+|--------|----------|
+| Arrow operations | {arrow_ops_time:.3f} |
+| Copy operations | {copy_ops_time:.3f} |
+| Speedup | {copy_ops_time/arrow_ops_time:.1f}x |
+
+> **Note:** Memory usage statistics are not available in this environment.
+> Arrow's zero-copy design typically uses 20–40% less memory than Pandas copies.
+"""),
+    ])
     return
 
 
@@ -608,8 +602,7 @@ def _():
     import pandas as pd
     import duckdb
     import sqlglot
-    import psutil
-    return duckdb, mo, pa, pd, pl, psutil
+    return duckdb, mo, pa, pd, pl
 
 
 if __name__ == "__main__":

duckdb/DuckDB_Loading_CSVs.py

@@ -2,12 +2,11 @@
 # requires-python = ">=3.10"
 # dependencies = [
 #     "marimo",
-#     "plotly.express",
+#     "duckdb==1.4.4",
 #     "plotly==6.0.1",
-#     "duckdb==1.3.2",
-#     "sqlglot==26.11.1",
-#     "pyarrow==19.0.1",
 #     "polars==1.27.1",
+#     "pyarrow==19.0.1",
+#     "sqlglot==27.0.0",
 # ]
 # ///
 

duckdb/README.md

@@ -1,37 +0,0 @@
----
-title: Readme
-marimo-version: 0.18.4
----
-
-# Learn DuckDB
-
-_🚧 This collection is a work in progress. Please help us add notebooks!_
-
-This collection of marimo notebooks is designed to teach you the basics of
-DuckDB, a fast in-memory OLAP engine that can interoperate with Dataframes.
-These notebooks also show how marimo gives DuckDB superpowers.
-
-**Help us build this course! ⚒️**
-
-We're seeking contributors to help us build these notebooks. Every contributor
-will be acknowledged as an author in this README and in their contributed
-notebooks. Head over to the [tracking
-issue](https://github.com/marimo-team/learn/issues/48) to sign up for a planned
-notebook or propose your own.
-
-**Running notebooks.** To run a notebook locally, use
-
-```bash
-uvx marimo edit <file_url>
-```
-
-You can also open notebooks in our online playground by appending marimo.app/ to a notebook's URL.
-
-
-**Authors.**
-
-Thanks to all our notebook authors!
-
-* [Mustjaab](https://github.com/Mustjaab)
-* [julius383](https://github.com/julius383)
-* [thliang01](https://github.com/thliang01)

duckdb/index.md

@@ -0,0 +1,16 @@
+---
+title: Learn DuckDB
+description: >
+    These notebooks teach you the basics of DuckDB,
+    a fast in-memory database engine that can interoperate
+    with dataframes, and show how marimo gives DuckDB superpowers.
+tracking: 48
+---
+
+## Contributors
+
+Thanks to our notebook authors:
+
+* [Mustjaab](https://github.com/Mustjaab)
+* [julius383](https://github.com/julius383)
+* [thliang01](https://github.com/thliang01)

{functional_programming → functional}/05_functors.py

@@ -875,7 +875,7 @@ def _(mo):
 
 @app.cell(hide_code=True)
 def _(mo):
-    mo.md("""
+    mo.md(r"""
     ## Functor laws, again
 
     Once again there are a few axioms that functors have to obey.

{functional_programming → functional}/06_applicatives.py

@@ -14,7 +14,7 @@ app = marimo.App(app_title="Applicative programming with effects")
 @app.cell(hide_code=True)
 def _(mo):
     mo.md(r"""
-    # Applicative programming with effects
+    # Applicative Programming with Effects
 
     `Applicative Functor` encapsulates certain sorts of *effectful* computations in a functionally pure way, and encourages an *applicative* programming style.
 

functional/_index.md

@@ -0,0 +1,25 @@
+---
+title: Learn Functional Programming
+description: >
+    These notebooks introduce powerful ideas from functional programming
+    in Python, taking inspiration from Haskell and category theory.
+tracking: 51
+---
+
+Using only Python's standard library, these lessons construct
+functional programming concepts from first principles.
+Topics include:
+
+-   Currying and higher-order functions
+-   Functors, Applicatives, and Monads
+-   Category theory fundamentals
+
+## Contributors
+
+Thanks to our notebook authors:
+
+- métaboulie
+
+and reviewers:
+
+- [Srihari Thyagarajan](https://github.com/Haleshot)

functional_programming/CHANGELOG.md

@@ -1,129 +0,0 @@
----
-title: Changelog
-marimo-version: 0.18.4
----
-
-# Changelog of the functional-programming course
-
-## 2025-04-16
-
-**applicatives.py**
-
-- replace `return NotImplementedError` with `raise NotImplementedError`
-
-- add `Either` applicative
-- Add `Alternative`
-
-## 2025-04-11
-
-**functors.py**
-
-- add `Bifunctor` section
-
-- replace `return NotImplementedError` with `raise NotImplementedError`
-
-## 2025-04-08
-
-**functors.py**
-
-- restructure the notebook
-- replace `f` in the function signatures with `g` to indicate regular functions and
-  distinguish from functors
-- move `Maybe` funtor to section `More Functor instances`
-
-- add `Either` functor
-
-- add `unzip` utility function for functors
-
-## 2025-04-07
-
-**applicatives.py**
-
-- the `apply` method of `Maybe` _Applicative_ should return `None` when `fg` or `fa` is
-  `None`
-
-- add `sequenceL` as a classmethod for `Applicative` and add examples for `Wrapper`,
-  `Maybe`, `List`
-- add description for utility functions of `Applicative`
-
-- refine the implementation of `IO` _Applicative_
-- reimplement `get_chars` with `IO.sequenceL`
-
-- add an example to show that `ListMonoidal` is equivalent to `List` _Applicative_
-
-## 2025-04-06
-
-**applicatives.py**
-
-- remove `sequenceL` from `Applicative` because it should be a classmethod but can't be
-  generically implemented
-
-## 2025-04-02
-
-**functors.py**
-
-- Migrate to `python3.13`
-
-    - Replace all occurrences of
-
-        ```python
-        class Functor(Generic[A])
-        ```
-
-        with
-
-        ```python
-        class Functor[A]
-        ```
-
-        for conciseness
-
-- Use `fa` in function signatures instead of `a` when `fa` is a _Functor_
-
-**applicatives.py**
-
-- `0.1.0` version of notebook `06_applicatives.py`
-
-## 2025-03-16
-
-**functors.py**
-
-- Use uppercased letters for `Generic` types, e.g. `A = TypeVar("A")`
-- Refactor the `Functor` class, changing `fmap` and utility methods to `classmethod`
-
-    For example:
-
-    ```python
-    @dataclass
-    class Wrapper(Functor, Generic[A]):
-        value: A
-
-        @classmethod
-        def fmap(cls, f: Callable[[A], B], a: "Wrapper[A]") -> "Wrapper[B]":
-            return Wrapper(f(a.value))
-
-    >>> Wrapper.fmap(lambda x: x + 1, wrapper)
-    Wrapper(value=2)
-    ```
-
-- Move the `check_functor_law` method from `Functor` class to a standard function
-
-- Rename `ListWrapper` to `List` for simplicity
-- Remove the `Just` class
-
-- Rewrite proofs
-
-## 2025-03-13
-
-**functors.py**
-
-- `0.1.0` version of notebook `05_functors`
-
-Thank [Akshay](https://github.com/akshayka) and [Haleshot](https://github.com/Haleshot)
-for reviewing
-
-## 2025-03-11
-
-**functors.py**
-
-- Demo version of notebook `05_functors.py`

functional_programming/README.md

@@ -1,77 +0,0 @@
----
-title: Readme
-marimo-version: 0.18.4
----
-
-# Learn Functional Programming
-
-_🚧 This collection is a [work in progress](https://github.com/marimo-team/learn/issues/51)._
-
-This series of marimo notebooks introduces the powerful paradigm of functional
-programming through Python. Taking inspiration from Haskell and Category
-Theory, we'll build a strong foundation in FP concepts that can transform how
-you approach software development.
-
-## What You'll Learn
-
-**Using only Python's standard library**, we'll construct functional
-programming concepts from first principles.
-
-Topics include:
-
-+ Currying and higher-order functions
-+ Functors, Applicatives, and Monads
-+ Category theory fundamentals
-
-## Running Notebooks
-
-### Locally
-
-To run a notebook locally, use
-
-```bash
-uvx marimo edit <URL>
-```
-
-For example, run the `Functor` tutorial with
-
-```bash
-uvx marimo edit https://github.com/marimo-team/learn/blob/main/functional_programming/05_functors.py
-```
-
-### On Our Online Playground
-
-You can also open notebooks in our online playground by appending `marimo.app/` to a notebook's URL like:
-
-https://marimo.app/https://github.com/marimo-team/learn/blob/main/functional_programming/05_functors.py
-
-### On Our Landing Page
-
-Open the notebooks in our landing page page [here](https://marimo-team.github.io/learn/functional_programming/05_functors.html)
-
-## Collaboration
-
-If you're interested in collaborating or have questions, please reach out to me
-on Discord (@eugene.hs).
-
-## Description of notebooks
-
-Check [here](https://github.com/marimo-team/learn/issues/51) for current series
-structure.
-
-| Notebook | Title | Key Concepts | Prerequisites |
-|----------|-------|--------------|---------------|
-| [05. Functors](https://github.com/marimo-team/learn/blob/main/functional_programming/05_functors.py) | Category Theory and Functors | Category Theory, Functor, fmap, Bifunctor | Basic Python, Functions |
-| [06. Applicatives](https://github.com/marimo-team/learn/blob/main/functional_programming/06_applicatives.py) | Applicative programming with effects | Applicative Functor, pure, apply, Effectful programming, Alternative | Functors |
-
-**Authors.**
-
-Thanks to all our notebook authors!
-
-- [métaboulie](https://github.com/metaboulie)
-
-**Reviewers.**
-
-Thanks to all our notebook reviews!
-
-- [Haleshot](https://github.com/Haleshot)

optimization/01_least_squares.py

@@ -1,9 +1,9 @@
 # /// script
 # requires-python = ">=3.11"
 # dependencies = [
-#     "cvxpy==1.6.0",
+#     "cvxpy-base",
 #     "marimo",
-#     "numpy==2.2.2",
+#     "numpy==2.4.3",
 # ]
 # ///
 
@@ -22,7 +22,7 @@ def _():
 @app.cell(hide_code=True)
 def _(mo):
     mo.md(r"""
-    # Least squares
+    # Least Squares
 
     In a least-squares problem, we have measurements $A \in \mathcal{R}^{m \times
     n}$ (i.e., $m$ rows and $n$ columns) and $b \in \mathcal{R}^m$. We seek a vector

optimization/02_linear_program.py

@@ -1,11 +1,11 @@
 # /// script
 # requires-python = ">=3.13"
 # dependencies = [
-#     "cvxpy==1.6.0",
+#     "cvxpy-base",
 #     "marimo",
-#     "matplotlib==3.10.0",
-#     "numpy==2.2.2",
-#     "wigglystuff==0.1.9",
+#     "matplotlib==3.10.8",
+#     "numpy==2.4.3",
+#     "wigglystuff==0.2.37",
 # ]
 # ///
 
@@ -24,7 +24,7 @@ def _():
 @app.cell(hide_code=True)
 def _(mo):
     mo.md(r"""
-    # Linear program
+    # Linear Program
 
     A linear program is an optimization problem with a linear objective and affine
     inequality constraints. A common standard form is the following:

optimization/03_minimum_fuel_optimal_control.py

@@ -1,7 +1,11 @@
 # /// script
 # requires-python = ">=3.13"
 # dependencies = [
+#     "cvxpy-base",
 #     "marimo",
+#     "matplotlib==3.10.8",
+#     "numpy==2.4.3",
+#     "wigglystuff==0.2.37",
 # ]
 # ///
 import marimo
@@ -19,7 +23,7 @@ def _():
 @app.cell(hide_code=True)
 def _(mo):
     mo.md(r"""
-    # Minimal fuel optimal control
+    # Minimal Fuel Optimal Control
 
     This notebook includes an application of linear programming to controlling a
     physical system, adapted from [Convex
@@ -128,14 +132,14 @@ def _():
 
 
 @app.cell
-def _(A, T, b, cp, mo, n, x0, xdes):
+def _(A, T, b, cp, mo, n, np, x0, xdes):
     X, u = cp.Variable(shape=(n, T + 1)), cp.Variable(shape=(1, T))
 
     objective = cp.sum(cp.maximum(cp.abs(u), 2 * cp.abs(u) - 1))
     constraints = [
         X[:, 1:] == A @ X[:, :-1] + b @ u,
-        X[:, 0] == x0,
-        X[:, -1] == xdes,
+        X[:, 0] == np.array(x0).flatten(),
+        X[:, -1] == np.array(xdes).flatten(),
     ]
 
     fuel_used = cp.Problem(cp.Minimize(objective), constraints).solve()

optimization/04_quadratic_program.py

@@ -1,11 +1,11 @@
 # /// script
 # requires-python = ">=3.13"
 # dependencies = [
-#     "cvxpy==1.6.0",
+#     "cvxpy-base",
 #     "marimo",
-#     "matplotlib==3.10.0",
-#     "numpy==2.2.2",
-#     "wigglystuff==0.1.9",
+#     "matplotlib==3.10.8",
+#     "numpy==2.4.3",
+#     "wigglystuff==0.2.37",
 # ]
 # ///
 
@@ -24,7 +24,7 @@ def _():
 @app.cell(hide_code=True)
 def _(mo):
     mo.md(r"""
-    # Quadratic program
+    # Quadratic Program
 
     A quadratic program is an optimization problem with a quadratic objective and
     affine equality and inequality constraints. A common standard form is the

optimization/05_portfolio_optimization.py

@@ -1,12 +1,12 @@
 # /// script
 # requires-python = ">=3.13"
 # dependencies = [
-#     "cvxpy==1.6.0",
+#     "cvxpy-base",
 #     "marimo",
-#     "matplotlib==3.10.0",
-#     "numpy==2.2.2",
-#     "scipy==1.15.1",
-#     "wigglystuff==0.1.9",
+#     "matplotlib==3.10.8",
+#     "numpy==2.4.3",
+#     "scipy==1.17.1",
+#     "wigglystuff==0.2.37",
 # ]
 # ///
 
@@ -25,7 +25,7 @@ def _():
 @app.cell(hide_code=True)
 def _(mo):
     mo.md(r"""
-    # Portfolio optimization
+    # Portfolio Optimization
     """)
     return
 
@@ -145,7 +145,7 @@ def _(mo, np):
 def _(mu_widget, np):
     np.random.seed(1)
     n = 10
-    mu = np.array(mu_widget.matrix)
+    mu = np.array(mu_widget.matrix).flatten()
     Sigma = np.random.randn(n, n)
     Sigma = Sigma.T.dot(Sigma)
     return Sigma, mu, n
@@ -153,7 +153,7 @@ def _(mu_widget, np):
 
 @app.cell(hide_code=True)
 def _(mo):
-    mo.md("""
+    mo.md(r"""
     Next, we solve the problem for 100 different values of $\gamma$
     """)
     return
@@ -187,7 +187,7 @@ def _(cp, gamma, np, prob, ret, risk):
 
 @app.cell(hide_code=True)
 def _(mo):
-    mo.md("""
+    mo.md(r"""
     Plotted below are the risk return tradeoffs for two values of $\gamma$ (blue squares), and the risk return tradeoffs for investing fully in each asset (red circles)
     """)
     return

optimization/06_convex_optimization.py

@@ -1,9 +1,9 @@
 # /// script
 # requires-python = ">=3.13"
 # dependencies = [
-#     "cvxpy==1.6.0",
+#     "cvxpy-base",
 #     "marimo",
-#     "numpy==2.2.2",
+#     "numpy==2.4.3",
 # ]
 # ///
 
@@ -22,7 +22,7 @@ def _():
 @app.cell(hide_code=True)
 def _(mo):
     mo.md(r"""
-    # Convex optimization
+    # Convex Optimization
 
     In the previous tutorials, we learned about least squares, linear programming,
     and quadratic programming, and saw applications of each. We also learned that these problem