1 of 33

The new Jupyter Book stack�a more powerful, modular, re-usable toolchain

Chris Holdgraf, 2i2c and UC Berkeley

�

2 of 33

bit.ly/2020-jc-jupyter-book-stack

�

3 of 33

The Executable Book Project

executablebooks.org

IAB

readiab.org/

QuantEcon

quantecon.org/

ipypublish

ipypublish.readthedocs.io/

�

4 of 33

What is Jupyter Book?

Build an online book with

Jupyter Notebooks and Markdown

​

​

jupyterbook.org

�

5 of 33

To learn more about Jupyter Book, see jupyterbook.org

​

This talk is about the tech stack underneath the re-write

6 of 33

  • In January 2020, we began an entire re-write of Jupyter Book
  • Our goals:
    • Make it more modular
    • Make it more extensible
    • Make it support more publishing features
    • Make easier to install and to use
    • Integrate it with pre-existing open source ecosystems

The Jupyter Book re-write

�

7 of 33

nbconvert

Old Jupyter Book engine

New Jupyter Book engine

�

8 of 33

  • More modular. Each component stands alone and can be used outside of Jupyter Book
  • Extensible. Sphinx is highly extensible with many pre-existing extensions.
  • Streamlined. No more installation of Jekyll (!), Jupyter Book is pure Python
  • More powerful. Sphinx is a documentation engine that brings more features needed for publishing

The new Jupyter Book stack is...

�

9 of 33

an overview of the stack

10 of 33

6. Customization

sphinx book theme

sphinx book latex

4. build output

  • PDF Builder
  • HTML Builder

.ipynb

.md

Partial Sphinx

AST

HTML

Latex

PDF

Website

Full Sphinx AST

execute

parse

  1. author

2. cache and execute

Jupyter Cache

3. parse to sphinx

MyST NB

MyST Parser

convert

insert

outputs

7. controller process

Jupyter Book

jupytext

ipynb+assets

build

5. Sphinx Extensions

sphinx-panels

sphinx-comments

sphinx-thebe

etc

An overview of the Jupyter Book build process

�

11 of 33

parsing and writing markdown in Sphinx

12 of 33

Brings the features of reStructuredText to Markdown

Is a superset of CommonMark -> all pre-existing Jupyter markdown is valid MyST Markdown

Has extended syntax for extra functionality for publishing and computational documents

MyST Markdown

�

13 of 33

(some) extended MyST syntax

More on this later

�

14 of 33

A Python implementation of the popular markdown-it JS package

Configurable syntax: you can add new rules and even replace existing ones.

Pluggable: Adds syntax extensions to extend the parser (see the plugin list)

markdown-it-py

markdown-it-py.readthedocs.io

�

15 of 33

Example of markdown-it-py

�

16 of 33

myst-parser.readthedocs.io

A (CommonMark / MyST) Markdown parser for Sphinx

Translates MyST roles and directives into Sphinx

Uses markdown-it-py for parsing

myst-parser

conf.py:

​

extensions = [

…

“myst_parser”,

…

]

�

17 of 33

```{directivename} args

:key: val

Some content

```

​

​

Roles and directives�kind of like functions, but in markdown

{rolename}`interpreted text`

​

​

​

​

�

18 of 33

```{directivename} args

:key: val

Some content

```

​

```{figure} myfig.png

:caption: my caption

:label: my_label

:name: Figure 1

```

Roles and directives�kind of like functions, but in markdown

{rolename}`interpreted text`

​

​

​

​

{ref}`my_label`

​

​

�

19 of 33

working with�notebooks in Sphinx

20 of 33

Parse ipynb files in Sphinx. Directly convert Jupyter Notebooks into Sphinx documents.

Execute and Cache your notebook content. Save time building your documentation without needing to commit your notebook outputs directly into git.

Write MyST Markdown. MyST Markdown allows you to write Sphinx roles and directives in markdown.

Insert notebook outputs into your content. Generate outputs as you build your documentation, and insert them across pages.

Write Jupyter Notebooks entirely with Markdown. Write notebooks in pure-text and convert with Jupytext at build-time.

  • ​

myst-nb

myst-nb.readthedocs.io

�

21 of 33

Examples of myst-markdown in notebook

�

22 of 33

Markdown notebooks with

�

23 of 33

An API and CLI for staging, executing, and cacheing Jupyter Notebooks.

Be smart about when to re-execute (code cell changes, not markdown)

Parallel access to notebooks (for execution)

Execution statistics/reports

jupyter-cache

jupyter-cache.readthedocs.io

�

24 of 33

�

25 of 33

extending sphinx

design and features

26 of 33

Bootstrap 4 for visual elements and functionality.

Flexible content layout that is inspired by beautiful online books, such as the Edward Tufte CSS guide

Visual classes designed for Jupyter Notebooks. Cell inputs, outputs, and interactive functionality are all supported.

Launch buttons for online interactivity. For pages that are built with computational material, connect your site to an online BinderHub for interactive content.

​

sphinx-book-theme

sphinx-book-theme.readthedocs.io

�

27 of 33

�

28 of 33

sphinx-thebe

sphinx-thebe.readthedocs.io

�

29 of 33

sphinx-panels

sphinx-panels.readthedocs.io/

�

30 of 33

sphinx-comments

sphinx-comments.readthedocs.io

�

31 of 33

Jupyter Book ties�it all together

32 of 33

6. Customization

sphinx book theme

sphinx book latex

4. build output

  • PDF Builder
  • HTML Builder

.ipynb

.md

Partial Sphinx

AST

HTML

Latex

PDF

Website

Full Sphinx AST

execute

parse

  • author

2. cache and execute

Jupyter Cache

3. parse to sphinx

MyST NB

MyST Parser

convert

insert

outputs

jupytext

ipynb+assets

build

5. Sphinx Extensions

sphinx-panels

sphinx-comments

sphinx-thebe

etc

Jupyter Book is a light wrapper and CLI around these tools

�

33 of 33

Try Jupyter Book

jupyterbook.org

​

​

Follow the project

executablebooks.org

github.com/executablebooks

​

Connect!�discuss.executablebooks.org

@choldgraf

​

�