Skip to content
openteams-aiPublic

Latest commit

 

History

131 Commits

Folders and files

Repository files navigation

Whybook

Github Actions Status Sandbox PyPI Binder

A research prototype that explores how to close the understanding gap in data analysis: the divide between what AI produces and what the analyst understands of it. Try the interactive demos on whybook.dev, and read why the gap matters.

In Whybook, the column IL6 of olink is dragged onto the mixed model of cell [5]. Of the questions that Whybook offers, "Add IL6 as a covariate" is picked; Whybook edits the cell and runs it, and its code change shows as a diff. Click to open the demo on Binder.

You ask the questions. Agents write the code.

Whybook is a JupyterLab extension: a view of a Jupyter notebook for asking questions about data and understanding every result. It has no chat box. The analyst picks one of the questions it offers, or types or says their own. The offers follow what the analyst points at (a column dragged onto a cell or clicked in Click mode, a region of a plot, a chip on a cell). The code comes from a template or from the AI model of the analyst's choice, and Whybook places the cell and runs it. Every constant, default and choice that Whybook finds behind the result is shown on the cell, where the analyst can question it.

Any .ipynb file opens in Whybook and stays an ordinary notebook: the questions, choices and assumptions go into its metadata. Whybook opens next to JupyterLab's notebook editor, and both share one notebook model and one kernel, so an edit in one shows in the other.

Three videos show Whybook at work:

whybook.dev also gives the evidence behind the idea, and the roadmap sets out the path to v1.

Whybook showing the demo notebook in JupyterLab after Run all. Cell [4] carries the chips MIN_DAYS 14, inner join, ci bootstrap and n_boot 1000, and the Exploration panel counts the questions asked by type.

What it does

Note

Whybook's features, settings and notebook metadata can change without deprecation while it remains in the prototype phase.

  • Ask by pointing at the data. The analyst drags a variable, a column, a file or a table of a SQLite file onto a cell, or clicks the two in Click mode. Whybook then offers up to 12 ranked questions of five types (descriptive, data quality, association, causal and model check), each with a place in the notebook. Most of them run code from a template at once, with no model call.
  • Every choice in plain sight. After a cell runs, Whybook reads its code against the live kernel and shows each value that it finds the result depends on as a chip, such as MIN_DAYS 14, or inner join ×2 for two merges that keep pandas' default. A constant defined in an imported file shows on the cell that uses it.
  • Question a plot. Every mark of a plot drawn with the whybook helpers maps to rows of its data frame, and so does a box drawn on a Plotly scatter chart, or on a matplotlib figure whose axes hold numbers and whose x label names a column. Whybook counts the rows behind a selection, reports what differs there and offers questions about it. A question about any other picture sends the picture to the AI model.
  • Branch in parallel. Shift+drop turns a question into a branch, and Alt+drop starts several branches at once. Each branch runs in its own kernel subshell (JEP 91, ipykernel 7), on the same data, and the notebook stays one list of cells.
  • Answers that need AI. For a question that needs a new cell, an agent adds and runs the cells it needs, and ends with a short answer that links the cells that show it. An edit, a branch or a preview gets one cell. Every cell that a model wrote is marked, and one click removes the cells of a run.
  • Python and R. In an R kernel, Whybook lists the variables, shows the chips and offers the questions, and a model writes the code of each answer in R. The templates write Python only.
  • A kernel in a sandbox. "Python 3 (sandboxed)" sees the notebook's folder and no network, under bubblewrap on Linux and Seatbelt on macOS. python -m whybook.sandbox <kernel name> makes a sandboxed copy of any other kernel, such as R.

A selection of weeks 14 to 24 on the plot of cell [4]. Whybook counts 2,050 rows and 262 patients behind it, reports that A minus B is +1.86 there and +1.33 elsewhere, and offers five questions, each with its place in the notebook.

The data in the screenshots is synthetic: the demo cohort of examples/pain_diary.

Install

Whybook needs Python 3.11 or later, and JupyterLab 4.6.6 or a later 4.x release.

pip install "whybook[kernel,demo,models]"
jupyter lab

To run small AI models in the Jupyter server too, add local: pip builds llama-cpp-python with the system's C++ compiler, in about 4 minutes on 8 virtual CPUs.

pip install "whybook[kernel,demo,local,models]"
jupyter lab

Add speech for spoken questions transcribed by Moonshine in the Jupyter server, on Linux x86_64 or macOS 15 on Apple silicon. On Linux, speech needs glibc 2.34 or later, as in Ubuntu 22.04. The setting Spoken questions in the AI models panel is off by default: choose Moonshine Medium or Small there. The first click on the microphone of a "Your own question" box then downloads the model.

Whybook reads the data in the kernel. When a kernel runs in another environment, install whybook[kernel] there too, and demo for the demo. kernel installs narwhals, and without it the Variables panel shows data frames without their columns.

To open a notebook in Whybook, right-click it in the file browser and choose Open With, then Whybook. With a notebook open in JupyterLab's notebook editor, "Open Notebook in Whybook" in the command palette does the same. A new notebook starts from the Whybook section of the Launcher, or from File, New, Whybook. A double-click still opens JupyterLab's notebook editor. To open notebooks in Whybook by default, set Default Viewers of the Document Manager to {"notebook": "Whybook"} in the Settings Editor. The sections of the Whybook panel can also move to the file browser.

Whybook also runs in Jupyter Notebook 7: on a notebook's page, the switch Notebook | Whybook in the menu bar shows Whybook in place of the notebook editor. Whybook has some limitations in Notebook 7 that it does not have in JupyterLab. They are bugs and improvements, tracked in issues of their own, often in upstream projects.

Try the demo

Open the demo on Binder: it runs in the browser and needs no install and no AI model.

To run it on this machine, install Whybook as above, then write the demo into a folder and open its notebook:

python -m whybook.demo pain_diary
jupyter lab pain_diary/pain_diary_cohort.ipynb

In JupyterLab, choose "Open Notebook in Whybook" in the command palette, then press Run all. python -m whybook.demo writes three files of the installed release: the notebook, prep.py and clinic.sqlite. The notebook imports its helpers and data from prep.py, and clinic.sqlite holds the clinic's records as a SQLite file. In a folder that already has one of these files, the command writes none of them. The same analysis after six more hours of work is examples/pain_diary/pain_diary_cohort_6h.ipynb on GitHub.

The questions from templates work with no AI model. Questions marked needs AI, and questions that the analyst types, need a model connected in the AI models panel: OpenRouter after a sign-in, a Hugging Face access token, an API key of Anthropic, OpenAI, Google or Mistral AI, or a model server on this machine.

What leaves the machine

Nothing leaves the machine until a remote AI model is connected. With c.Whybook.way_to_use_models = "all-local", Whybook refuses every model outside the machine and its network, so no prompt leaves them. With the setting "Keep data on this machine", only models on this machine read the data, and no tables, printed outputs or pictures leave it; c.Whybook.keep_data_local = True enforces the setting for every user. Otherwise a connected remote model receives the code of the cells and what each task needs, such as the tables shown as tiles, to label them. Every use of AI in Whybook lists what each task sends.

Report a problem or contribute

Report a problem or ask a question in the issues. CONTRIBUTING.md explains how to report a security problem privately, set up a checkout and run the tests.

Uninstall

pip uninstall whybook

pip leaves Whybook's folder in Jupyter's data folder, with the keys and sign-ins of the AI models panel in keys.json. To remove them, delete the whybook folder in the folder that jupyter --data-dir prints. The models that Whybook downloaded stay in the caches of Hugging Face and moonshine-voice, and in ~/.cache/whybook. A kernel copied with python -m whybook.sandbox stays until jupyter kernelspec remove <name>. Whybook also installs jupyterlab-unfold, and the file browser stays a tree until pip uninstall jupyterlab-unfold.

Troubleshoot

  • pip fails while it builds llama-cpp-python: install a C++ compiler, or leave local out of the list.
  • The Variables panel shows data frames without their columns: install whybook[kernel] in the kernel's environment.
  • Questions marked needs AI are greyed: connect a model in the AI models panel. The item of the status bar that starts with "AI:" opens it.
  • Python 3 (sandboxed) does not start: on Linux it needs bubblewrap, and on Ubuntu 24.04 also an AppArmor profile for bubblewrap. The "Error Starting Kernel" dialog gives the commands.

If you are seeing the frontend extension, but it is not working, check that the server extension is enabled:

jupyter server extension list

If the server extension is installed and enabled, but you are not seeing the frontend extension, check the frontend extension is installed:

jupyter labextension list

License

Whybook is under the Apache License 2.0: see LICENSE. The Seatbelt profiles in whybook/sandbox/codex/ come from OpenAI Codex, under the same licence, with its LICENSE and NOTICE.

Releases

Used by

Contributors

Languages