Skip to content

Allow null execution_count in execute_result. [API change] - #4

Open
JayOfTheKeyboard wants to merge 1 commit into
jgm:masterfrom
JayOfTheKeyboard:execute-result-null-count
Open

JayOfTheKeyboard wants to merge 1 commit into
jgm:masterfrom
JayOfTheKeyboard:execute-result-null-count

Conversation

@JayOfTheKeyboard

Copy link
Copy Markdown

See jgm/pandoc#12008.

The nbformat v4 schema allows null for an execute_result's execution_count, as it does for a code cell's ("type": ["integer", "null"] in every schema from nbformat.v4.0 to v4.5). executeCount is an Int, so a notebook with one of these fails to decode as v4, and pandoc then reports the error from its v3 attempt:

% pandoc -f ipynb -t markdown null-ec.ipynb
ipynb decoding error: Error in $: expected nbformat <= 3
% echo $?
93

with this input:

{"nbformat": 4, "nbformat_minor": 5, "metadata": {},
 "cells": [{"cell_type": "code", "id": "a1", "execution_count": null, "metadata": {}, "source": ["1+1"],
   "outputs": [{"output_type": "execute_result", "execution_count": null, "metadata": {}, "data": {"text/plain": ["2"]}}]}]}

That was pandoc 3.13. The file passes nbformat.validate (nbformat 5.11.1). Four of the Ray project's Lightning and Tune example notebooks have such outputs (for example doc/source/tune/examples/tune-pytorch-lightning.ipynb), and all four validate.

executeCount is now Maybe Int, like codeExecutionCount. In v4, null reads as Nothing and is written back as null, so the key the schema requires stays in place. In v3, a pyout's prompt_number now works the way a code cell's does: it is read with .:? and left out when there is no count. Writing null there would break the v3 schema, which wants an integer, and the reader can still read what the writer produces.

Because this changes an exported type, pandoc needs two edits. The writer stops compiling, which makes that one easy to spot. The reader still compiles but would quietly produce execution_count="Just 3":

  • Readers/Ipynb.hs, outputToBlock: replace [("execution_count",tshow ec)] with maybe [] (\n -> [("execution_count", tshow n)]) ec, as the code-cell case already does.
  • Writers/Ipynb.hs, blockToOutput: drop the fromMaybe 0, so an execute_result div with no count is written as null rather than 0. None of the files in test/ipynb has an execute_result without a count.

I didn't touch the version or Changelog.md, since you have always updated those yourself at release.

Test: test/rt-files/test4execute_result_null.ipynb, a one-cell v4.5 notebook, goes through the existing round trip. The round trip compares decoded values, so it can't see whether null survives in the JSON. I added two small cases to test/roundtrip.hs for that: one checks that encoding the new notebook gives "execution_count": null on the output, and one checks that a v3 pyout with "prompt_number": null is written without prompt_number and reads back the same. Without the change all three fail with parsing Int failed, expected Number, but encountered Null. With it, all 8 tests pass (GHC 9.6.7, aeson 2.3.2.0), and -Wall reports only the existing unused x at line 241. I also broke the change four ways (v4 writes 0, v4 leaves out the key, v3 writes null, v3 reads with .:), and each one makes a test fail. The four Ray notebooks decode and round-trip with the change and fail without it.

Prepared with Claude Code.

The nbformat v4 schema gives an execute_result's execution_count the
type ["integer", "null"], the same as a code cell's. Such notebooks
failed to decode as v4. executeCount is now Maybe Int, like
codeExecutionCount, and null is written back as null.

For v3, a pyout's prompt_number is now handled like a code cell's:
it is read with .:? and left out when there is no count.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant