More Like this

1. A Framework to capture and reproduce the Absolute State of Jupyter Notebooks

   https://doi.org/10.1145/3491418.3530296  

   Wannipurage, Dimuthu; Marru, Suresh; Pierce, Marlon (July 2022, PEARC '22: Practice and Experience in Advanced Research Computing)

   Jupyter Notebooks are an enormously popular tool for creating and narrating computational research projects. They also have enormous potential for creating reproducible scientific research artifacts. Capturing the complete state of a notebook has additional benefits; for instance, the notebook execution may be split between local and remote resources, where the latter may have more powerful processing capabilities or store large or access-limited data. There are several challenges for making notebooks fully reproducible when examined in detail. The notebook code must be replicated entirely, and the underlying Python runtime environments must be identical. More subtle problems arise in replicating referenced data, external library dependencies, and runtime variable states. This paper presents solutions to these problems using Juptyer’s standard extension mechanisms to create an archivable system state for a running notebook. We show that the overhead for these additional mechanisms, which involve interacting with the underlying Linux kernel, does not introduce substantial execution time overheads, demonstrating the approach’s feasibility. 

   more » « less
2. Elevating Jupyter Notebook Maintenance Tooling by Identifying and Extracting Notebook Structures

   https://doi.org/10.1109/ICSME55016.2022.00047  

   Jiang, Yuan; Kastner, Christian; Zhou, Shurui (October 2022, 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME))

   Data analysis is an exploratory, interactive, and often collaborative process. Computational notebooks have become a popular tool to support this process, among others because of their ability to interleave code, narrative text, and results. However, notebooks in practice are often criticized as hard to maintain and being of low code quality, including problems such as unused or duplicated code and out-of-order code execution. Data scientists can benefit from better tool support when maintaining and evolving notebooks. We argue that central to such tool support is identifying the structure of notebooks. We present a lightweight and accurate approach to extract notebook structure and outline several ways such structure can be used to improve maintenance tooling for notebooks, including navigation and finding alternatives. 

   more » « less
3. Automated Assessment of Quality of Jupyter Notebooks Using Artificial Intelligence and Big Code

   https://doi.org/10.32473/flairs.v34i1.128560  

   Oli, Priti; Banjade, Rabin; Tamang, Lasang Jimba; Rus, Vasile (May 2021, The International FLAIRS Conference Proceedings)

   We present in this paper an automated method to assess the quality of Jupyter notebooks. The quality of notebooks is assessed in terms of reproducibility and executability. Specifically, we automatically extract a number of expert-defined features for each notebook, perform a feature selection step, and then trained supervised binary classifiers to predict whether a notebook is reproducible and executable, respectively. We also experimented with semantic code embeddings to capture the notebooks' semantics. We have evaluated these methods on a dataset of 306,539 notebooks and achieved an F1 score of 0.87 for reproducibility and 0.96 for executability (using expert-defined features) and an F1 score of 0.81 for reproducibility and 0.78 for executability (using code embeddings). Our results suggest that semantic code embeddings can be used to determine with good performance the reproducibility and executability of Jupyter notebooks, and since they can be automatically derived, they have the advantage of no need for expert involvement to define features. 

   more » « less
4. Exploring Organization of Computational Notebook Cells in 2D Space

   https://doi.org/10.1109/VL/HCC53370.2022.9833128  

   Harden, Jesse; Christman, Elizabeth; Kirshenbaum, Nurit; Wenskovitch, John; Leigh, Jason; North, Chris (October 2022, 2022 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC))

   Representing branching and comparative analyses in computational notebooks is complicated by the 1-dimensional (1D), top-down list arrangement of cells. Given the ubiquity of these and other non-linear features, their importance to analysis and narrative, and the struggles current 1D computational notebooks have, enabling organization of computational notebook cells in 2 dimensions (2D) may prove valuable. We investigated whether and how users would organize cells in such a “2D Computational Notebook” through a user study and gathered feedback from participants through a follow-up survey and optional interviews. Through the user study, we found 3 main design patterns for arranging notebook cells in 2D: Linear, Multi-Column, and Workboard. Through the survey and interviews, we found that users see potential value in 2D Computational Notebooks for branching and comparative analyses, but the expansion from 1D to 2D may necessitate additional navigational and organizational aids. 

   more » « less
5. The Right Tool for the Job: Data-Centric Workflows in Vizier

   Oliver Kennedy, Boris Glavic (September 2022, Bulletin of the Technical Committee on Data Engineering)

   Sudeepa Roy and Jun Yang (Ed.)

   Data scientists use a wide variety of systems with a wide variety of user interfaces such as spreadsheets and notebooks for their data exploration, discovery, preprocessing, and analysis tasks. While this wide selection of tools offers data scientists the freedom to pick the right tool for each task, each of these tools has limitations (e.g., the lack of reproducibility of notebooks), data needs to be translated between tool-specific formats, and common functionality such as versioning, provenance, and dealing with data errors often has to be implemented for each system. We argue that rather than alternating between task-specific tools, a superior approach is to build multiple user-interfaces on top of a single incremental workflow / dataflow platform with built-in support for versioning, provenance, error & tracking, and data cleaning. We discuss Vizier, a notebook system that implements this approach, introduce the challenges that arose in building such a system, and highlight how our work on Vizier lead to novel research in uncertain data management and incremental execution of workflows. 

   more » « less