Google Summer of Code NumFOCUS PyBaMM

Blogs Project page on the Google Summer of Code website Proposal on Google Docs


View my Twitter profile View my GitHub profile View my LinkedIn profile

This page serves as a permalink to the final report that provides details of Agriya Khetarpal’s project in the Google Summer of Code 2023 programme.

📖 TL;DR

A summary of some of the tasks I achieved throughout the community bonding and the coding periods is given below.

  • A revamped static PyBaMM website, at https://pybamm.org
  • Documentation infrastructure improvements with Sphinx extensions and new functionality to make it more robust, readable, and maintainable
  • Various improvements to the general infrastructure, DevOps-related issues, and repository maintenance-related tasks: tweaks to the CI/CD pipelines, issues related to the migration of the test suite from tox to nox, caching dependencies for speed-ups in the jobs matrices, removing unused dependencies, addressing deprecations, and more
  • Some features integrated into the core PyBaMM functionality

🔋 About PyBaMM

PyBaMM is an open-source Python package for the mathematical modelling of batteries and running fast, flexible, and accurate simulations for myriads of battery models. Its mission is to accelerate battery modelling research by providing open-source tools for multi-institutional, interdisciplinary collaboration; it is fiscally sponsored by NumFOCUS and The Faraday Institution in the U.K. The use of the software in academia and industry has been prolific since its inception—it has been used at a multitude of universities, research institutions, and commercial research labs in collaborative settings.

📄 Project abstract

This project aims to enhance the documentation infrastructure for PyBaMM and consolidate all of it in one place, which includes but will not be limited to: the API documentation, user guides, the example Jupyter notebooks, and the PyBaMM Wiki. This project will also introduce a new pybamm.org website with the Hugo static site generator, and improvements to the existing documentation built with Sphinx – based on other popular Python packages and libraries in order to make it more robust and maintainable.

🚀 Motivation

A robust and up-to-date documentation infrastructure is essential to PyBaMM’s development and usage as the package transitions into a modern, open-source software suite to model battery simulations and provide unparalleled flexibility to its end users. It will help not only future contributors and maintainers but also users across various domains involved in the battery science ecosystem.

🍀 Benefits to the community

My work shall make it convenient to maintain an ever-expanding documentation resource by implementing a series of both short and major quality-of-life improvements to prettify it, and by adding new features that have been requested in the past. The aim is to make it easier not only for users and learners of PyBaMM but also for new contributors towards its development in the future.

🧑‍💻 Work done

⚡️ A new website for PyBaMM

PyBaMM has received a revamp on the realms of the internet—the new static website can be accessed on the same domain as the previous one: at https://pybamm.org. It uses the Scientific Python theme for Hugo and Netlify for the deployment infrastructure and is inspired by the NumPy and the SciPy websites. The content is written in Markdown syntax. The pages of the website comprise but are not limited to: a homepage, an about page, a teams page to list maintainers and contributors, the governance and roadmap pages, a GSoC page, a newsroom, and more. The site has an impeccable SEO score and is WCAG-compliant on most pages and elements as much as it is possible without significant alterations to the theme itself.

The website incorporates a Makefile to be used with make as a task runner. The CI includes workflows for linting and autoformatting, and for updating the teams page at https://pybamm.org/teams/ weekly via the GitHub API.

The pinned summary issue lists all the tasks that were pursued in the form of issues and the pull requests that tracked them, as a part of getting the website ready for deployment: https://github.com/pybamm-team/pybamm.org/issues/6/

🚇 Documentation infrastructure improvements

  1. A custom domain was added on Read the Docs, so the documentation hosted on https://pybamm.readthedocs.io now redirects to and can be accessed at https://docs.pybamm.org.

  2. PyBaMM uses a citation file to keep track of relevant papers for battery models and parameter sets. With the sphinxcontrib-bibtex extension; the CITATIONS.bib file can be parsed to extract references from it, and the directives provided such as :footcite:t: and .. footbibliography:: are used to display all references on a page programmatically. This eliminates the possibilities of errors in spelling and BibTeX validation errors.

  3. Pages now display when they were last updated in the docs via the sphinx-last-updated-by-git Sphinx extension.

  4. Added configuration for multithreaded documentation builds to speed up the documentation builds locally.

  5. Added nbsphinx to display and render Jupyter notebooks and their galleries with sphinx-gallery: it was used as a Sphinx extension to convert .ipynb files to .rst format that can be parsed by Sphinx’s HTML builder. The notebooks can be viewed in a gallery format on the Examples page of the website. Functionality was added such that each notebook can be separately downloaded from the website itself or opened in Google Colab from the main or the develop branches. Similarly, the example scripts will soon be incorporated in the form of a thumbnail gallery on the same page.

  6. Added floating window tooltips for cross-references in the docs using the sphinx-hoverxref extension, which enables external tooltips for classes, methods, and self-contained links inside the documentation and references intersphinx inventories for external links. Users can hover over a link to display the contents of the link.

  7. Inheritance diagrams to display class hierarchies for models and submodels: PyBaMM ships with an extensive library of models and submodels for various battery chemistries—such as lithium-ion models, lead-acid models, and more. The sphinx.ext.inheritance_diagram extension was added and customised through a custom extension in the docs/sphinxext/ folder that automatically creates inheritance diagrams via graphviz for a particular model or submodel to show the qualified name(s) of the base model class(es) it inherits from.

  8. Algolia DocSearch: the default Sphinx search was relatively slower than other third-party vector search solutions. Algolia offered a search-as-you-type service for open-source technical websites, which was configured and added to the Sphinx configuration via the sphinx-docsearch extension. It can be accessed on the latest version of the documentation using the ⌘ + K or Ctrl + K keyboard shortcuts.

There were many more experiments with the documentation infrastructure which were tried, such as the use of sphinx.ext.autosummary for automatically picking up relevant bits of documentation and reST-formatted docstrings from Python modules and classes instead of referencing them, and that of sphinx.ext.coverage for automatically checking the if all public API elements were included in the documentation and generating formatted documentation coverage reports. Both of them did not make it to the source code due to issues with public-private API behaviour (see pybamm-team/PyBaMM #2427) and the conflicting use of the sphinx.ext.autodoc extension respectively.

🍪 pybamm-cookiecutter: a template for battery modelling projects using PyBaMM

After the completion of the requirements and the stretch goals of my project, there is a new project named pybamm-cookiecutter that has been started to provide a template for battery modelling projects using PyBaMM. It is a cookiecutter template that can be used to generate a new project with a pre-defined file structure and a set of files that are required to get started with a new project. It is a work in progress and will be updated with more features and functionality in the future. The project can be accessed at https://github.com/pybamm-team/pybamm-cookiecutter.

The idea is to integrate it with features such as nox, pre-commit, and the necessary Sphinx boilerplate such that new projects, research showcases, battery models, and other examples can be integrated with PyBaMM’s documentation and CI/CD infrastructure with ease. At the time of writing, the project can support third-party parameter sets through entry points, which can be loaded from PyBaMM, which comes installed as a dependency. The project is statically typed with mypy and has a pyproject.toml file with support for hatch and hatch-vcs to manage releases using git tags and uploading wheels to PyPI. Support for the flit and setuptools621 backends is coming soon!

📑 A list of issues opened and pull requests submitted

This list contains issues and pull requests that are both specific to and not directly related to my project: they contain documentation infrastructure improvements, improvements to the infrastructure in general, bug fixes, miscellaneous enhancements, and relevant queries and/or discussions.

🛠️ Issues

👷 Pull requests

✍️ Blogs

I wrote a series of blogs entailing my journey throughout the coding period. They can be accessed at the following link: https://dev.to/agriyakhetarpal

🔮 Future work

  • A new project named pybamm-cookiecutter is under development. It started as a supernumerary project after the completion of the main requirements of the documentation infrastructure project. This will be a BSD-licensed Python package and a cookiecutter template for battery modelling projects and will rely on PyBaMM as a dependency, the embodiments of which will simplify the creation and usage of third-party parameter sets, add utility functions to use with PyBaMM for ease-of-use, and provide configuration options for the surrounding infrastructure for creating releases and hosting documentation. The development of pybamm-cookiecutter can be tracked on its GitHub repository at https://github.com/pybamm-team/pybamm-cookiecutter and the package will soon be pip-installable after the first release on PyPI.
  • Nightly release infrastructure: alongside creating Docker images for simplifying the developer installation of PyBaMM, a new goal is to create nightly builds and subsequent releases on a custom, public PyPI index. The envisioned releases will help install PyBaMM through pip, the package manager that ships with Python and provide convenience to users and developers in the case of bug fixes and improvements that are yet to make it to the next official release on PyPI. This process will require suitable versioning with timestamps of release tags and modifications to the existing infrastructure for building wheels on GNU/Linux, macOS, and Windows, and uploading them to the third-party PyPI index.
  • Migration to pyproject.toml: PyBaMM is not a pure-Python package and depends on pybind11 for interoperability between C++ and Python code. There are new build-backends available, such as meson-py and scikit-build, which can be used with or alongside the setuptools621 specification to migrate from setup.py to the new standard for modern Python packaging.
  • Adding example scripts to the hosted documentation: this will entail adding reST documentation for the Python scripts and configuring them to work with Sphinx and sphinx-gallery to display them on the website.
  • With all the new changes, we enabled PDF builds for the documentation on Read the Docs, however, the outcome has been error-prone. I will look to resolve the issues with Sphinx’s LaTeXBuilder and ensure that the PDF output works properly, and consider either self-hosting the documentation or migrating from Read the Docs to GitHub Pages.

Acknowledgements

I would like to thank the Google Summer of Code programme, NumFOCUS, and the PyBaMM Team for a rewarding experience and a productive summer. It has been the first time that I have contributed to the development and maintenance of a set of open-source, scientific software of this scale – I have learned a lot throughout the process.

I am honoured to have been mentored by the PyBaMM Steering Council members Valentin Sulzer and Saransh Chopra. They were always helpful in recharging my batteries (no pun intended) and amicably provided valuable feedback on both technical and non-technical issues; going forward, I shall truly cherish the camaraderie we shared.

I am also grateful to Ferran Brosa Planella and Robert Timms for their assistance in many stages of my contributions to PyBaMM. None of the work done would have been possible without the guidance of the members of the PyBaMM team. I also enjoyed working with Arjun, my co-student at GSoC and we took part in discussions on issues and PRs as a part of the collaborative process that the philosophy of working on open-source software advocates for. A part of me wants to inoculate the meticulous zeal of software development and design choices followed by the ethos of PyBaMM across my own life’s spheres.

🎬 Footnotes

I am enthralled to have completed the Google Summer of Code 2023 programme! I hope to lay down my contributions to PyBaMM and its infrastructure further in the future and help ameliorate the development and success of the PyBaMM ecosystem.

I also look forward to incorporating the technical skills I have learned into my repertoire through academic, personal, and professional projects in the future.

🔭 References