Management

Meetings, whether on-line or in-person are regarded as critical for good collaboration, and are discussed in Section 7.2. The other key collaborative element centres naturally on the software, where use of the git control system, see Section 7.2 and consequent use of repositories, see Section 7.2, is becoming universal.

Meetings and Workshops

To start the project and any notable identifiable piece of work within it, a kick-off meeting should bring together all partners who will contribute significant code to the project. The aim of this meeting will be to build personal links among the team, and to establish community practices consistent with the charter. Efforts should be made to build consensus and a community spirit within the project team.

A regular project planning and monitoring meeting should be set up, at least monthly. Its agenda should include short updates on progress of each project component, and focus on the project planning and coordination. In addition, a separate series of seminars and training should be organised, where each partner might give a longer talk on an aspect of their work, for example showing other partners how to use recently developed capabilities.

Development and collaboration mechanisms should include:

  • 1. A system of code repositories for version control (eg. github)

  • 2. Automated testing infrastructure (eg. github actions)

  • 3. Documentation infrastructure, ie. as a website

  • 4. A repository for long-term storage of large files, records of meetings, presentations etc. (eg. Google shared drive)

  • 5. A chat/messaging service such as Slack or Zulip, to facilitate interactions between developers

As these are established, a series of training workshops should be arranged. These should include talks on the “high level" objectives, on the near-term plans of each partner, and also hands-on training in the tools being used.

Version control

The standard git version control system should be used; there is no viable competitor to this in terms of capabilities, widespread adoption, or integration into other tools and services (eg. github).

A common complaint against git is the user interface, which can be intimidating to new users. There are very strong reasons why even programmers with plenty of other experience, should seek guidance and preferably training in use of the command line interface (CLI). For those who have time enough to attempt to do so without, a few hints are provided:

  • 1. The complexity of the interface can be mitigated by restricting usage to a few well-chosen subcommands such as clone, add, commit, push, pull, diff, log and status.

  • 2. Exercise caution before using other subcommands or new options to the core subcommands, eg. by first committing all files, adding a suboption which indicates what will be done without actually modifying any files, and avoiding forcing options.

  • 3. For the purpose of the key subcommands such as ‘pull’ and ‘push’, it is important to remember than these are are defined from the user’s point-of-view, so that ‘pull’ brings source from the repo towards the user, and ‘push’ sends it away. There are other non-intuitive aspects so that it is important to study very carefully the description of any new sub-command/option and particularly its ordering of options.

  • 4. Since the software is widely used, error messages can invariably be ‘googled’ for further explanation.

  • 5. Should conflicts occur, these are recorded by the insertion of strings ‘+++…’, ‘>>>>...’ and ‘<<<<...’ in disc files to indicate lines where the clashes lie. Many users find resolving conflicts very difficult on the basis of such information, however making up for the absence of a GUI mechanism within git to do this, it is possible to integrate GUIs such as meld, being aware of possible system dependences.

Otherwise, the experience of git can be mitigated through:

  • • Training: Links to training material for adopted tools should be made available as part of the project documentation. This should be supplemented by training, both one-to-one and as part of a programme of talks and training.

  • • Adoption of, and training in, tools to provide easier interfaces. github itself allows browsing of history; Magit is an excellent interface integrated into Emacs; and similar tools exist for eg. Visual Studio Code. The ITER organisation uses bitbucket and UKAEA uses gitlab.

Code repositories

The structure of ExCALIBUR will result in a number of different components, experimental proxyapps, and increasingly complex applications. There are two main different approaches as to how these different components could be split between git repositories, namely (1) Several large code bases are kept in a single repository (a ‘monorepo’) and (2) projects are kept in separate repositories, with dependencies being included as git submodules.

Adopted is a compromise approach whereby:

  • • A github ‘organisation’ https://github.com/ExCALIBUR-NEPTUNE was created to host new repositories. Organisations allow permissions for groups of administrators and developers to be managed, and this is currently restricted to community members, so that it is important to be ‘logged in’ to access their components.

  • • Individual components and proxyapps are hosted in separate repositories under this organisation. These contain the code, unit tests, documentation etc. specific to these components.

  • • Reports produced as part of the NEPTUNE project, unless they contain commercially sensitive information, are to be found in the repository ref. [7].

  • • A central repository under this organisation includes components as sub-modules. These could be organised into a directory structure, with documentation explaining the relations or coupling between components. In this repository should go:

    • – Integration tests which couple components and ensure that they work together

    • – Documentation of the interfaces between components, project conventions (eg. style guides), and overall project aims.

    Sub-modules are pinned to a particular git commit, so that at any point the versions included are those which are known to work with each other. A developer who wants the latest version of a component should clone the individual repository, while a user who wants something that “just works" should clone the central repository.

(There is the disadvantage of a tie specifically to github, but loss of the ‘organisation’ capability would be expected to be an inconvenience rather than a disaster for a project.)

Development workflow

The standard git work flow has been adopted, since this is widely familiar and has been developed as best practice based on industrial experience. Exceptions are allowed for minor issues, such as typographical errors and broken links in documentation.

Each code component maintains a main branch (often referred to as the ‘master’ as in ‘master copy’), which can only be modified through a pull request mechanism which ensures peer review and testing. Bug fixes and feature development are done in separate branches, either in the same repository, or in forked repositories. When someone encounters a bug, or wishes to develop a new feature, the recommended approach is:

  • 1. An issue is opened, describing the bug or feature request or proposal. This allows discussion of the issue, and possible approaches to addressing it.

  • 2. A pull request is opened as early as possible, marked “Work in progress" or similar. This can contain only minimal code or outline of the code structure. This links to the issue, lets other people know that it is being worked on, and enables peer review and input into the development direction.

  • 3. Once ready for merging, and consensus has been reached that the proposed change should be made, then it is merged.

If a code is sufficiently large, then a further degree of separation between the stable main branch and active development is needed. A common pattern is to only branch off and merge features into a next branch. Periodically this branch is merged into main as a new release, once the new features are judged to be sufficiently mature and tested.

Whether into main or next, pull requests should be reviewed using a checklist the remind the reviewers. Review involves testing, aspects of which are addressed in Section 9.2.

It must be stressed that code review is not a job separate from code development: All developers should be expected to participate in and carry out code reviews. Reviewing code benefits not only the original author, but also the reviewer. Through the discussion, it contributes to a sense of shared ownership of the code base, and spreads good practices. There is the implication that code should be written ‘for the other guy’, ie. so that the other guy can understand it without much difficulty. It also ensures that at least two developers know how each part of the code works.

Code release

Code releases should be a regular occurrence. Code release helps with project branding and user engagement, and ensures that the project is seen as active. It also helps project administration by ensuring new features are shared in a timely fashion, and by reducing the number of long-lived divergent branches.

The project NEPTUNE codebase contains proxyapps, and infrastructure code that interfaces them. A code release will consist of a version of this infrastructure code, plus commit hashes that fix the versions of the proxyapps. As proxyapps might be independent projects with their own established release cycle, the following release policy applies only to the infrastructure code. It is the recommended policy for new proxyapps written under Project NEPTUNE.

Release numbering should follow (a modified) Semantic Versioning approach ref. [100], summarized as

“Given a version number MAJOR.MINOR.PATCH, increment the:

  • 1. MAJOR version when you make incompatible API changes,

  • 2. MINOR version when you add functionality in a backwards compatible manner, and

  • 3. PATCH version when you make backwards compatible bug fixes.”

Here it is understood that “API” refers to user-facing interfaces; APIs to functions internal to proxyapps may break backwards compatibility in MINOR releases. There is however an absolute guarantee that no backwards incompatible changes are made for end users in MINOR and PATCH releases, except those that arise from fixing a bug. That is, physics results are permitted to change in such releases if the new release’s results are “correct” and the previous release’s results were “wrong”.

It is also understood that releases with MAJOR number 0 are considered beta releases, for which there are no guarantees of backwards compatibility.

Each release will be uploaded as a Zenodoref. [101] record with its own DOI. This gives a clear citation for the project (to be included in the project’s README or CITATION.cff file), while ensuring that developers receive credit for their work, without the need for associating each release with a publication. Encouraging researchers to use release versions and to cite by version number also aids scientific reproducibility.

While some technical aspects of the release process can be automated, many of the tasks, such as curating issues and writing release notes, are inherently manual. To prevent NEPTUNE relying on a single person to make releases, the exact workflow will be codified and included in the project documentation. An example of such a workflow for the GS2 project may be found online ref. [102].