CI/CD workflows
Ersilia relies heavily on GitHub Actions for CI/CD. Here is a high-level summary of workflows involved in maintaining the Ersilia CLI and code-bases from all the Ersilia maintained models.
Last updated
Was this helpful?
Ersilia relies heavily on GitHub Actions for CI/CD. Here is a high-level summary of workflows involved in maintaining the Ersilia CLI and code-bases from all the Ersilia maintained models.
Last updated
Was this helpful?
These workflows ensure that we push quality code to Ersilia, incorporating testing by individual components as well as end-to-end.
Objective: Ensure Ersilia continues to work with any new code merging from contributors and maintainers to the master branch
Workflow name: Deploy and test ersilia on PR
Workflow file:
Jobs:
Build: creates a conda environment (py3.10 currently) and installs Ersilia from the open PR. Then it tests the following functions: catalog and model running (fetch, serve, info and run) from GitHub, S3 and DockerHub. A single molecule is passed as input, and the output is analysed.
Run: automatically when a PR is open on the Ersilia master branch.
Objective: Ensure that Ersilia is installable across supported Python environments, supported platforms, both through source and while interacting with Dockerized models. This workflow also runs the unit and integration test suite for the repository.
Workflow name: Ersilia tests, installation checks, and cleanup of model request template
Workflow file:
Jobs:
install-ersilia, installs Ersilia from Python 3.8 to Python 3.12 on an Ubuntu base.
test-docker is the next step to follow after we confirm that Ersilia can be installed in environments created with different Python versions. This step tests whether dockerized models can be fetched using the ersilia CLI, ie the happy flow works without issues.
test-os Since Ersilia is supported only on Linux distributions, and MacOS, this job ensures that Ersilia can be installed and run on both of these platforms, presently only utilizing a Python 3.10 environment.
run-pytest This is the main step that carries out unit testing within Ersilia ensuring that different components continue to work without issues.
run-cli-test-single This an integration level test that ensures different components within Ersilia interact cohesively in running a single model using the Ersilia CLI. More details on how this is implemented can be found .
run-cli-test-multiple Similar to the job described above, this job tests the functionality in Ersilia to simultaneously run multiple models. More details around implementation of the specific tests and the testing frameworks used can be found .
test-colab-notebook This test ensures that Ersilia can run in IPython notebooks, such as Jupyter or Google Colab. Specifically, this test checks whether a notebook containing Ersilia code can be executed successfully.
update-model-request-template This job within the workflow only runs when the PR or the commit on master branch change the file containing . We expect model tags to be completely sorted because they are used to populate the (more on this below). The job sorts the tags file if it is changed, and then updates the Model Request Issue with the changes, and commits them back to the master branch.
Run: when a PR is opened against the master branch, and again when code is pushed to the master branch.
Objective: Package Ersilia into an image to use as a base image to containerise individual models as well as in development containers.
Workflow name: Upload Ersilia base image to DockerHub
Jobs:
upload_ersilia_base_to_dockerhub builds two versions of the Ersilia image, a lighter version tagged as ersiliaos/base-multistage-condapack , and a legacy much bloated version tagged as ersiliaos/base-legacy-bentoml .
Run: configured to trigger in multiple ways - whenever code is pushed on to the master branch, when a new release is created from Ersilia at the beginning of each month (more on this below), and also manually by developers and maintainers.
Objective: Distribute Ersilia as source, as a Docker image, as a PyPI package, and as a conda-forge package.
Workflow name: Ersilia Release
Jobs:
gh-release Utilizing the tag created in the previous job, a GitHub release is created in this job of the workflow. The GitHub CLI is used to automatically create this release and utilize the commit history from the master branch to generate release notes.
pypi-release Finally, we utilize the release artifacts created from the GitHub release job of the workflow to publish a new version of Ersilia on the Python Packaging Index (PyPI).
The completion of this release workflow triggers the Docker build workflow mentioned above.
Conda-Forge Release
Objective: Contribute new models to the Ersilia Model Hub
Workflow name: Approve Command Dispatch
Jobs:
Objective: Test a model using its source code.
Workflow name: Model test on push
Jobs:
Run: Whenever code is pushed to the main branch of the repository, either directly by a maintainer or by way of merging a pull request. This workflow can also be triggered manually by a maintainer.
Objective: Test a model using its source code.
Workflow name: Model Test on PR
Jobs:
test This workflow is quite similar to the one above. It tests a model by fetching it locally from source, and verifying that the model both produces outputs and that they are functionally correct. However, this workflow does not sync any metadata with Ersilia's metadata stores.
Run: Upon Pull Request creation by the model contributor from the forked model repository.
Objective: Upload the model source code as a zip file to Ersilia's S3 storage in AWS cloud.
Workflow name: Upload model to S3
Jobs:
Run: Only on the main repository, upon successful completion of the Model test on push workflow.
Objective: Upload a functioning dockerized image for the model
Workflow name: Upload model to DockerHub
Jobs:
Run: When the Upload Model to S3 workflow is successfully completed.
Workflow name: Upload Ersilia Pack Dockerized Model
Jobs:
Run: When called by Upload model to DockerHub workflow.
Objective: This workflow is called by other workflows and requires an input version to determine the type of BentoML Dockerized model to build.
Workflow name: Upload BentoML Dockerized Model with or without Multi-stage Conda Pack
Jobs:
Run: When called by Upload model to DockerHub workflow.
Objectives: Update metadata for the model, and notify users to test the model.
Workflow name: Post Model Upload actions
Jobs:
post-model-upload This job utilizes the arch.txt build artifact from the Upload Model to DockerHub job to update the model metadata file in the repository signifying the image architectures supported by the model, as well as updates that information in Ersilia's metadata stores such as Airtable, and read-only JSON files in S3. Thereafter, a test issue is created in the model's repository notifying an Ersilia maintainer to test the model. If the Ersilia maintainer finds the model to be working, then at this stage we conclude the model to have been successfully incorporated.
Run: Upon completion of the Upload model to DockerHub workflow.
Ersilia Statistics
Ersilia Maintenance
Workflow file:
Workflow file:
version This is the job that increments the version of the code-base, and creates a new release tag, while also updating this information in the , and files.
Ersilia is released on conda-forge using by way of updating its maintained within the conda-forge organization on GitHub. Upon the creation of a new GitHub release, an automated pull request is created in this feedstock repository which is then manually reviewed and merged by an Ersilia maintainer.
Run: At the beginning of each month, a release is executed at 3:00 AM UTC publishing a new release for Ersilia using the code present on the master branch.
Workflow file:
approve-command-dispatch A model contributor is expected to fill out several fields in the issue form such as the model's name, its slug, relevant tags, its publication and source code, etc. Once approved, the workflow creates an model repository from the utilizing information parsed from the issue body to name the repository, track information about the requested model within Ersilia's metadata stores such as Airtable, and read-only JSON files in S3. A mock LFS object is also instantiated and uploaded to the repository to facilitate model contributors and utilizing Ersilia's while adding checking model weights in the repository. Once these steps complete successfully, the original issue creator is notified that a repository has been successfully created for them to start contributing the model. Read on more about model contributions in our Model Contribution .
Run: When an Ersilia maintainer comments /approve on a issue submitted by a model contributor.
Workflow file:
test This workflow is responsible for testing the model source and verifying that the model produces outputs and those outputs are functionally correct. The model is first fetched using its local path on the runner, and is then run against a set of example inputs. The model output is checked for as a proxy for making sure that the model actually works. Thereafter, the ersilia test command is utilized to perform sanity checks like completeness of model dependencies, metadata, and necessary files, including testing for functional correctness of the model's output. The job is also responsible for updating the model's metadata in Ersilia's metadata stores such as Airtable and read-only JSON files in S3.
Workflow file:
Workflow file:
upload_model_to_s3 This workflow is responsible for zipping the model source and it to a S3 bucket in Ersilia's AWS cloud. The referenced script performs many actions such as removing git artifacts from the model source directory before preparing it for upload. The workflow also carries out updating metadata for the model in Ersilia's metadata stores, particularly with the URL of the AWS S3 object that gets created by running this workflow.
Workflow file:
upload-ersilia-pack This job uses the callable workflow to build and upload the Ersilia Pack Dockerized model. The called workflow inherits any necessary secrets from the repository.
upload-multi-stage-condapack This job runs if the upload-ersilia-pack job fails. It uses the upload-bentoml.yml workflow to build and upload a multi-stage CondaPack Dockerized model. Because it depends on the success or failure status of the upload-ersilia-pack it cannot start before the previous job is finished. This job uses the callable workflow with the input "multistage-condapack". More on how models are Dockerized within Ersilia can be found .
upload-legacy-bentoml This job runs if the upload-multi-stage-condapack job fails. It uses the callable workflow to build and upload a legacy BentoML Dockerized model. More information on this terminology (and some history) can be found .
Objective: This workflow is called by to build and upload the Ersilia Pack Dockerized model
Workflow file:
build-ersilia-pack-image The job fetches the based on the model's Python version requirement, and uses that Dockerfile to build an image for the model. Thereafter, the model is fetched using its locally built image, served with tracking enabled, and run against a set of example inputs. The output is checked to see if it contains any values as a proxy measure for testing whether the model image got built successfully or not. Note that we only build an AMD image for testing, and if the test passes, we build the ARM image as well, and push them both to DockerHub. Note that if the ARM image fails to build, we store that in arch.txt build artifact.
Workflow file:
build-bentoml-image This job is only run if the model repository structure supports building a model using BentoML, ie, it should not contain an install.yml file. More on why this happens can be found . This workflow is quite similar to the workflow above, in that it the Dockerfile for the model repository, and builds and tests the model image. First only the AMD image is built, which is then tested for outcomes as a proxy measure for a functional model image, and if this test passes, the ARM image also gets built. After successful building of both versions, the images get pushed to DockerHub. Note that if the ARM image fails to build, we store that in arch.txt build artifact.
Workflow file: