Ersilia Book
  • 🤗Welcome to Ersilia!
    • The Ersilia Open Source Initiative
    • Ten principles
    • Ersilia's ecosystem
  • 🚀Ersilia Model Hub
    • Getting started
    • Online inference
    • Local inference
    • Model contribution
      • Model template
      • Model incorporation workflow
      • Troubleshooting models
      • BioModels annotation
    • For developers
      • Command line interface
      • CI/CD workflows
      • Test command
      • Testing playground
      • Model packaging
      • Inputs
      • Codebase quality and consistency
      • Results caching
  • 💊Chemistry tools
    • Automated activity prediction models
      • Light-weight AutoML with LazyQSAR
      • Accurate AutoML with ZairaChem
      • Model distillation with Olinda
    • Sampling the chemical space
    • Encryption of AI/ML models
  • AMR chemical collections
  • 🙌Contributors
    • Communication channels
    • Tech stack
    • Internships
      • Outreachy Summer 2025
      • Outreachy Winter 2024
      • Outreachy Summer 2024
      • Outreachy Winter 2023
      • Outreachy Summer 2023
      • Outreachy Winter 2022
      • Outreachy Summer 2022
  • 📑Training materials
    • AI2050 intro workshop
    • AI2050 AI for Drug Discovery
    • Introduction to ML for Drug Discovery
    • Python 101
    • External resources
  • 🎨Styles
    • Brand guidelines
    • Slide and document templates
    • Scientific figures with Stylia
    • Coding style
  • 🌍About Us
    • Where to find us?
    • Diversity and inclusion statement
    • Code of conduct
    • Open standards and best practices
    • Ersilia privacy notice
    • Strategic Plan 2025-2027
    • Ersilia, the Invisible City
Powered by GitBook

2025, Ersilia Open Source Initiative

On this page
  • TL:DR
  • Structure overview
  • Ersilia playground flags
  • Additional flags for Ersilia's CLI
  • General setting flags

Was this helpful?

  1. Ersilia Model Hub
  2. For developers

Testing playground

The Testing Playground provides a flexible and robust framework for validating and profiling CLI commands that are being used for managing Ersilia models from various sources, such as GitHub, DockerHub, and local directories.

The Testing Playground only works with Linux systems, and is oriented towards Ersilia developers with experience in our tools.

TL:DR

To use the Test Playground, you need to have Ersilia installed in test mode and the Ersilia repository cloned into your local.

conda create -n ersilia python=3.12
conda activate ersilia
git clone https://github.com/ersilia-os/ersilia.git
pip install -e .[test]

The Playground runs on a nox environment, isolated from the local source. Each time we will activate a nox session from the test/playground folder in Ersilia:

cd ersilia/test/playground
nox -s execute # to start the session and run commands
nox -s clean #to clean up all folders and session files created by nox

The nox command nox -s execute will initiate a session. A few built-in flags can be passed to it:

Flag
Status
Description

-s

Required

Specifies the session to be run by Nox.

-p

Not required

Specifies the python environment. If none is specified, it will test everything in py3.8 to py3.12. More than one environment can be specified simply using nox -s execute -p 3.8 3.9.

-fb

Not required

Used to change python backends (conda, mamba, micromamba, virtualenv, venv, uv, none). Defaults to conda.

-v

Not required

Verbose output printed in the terminal.

The first time you use Nox in your system you will be required to grant sudo privileges so that actions like DockerHub activation can be performed

Please note that nox separates built-in arguments and custom flags using --. Hence, all arguments not referred in this table will need to follow the following structure:

nox [built-in flags] -- [custom flags]

Structure overview

The idea behind the playground is to cover all sorts of tests we might want to do on any model, more extensively than the test command itself. Therefore, it is a highly customizable functionality and only addressed to Ersilia developers.

Nox will create an isolated environment and store the files used for testing under ~/eos/playground/files and the logs generated under ~/eos/playground/logs. Those will be eliminated with the nox -s clean command.

The playground is adapted to many use cases, for example:

  • Test several models on python 3.12 fetching them from github

  • Test one single model across all python versions fetching from dockerhub

  • Evaluate if a model seems to have gotten slower

  • ...

Below we describe the flags you can use to combine all these custom-made tests.

Ersilia playground flags

Flag
Default
Description
Example

--cli

all

Specifies ersilia commands to run in order (fetch, serve, run, catalog, example, test, close, delete). Default is all, which executes commands in this order: "fetch", "serve", "run", "close", "catalog", "example", "delete", "test".

nox -s execute -- --cli fetch serve run nox -s execute -- --cli run

--fetch

--from_github

Fetches models from sources (from_github, from_dockerhub, from_s3, version)

nox -s execute -- --fetch from_s3

nox -s execute -- --fetch from_dockerhub version dev

--run

None

Run a model. It will try to fetch it from_dockerhub as this is Ersilia's default (depends on the activate_docker flag, see below). Best combined with the input and output flags (see below)

nox -s execute -- --run

--example

["-n", 10, "--random"]

Generates example input for a model (-n, --random, -f). If we specify a

nox -s execute -- --example -n 10 random/predefined -c -f example.csv

--catalog

["--more", "--local", "--as-json"]

Retrieves model catalog from local or hub.

nox -s execute -- --catalog hub

--test

["--shallow", "--from_github"]

Tests models at different levels (shallow and deep) and from_github, from_dockerhub or from_s3.

nox -s execute -- --test deep from_dockerhub/from_s3/from_github

--delete

None

Used to delete models. It has only one flag: all

nox -s execute -- --delete all

Additional flags for Ersilia's CLI

Flag
Default
Description
Example

--outputs

[results.{csv, json, h5}]

This is used with run command and used to specify output file types. Note that we only specified the file name, the path will be automatically set to ~/eos/playground/files/{file_name.{csv, json, h5}}

nox -s execute -- --outputs result.csv result.h5

--input_types

List of (str, list, csv)

This is also used with run command to define input formats (str, list, csv).

nox -s execute -- --input_types str list csv

--runner

single

Specifies execution mode (single, multiple). The single mode is used to execute commands using one model whereas the multiple mode will use multiple models to execute the given commands.

nox -s execute -- --runner multiple

--single

eos3b5e

Used to specify or override the default model ID used for single running mode.

nox -s execute -- --single model_id

--multiple

[eos5axz, eos4e40, eos2r5a, eos4zfy, eos8fma]

Used to specify or override the default model IDs used for multiple running mode.

nox -s execute -- --multiple model_id1 model_id2

General setting flags

Flag
Default
Description
Example

--activate_docker

true

Activates or deactivates Docker. It allows to test for example if autofetcher will decide not to fetch from Docker (default) when Docker is not active

nox -s execute -- --activate_docker false

--log_error

true

Enables or disables logging of errors as file, which will be stored in ~/eos/playground/logs/. Each command failures will create a standalone file, with datetime on it in a string format. For instance catalog_20250129_145802.txt

nox -s execute -- --log_error false

--silent

true

Enable or disable logs from ersilia command execution

nox -s execute -- --silent false

--show_remark

false

Displays a remark column in the final execution summary table that allows to quickly see if the ersilia commands are executed successfully

nox -s execute -- --show_remark true

--max_runtime_minutes

10

Sets the maximum execution time for a run command, to test model speed if seems to be slow.

nox -s execute -- --max_runtime_minutes 5

--num_samples

10

Sets the sample size to create input for run command.

nox -s execute -- --num_samples 5

PreviousTest commandNextModel packaging

Last updated 3 months ago

Was this helpful?

If no flag is specified, the command nox -s execute will run its default test: it will try to fetch, serve and run the model eos3b5e (molecular weight) in all python environments from 3.8 to the latest maintained by Ersilia. The model will be fetched from github by default and a shallow test will also be performed (see section). More granularity can be specified using the built-in flags (for python versions and environments) as well as the flags specific to the Ersilia CLI:

🚀
model test