Compute Environments

The Fornax Science Console offers several pre-installed environments. The specific environments that are available depend on the Base Environment (container image) you selected when starting your server. All container images use an x86_64 Ubuntu Linux system. Each comes pre-installed with JupyterLab extensions as well as one or more Python software environments. You can customize your experience by installing additional Python software in the pre-installed environments as well as creating new environments, installing additional JupyterLab extensions, and installing non-Python software. This page describes the details.

Working with Python Environments¶

python3 is the default Python environment. It has general astronomy and plotting software.

Each of the Fornax demo notebooks has its own environment with a name of the form py-{notebook-name} (e.g. py-light_curve_collector and py-multiband_photometry). Each environment has the packages required to run the notebook pre-installed (see View Pre-installed Software). When opening the notebook, the corresponding kernel should automatically start. You can also select it from the drop down kernel menu at the top-right of an open notebook.

To activate a specific pip-based environment (see Environment Types for details) from the terminal, run: source $ENV_DIR/{environment-name}/bin/activate. For example, to activate the py-light_curve_classifier environment, run:

source $ENV_DIR/py-light_curve_classifier/bin/activate

and the following to deactivate it:

deactivate

Environment Types¶

There are two types of Python environments, pip-based and conda-based.

pip-based

The pip-based environments use uv to manage the packages. These environments contain pip-installable packages and are used in most cases. The default environments are installed under $ENV_DIR. They are activated as indicated above with source $ENV_DIR/{env-name}/bin/activate.

conda-based

These use micromamba to manage the packages (similar to conda/mamba): The conda-based environments are used with packages that are not pip-installable. Examples of this include heasoft and ciao in the high-energy container image. These are activated with micromamba activate {env-name} and deactivated with micromamba deactivate. These are also installed under $ENV_DIR.

Pre-installed Environments¶

The following environments are pre-installed:

python3

This is the default Python environment. It has general astronomy and plotting software.

py-{notebook-name}

Each of the Fornax demo notebooks has its own environment with a name of the form py-{notebook-name} (e.g. py-light_curve_collector and py-multiband_photometry). Each environment has the packages required to run the notebook pre-installed. When opening the notebook, the corresponding kernel should automatically start. You can also select it from the drop down kernel menu at the top-right of an open notebook.

See View Pre-installed Software to learn about specific libraries each environment contains.

Select an Environment¶

Notebook: To activate a specific environment from a notebook, click on the name of the notebook’s current environment at the top right and then select your desired environment from the kernel drop down menu. If you open a Fornax demo notebook and get a popup window asking you to select a kernel, choose the kernel from the drop down menu with the same name as the notebook you are opening. If you open any other notebook and get a popup window asking you to select a kernel, python3 is usually the best choice.

Terminal: To activate a specific environment from the terminal, run: source $ENV_DIR/{environment-name}/bin/activate. For example, to activate the py-light_curve_classifier environment, run:

source $ENV_DIR/py-light_curve_classifier/bin/activate

and the following to deactivate it:

deactivate

Install Additional Software¶

To install additional Python software, you can either update an existing environment or create a new one.

Update an Existing Environment¶

To add packages to a currently installed environment, you install them with pip (or the faster uv pip) after activating the relevant environment.

• Inside a notebook running the relevant environment, run !uv pip install ..., passing the extra packaged needed.
• In the terminal, after activating the environment run: uv pip install ....

Note that packages installed this way are added in the $ENV_DIR folder, and therefore are not saved for the next session. To ensure the packages are available in the next session, you can install them in your home directory by adding --user to the pip command.

Note also that if the container image is updated, packages installed with the --user option may not be compatible with the new image and package conflicts may arise. The solution is this case is to create your own environments that are independent of the container image (next section).

Create a New Environment¶

To create a new environment that persists between sessions, create a folder in your home directory where user environments will be installed. Say mkdir ~/user-envs. Then inside that folder run the following to create an environment:

cd ~/user-envs
uv venv myenv --python=3.11
source myenv/bin/activate
# add numpy for example
uv pip install "numpy<2"

This will create a new environment with Python version 3.11, activate it, and then install a “numpy<2”.

In order to use this new environment in a notebook, you’ll need to install ipykernel inside the environment and then register it with JupyterLab.

uv pip install ipykernel
python -m ipykernel install --name myenv --user

The kernel should show up in the JupyterLab main launcher page and in the kernel selection dropdown menu inside a running notebook.

The same can be done for conda environments. A conda environment can be created by:

micromamba create -p ~/user-envs/my-conda-env python=3.12 pandas
micromamba activate -p ~/user-envs/my-conda-env

Similarly, to use this environment in a notebook, you’ll need to install ipykernel and register it with JupyterLab like for pip-installed environments.

Note: It is recommended that you remove user environments that are no longer needed, as they may deplete your home storage.

JupyterLab Extensions¶

Pre-installed extensions are described on the JupyterLab page.

Install a New Extension¶

Instructions on how to find and install extensions can be found at JupyterLab: Extensions. Extensions may include a front-end component, a server-side component, or both. You can install front-end extensions after JupyterLab starts, and they can show up if you refresh the page, as long they are installed in the environment running JupyterLab (/opt/jupyter/). Extensions that include a server-side component cannot be installed by individual users because they must be installed before JupyterLab starts. In that case, please open a helpdesk request on the Fornax Community Forum.

Compilers and General Software¶

As part of the system optimization and to allow for users to manage their own software, the list of packages installed in the system (using ubuntu apt) is kept to a minimum. Many of the useful packages (vim, htop, git, awscli, etc) are installed from conda-forge into the base conda environment under $ENV_DIR/base. You can add packages to this environment by doing:

micromamba install package_name

You can also include compilers. For example, to install C, C++ and Fortran compilers, you can do:

micromamba install c-compiler cxx-compiler fortran-compiler

For non-Python tools (e.g. htop, vim etc), they can be run directly from the terminal without a need for activating the base environment as they are included in the PATH by default.