1. Introduction¶

SciExp²-ExpRun (aka Scientific Experiment Exploration - Experiment Running) provides a framework for easing the workflow of executing experiments that require orchestrating multiple processes in local and/or remote machines.

The contents of the framework are organized into:

An extension of spur, to run processes on both local and remote machines, with improvements on error handling and cleanup.
Various helper functions to manage package installation, file copying, process management (pinning, setting limits, etc.), or CPU frequency and interrupt configuration, among others.

2. Installing¶

The simplest way to install SciExp²-ExpRun is using the official package [1] and a virtual environment [2]. This will ensure you have a controlled local installation of a specific version for each of your projects, making sure your scripts always use the same SciExp²-ExpRun version to avoid any possible future version incompatibilities.

Create a virtual environment:
```
python3 -m venv ~/my-exprun
```

You can create as many as you want; e.g., one for each different version of the SciExp²-ExpDef package that you want to test.

Install SciExp²-ExpRun:

. ~/my-exprun/bin/activate
pip install sciexp2-exprun

Or to install a specific version (version 0.0.0 in the example):

. ~/my-exprun/bin/activate
pip install "sciexp2-exprun==0.0.0"

[1]	https://pypi.org/project/sciexp2-exprun

[2]	https://docs.python.org/tutorial/venv.html

3. Running¶

3.1. SciExp²-ExpRun and Make¶

In you use make to automate the execution of your scripts, here is a simple snippet you can add to your Makefile (the example uses version 0.0.0 of SciExp²-ExpRun):

all: deps/exprun
      # run our script using exprun
      ( . deps/exprun/bin/activate && /path/to/my/script.py )

# the ".done" file ensures a partial installation will not count as a success
deps/exprun: deps/exprun/.done
      $(RM) -R $@
      mkdir -p $(dir $@)
      python3 -m venv $@
      ( . $@/bin/activate && pip install "sciexp2-exprun==0.0.0" )
      touch $<

3.2. Debugging aids¶

You can start your scripts with ipython --pdb /path/to/my/script.py to get into a debugging shell whenever an error occurs [3]. Sometimes it is also useful to start an IPython shell [4] somewhere deep in your code to interactively evaluate its current state:

from IPython import embed
embed()

Exiting the debugging shell will returning to normal execution.

[3]	http://ipython.org/ipython-doc/stable/interactive/reference.html#post-mortem-debugging

[4]	http://ipython.org/ipython-doc/stable/interactive/reference.html#embedding-ipython