Parsl - Parallel Scripting Library
Parsl extends parallelism in Python beyond a single computer.
You can use Parsl just like Python's parallel
executors
but across multiple cores and nodes. However, the real power of Parsl
is in expressing multi-step workflows of functions. Parsl lets you chain
functions together and will launch each function as inputs and computing
resources are available.
import parsl
from parsl import python_app
# Start Parsl on a single computer
parsl.load()
# Make functions parallel by decorating them
@python_app
def f(x):
return x + 1
@python_app
def g(x):
return x * 2
# These functions now return Futures, and can be chained
future = f(1)
assert future.result() == 2
future = g(f(1))
assert future.result() == 4
Start with the configuration
quickstart
to learn how to tell Parsl how to use your computing resource, then
explore the parallel computing
patterns
to determine how to use parallelism best in your application.
Quickstart
Parsl is now available on PyPI, but first make sure you have Python3.7+
:
$ python3 --version
Install Parsl using pip:
$ pip3 install parsl
To run the Parsl tutorial notebooks you will need to install Jupyter:
$ pip3 install jupyter
Detailed information about setting up Jupyter with Python is available
here
Note: Parsl uses an opt-in model to collect anonymous usage statistics
for reporting and improvement purposes. To understand what stats are
collected and enable collection please refer to the usage tracking
guide
Documentation
The complete parsl documentation is hosted
here.
The Parsl tutorial is hosted on live Jupyter notebooks
here
For Developers
-
Download Parsl:
$ git clone https://github.com/Parsl/parsl -
Build and Test:
$ make # show all available makefile targets $ make virtualenv # create a virtual environment $ source .venv/bin/activate # activate the virtual environment $ make deps # install python dependencies from test-requirements.txt $ make test # make (all) tests. Run "make config_local_test" for a faster, smaller test set. $ make clean # remove virtualenv and all test and build artifacts -
Install:
$ cd parsl $ python3 setup.py install -
Use Parsl!
Requirements
Parsl is supported in Python 3.7+. Requirements can be found
here. Requirements for running tests can be found
here.
Code of Conduct
Parsl seeks to foster an open and welcoming environment - Please see the
Parsl Code of
Conduct for more
details.
Contributing
We welcome contributions from the community. Please see our
contributing
guide.