Python virtual environments for scientists with conda part2
In our previous post we learned about Python virtual environments and how conda makes it easy for us as scientists to use virtual environments and access hundreds of useful packages.
Despite the number of packages directly available through
conda, not all packages can be downloaded this way. In this post we will learn how to add such packages to our Python virtual environments.
What if conda cannot find the package I am looking for? Conda channels
Thankfully, there is a large community of Anaconda users that help fill the gap by providing different channels from which we can download our desired package. Think of channels as branches of the Anaconda library. The same Python packages can be contained in several channels; pick a channel that contains the version of the package that is compatible with your system (32-bit vs 64-bit; linux, win, os-x), preferably one that has been downloaded many times as this provides some reassurance that the package works reliably.
As an example, lets pretend we wanted to include the
pygame package to our environment. Note that we start by activating the virtual environment we created in our previous post – it is called
(base) $ conda activate sci_sound (sci_sound) $ conda install pygame Collecting package metadata: done Solving environment: failed PackagesNotFoundError: The following packages are not available from current channels: - pygame Current channels: - https://conda.anaconda.org/conda-forge/linux-64 - https://conda.anaconda.org/conda-forge/noarch - https://repo.anaconda.com/pkgs/main/linux-64 - https://repo.anaconda.com/pkgs/main/noarch - https://repo.anaconda.com/pkgs/free/linux-64 - https://repo.anaconda.com/pkgs/free/noarch - https://repo.anaconda.com/pkgs/r/linux-64 - https://repo.anaconda.com/pkgs/r/noarch To search for alternate channels that may provide the conda package you're looking for, navigate to https://anaconda.org and use the search bar at the top of the page.
As you can see,
pygame is not available for direct download. But if we navigate to
https://anaconda.org and search for
pygame, we see the following search results:
If we click the most popular channel, called
CogSci, we see the following information:
We need to run
conda install -c cogsci pygame to install pygame. Let’s do that!
(sci_sound) $ conda install -c cogsci pygame Collecting package metadata: done Solving environment: done ## Package Plan ## environment location: /home/martin/anaconda3/envs/sci_sound added / updated specs: - pygame The following packages will be downloaded: package | build ---------------------------|----------------- numpy-1.16.3 | py27he5ce36f_0 4.2 MB conda-forge pip-19.1 | py27_0 1.8 MB conda-forge setuptools-41.0.1 | py27_0 650 KB conda-forge ------------------------------------------------------------ Total: 6.7 MB The following NEW packages will be INSTALLED: pygame cogsci/linux-64::pygame-1.9.2a0-py27_0 The following packages will be DOWNGRADED: certifi 2019.3.9-py37_0 --> 2019.3.9-py27_0 numpy 1.16.3-py37he5ce36f_0 --> 1.16.3-py27he5ce36f_0 pip 19.1-py37_0 --> 19.1-py27_0 python 3.7.3-h5b0a415_0 --> 2.7.15-h721da81_1008 setuptools 41.0.1-py37_0 --> 41.0.1-py27_0 wheel 0.33.1-py37_0 --> 0.33.1-py27_0 Proceed ([y]/n)?
Who-oh! That is more than we bargained for. Do you see what went wrong? The package that we were going to download is listed as:
The key thing to notice is the
py27_0 part. It means the
pygame package we were about to download was written to work with Python 2.7. As you may or may not known, the creators of Python made a breaking change from Python 2 to Python 3. That means that packages coded using Python 2 often won’t work when they are run using Python 3. This is why, to create a virtual environment that is compatible with the version of
pygame we asked
conda to download,
conda is telling us that it has to downgrade several packages, including Python! Specifically,
conda wants to downgrade Python from version 3.7.3 to version 2.7.15.
No one from the Anaconda community has included a version of
pygame in one of the channels that is compatible with Python 3. While we could change everything over to Python 2.7, this is not a great solution.
Is there a way to easily obtain a version of
pygame that is compatible with Python 3?
What if conda cannot find the package I am looking for? Pip install
pip is a package-management system used to install and manage software packages written in Python. Many packages can be found in the default source for packages and their dependencies — Python Package Index (PyPI). The nice thing of working with
Anaconda is that, once we have activated our Python virtual environment,
conda is aware of
pip and packages that are installed using it.
Thus, to install a Python 3 compatible version of
pygame, we can run a simply
(sci_sound) $ pip install pygame Collecting pygame Downloading https://[...]/pygame-1.9.6-cp37-cp37m-manylinux1_x86_64.whl (11.4MB) |████████████████████████████████| 11.4MB 20.6MB/s Installing collected packages: pygame Successfully installed pygame-1.9.6
In this post we learned how to look for a Python package that is not directly available throught the
conda install command. Specifically, we learned about
https://anaconda.org and how to search and (almost) install packages available in
conda channels. Finally, we learned how to install Python packages using the
pip command. In our next post, we will review how to create a new conda environment, and learn how to see what environments exists on our system, see what packages are install in each environment, and also delete unwanted environments.