Workspaces and libraries in Python

PIP

pip is the native Python package manager. It is used to install packages from several repositories, mainly pypi.org. It should be already on your system with Python 2 >=2.7.9 or Python 3 >=3.4. If you don't have it, follow the installation instructions on the official website, or install it with a conda distribution.

Basic commands

Install package

$ pip install <pkg>

Upgrade a package

$ pip install --upgrade <pkg>

Uninstall a package

$ pip uninstall <pkg>

Show install locations and files

$ pip show --files <pkg>

Export all installed packages

$ pip freeze > requirements.txt

Install from list

$ pip install -r requirements.txt

Workspace separation

For working in Python on multiple projects one inevitably faces the problem of having to maintain two or more different sets of libraries and interpreters, each of their respective versions, that don't have to be compatible. For solving this issue, the idea of environments was implemented. The environment is basically a set of all dependencies of a project, including an interpreter and all needed libraries. Such an environment is therefore easily manageable separately, so changing a version of the library doesn't break any other project. When running a code “inside”, the only thing needed is then to set all relevant $PATH variables. For using environments, many tools have been developed, so they pose no great additional inconvenience.

There are two mainstream ways for managing Python environments, each effectively presenting its implementation of the idea

virtualenv

virtualenv is a low-level native tool for managing virtual environments. That is all. No rocket science.

The environment is created in a dedicated directory, usually in the root of your project. You basically need only a few commands for setting up and then you turn to pip for package management.

Basic commands

To create a new environment, move to the root of your project directory and run

$ virtualenv env

env stands for the newly created environment directory and it's also a common practice to name it this way

Activating the environment

$ source env/bin/activate

Deactivating

$ deactivate

After activation, you can use pip normally without the need for sudo because it installs the packages locally to the environment.

conda

Conda is an open-source package management system and environment management system. That means that it combines both discussed problems in one tool. It is a popular tool, which is strongly supported and even includes repositories with over 7,500+ packages. The possibility to very easily run and manage your own repositories might not seem important right now (and is definitely not a part of this course), but it is an important business feature that you might encounter someday. For using conda for managing virtual environments, you have two options for installation.

  • Installing Anaconda, which is a complete open-source distribution including about 250+ default packages (like NumPy, SciPy, etc.), which may be a way to go for a scientific project. On the other hand, in many cases, this turns out to be very tiresome and disk-space intensive.
  • Installing Miniconda, which is a distribution containing only the conda and its basic dependencies.

In both of these cases, one gets full powers of conda, the only thing that differs is the set of default packages for environments. Thus we recommend installing Miniconda, which comes with lower overhead. Official installation instructions are here. The prefered version is Miniconda Linux 64-bit and Python 3. To deactivate automatic configuration of default base environment:

$ conda config --set auto_activate_base false

Basic commands

Here we list the most important commands you might need for starting with conda. See the documentation or the cheat sheet for more.

Information about conda and its version

$ conda info

Create a new environment

$ conda create --name <name> <pkgs>

Using an environment

$ conda activate <name>

Switching out of the environment

$ conda deactivate

List packages in the current environment

$ conda list

Search for a package in the current environment

$ conda search <pkg>

Install a package in the current environment

$ conda install <pkg>

List all environments

$ conda info --envs

Delete an environment

$ conda remove --name <name> --all
courses/b4b36zui/materials/python.txt · Last modified: 2021/02/14 14:43 by janisjar