Creating a virtual environment with Anaconda

Anaconda

Anaconda comes with Conda and Anaconda navigator. Conda is an open source package management system and environment management system that runs on Windows, macOS and Linux. Conda helps to find and isntall packages.

Download anaconda installer and install anaconda

• Latest Anaconda installers are available for download

List of packages

• Anaconda package list is available for users to review
• From DS and ML point of view, considering a latest conda version for 64-bit Windows with Python 3.7:
  • Installer Built in packages: async_generator, beautifulsoup4, bkcharts, bokeh, curl, cython, dask, flask, hdf5, ipython, jedi, jupyter, matplotlib, networkx, nltk, numpy, pandas, pathlib2, pep8, pytest, regex, scipy, sqlalchemy, tqdm
  • Available packages: arrow, azure, blaze, chainer, coverage, cudnn, dask-ml, django, elasticsearch, fastparquet, git, graphviz, mysql, nginx, nodejs, opencv, plotly, pyspark, pytorch, pytorch-gpu, scikit-learn, seaborn, torchvision, tensorflow, tensorflow-gpu, teradata, virtualenv

Creating conda environment

• Anaconda guide for creating a python virtual environment can be found here
• Once acustomed with conda, users can refer to cheet sheet for quick reference.

Example For creating conda environment with pytorch on windows 10

Create conda venv

I want to create virtual environment by name ‘torch’, as I wish to install pytorch. Type following command in command prompt.

C:\Users\rasik>conda create --name torch

conda asks for permission to proceed

## Package Plan ##
  environment location: C:\Users\rasik\Anaconda3\envs\torch
Proceed ([y]/n)? y

On successful creation, conda prompts user to activate new environment

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
# To activate this environment, use
#     $ conda activate torch
# To deactivate an active environment, use
#     $ conda deactivate

After activation, user can find out externally installed packages. As expected, it is empty.

C:\Users\rasik>conda activate torch
(torch) C:\Users\rasik> conda list -n torch
# packages in environment at C:\Users\rasik\Anaconda3\envs\torch:
# Name                    Version                   Build  Channel

Install pytorch [for deep Neural Networks]

(torch) C:\Users\rasik>conda install pytorch
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
  environment location: C:\Users\rasik\Anaconda3\envs\torch
  added / updated specs:
    - pytorch
The following packages will be downloaded:
    package                    | build
    ---------------------------|-----------------
    _pytorch_select-0.1        | cpu_0           4 KB
....
    pytorch-1.6.0              | cpu_py37h538a6d7_0       101.1 MB
    vs2015_runtime-14.27.29016 | h5e58377_2        1007 KB
    wheel-0.36.2               | pyhd3eb1b0_0          33 KB
    ------------------------------------------------------------
                                           Total:       123.0 MB

The following NEW packages will be INSTALLED:
  _pytorch_select  pkgs/main/win-64::_pytorch_select-0.1-cpu_0
  blas             pkgs/main/win-64::blas-1.0-mkl
....
....
  pytorch          pkgs/main/win-64::pytorch-1.6.0-cpu_py37h538a6d7_0
  wincertstore     pkgs/main/win-64::wincertstore-0.2-py37_0
  zlib             pkgs/main/win-64::zlib-1.2.11-h62dcd97_4

Proceed ([y]/n)? y
Downloading and Extracting Packages
mkl_random-1.0.4     | 287 KB    | #################### | 100%
_pytorch_select-0.1  | 4 KB      | #################### | 100%
mkl-service-2.3.0    | 45 KB     | #################### | 100%
....
pytorch-1.6.0        | 101.1 MB  | #################### | 100%
libmklml-2019.0.5    | 17.5 MB   | #################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

Launch Jupyter notebook

(torch) C:\Users\rasik>jupyter notebook
    To access the notebook, open this file in a browser:
        file:///C:/Users/rasik/AppData/Roaming/jupyter/runtime/nbserver-21512-open.html
.....
[W 16:21:25.895 NotebookApp] 404 GET /nbextensions/widgets/notebook/js/extension.js?v=20210109162058 (::1) 2.99ms referer=http://localhost:8888/notebooks/Untitled.ipynb?kernel_name=python3
[I 16:23:25.811 NotebookApp] Saving file at /Untitled.ipynb

Handling error in loading environment

Often we note that despite launching jupyter from virtual environment, jupyter launches in base environment

• So, as noted below, import torch is not recognised inside a newly opened jupyter notebook
• This happens because jupyter kernel attached with new conda environment does not exist. To overcome this situation, we have to manually add a kernel with a different version of Python or a virtual environment.

Install ipykernel manually

(torch) C:\Users\rasik>pip install --user ipykernel
Collecting ipykernel
  Downloading ipykernel-5.4.2-py3-none-any.whl (119 kB)
     |████████████████████████████████| 119 kB ...
Collecting ipython>=5.0.0
  Downloading ipython-7.19.0-py3-none-any.whl (784 kB)
     |████████████████████████████████| 784 kB ...
.....
.....     

Solve common hurdles

Add conda environment to Jupyter kernel

• As noticed, after running following commands, new kernel “torch” is detected and command import torch works successfully

  (torch) C:\Users\rasik>python -m ipykernel install --user --name=torch
  Installed kernelspec torch in C:\Users\rasik\AppData\Roaming\jupyter\kernels\torch
  

  

Add Code completion on TAB

Sometimes, code completion/ suggestions with TAB does not work Follwing workaround works in most cases:

pip install jupyter_contrib_nbextensions
jupyter contrib nbextension install --user
jupyter nbextensions_configurator enable --user

• If jedi is installed and user wants to keep it, it may cause problem with autocompletion.
• Include following codeline in your notebook to ignore jedi

  %config Completer.use_jedi = False
  

Conda environment - common commands

Update all environment packages

conda update -n name_of_environment --all