Environment Tools: pyenv & pipx

Before we get into the heavy duty tools, I wanted to mention two tools that are used independently of virtual environments: pyenv and pipx.

If you have been programming in python for a while, you probably have many projects on your computer. Most likely, each of these projects use different versions of Python. At this point you encounter the first pain point as a developer - how to easily switch versions between projects.

Pyenv

pyenv to the rescue. pyenv allows us to manage our python installations, easily installing different versions and switching between them. So when you need Python 3.7 for a work project, but want to play around with Python 3.12 on your side project then pyenv can help you do that.

Get pyenv from the link below.

GitHub - pyenv/pyenv: Simple Python version management

Simple Python version management. Contribute to pyenv/pyenv development by creating an account on GitHub.

GitHubpyenv

There is one problem if you are working on Windows - pyenv does not support Windows (though you can still use it on Windows Subsystem for Linux if you work in that environment). But not to worry; there is a fork called pyenv-win that provides the same functionality for windows devs.

GitHub - pyenv-win/pyenv-win: pyenv for Windows. pyenv is a simple python version management tool. It lets you easily switch between multiple versions of Python. It’s simple, unobtrusive, and follows the UNIX tradition of single-purpose tools that do one thing well.

pyenv for Windows. pyenv is a simple python version management tool. It lets you easily switch between multiple versions of Python. It's simple, unobtrusive, and follows the UNIX tradition of s...

GitHubpyenv-win

Pyenv Usage

Using pyenv is quite straightforward.

• Use pyenv install xyz to install Python version xyz. You can install multiple versions of Python this  way.
• When you want to select a particular python version, you can do pyenv global xyz to set the default version  of python globally,
• Another useful feature is to use pyenv local xyz4 to set the python version to be used for the current directory. This makes it really easy to use different versions of python as required by each project
• And if you need to just temporarily use a version of python, you can use the command pyenv shell xyz to get a shell configured with that version of python. The old configuration will still be in use outside the shell

pipx

Another situation that comes up from time to time is when you want to run a command line tool written in python.

To do this you would normally do a pip install <tool> into the global python site-packages folder. But that tool might in turn require dependencies that also need to be downloaded and installed, and this can end up with many packages getting installed in the global site-packages

Having many packages installed globally is problematic because you might end up with version conflicts with other dependencies for other tools. You can also mistakenly end up importing the wrong package than what you indended to.

The best solution is if you could create a separate virtual environment for each tool so that they would all be isolated from each other. This is exactly what pipx does.

pipx

execute binaries from Python packages in isolated environments

Using pipx is just like using pip. You do pipx install <tool> and that will create a new virtualenv just for this tool. The tool and all its dependencies will be installed into this virtualenv. Finally, the tool's binary will be added to the Path variable so that it can be run just as if it was installed globally.

Difference between pip and pipx

It might look like pip and pipx do the same thing. You say pip/pipx install <dependency> and ig will download and install the dependency. While they look similar, the intent for both tools is quite different.

pip will download the package and all its dependencies into the current environment (whether global or within an activated virtual environment)

pipx will create a new virtual environment and download the tool and its dependencies into the newly created environment. So every tool will have its own environment and will be isolated from other tools as well as the project environment.

pip is mainly used to install packages that are required for your project. pipx is used to install command line tools that live outside of any project environment.

Summary

There is nothing more frustrating than facing hard to debug bugs that turn out to be due to dependencies from different projects interfering with each other. pyenv and pipx are two tools that are super useful for setting up your python environments in a clean way.