Preface #
I am not an expert. Wrote this section just for completion.
Why not simply pip #
pip install is simple and naive. You will get crazy if you come from JavaScript world where both npm and yarn are excellent package managers. It even takes effort to manage production and development dependecies in two separate list. A tool for handling all the dependencies and virtual env works is handy, and can simplify workflows.
What do they actuallly do #
Pretty much like mentioned above. Quoting from pipenv's doc for some detailed feature:
- Enables truly deterministic builds, while easily specifying only what you want.
- Generates and checks file hashes for locked dependencies.
- Automatically install required Pythons, if pyenv is available.
- Automatically finds your project home, recursively, by looking for a Pipfile.
- Automatically generates a Pipfile, if one doesn’t exist.
- Automatically creates a virtualenv in a standard location.
- Automatically adds/removes packages to a Pipfile when they are installed or uninstalled.
- Automatically loads .env files, if they exist.
Major Problem: Speed #
Resolving and locking dependencies are too slow. A typical jupyterlab workspace takes hours to finish.
Why this happens #
In node package ecosystem a project has hundreds of dependencies to resolve, install and lock, but still faster than pipenv or poetry. That's because in python package ecosystem, many packages are not properly formed and it's impossible to get it's dependencies without downloading it.
See also https://python-poetry.org/docs/faq/#why-is-the-dependency-resolution-process-slow
How to solve in pipenv #
pipenv is slow at locking, so you may want use --skip-lock flag or set PIPENV_SKIP_LOCK env when the network is poor, then pipenv lock when network connection is good.
How to solve in poetry #
Poetry is slow at dependency resolution. Quoting from poetry doc:
At the moment there is no way around it.
Other pipenv problems #
Lock file not cross platform #
Ironically, deterministic builds actually leads to wrong packages installed on different platforms.
Lock files may be version-dependent. Take pytest for example, if you install and lock on python 3.8, the importlib_metadata is not installed because py3.8 already have importlib.metadata included. But when you install based on the lock file on py3.6, importlib_metadata is not found. This is not happening when directly install pytest on py3.6
How to solve #
- Use markers for subdependencies:
importlib_metadata = {version="~=1.7.0", python_version="<'3.8'"} - Or use
pyenv/pyenv-win
apt install may be outdated #
Better install via pip(x)
Script shortcut cannot combine multi commands #
pypa/pipenv#2283
You can write command A && command B in package.json, but not Pipfile. All you can do is write in a shell script.
No python version range #
pypa/pipenv#1050
If you are sure that your program and Pipfile.lock works for python > 3.5.2, remove python requirement 😏
No extra denpendencies #
This can be frustrating if you want uWSGI to be installed only on production machine, but not on your PC.
Other poetry problems #
Prefer python
#
Use poetry with pyenv if python -V outputs 2.7.x. Or it will be an awful develop experience.
Or if you are sure you just need to create venv with system's python3, you can choose pip(x) install it.
No built-in dot-env and handy scripts #
But you can make use of the power of pyproject.toml and use tools like taskipy and poethepoet
Not supported by github dependency graph #
But dependabot does support it.
More info and solutions welcome!