Python how to install requirements

Use requirements.txt

Define requirements

Select the method of handling versions of the required libraries. The version numbers can be defined:

Greater or equal

Define the requirements management policy:

Remove unused requirements

Deletes records that correspond to unused libraries and packages.

Modify base files

Allows modifications in the base requirements files (if any is referenced in the requirements.txt file).

Keep existing version specifier if it matches the current version

Leaves the version number unchanged if it satisfied the selected method versions handling.

Click OK and inspect the generated file.

You can also run pip freeze > requirements.txt in the command line to generate a requirements.txt file for your project. See https://pip.pypa.io/en/stable/reference/pip_freeze/ for more details.

If the name of the requirements file differs from requirements.txt or when you have several requirements files in one project, you have to notify PyCharm about the requirements file you want to apply.

Configure the default requirements file

In the Package requirements file field, type the name of the requirements file or click the browse button and locate the desired file.

Click OK to save the changes.

Though you can always run the Sync Python Requirements to update the requirements file, PyCharm provides quick fixes that enable populating this file.

Update a requirements file

In an import statement of a Python file, click a package which is not yet imported. PyCharm suggests a quick-fix:

Select and apply the suggested quick-fix. The package is added to the dependency management file.

PyCharm provides quick fixes and notifications related to the unsatisfied dependencies.

Install the required packages

The notification bar is displayed when the Unsatisfied package requirements inspection is enabled. You can enable it in the Preferences/Settings | Editor | Inspections dialog.

Open a project with the requirements file specified, a notification bar is displayed on top of any Python or requirements file opened in Editor :

Click one of the provided links to satisfy or ignore requirements.

If you have selected the Ignore option, you can always change your mind, and remove the package from the list of the ignored packages.

Add the ignored dependencies

Preview the list of the ignored requirements and click the Add icon () to add them.

Better Python dependency while packaging your project

I have been cooking this blog topic idea for a long time. I searched, read, and tried a lot while working on different projects. But even today after publishing it I don’t think I’m 100% satisfied with the provided solution on how to manage python project dependencies efficiently.

What is package and dependency management?

Software is released in bundled packages; this way, it’s easier to manage installed programs.

The package manager is a collection of libraries packaged together, making it easier to download the entire package rather than each library.

Almost every library in the package has a dependency managed by the d ependency manager.

Dependency management helps manage all the libraries required to make an application work. It’s incredibly beneficial when you’re dealing with complex projects and in a multi-environment. Dependency management also helps to keep track, update libraries faster and easier, and solve the problem then one package will depend on another package.

Every programming language has its flavor of dependency manager.

To summarize all the above :

Typical way of managing project dependency today

Today the most used Python package manager is pip, used to install and manage python software packages, found in the Python Package Index. Pip helps us, python developers, effortlessly “manually” control the installation and lifecycle of publicly available Python packages from their online repositories.

Pip also can upgrade, show, uninstall project dependencies, etc.

To install the package, you can just run pip install that will build an extra Python library in your home directory.

Running pip freeze, can help check installed packages and packages versions listed in case-insensitive sorted order.

Project setup

After building your application, you will need to perform some set of actions(steps) to make application dependencies available in the different environments.

Actions will be similar to the one below:

Install project dependencies

. But how informative is the output?

How can project dependencies be easily maintained?

Personally, I think the above setup is not easy to maintain for a variety of reasons:

Are there any better alternatives?

Option 1: multiple requirements.txt files?

There are many examples of projects with multiple requirements.txt files. Developers have different versions of requirements.txt file for example for different environments (e.g., test or local ) or files for different users (e.g., machines vs. people).

Multiple requirements.txt is a good solution for managing project dependencies? I disagree…managing manually various requirements.txt files is not a good solution, and it will be not easy if they grow more than even

Option 2: can Pipreqs and Pipdeptre make it better?

I recently tried pipreqs utility, which generates requirements.txt file based on project imports. It’s simple to use.

To develop a requirements.txt file, you can run pipreqs /your_project/path

Pipdeptree

I thought of combining it with pipdeptree another cool and “handy” command-line utility that will help to display the installed python packages in the form of a dependency tree.

After executing pipdeptree command in your terminal window in the virtualenv directory of the project, all the installed python packages of a dependency tree will be displayed like so:

Cool bonus that pipdeptree will warns you when you have multiple dependencies where versions don’t exactly match.

I found it’s handy in some cases, like:

There are some downsides too, Pipreq will not include the plugins required for specific projects, and you will end up adding plugins information manually in a requirement.txt. It’s not yet a very mature utility.

Option 3: have you tried pip-compile?

-generate-hashes flag helps to generate-hashes. In this case pip-compile consults the PyPI index for each top level package required, looking up the package versions available.

pip-sync command used to update your virtual environment to reflect precisely what’s in there. Command will install/upgrade/uninstall everything necessary to match the requirements.txt contents.

Software dependencies are often the largest attack surface

To know what dependencies your project contains is very useful to find out after the fact what packages were installed and what dependencies your project has.

Organizations usually assume most risks come from public-facing web applications. That has changed. With dozens of small components in every application, threats can come from anywhere in the codebase.

Currently using pip freeze will only show the final package list of dependencies.

I would recommend using pipdeptreea module which helps to find possible dependency conflicts and display an actual dependency in the project.

Another good piece of advice is to start building applications using Docker. Every tool we run in Docker is one less tool we have to install locally, so get up-and-running phase will be much faster. But that is a different topic.

Управление необходимыми пакетами Python с помощью requirements.txt

Если вы загрузили проект, содержащий файл requirements.txt, и хотите установить все указанные в нем пакеты, разверните узел Среды Python в обозревателе решений, щелкните правой кнопкой мыши узел среды и выберите Установить из файла requirements.txt:

Если вы хотите установить зависимости в виртуальном окружении, сначала создайте и активируйте окружение, а затем воспользуйтесь командой Установка из файла requirements.txt. Дополнительные сведения о создании виртуального окружения см. в статье Выбор окружения Python для проекта.

Если в среде установлены все необходимые пакеты, можно щелкнуть среду правой кнопкой мыши в обозревателе решений и выбрать Создать файл requirements.txt, чтобы создать необходимый файл. Если файл уже существует, отображается запрос с вариантами обновления:

requirements.txt vs setup.py in Python

Understanding the purpose of requirements.txt, setup.py and setup.cfg in Python when developing and distributing packages

Introduction

Managing dependencies in Python projects could be quite challenging, especially for people new to the language. When developing a new Python package, the chances are you will also need to utilise some other packages that will eventually help you write less code (in less time) so that you don’t have to re-invent the wheel. Additionally, your Python package may also be used as dependency in future projects.

In today’s article we will discuss about how to properly manage dependencies for Python projects. More specifically, we will discuss about the purpose of requirements.txt file and how to use setuptools in order to distribute your Python package and let other users develop it further. Therefore, we will also be discussing about the purpose of setup files (namely setup.cfg and setup.py ) and how they can be used along with requirements file in order to make package development and redistribution easier.

What are dependencies of Python projects

First, let’s start our discussion with package dependencies; what they really are and why it’s important to manage them properly in order for your Python project to be maintained easier.

In very simple terms, dependencies are external Python packages that your own project relies onto, in order to do the job is intended to. In the context of Python, these dependencies are usually found on the Python Package Index (PyPI) or in other repository management tools, such as Nexus.

As an example, let’s consider a Python project that makes use of pandas DataFrames. In this case, this project has a dependency on pandas package since it cannot work properly without pre-installing pandas.

Every dependency — which is in turn a Python package on its own — may also have other dependencies. Therefore, dependency management can sometimes get quite tricky or challenging and needs to be handled properly in order to avoid issues when installing or even enhancing the package.

The most common way for handling dependencies and instructing package management tools about what specific versions we need in our own project is through a requirements text file.

The requirements.txt file

The requirements.txt is a file listing all the dependencies for a specific Python project. It may also contain dependencies of dependencies, as discussed previously. The listed entries can be pinned or non-pinned. If a pin is used, then you can specify a specific package version (using == ), an upper or lower bound or even both.

Finally, you could install these dependencies (normally in a virtual environment) through pip using the following command:

In the example requirements file above, we specified a few dependencies using various pins. For example, for the pandas package that has no pin associated to it, pip will normally install the latest version, unless one of the other dependencies has any conflict with this (in this case pip will normally install the latest pandas version that satisfies the conditions specified by the remaining dependencies). Now going forward, for pytest the package manager will install the specific version (i.e. 4.0.1 ) while for matplotlib, will install the latest version, which is at least greater or equal than 2.2 (again this depends on whether another dependency specifies otherwise). Finally, for numpy package, pip will attempt to install the latest version between versions 1.15.0 (inclusive) and 1.21.0 (non-inclusive).

The requirements file is extremely useful but in most of the cases, it must be used for development purposes. If you are planning to distribute your package so that it is widely available (say on PyPI), you may need something more than just this file.

setuptools in Python

setuptools is a package built on top of distutils that allows developers to develop and distribute Python packages. It also offers functionality that makes dependency management easier.

When you want to release a package, you normally need some metadata including the package name, version, dependencies, entry points, etc. and setuptools offers the functionality to do exactly this.

The project metadata and options are defined in a setup.py file, as shown below.

In fact, this is considered to be a somewhat bad design given that the file is purely declarative. Therefore, a better approach is to define these options and metadata in a file called setup.cfg and then simply call setup() in your setup.py file. An example setup.cfg file is illustrated below:

and finally, you can have a minimal setup.py file:

requirements.txt vs setup.py

I started working with Python. I’ve added requirements.txt and setup.py to my project. But, I am still confused about the purpose of both files. I have read that setup.py is designed for redistributable things and that requirements.txt is designed for non-redistributable things. But I am not certain this is accurate.

How are those two files truly intended to be used?

4 Answers 4

Trending sort

Trending sort is based off of the default sorting method — by highest score — but it boosts votes that have happened recently, helping to surface more up-to-date answers.

It falls back to sorting by highest score if no posts are trending.

Switch to Trending sort

requirements.txt :

This helps you to set up your development environment.

Programs like pip can be used to install all packages listed in the file in one fell swoop. After that you can start developing your python script. Especially useful if you plan to have others contribute to the development or use virtual environments. This is how you use it:

It can be produced easily by pip itself:

pip automatically tries to only add packages that are not installed by default, so the produced file is pretty minimal.

setup.py :

This helps you to create packages that you can redistribute.

The dependencies of your project are listed in both files.

Because both files list dependencies, this can lead to a bit of duplication. Read below for details.

requirements.txt

If multiple requirements-X.txt variants are present, then usually one will list runtime dependencies, and the other build-time, or test dependencies. Some projects also cascade their requirements file, i.e. when one requirements file includes another file (example). Doing so can reduce repetition.

setup.py

Do I really need both?

Short answer is no, but it’s nice to have both. They achieve different purposes, but they can both be used to list your dependencies.