How to create requirements txt python

requirements.txt — что это и зачем?

В исходниках множества Python-проектов можно встретить этот странный текстовый файл. Например, им пользуются urllib3, numpy, pandas, flake8 и куча других проектов. Давайте разберемся, что это такое, как этим пользоваться и зачем нам это нужно.

Гипотетическая предыстория

Давайте представим, что вы написали замечательный скрипт, который спрашивает у пользователя название города и выводит текущую температуру и общее состояние погоды:

Скрипт получился настолько хорош, что вы хотите поделиться им со всеми своими друзьями. К сожалению, друзья при попытке запустить вашу программу получают следующую ошибку:

Кажется, что скинуть только код недостаточно.

Или, допустим, что вы сами через полгода-год попытаетесь запустить эту же программу. За это время вы успели пару раз переустановить Python, переустановить ОС, отформатировать свой магнитный накопитель (используйте SSD — нет, я серьёзно!) или может быть вообще сменили компьютер. Почти уверен, что при запуске скрипта вы получите ровно ту же самую ошибку.

Зачастую, когда мы пишем код, мы полагаемся на какие-либо библиотеки или фреймворки. Это со всех сторон хорошо — это удобно, уменьшает размер программы во много раз, позволяет не думать о мелких деталях, а решать свою конкретную задачу, опираясь на высокоуровневые абстракции. Но, к сожалению, есть «но» — такие библиотеки становятся частью вашей программы, ваш код становится зависим. Это значит, что ваш код больше не сможет работать сам по себе, для его работы должны быть установлены все зависимости.

Я хочу сказать, что намного мудрее составлять этот список зависимостей сразу же и просто поддерживать его в актуальном состоянии по мере развития проекта.

requirements.txt — это список внешних зависимостей

Сообщество Python исповедует идеологию «простое лучше, чем сложное». Наверное, поэтому для хранения списка зависимостей сообщество выбрало самый простой из возможных форматов — текстовый файл, где на каждой строке перечислено ровно по одной зависимости.

Стоит отметить, что requirements.txt не является стандартом, т.е. нет документа, который описывал бы требования к этому файлу. Скорее, это просто распространённая практика в сообществе, которая, наверное, возникла спонтанно и хорошо прижилась.

Вот пример самого простого такого файла (кстати, именно этим файлом можно описать зависимости, которые нужны для запуска нашего скрипта с погодой):

Если бы было несколько зависимостей, то файл выглядел бы так:

Можно указать конкретную версию зависимости. Если версия не указана, то считается, что нужна последняя доступная:

Можно указывать диапазоны и другие более сложные «спецификаторы версий». В целом, в requirements.txt можно писать любые «запросы», которые понимает команда pip install :

Как пользоваться

Команда pip install умеет читать такие файлы, если передать специальный флаг:

Таким образом, если requirements.txt будет иметь вот такое содержимое:

То следующие две команды будут иметь одинаковое действие:

Преимущества использования requirements.txt :

На таком маленьком примере разница может быть не очевидна, но когда список зависимостей разрастётся до определенного размера, то вам не захочется больше перечислять его в pip install напрямую.

Так как это распространённое соглашение, то другим разработчикам будет достаточно увидеть этот файл, чтобы понять, что нужно сделать. Это здорово экономит время на чтении инструкций.

Как создать

Есть два подхода:

Но можно использовать и встроенную в pip функциональность:

Команда pip freeze выводит все установленные в интерпретатор сторонние пакеты. Заметьте, что в список попали не только прямые зависимости ( pyowm ), но и подзависимости — это даже лучше, потому что вы сможете более точно воссоздать окружение по этому файлу.

Можно перенаправить вывод этой команды в файл при помощи стандартного консольного приема (работает и на Windows), и получить валидный файл requirements.txt :

Обратите внимание, что pip freeze выведет список пакетов в том окружении, в котором он запущен. Не забудьте активировать виртуальное окружение перед запуском этой команды, иначе получите список пакетов, установленных в глобальный интерпретатор. Кстати, у меня есть пост про виртуальные окружения, где объясняется как ими пользоваться.

Подытожим плюсы и минусы ручного и автоматического подходов:

Можно использовать и смешанный подход: сгенерировать начальную версию файла при помощи pip freeze и допилить её затем руками, если у вас какая-то сложная нестандартная ситуация.

Проблемы requirements.txt

Почему «хотя бы примерно»? Практика показывает, что зафиксировать версию пакета недостаточно. Иногда случается, что под одной версией пакета в разное время может находиться совершенно разный код. PyPI, конечно, не позволит перезаписать уже опубликованную версию, но, например, ваш приватный корпоративный индекс пакетов может быть не таким строгим.

Заключение

Для тренировки можно попытаться запустить скрипт с погодой. Все исходники лежат здесь.

Дополнительное чтение

Подпишитесь!

Чтобы получить уведомление о новом посте можно:

Create requirements.txt in Python

When developing Python applications, we have to use a bunch of modules for a variety of features. The number of modules being used by an application can be a lot. Generally, when developing such gigantic applications and even the smaller ones, creating a virtual environment specific to the project is recommended because it lets us install whatever we wish to and of whichever version without brimming the global package space.

If our friends, family, or colleagues wish to use the developer on their system, they would also require the code and dependencies installed on their end. Since the dependencies are installed in a virtual environment, sharing the whole virtual environment does not make sense because the folder size will be massive, and they can face errors due to integrity issues.

In such cases, developers add a requirements.txt file to a project containing a list of all the dependencies installed in the virtual environment and the details of the version being used. This way, the borrower or the end-user only has to create a virtual environment and install the dependencies to use the application.

This article will guide us on creating the requirements.txt file and installing dependencies from the requirements.txt file.

Create the requirements.txt Using pip Package Installer

To generate a requirements.txt file, we can use the pip package installer or package management system from the command line. Refer to the following commands for the same.

Please enable JavaScript

Install Dependencies From requirements.txt Using pip Package Installer

Once we have generated a requirements.txt file, we can use this file to install all the dependencies mentioned inside it. Refer to the following command for the same.

Generally, it is recommended to create a virtual environment before starting any new project and installing any dependency. This makes sure that you do not clutter your global package space with random and uncommon packages. The workflow for the same would be as follows.

Refer to the following commands for the same.

How to Use requirements.txt Files in Python

In this tutorial, you’ll learn how to use a requirements.txt file in Python to manage and handle the dependencies of your project. Using a requirements.txt file is particularly helpful when sharing your code with others via source code management tools, such as Github. The file provides the ability to easily track and identify the packages that you use in a project. Similarly, it makes it easy to install all the packages required for a project.

By the end of this tutorial, you’ll have learned:

Table of Contents

What is a requirements.txt file in Python?

This makes it easier to install the required packages, whether for yourself or for other users of your code. It makes it much simpler to review these requirements, rather than looking through different pieces of code to try and piece it all together.

Similarly, when you share your project code with others, they’re able to see (and install, as you’ll learn later) your project’s requirements. This is particularly useful as the file also keeps track of the versions of these libraries, ensuring that the minimum requirements are met!

How to Create a requirements.txt File in Python

When working in a virtual environment, you can simply use the command below:

When you run this code, a new file will be created in the directory you’re working in. This file will list the different packages and modules in your virtual environment. That’s really all there is to it!

In addition, running this code will also include the version numbers in the file.

How to Install All Packages from a requirements.txt File Using pip and Python

One of the benefits of using a requirements.txt file is that users of your code can install all required libraries and modules in one command. This can be really helpful when working in a new virtual environment, meaning that you don’t need to install the libraries one by one.

If a requirements.txt file is present in the root folder of the project, you can run the code below to install all of the packages in the file:

This will read the file and install all the packages in that file. Your terminal will output a list of all the libraries that have been installed and raise any errors if they occur.

How to Maintain a requirements.txt File in Python

Now that you’ve created a requirements.txt file, you may be wondering how you maintain it. If you want to check what packages are out of date, you can use the following command:

This will list out all the packages where your package version is out of date, listing out your version and the most current version.

In order to update to the latest version of a package, you can use the command below:

This will upgrade your version to the latest version. Once this is done, you can refreeze your requirements by re-running the pip freeze > requirements.txt command.

Conclusion

In this tutorial, you learned how to use a requirements.txt file in Python using the pip package manager. You learned why this file is useful in developing your Python programs, especially if you’re sharing them with others via a SCM like Github. You learned how easy it is to create a requirements.txt file using the pip package manager. Then, you learned how to install all packages from the file using a terminal command. Finally, you learned how to maintain the file by checking which packages need updating.

Additional Resources

To learn more about related topics, check out the tutorials below:

The Python Requirements File and How to Create it

Python requirements files are a great way to keep track of the Python modules. It is a simple text file that saves a list of the modules and packages required by your project. By creating a Python requirements.txt file, you save yourself the hassle of having to track down and install all of the required modules manually.

In this article, we will learn how to create Python requirements files along with the best practices and the benefits of using them. It’s often used together with virtual environments in Python projects, but that is outside the scope of this article.

Before we go into the details on how to create a Python requirements file, make sure to check our list of Python IDEs and code editors here if you are serious about learning Python. It makes your life easier and increases your productivity.

Using a Python requirements file comes with a lot of benefits.

First, it allows you to keep track of the Python modules and packages used by your project. It simplifies the installation of all of the required modules on any computer without having to search through online documentation or Python package archives. It is used to install all of the dependencies on another computer so that they are compatible with one another.

Second, it makes it easy to share your project with others. They install the same Python modules you have listed in your requirements file and run your project without any problems.

Third, if you ever need to update or add a Python module to your project, you simply update the requirements file rather than having to search through all of your code for every reference to the old module.

Next, let’s learn how to create one!

How to Create a Python Requirements File

There’s not much else we need to do to create a Python requirements file at this point, but we will cover how to install specific packages manually in the terminal.

You can also generate a Python requirements.txt file directly from the command line with:

pip freeze outputs a list of all installed Python modules with their versions.

Adding Modules to Your Python Requirements File

Now that we have created a Python requirements file, it’s time to start adding some modules! The first step is to open the text document and add the names of the modules you would like to install.

For example, if I want to install the tensorflow library into my project, I type in tensorflow on its own line along with the required version. Let’s type an example in your newly created Python requirements.txt file:

Once you add all of the modules you need, save the document and exit!

Installing Python Packages From a Requirements File

Now that our Python requirements file is all set up, let’s take a look at how to install packages from it. To do this, we will use the pip package manager.

The pip utility is used to install, upgrade, and uninstall Python packages. It is also used to manage Python virtual environments and more.

To start, open up a terminal or a command prompt and navigate to the directory of your Python project. Once you are there, type the following command:

This installs all of the modules listed in our Python requirements file into our project environment.

Output:

It is a good practice to set a new environment before installing packages with your Python requirements file. pyenv and venv to help you with this process in my next article, here and here.

As mentioned before, use the pip freeze command to output a list of the Python modules installed in your environment.

How to Maintain a Python Requirements File

If you created a Python requirements.txt file at one point but have failed to maintain it for some reason, fear not! You can do it as follows.

Output:

As an example, let’s update fastapi :

Output:

Step 3: Check to see if all of the tests pass.

Step 4: Run pip freeze > requirements.txt to update the Python requirements file.

Step 5: Run git commit and git push to the production branch.

Freezing all your dependencies helps you have predictable builds.

If you need to check for missing dependencies, you can do so with the following command:

Output:

In our case, we are good to go!

How to Create Python Requirements File After Development

While it is possible to create it manually, it is a good practice to use the pipreqs module. It is used to scan your imports and build a Python requirements file for you.

According to the documentation, once installed with the following command:

running pipreqs in the command line generates a requirements.txt file automatically:

Why You Should Use a Python Requirements File

Create a Python requirements.txt file when starting a new data science project. It is always a good idea to include one in your project, particularly in the context of version control.

If you are unsure about version control, read more about it here. And if you are interested in writing better Python code, you can find more information here.

Using Python requirements files is among Python development best practices. It dramatically reduces the need for managing and supervising different libraries. Managing your library dependencies from one place makes it easier, more convenient, and faster. It helps keep everything organized and easy for everyone involved.

Compared to pasting a list of dependency paths into the command line every time you want to install or update them, it makes installing your Python applications on another system easier. It is a great way to ensure you have all the necessary dependencies installed for your project.

Also, GitHub provides automated vulnerability alerts for dependencies in your repository. By uploading a requirements.txt with your code, GitHub checks for any conflict and sends an alert to the administrator if it detects any. It can even resolve the vulnerabilities automatically!

Best Practices for Using a Python Requirements File

There are several best practices to follow in using a Python requirements.txt file:

Looking for data science project ideas to experiment with creating and maintaining a Python requirements file? Feel free to check this article to find some inspiration!

How to create & use requirements.txt for python projects

What is requirements.txt & it’s advantage

When a person is working on a python project and added some python libraries and then shares the project in the team or when a new team member joins the team, it’s always difficult to tell what libraries used in the project, either you have to specify in a text document or write somewhere.

but during the time, if you upgrade the libraries, you have to manually update the version details as well in the documentation, so the next person who picks up the project can install specific version of the libraries.

so requirements.txt file will automatically updates the package details with version when someone installs a new package, also python provides a way to install those version packages directly from the requirements.txt file, so you need not to manually install each & every package.

How to create requirements.txt

Navigate to your project directory in terminal & type

pip freeze > requirements.txt

This will keep all the installed package names with the versions installed for the current project.

Important note –

Sometimes handcraft or relook into the requirements.txt is a good idea after doing a pip freeze

Scenario – Sometimes when developers try with different libraries for experiment, but they tend to forget to remove from the project, but those libraries left inside the requirements.txt file, so when other developer use the requirements.txt to install the packages might not have idea where it’s used inside the project.

Another scenario – few libraries when we install, requirements.txt add extra dependencies specific to the libraries, but when we upgrade the libraries during course of time, the dependencies might not be required, but doesn’t remove from the requirements.txt

so we need to remove those specific unnecessary dependencies manually so they don’t create any issues when we install using requirements.txt

When we upgrade the package, the futures dependency is not needed, but stays in the file, so we need to remove manually.

How to install from requirements.txt

Navigate to the project folder where you want to install the packages from the requirements.txt file

Note – make sure requirements.txt file is present under the parent project directory

Open the terminal from the current directory and type

This will install all the libraries under the project directory [recommended to use the virtual environment]

You can use below command to know more install options –

Bonus –

If you are using pycharm editor, then you can directly create or install the packages from the requirements.txt

Create requirements.txt

Open the project from the editor > Tools > Sync Python Requirements… > Ok

Will generate requirements.txt under the project directory

This file can be shipped with the project, so other person can install the libraries

To install from requirements.txt

Open the requirements.txt file using pycharm editor

You will get a notification to install the libraries on top of the file, click on the link to install!