Automatically create requirements.txt
Sometimes I download the python source code from github and don’t know how to install all the dependencies. If there is no requirements.txt file I have to create it by hands.
The question is:
Given the python source code directory is it possible to create requirements.txt automatically from the import section?
21 Answers 21
You can use the following code to generate a requirements.txt file:
more info related to pipreqs can be found here.
Sometimes you come across pip freeze , but this saves all packages in the environment including those that you don’t use in your current project.
Use Pipenv or other tools is recommended for improving your development flow.
If you do not use a virtual environment, pigar will be a good choice for you.
For python3: (I have both python 2 and 3 on my machine, where python2 is the default)
To check your python version:
In my case, I use Anaconda, so running the following command from conda terminal inside my environment solved it, and created this requirements.txt file for me automatically:
This was taken from this Github link pratos/condaenv.txt
If an error been seen, and you are using anaconda, try to use the .yml option:
For other person to use the environment or if you are creating a new enviroment on another machine:
To create a requirement Python module list, write this code in your terminal:
Kinda mind-blowing how this simple task is so complicated in Python. Here is what I think is the best way to do it automatically.
You need two tools:
pip3 install pipreqs
pipreqs will go through your project and only install the packages that your project use. Instead of all the packages in your python environment as pip freeze would do.
But there’s a problem with this approach. It does not install the sub-packages.
For example, your project uses pandas==1.3.2 . pandas itself uses numpy==1.21.2 among other packages. But pipreqs itself does not write the sub-packages (i.e. numpy) in requirments.txt
This is where you need to combine pipreqs with the second tool.
pip3 install pip-tools
pip-tools will take the packages in requirements.in and generate the requirements.txt with all the sub-packages. For example, if you have pandas==1.3.2 in requirements.in , pip-tools would generate
numpy==1.21.2 # via pandas in requirements.txt .
But you need to manually add the package in requirements.in . Which is prone to mistake and you might forget to do this once in a while.
This is where you can use the first tool.
But both the tools write to requirements.txt . So how do you fix it?
Use the —savepath for pipreqs to write in requirements.in instead of the default requirements.txt .
To do it in one command; just do
pipreqs —savepath=requirements.in & pip-compile
There you go. Now you don’t need to worry about manually maintaining the packages and you’re requirements.txt will have all the sub-packages so that your build is deterministic.
TL;DR
- pip3 install pipreqs
- pip3 install pip-tools
Use the following to build a deterministic requirements.txt
pipreqs —savepath=requirements.in && pip-compile
Firstly, your project file must be a py file which is direct python file. If your file is in ipynb format, you can convert it to py type by using the line of code below:
Then, you need to install pipreqs library from cmd (terminal for mac).
Now we can create txt file by using the code below. If you are in the same path with your file, you can just write ./ . Otherwise you need to give path of your file.
That will create a requirements.txt file for your project.
I blindly followed the accepted answer of using pip3 freeze > requirements.txt
It generated a huge file that listed all the dependencies of the entire solution, which is not what I wanted.
So you need to figure out what sort of requirements.txt you are trying to generate.
If you need a requirements.txt file that has ALL the dependencies, then use the pip3
However, if you want to generate a minimal requirements.txt that only lists the dependencies you need, then use the pipreqs package. Especially helpful if you have numerous requirements.txt files in per component level in the project and not a single file on the solution wide level.
Make sure to run pip3 for python3.7.
Before executing the above command make sure you have created a virtual environment.
python3:
python2:
After that put your source code in the directory. If you run the python file now, probably it won’t launch if you are using non-native modules. You can install those modules by running pip3 install <module> or pip install <module> .
This will not affect you entire module list except the environment you are in.
Now you can execute the command at the top and now you have a requirements file which contains only the modules you installed in the virtual environment. Now you can run the command at the top.
I advise everyone to use environments as it makes things easier when it comes to stuff like this.
If Facing the same issue as mine i.e. not on the virtual environment and wants requirements.txt for a specific project or from the selected folder(includes children) and pipreqs is not supporting.
You can use :
P.S: It may have a few additional libraries as it checks on fuzzylogic.
best way for Python 3 is:
it worked for me.
If you have installed many dependencies in your system and you need requirements.txt for a specific project, you can install first pipreqs:
and execute the below command under the project folder.
This command will generate requirements.txt file for the particular project.
Simple Pythonic Way
To get a list of all the REQUIREMENTS in a standard requirements.txt file, you can use the following command.
Now, this should automatically create a standard requirements file with all of the packages installed alongside their corresponding versions.
Pretty Print on Terminal
If you just want to get a pretty print on the terminal you can use the following approach.
This lists all of the installed packages, in a pretty print format.
Custom Dependency
If you have a project folder like say, a Github Repo, and you want to get a custom requirements.txt for project You can use the following Package. https://pypi.org/project/pipreqs/ pipreqs
Usage
Contents of requirements.txt
Automatic requirements.txt updating approach
While developing a python application with requirements.txt we have several choices:
- Generate requirements.txt after development, when we want to deploy it. It is performed by pip freeze > requirements.txt or pipreqs for less messy result.
- Add every module to requirements.txt manually after each install.
- Install manager that will handle requirements.txt updates for us.
There are many answers for the 1-st option, the 2-d option is self-explanatory, so I would like to describe the 3-d approach. There is a library called to-requirements.txt. To install it type this:
If you read the whole command at once you would see, what it does. After installing you should setup it. Run:
Как создать requirements.txt файл для python программы
Требование наличия в корне файла requirements.txt — достаточно распространенное среди заказчиков. Это и не удивительно — ведь для того чтобы быстро установить все требуемые библиотеки python в новом окружении достаточно выполнить команду pip install -r requirements.txt .
Рассмотрим несколько вариантов.
Вариант первый — вручную. Минусы думаю объяснять не надо, мало того что вы можете ошибиться в названиях и версиях, но также легко и пропустить библиотеку.
requirements.txt в Python. Установка и версионирование зависимостей.
В своих проектах на Python я конечно пользуюсь сторонними пакетами. Их легко установить с помощью pip install, но что делать если проект нужно распространять или показывать все в видео (как делаю я на своем канале Azzrael Code). Или если нужно чтобы в проекте использовались пакеты нужных версий.
В стандартном менеджере пакетов pip, который идет в комплекте с установочными пакетом Python, есть возможность перечислить необходимые пакеты в файле requirements.txt . И не просто перечислить, а также указать какие конкретно версии пакетов нужны (или даже выбрать диапазон версий, или исключить какие-то версии). А затем установить все эти пакеты одной командой python -m pip install -r requirements.txt. Подробнее о том как я использую requirements.txt в видео:
Как создать requirements.txt
Кстати совсем не обязательно называть файл с зависимостями именно requirements.txt. Можно использовать любое имя. А requirements.txt просто общепринятое название, ну и IDE могут ожидать такого именования.
Как установить зависимости с использованием requirements.txt
Документация по requirements.txt
Если у вас есть вопросы или есть мнения как сделать лучше, то пишите в комментария к python случайное