Python has not had a de facto standard project management and construction tool for such a long time, resulting in a variety of Python project structures and construction methods. This may reflect Python's free will.
Unlike Java, which went through the initial manual construction, from semi-automatic Ant to maven, it is basically a de facto standard. Meanwhile, Maven also accepted the challenges of other gradle (the main push of Android project), SBT (mainly Scala project), Ant + ivy, builder, etc., but it was difficult to shake Maven's Jianghu position, and others almost followed Maven's directory layout.
Back to Python, package management tools such as pip, pipenv and CONDA have been generated, but there is no agreement on the directory layout of the project.
For building a lot, it still continues the traditional Makefile method, and adds setup Py and build Py uses program code to install and build. As for the project directory layout, you can make a project template, and then make a tool to apply the project template.
Let's take a look at the use of the four tools
- CookieCutter
- PyScaffold
- PyBuilder
- Poetry
CookieCutter is a classic Python project directory structure
$ pip install cookiecutter $ cookiecutter gh:audreyr/cookiecutter-pypackage #Take "audreyr / cookiecutter py package" on "github" as the template, and then answer a bunch of questions to generate a "Python" project ...... project_name [Python Boilerplate]: sample ......
Finally, the project template generated by cookie cutter looks like the following:
$ tree sample sample ├── AUTHORS.rst ├── CONTRIBUTING.rst ├── HISTORY.rst ├── LICENSE ├── MANIFEST.in ├── Makefile ├── README.rst ├── docs │ ├── Makefile │ ├── authors.rst │ ├── conf.py │ ├── contributing.rst │ ├── history.rst │ ├── index.rst │ ├── installation.rst │ ├── make.bat │ ├── readme.rst │ └── usage.rst ├── requirements_dev.txt ├── sample │ ├── __init__.py │ ├── cli.py │ └── sample.py ├── setup.cfg ├── setup.py ├── tests │ ├── __init__.py │ └── test_sample.py └── tox.ini 3 directories, 26 files
This is probably the main framework of the popular directory structure. The main elements are:
$ tree sample sample ├── Makefile ├── README.rst ├── docs │ └── index.rst ├── requirements.txt ├── sample │ ├── __init__.py │ └── sample.py ├── setup.cfg ├── setup.py └── tests ├── __init__.py └── test_sample.py
Repeat in the project sample directory, place Python source files in the sample directory, test files in the tests ， directory, and add a ， docs ， directory to place documents, readme Rst, other setup for building, setup CFG and Makefile files.
This is actually a very classic Python project structure. The next build will use the command "make". Enter "make" to see the instructions defined in the Makefile file
$ make clean remove all build, test, coverage and Python artifacts clean-build remove build artifacts clean-pyc remove Python file artifacts clean-test remove test and coverage artifacts lint check style test run tests quickly with the default Python test-all run tests on every Python version with tox coverage check code coverage quickly with the default Python docs generate Sphinx HTML documentation, including API docs servedocs compile the docs watching for changes release package and upload a release dist builds source and wheel package install install the package to the active Python's site-packages
In order to use the above build process, you need to install corresponding packages, such as {tox, wheel, coverage, sphinx and flake8, which can be installed through {pip}. Then you can make test, make coverage, make docs, make dist, etc. Make docs can generate a beautiful Web document.
PyScaffold create a project
PyScaffold, as its name implies, is a tool used to create a Python project scaffold. It is installed and used:
$ pip install pyscaffold $ putup sample
In this way, a Python project is created. The directory structure is similar to that of the template selected by {cookiecutter, except that it places the source file in the} src} directory instead of the} sample} directory.
$ tree sample sample ├── AUTHORS.rst ├── CHANGELOG.rst ├── CONTRIBUTING.rst ├── LICENSE.txt ├── README.rst ├── docs │ ├── Makefile │ ├── _static │ ├── authors.rst │ ├── changelog.rst │ ├── conf.py │ ├── contributing.rst │ ├── index.rst │ ├── license.rst │ ├── readme.rst │ └── requirements.txt ├── pyproject.toml ├── setup.cfg ├── setup.py ├── src │ └── sample │ ├── __init__.py │ └── skeleton.py ├── tests │ ├── conftest.py │ └── test_skeleton.py └── tox.ini
The tool {tox} will be used for the construction of the whole project. Tox # is an automated test and build tool that creates a Python virtual environment during the build process, which enables a clean environment for testing and building.
tox -av ， can display the definition in ， tox All tasks in ini:
$ tox -av default environments: default -> Invoke pytest to run automated tests additional environments: build -> Build the package in isolation according to PEP517, see https://github.com/pypa/build clean -> Remove old distribution files and temporary build artifacts (./build and ./dist) docs -> Invoke sphinx-build to build the docs doctests -> Invoke sphinx-build to run doctests linkcheck -> Check for broken links in the documentation publish -> Publish the package you have been developing to a package index server. By default, it uses testpypi. If you really want to publish your package to be publicly accessible in PyPI, use the `-- --repository pypi` option.
To execute which command, use tox -e build, tox -e docs, etc
In the process of experiencing the tox command, every step seems to be slow. It should take some time to create the virtual machine.
PyBuilder
It's best to look at another build tool PyBuilder, which creates a directory structure very close to Maven. Let's take a look
$ pip install pybuilder $ mkdir sample && cd sample #The project directory needs to be created manually $ pyb --start-project #After answering some questions, create the required directories and files
After that, take a look at its directory structure:
$ tree sample . ├── build.py ├── docs ├── pyproject.toml ├── setup.py └── src ├── main │ ├── python │ └── scripts └── unittest └── python
The build process still uses the pyb command. You can use pyb -h to view help. pyb -t lists all tasks. PyBuilder's tasks are added in the form of plug-ins. The plug-ins are configured in pyb build Py file.
$ pyb -t sample Tasks found for project "sample": analyze - Execute analysis plugins. depends on tasks: prepare run_unit_tests clean - Cleans the generated output. compile_sources - Compiles source files that need compilation. depends on tasks: prepare coverage - <no description available> depends on tasks: verify install - Installs the published project. depends on tasks: package publish(optional) package - Packages the application. Package a python application. depends on tasks: compile_sources run_unit_tests(optional) prepare - Prepares the project for building. Creates target VEnvs print_module_path - Print the module path. print_scripts_path - Print the script path. publish - Publishes the project. depends on tasks: package verify(optional) coverage(optional) run_integration_tests - Runs integration tests on the packaged application. depends on tasks: package run_unit_tests - Runs all unit tests. Runs unit tests based on Python's unittest module depends on tasks: compile_sources upload - Upload a project to PyPi. verify - Verifies the project and possibly integration tests. depends on tasks: run_integration_tests(optional) $ pyb run_unit_tests sample
PyBuilder also creates a virtual environment before building or testing. Starting from version 0.12.9, you can skip the step of creating a virtual environment through the parameter -- no VENVS. With -- no VENVS ， Python code will be executed in the current Python environment running ， pyb ， and the required dependencies will be installed manually.
Project dependencies should also be defined in build Py file
@init
def set_properties(project):
project.depends_on('boto3', '>=1.18.52')
project.build_depends_on('mock')The above dependencies are then installed and tests and builds run on them when you execute pyb} create the virtual environment.
Poetry
The last one, Poetry, feels like a more mature Python construction with higher project activity. It has a more powerful trust management function. You can add dependencies with {poetry add boto3}, and poetry show --tree shows the dependency tree. See how to install and create a project
$ pip install poetry $ poetry new sample
It creates projects that are simpler than the above
$ tree sample sample ├── README.rst ├── pyproject.toml ├── sample │ └── __init__.py └── tests ├── __init__.py └── test_sample.py
If you add the -- src} parameter to ， poetry new ， the source file directory ， sample ， will be placed in the ， src ， directory, that is, ， sample / src / sample Poetry init will generate pyproject in the current directory The generation of toml# files, directories, etc. needs to be completed manually.
It does not pay attention to document generation, code specification inspection, and code coverage. Its project configuration is more centralized, all in pyproject What is toml ？ in the toml ？ file? It is a configuration file format Tom's obvious, minimum language( https://github.com/toml-lang/toml).
pyproject.toml ， some similar to NodeJS ， package JSON file, such as the command line of poetry add and poetry install
#Go to pyproject Add and install the dependency on {boto3} in toml {(add} can also install the dependency locally or {git}),
poetry add boto3
#Will follow pyproject The toml} file defines how to install the corresponding dependencies into the current Python} virtual environment
#For example, in < test venv > / lib / Python 3 In the 9 / site packages directory, test cases can also be used after modules are installed
poetry install Other major
1. poetry build #Build installable * whl ， and ， tar GZ} file
2. poetry shell #It will be defined in pyproject Create and use virtual environments based on dependencies in the toml} file
3. poetry run pytest #Run test cases that use {pytest}, such as} tests/test_sample.py
4. poetry run python -m unittest tests/sample_tests.py #Run the # unittest # test case
5. poetry export --without-hashes --output requirements.txt #Export} requirements Txt file, - dev - export dependencies containing - Dev, or use - poetry - export -- without hashes > requirements txtPoetry run can execute any system command, but it will execute in the virtual environment it wants. Therefore, it can be imagined that the project of "poetry" must use "poetry run" to generate documents or coverage Command to support ， sphinx, coverage ， or ， flake8.
Create a file in the sample directory (at the same level as the pyproject.toml file)_ module. Py, the content is
def main():
print('hello poetry')Then in pyproject Write in toml #
[tool.poetry.scripts] my-script="sample.my_module:main"
Re execution
$ poetry run my-script
It will output "hello poetry".
Through the understanding of the above four tools, the complexity of the project structure is reduced from cookie cutter py project - > pyscaffold - > pybuilder - > poetry, and the difficulty of use is roughly the same order.