A perfect application of Python "user environment"

I wrote an article about the use of virtual environment before.

However, I haven't introduced the Python User environment well because I haven't encountered it to use   User environment   So I've been too lazy to write.

It happened that these two days, I met a case where the experience of using the user environment can completely explode the virtual environment, so I took it out and shared it.

1. My background

The company has tens of thousands of servers. In order to realize the centralized management of access records and for security reasons, each server has access restrictions, and the company's springboard machine must be used to log in.

Each company's employees have their own user and home directory on the springboard machine, which is highly restricted for many operations requiring root permission.

For example, I now want to remotely log in to a large number of machines on the springboard machine for some maintenance. Of course, I still use Python here. There are some dependent libraries in the python script (such as the artifact paramiko introduced earlier), which are not installed on the springboard machine.

As an ordinary user, you do not have permission to install third-party packages.

The question is, how can I use the paramiko package in the springboard machine?

2. Why not use a virtual environment?

Since the global Python environment cannot be changed, I can create another environment myself, as long as the paramiko package is installed in this environment in advance.

Therefore, using virtual environment is a solution, but it is not a perfect solution.

The reasons are as follows:

1. The process of creating a virtual environment involves many steps and is complex. The complexity here is relative to the user environment I will use later.

2. The virtual environment contains an entire Python interpreter. There are a large number of packages that duplicate the system. The size is relatively large and not portable.

3. If you use console mode for debugging, it is very inconvenient to enter

Even if you don't use the console mode, the way you call the script will be strange. You have to do this

$ zabbix_env/bin/python demo.py

If you don't want to use this, you can add an executable permission to the script and specify your interpreter in the first line of the script, which saves a little trouble, but even so, I still feel very awkward.

[wangbm@35ha02 ~]$ cat demo.py 
#!/home/wangbm/zabbix_env/bin/python

import zabbix_api
[wangbm@35ha02 ~]$ 
[wangbm@35ha02 ~]$ 
[wangbm@35ha02 ~]$ chmod +x demo.py
[wangbm@35ha02 ~]$ 
[wangbm@35ha02 ~]$ ./demo.py     # It can be executed without error
[wangbm@35ha02 ~]$

You may ask me: why not use virtualenv + virtualenv wrapper so that you can use workon to enter the virtual environment.

The reason is that there are very old packages in the springboard machine. You see, Python above is still 2.7.5, so all the tools you mentioned are not available.

3. User environment principle

The scheme (user environment) to be introduced here may not have been used or even heard by many people. It is a popular but very easy-to-use function.

Before operation, briefly introduce it.

First of all, when Python looks for the import package, if we Mobile phone number purchase Multiple paths have this package. How does Python determine which path to import from?

The answer is that the search import path has priority, and you can view it through sys.path.

>>> import sys
>>> from pprint import pprint
>>> pprint(sys.path)
['',
 '/usr/lib64/python27.zip',
 '/usr/lib64/python2.7',
 '/usr/lib64/python2.7/plat-linux2',
 '/usr/lib64/python2.7/lib-tk',
 '/usr/lib64/python2.7/lib-old',
 '/usr/lib64/python2.7/lib-dynload',
 '/home/wangbm/.local/lib/python2.7/site-packages',
 '/usr/lib64/python2.7/site-packages',
 '/usr/lib64/python2.7/site-packages/gtk-2.0',
 '/usr/lib/python2.7/site-packages',
 '/usr/lib/python2.7/site-packages/pip-18.1-py2.7.egg',
 '/usr/lib/python2.7/site-packages/lockfile-0.12.2-py2.7.egg']
>>>

You can see the path  / home/wangbm/.local/lib/python2.7/site-packages   Is prior to  / usr/lib64/python2.7/site-packages   Path.

This is it.   User environment   As long as we package it in our own home directory, we can find it in the global environment first.

It can be used without user perception, which is no different from using the native global environment.

4. Specific operation methods

Creating a user environment and installing the packages you need can be done with one command, which is much simpler and more convenient than a virtual environment.

So how?

As long as you use the pip installation package, add  -- user   Parameter, pip will install it in the current user's  ~/. local/lib/python2.x/site-packages   The python of other users will not be affected.

$ pip install --user pkg

It should be noted here that this method cannot be used. The personal test will package it into the global environment. I haven't studied the specific reasons yet.

$ python -m pip install --user pkg

In order to let you understand this process, let me give an example and verify whether it can achieve user isolation.

# requests is not installed in the global environment
[root@localhost ~]$ pip list | grep requests
[root@localhost ~]$ su - wangbm

# Since the user environment inherits from the global environment, it is not installed here
[wangbm@localhost ~]$ pip list | grep requests
[wangbm@localhost ~]$ pip install --user requests
[wangbm@localhost ~]$ pip list | grep requests
requests (2.22.0)
[wangbm@localhost ~]$

# From the Location attribute, it can be found that requests is only installed in the current user environment
[wangbm@localhost ~]$ pip show requests
---
Metadata-Version: 2.1
Name: requests
Version: 2.22.0
Summary: Python HTTP for Humans.
Home-page: http://python-requests.org
Author: Kenneth Reitz
Author-email: me@kennethreitz.org
Installer: pip
License: Apache 2.0
Location: /home/wangbm/.local/lib/python2.7/site-packages
[wangbm@localhost ~]$ exit
logout

# Exit the wangbm user and find that requests is not installed in the root user environment
[root@localhost ~]$ pip list | grep requests
[root@localhost ~]$

With this idea, I can first create a user environment on other machines (on the premise that I must have administrator privileges) and install the paramiko package.

Then compress and copy the user environment to the home directory of the springboard machine  . local/lib   Directory and unzip.

Then directly use python to enter the console mode. Now you can directly use the paramiko package.

This is my sharing about the user environment. If it is helpful to you, please give me a praise and support.

Keywords: Python Back-end

Added by penkomitev on Thu, 28 Oct 2021 10:41:09 +0300