Multiprocessing is the main shared memory in python Manager (), multiprocessing shared_memory method, both of which are python built-in modules, in which shared_memory is python 3 If you want to use the new functions added after 8, you must use python 3 Version above 8. The following describes the differences between the two methods:

1.multiprocessing.Manager()

Multiple data types can be used, including list,dict,Namespace,Lock,RLock,Semaphore,BoundedSemaphore,Condition,Event,Queue,Value, Array and other data types supported by the Manager.

Sample code segment:

# -*- coding:utf-8 -*-
from multiprocessing import Process, Manager
import time
import random
def kkk(a_list, number):
  for i in range(10):
    a_list.append(i)
    time.sleep(random.randrange(2))
    print('This is a process{} {}'.format(number, a_list))
  print('This is a process{} {}'.format(number, a_list))
def jjj(a_list):
  for i in range(2):
    process = Process(target=kkk, args=(a_list, i))
    process.start()
if __name__ == '__main__':
  manager = Manager()
  a_list = manager.list()
  # a_list = []
  process_0 = Process(target=jjj, args=(a_list,))
  process_0.start()
  process_0.join()
  print(a_list)
  print(len(a_list))
  print('it\'s ok')

2.shared_memory and shareableList

 SharedMemory Allocation and management of shared memory accessed by one or more processes on a multicore or symmetric multiprocessor (SMP) computer. To help with shared memory lifecycle management, especially in different processes, a BaseManager The subclass, SharedMemoryManager, is also in {multiprocessing Managers module.

In this module, shared memory refers to "System V style" shared memory blocks (although they do not have to be implemented explicitly), rather than "distributed shared memory". This type of shared memory allows different processes to potentially read and write to a common (or shared) area of volatile memory. Generally, processes can only access their own process memory space, but shared memory allows data to be shared between processes, thus avoiding the need to send messages between processes containing the data. Sharing data directly through memory provides significant performance advantages over sharing data through disk or socket or other communications that require data serialization / deserialization and replication.

Example code:

from multiprocessing import shared_memory
shm_a = shared_memory.SharedMemory(create=True, size=10)
type(shm_a.buf)
<class 'memoryview'>
buffer = shm_a.buf
len(buffer)
10
buffer[:4] = bytearray([22, 33, 44, 55])  # Modify multiple at once
buffer[4] = 100                           # Modify single byte at a time
# Attach to an existing shared memory block
shm_b = shared_memory.SharedMemory(shm_a.name)
import array
array.array('b', shm_b.buf[:5])  # Copy the data into a new array.array
array('b', [22, 33, 44, 55, 100])
shm_b.buf[:5] = b'howdy'  # Modify via shm_b using bytes
bytes(shm_a.buf[:5])      # Access via shm_a
b'howdy'
shm_b.close()   # Close each SharedMemory instance
shm_a.close()
shm_a.unlink()  # Call unlink only once to release the shared memory

3. Summary and comparison

Through experimental comparison, I summarized the following experience:

1 sharedMemory and shareableList are essentially the same implementation mechanism, and the access time will increase significantly with the increase of shared memory space; multiprocessing. As the shared memory space increases, the access time of manager () is almost the same.

2 shareableList only supports small cache (within 10M), and the effect of shareableList is 10 times better than that of Manager under the same cache size

3. shareableList is recommended for small parts of shared memory space; Most of the shared memory space is required, and Manager is recommended

If you have a better understanding of python multi process sharing mechanism, you can tell me in the comments below.