modular

The main function is to monitor the folder. If there is a new file, upload it

inotify

Because I want to write in Python, I mainly studied pyinotify
Document
Some information

I basically have all the functions I want (e.g: in_close_write for operation after copying)
But the hard thing is that you can't monitor across servers. You can only monitor local operations. Monitoring shared files will not respond, sad.

A small example:

import pyinotify


class TestEventHandler(pyinotify.ProcessEvent):

    def process_IN_CREATE(self, event):
        print('IN_CREATE ',event.pathname)
        
    def process_IN_CLOSE_WRITE(self, event):
        print('IN_CLOSE_WRITE ',event.pathname)

wm = pyinotify.WatchManager()
event = pyinotify.IN_CREATE | pyinotify.IN_CLOSE_WRITE
wm.add_watch('/watchfolder', events, rec=True, auto_add=True)  # rec: whether to monitor sub file price, auto_add: whether to automatically add the new subfolder to the monitoring list 
handler = TestEventHandler(corpusManager)
notifier = pyinotify.Notifier(wm, handler)
notifier.loop()

Systemd Path Units

An introduction
The advantage is that it is simple and convenient, and there is no need to specially pack any package, but the disadvantage is that the ductility is not strong. For these States, it is impossible to do complex operations. And this is mainly based on inotify, so there will be problems encountered by inotify. It cannot cross servers, and if the two operations are too close, the latter may be lost.

A small example:
In short, there are three scripts:
One is the script (test.py) that you need to operate when you detect file changes, or directly an application
Another is the systemd file (filemonitor_event.service) that executes the script (test.py)

[Unit] 
Description="Run script to send email alert"

[Service]
ExecStart=/usr/bin/python /usr/local/test.py #Full path required

Finally, there is a systemd file (unit_file.path) for monitoring

[Unit]
Description="Monitor the /etc/passwd file for changes"

[Path]
PathModified=/etc/passwd # Monitoring path
Unit=passwd-mon.service

[Install]
WantedBy=multi-user.target

PathModified is the status of the monitoring file, which can be changed into PathExists, PathExistsGlob, PathChanged, PathModified and DirectoryNotEmpty

The last two files (filemonitor_event.service, unit_file.path) need to be placed under / etc/systemd/system /

watchdog

It also uses a module of python,
Official documents

import sys
import time
import logging
from watchdog.observers import Observer
from watchdog.events import LoggingEventHandler

if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO,
                        format='%(asctime)s - %(message)s',
                        datefmt='%Y-%m-%d %H:%M:%S')
    path = sys.argv[1] if len(sys.argv) > 1 else '.'
    event_handler = LoggingEventHandler()
    observer = Observer()
    observer.schedule(event_handler, path, recursive=True)
    observer.start()
    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        observer.stop()
    observer.join()

If you need to operate, write a class to inherit, such as

class FileEventHandler(FileSystemEventHandler):
	def on_created(event):
		print("create a new file")

Set from watchdog Replace observers import observer with from watchdog observers. Polling import pollingobserver can perfectly solve the cross platform problem, but there is a problem with uploading. All of its triggering mechanisms are triggered. There is no distinction between triggering before or after the event is completed. If I want to upload a new file, I can't guarantee that the file will be uploaded after copying, Read a lot of information This method of comparing file sizes is not very good, but it can also meet my needs

copying = True
size2 = -1
while copying:
    size = os.path.getsize('name of file being copied')
    if size == size2:
        break
    else:
        size2 = os.path.getsize('name of file being copied')

summary

To sum up, I finally chose watchdog, but all my tests are based on linux. If there is no cross platform test, will there be a problem.