Preface
In the United States, I just got off the plane and wrote this blog while waiting for my luggage to commemorate my two-day history of trampling pits, as well as a new way of thinking, programming for Baidu. This article introduces how to use python to make piano scripts, using modules.

• PIL module
• numpy module
• ctypes module (using dynamic link libraries)

text
Okay, then we're going to have a long history of trampling, Are you ready?
ADB pit
At first, I thought about touchscreen operation on my mobile phone, let alone that.

I packed the kit. click here to download Password: xsx3

Put it in your work directory (you don't need to put it all in, but it's sure it's okay to put it all in). Turn on the usb and debug it.
Then open ADBDriverInstaller.exe:

After clicking, you can close it.

Enter the working directory, cmd, and enter the command:

adb shell

A bunch of parameters means no problem.

Introduce several adb commands you will use:

//(Save to SDCard)
adb shell /system/bin/screencap -p /sdcard/screenshot.png
// Export from SD card to computer. Note that F:\mvp is the computer path and must exist.
adb pull /sdcard/screenshot.png F:\\mvp(Save to computer)
//Simulate clicks at (x,y)
adb shell input tap x y

Then the PIL module is introduced.

from PIL import Image
import numpy as np

path=""	#Path is the picture path
img=np.array(Image.open(path))#img is the three-dimensional matrix of this picture [y][x][r,g,b]

If the rgb sum at a certain point is less than 100, then it is the place to click, that is, the black box. Orthogonal coordinates fixed, abscissa check four points corresponding to four columns, otherwise it would take two seconds for me to test a screenshot to be transmitted to the computer, and one second for a click, I would almost be excited, how to play, three seconds at a time, joke, or hand play, decisively agree to give up ADB.

So I meditated and thought hard, and at last I had a flash of inspiration. Is there a way to operate the mobile phone directly on the computer without having to be so slow as ADB? Fortunately, the answer is yes!!! Proud Soft Screen To meet all your needs and support reverse operation!! The interface is as follows:

Using the mouse can simulate mobile phone clicks, decisive Baidu python how to operate the mouse, there is a library is pyhook, just to meet this demand, as a result, I failed to install the compiler, Baidu see

Links to the original text https://www.cnblogs.com/nymrli/p/9557023.html
Hahaha, if a hook is evil, isn't the windows API a great evil? When giving up, we should give up. After a little reflection, we decided to write a dynamic link library to operate the mouse click, create a new folder named Pro1, and create a new file named Click.c under the Pro1 folder. Click.c reads as follows

#include<windows.h>
#include<stdio.h>

void click(int x,int y){//Define a method to transfer coordinate parameters
	SetCursorPos(x, y);//Move the mouse to (x,y)
	mouse_event(MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTUP, 0, 0, 0, 0);//Left mouse click

}

Here is a brief introduction to mouse_even. https://blog.csdn.net/THmen/article/details/78249422

Then it is compiled into dynamic link library with gcc. The steps are as follows:

1. Open cmd and enter Pro1 directory
2. Enter commands
3. Gcc-m64-shared Click.c-o Click.dll//Use-m32 if python is 32-bit
4. Enter

Then you find a Click.dll file in your Pro1 directory.

How to use this dynamic library?

Install the ctypes module. The pycharm interface is as follows:

To test the speed of this click, create a new test.py under the Pro folder.

from ctypes import  *
import  _thread	#This module needs to be installed
import time

test=CDLL("./Click.dll")
class date:
	Count=0	#Storage clicks

def t(name,n):#At least two parameters, fill in
	while True:
		test.click(500,500)
		date.Count+=1
_thread.start_new_thread(t,("ll",1))
time.sleep(1)#One second
print("Clicked in a second.{n}lower".format(n=date.Count))

Apart from loading pictures, it's more than enough time for Muyou, Haha, a good start. In order to avoid detours and simplify data calculation, we need to be smart and drag the Ao Soft Record Screen to the upper left corner, as shown in the figure:


Keep the upper left corner of the mobile phone and the upper left corner of the computer screen aligned as far as possible. At this time, the coordinates are (0,0). After testing, the lower right coordinates are (450,805). The data come out and go directly to the whole code.

import numpy as np
from PIL import ImageGrab
from ctypes import *

BOX=(0,699,450,701)
screen=np.array(ImageGrab.grab(BOX)#Intercept screen areas with longitudinal coordinates from 699 to 701 and abscissa from 0 to 450
click=CDLL("./Click.dll")
def click(x,y):#Customize mouse click method and refer to Click Dynamic Link Library
	click.click(x,y)

def judge(x,y):#Determine whether (x,y) is a dark pixel
	sum=int(screen[y][x][0])+int(screen[y][x][1])+int(screen[y][x][2])	#In order to transform strongly into the type of int prototype 0-255, the rgb sum of (x,y) pixel is (0, 0, 0) in pure black.
	if(sum<=100):#If dark

        return True
    return False
while True:
	screen=np.array(ImageGrab.grab(BOX)#Continuous updating of screen selection area
	for n in range(0,4):
		if(judge(int(56.25+N*113),1)):#Through continuous calculation, it is found that the abscissa is the most suitable for these four points, and the ordinate is actually 699+1.
			click(45 + 90 * n, 565)#The unit conversion is done here. The unit above is the pixel point. I don't know what it is here. The data is calculated. I'm fine with it.

So far, I can only say that it's probably finished, but there are still bug s. When I press a black box, when I press the next frame, I will press that pressed box again. So to stop this happening, the code is modified as follows:

import numpy as np
from PIL import ImageGrab
from ctypes import *

BOX=(0,699,450,701)
screen=np.array(ImageGrab.grab(BOX)#Intercept screen areas with longitudinal coordinates from 699 to 701 and abscissa from 0 to 450
click=CDLL("./Click.dll")
def click(x,y):#Customize mouse click method and refer to Click Dynamic Link Library
	click.click(x,y)

def judge(x,y):#Determine whether (x,y) is a dark pixel
	sum=int(screen[y][x][0])+int(screen[y][x][1])+int(screen[y][x][2])	#In order to transform strongly into the type of int prototype 0-255, the rgb sum of (x,y) pixel is (0, 0, 0) in pure black.
	if(sum<=100):#If dark

        return True
    return False
N=-1#Which column was clicked in the previous frame with an initial value of -1
while True:
	screen=np.array(ImageGrab.grab(BOX)#Continuous updating of screen selection area
	for n in range(0,4):
		if(judge(int(56.25+N*113),1)):#Through continuous calculation, it is found that the abscissa is the most suitable for these four points, and the ordinate is actually 699+1.
			if(n!=N):#If the column in the previous frame is different from the current one
				N=n#Change N
				click(45 + 90 * n, 565)#The unit conversion is done here. The unit above is the pixel point. I don't know what it is here. The data is calculated. I'm fine with it.

It's almost done here, but if the score of ten thousand is higher, a problem will arise.
For example, this time I clicked on the third column, but after I got the gold coin, the next box came down from the third column, so according to its logic, it's game over!! So I have to stop it, change it as follows

import numpy as np
from PIL import ImageGrab
from ctypes import *

BOX=(0,699,450,701)
screen=np.array(ImageGrab.grab(BOX)#Intercept screen areas with longitudinal coordinates from 699 to 701 and abscissa from 0 to 450
click=CDLL("./Click.dll")
def click(x,y):#Customize mouse click method and refer to Click Dynamic Link Library
	click.click(x,y)

def judge(x,y):#Determine whether (x,y) is a dark pixel
	sum=int(screen[y][x][0])+int(screen[y][x][1])+int(screen[y][x][2])	#In order to transform strongly into the type of int prototype 0-255, the rgb sum of (x,y) pixel is (0, 0, 0) in pure black.
	if(sum<=100):#If dark

        return True
    return False
N=-1#Which column was clicked in the previous frame with an initial value of -1
canC=False#Can I click on it?
while True:
	screen=np.array(ImageGrab.grab(BOX)#Continuous updating of screen selection area
	for n in range(0,4):
		if(judge(int(56.25+N*113),1)):#Through continuous calculation, it is found that the abscissa is the most suitable for these four points, and the ordinate is actually 699+1.
			if(n!=N):#If the column in the previous frame is different from the current one
				N=n#Change N
				canC=True
				click(45 + 90 * n, 565)#The unit conversion is done here. The unit above is the pixel point. I don't know what it is here. The data is calculated. I'm fine with it.
		if (canC):
            if(judge(int(56.25+N*113),1)==False ):
                N=-1
                canC=False

So far, there's no big problem. It's suggested that the computer should be running smoothly. I ran the Star 1073 at the top.

In fact, I don't use python very much. I used python when I built the neural network. Then I used the left string animation in my freshman semester. Now I'm almost sophomore. One of my junior college students can live up to now. Thanks to Du Niang, these two days'left thing is a sudden fantasy. Baidu has two days. I was so excited when I got to over 1000. I was thinking about plugging in USB, struggling to death, and then going to wash and sleep. I almost cried with joy after a slight change. Thank you, it's incoherent. If I could help you with this blog, I would be happier.