Crawler project 17: easily climb down Zhilian recruitment with Python to recruit 7000 recruitment messages
preface
Use Python requests + selenium to crawl the national recruitment data in Zhilian recruitment. If you read my previous articles, you should know that we have written a crawler that uses selenium to climb Zhilian recruitment
Tip: the following is the main content of this article. The following cases can be used for reference
My ...
Added by mikeglaz on Tue, 08 Mar 2022 17:08:15 +0200
Crawler project 18: use Python to crawl all recruitment information of all cities in the pull hook network
preface
Use selenium+requests to visit the page and crawl the hook bar recruitment information
Tip: the following is the main content of this article. The following cases can be used for reference
1, Analyze url
By observing the page, we can see that the page data belongs to dynamic loading, so now we get the data packet through the ...
Added by ashu.khetan on Tue, 08 Mar 2022 04:27:14 +0200
Three kinds of waiting in selenium
When we are doing WEB automation, we generally have to wait for the page elements to load before we can perform the operation, otherwise we will report the error that the elements cannot be found, which requires us to add the waiting time in some scenarios.
We usually use three waiting methods:
Forced waitingImplicit waitingDisplay wait
1, Fo ...
Added by jej1216 on Tue, 01 Mar 2022 04:26:32 +0200
Python crawlers - requests and selenium camouflage headers and agents to deal with anti crawling mechanism
catalogue
1. requests headers Send request2. selenium simulation uses browser camouflage headers3. requests use ip Proxy send request4. selenium webdriver uses proxy ip
In the process of writing crawlers, some websites will set up anti crawling mechanism to refuse to respond to non browser access; Or frequent crawling in a short time will ...
Added by hnxuying on Tue, 22 Feb 2022 07:58:13 +0200
python+appium automation
1. Download Android SDK( https://www.androiddevtools.cn/ )If you want to try Android development, you can download Android studio. If you want to be concise, you can download SDK tools. Download the Windows version of the official installer_r24.4.1-windows.exe. After installation, there will be two startup items: AVD manager and SDK manager. 2. ...
Added by renno on Sun, 20 Feb 2022 07:03:21 +0200
Construction of web automated testing framework (python+selenium+pytest+pom+ddt)
This document uses the popular pom design pattern to set up the test framework, write test cases, generate test reports, and finally integrate jenkins.
1, selenium
selenium is an open source automatic testing tool for web ui, which will not be introduced in detail.
2, Environment construction
The environment construction is very simple. P ...
Added by Random on Fri, 18 Feb 2022 05:46:01 +0200
selenium python easy to get started
selenium python easy to get started
Hello everyone, I believe you want to automatically obtain the things you can only see by clicking the web content manually, including saving the knowledge content, collecting data, calculating the price, etc., but it is a little difficult due to the past methods. You know that the emergence of browsers such ...
Added by wedge00 on Thu, 17 Feb 2022 04:07:35 +0200
selenium realizes the simulated Login of QQ space and b station
After working all night, it's too unfriendly for me as a novice. I'm going to cry. I finally succeeded. Come on, come on, rush, release it and leave a souvenir for myself
Let's talk about the idea first. When we use selenium automation, it's not very different from when you open the website. So why can you log in when you open the website? Bec ...
Added by gernot on Tue, 15 Feb 2022 22:03:36 +0200
Python+selenium to realize automatic hospital registration
Python+selenium to realize automatic hospital registration
In some hospitals, some experts Daniel's number is always "No. 1 is difficult to find", and the ticket is basically empty for seconds. In view of this situation, small partners who master certain technology can register using automatic methods
First, declare that the script ...
Added by matanoosh on Mon, 14 Feb 2022 16:27:58 +0200
Selenium has three waiting modes (forced waiting, implicit waiting and display waiting)
1. Forced wait (sleep)
from time import sleep
sleep(3) # Forced wait for 3 seconds
Disadvantages: because the speed of Web loading depends on the hardware, network speed, server response time and other factors. If the waiting time is too long, it is easy to waste time. If the waiting time is too short, it may result in an error when the web ...
Added by r-it on Mon, 14 Feb 2022 13:39:04 +0200