How to integrate selenium crawling web pages in scratch

1. Background We usually use three crawler libraries when crawling web pages: requests, scratch and selenium. Requests are generally used for small crawlers, while scratch is used to build large crawler projects, while selenium is mainly used to deal with responsible pages (complex js rendered pages, requests are very difficult to construct, o ...

Added by Pi_Mastuh on Sun, 16 Jan 2022 06:16:34 +0200

Other ways to use Python Selenium

Knowledge points: Master selenium control tab switching; Master the switching of iframe controlled by selenium; Master the method of using selenium to obtain cookie s; Master manual page waiting; Master the method of selenium controlling the browser to execute js code; Master selenium to open no interface mode; Understand how selenium uses prox ...

Added by Steveo31 on Wed, 05 Jan 2022 12:39:24 +0200

[software test report] Selenium+TestNG comprehensive exercise

[software test report] Selenium+TestNG comprehensive exercise Student No.: 04191315 Name: He Xiang College: Computer College Major: Software Engineering Full code: https://github.com/He-Xiang-best/Software-Quality-Assurance-and-Testing 1, Test requirements 1.1 test module Login function module 1.2 test contents Use [Selenium ...

Added by MrOnline on Wed, 05 Jan 2022 02:46:29 +0200

Use Python and selenium to automatically simulate and log in to 12306 official website

preface: In recent years, the anti crawling of 12306 has become more and more serious. From obtaining tk parameters a year ago to now, JS, CSS and other encryption methods have been added! At present, most people use selenium to log in, and this article is no exception. Environmental Science: Windows **  python 3.6.5** modular: seleniu ...

Added by hongco on Tue, 04 Jan 2022 09:26:24 +0200

Learn Python web crawler in five minutes

But anyway, crawler technology is innocent, and it is still worth our developers to learn about it. Before learning, we still need to understand the relevant concepts. What is a reptile Web crawler: also known as web spider, web robot, is a program or script that automatically grabs World Wide Web information according to certain rules. The ...

Added by CyberShot on Tue, 04 Jan 2022 02:39:52 +0200

python + selenium + webdriver automation case QQ mailbox sends e-mail automatically. Beginners will read it once

python + selenium + webdriver automation case QQ Email automatically sends email The overall code is at the end, the download link is also at the end, and the functions used are summarized at the top of the overall code Let's start with step analysis: Log in to qq mailboxEnter the letter writing functionEnter the recipient's email name and th ...

Added by Kyrillos on Sun, 02 Jan 2022 12:00:48 +0200

Automated testing - selenium launch browser

In the process of automated testing, when you start the browser through selenium, you may need to load plug-ins (such as firebug for testing, or a plug-in must be added in the product), read user data (your own browser configuration file / browser configuration file directly given by others), and set the browser (do not load pictures, etc.). S ...

Added by LoneTraveler on Fri, 31 Dec 2021 15:25:26 +0200

Selenium | after page Jump, you can't locate the element. How to break it?

This article is excerpted from the internal textbook of Hogwarts Testing Institute When you want to locate an element, if you can't locate it, you should consider whether the browser has embedded a frame window or the element you want to find is in the newly opened window. At this time, frame is required Switch between or windows. frame is ...

Added by shivabharat on Mon, 27 Dec 2021 19:45:45 +0200

Automatic measurement foundation operation

1, Automated testing 1.1 automated testing Automated testing is the process of transforming human driven testing into machine execution. 1.2 what kind of projects are suitable for automated testing 1) Software requirements change infrequently 2) The project cycle is relatively long (at least one year, preferably self-developed products) 3 ...

Added by DChiuch on Thu, 23 Dec 2021 16:29:23 +0200

qq mailbox login-mail for selenium

1. Browser opens QQ mailbox login web address QQ mailbox login address is: https://mail.qq.com/ driver = webdriver.Chrome() driver.get('https://mail.qq.com/') 2. Element Positioning, Enter QQ Account and QQ Password Manually enter the QQ mailbox login page, press option+command+i to open the developer tools, click View Elements, select e ...

Added by Eugene on Tue, 21 Dec 2021 16:00:09 +0200