python reptile selenium crawls cat's eye and Jingdong

Selenium case exercise 1. Crawl cat's eye movie data Page requirements Use selenium to crawl the top 100 of cat's eye, the ranking, film name, starring, release time and score of each film. Page analysis By analyzing the page structure, select the appropriate access point. selenium can directly check the page without analyzing whether the ...

Added by timj on Sun, 05 Sep 2021 05:32:03 +0300

Python's advanced approach: extracting official account background data and data visualization

The official account is rich in data statistics, but there may still be no statistical indicators that I want. For example official account official account is not high but the reading rate is very high. Can I draw a scatter map according to the reading rate of each article in my official account to sho ...

Added by kubis on Sat, 27 Jun 2020 10:23:27 +0300

[reptile] Selenium actual combat notes

selenium initialization Note that if Chrome is not installed in the default path, you need to use the option.binary_location sets the path of chrome. If the chrome driver is not in the same path as the current code, you also need to set the path of the chrome driver. option = webdriver.ChromeOptions() ...

Added by rachae1 on Tue, 09 Jun 2020 08:30:54 +0300

Mouse and keyboard operation events

Mouse events Mouse events are included in the ActionChains class. When importing, you only need to: from selenium.webdriver.common.action_chains import ActionChains Import the class to: Common methods are: context_click() -- right click double_click() -- double click drag_ and_ Drop - drag move_to_element() -- mouse over an element cl ...

Added by snipesh0tz on Mon, 01 Jun 2020 04:12:15 +0300

Golang uses selenium to operate Chrome

Golang uses selenium to operate Chrome 1. Demand Solve the problem of automatic login, and solve the problem of crawler by the way. 2. Basic concepts Selenium: selenium is a tool for Web application testing. Selenium testing runs directly and automatically in the browser, just like real users operate manually. Webdriver: Chrome driver is an au ...

Added by Kold on Fri, 29 May 2020 04:15:44 +0300

appium -- positioning package

Automated testing is simple, fast and easy to maintain. The most important part of automatic maintenance is element positioning. thinking Before we write code, we should make clear our thinking, so that when we write code again, we won't know how to start. 1. Manage elements and positioning methods through yaml (or other tools) 2. Encapsul ...

Added by NEWDAY on Sat, 16 May 2020 08:18:03 +0300

Thousands of yuan of Python crawler outsourcing cases, learn to earn

    Preface With the advent of the Internet age, people are more inclined to Internet shopping. A treasure is also a giant in the e-commerce industry. There are many business data in a treasure platform. Today, I will take you to use python+selenium tool to obtain these public Suitable for: Python zero foundation, interested in reptile data c ...

Added by arie_parie on Wed, 06 May 2020 17:36:24 +0300

Scrape crawls data using phantom JS

Environment: python2.7 + scratch + selenium + phantomjs Content: test summary + phantomjs Crawling content: involving js to load more pages Principle: open middleware in configuration file + modify process ﹣ request function (add phantom JS operation in it) Step 1: settings.py DOWNLOADER_MIDDLEWARES = { 'dbdm.middlewares.DbdmSpiderMiddlew ...

Added by skylert on Fri, 01 May 2020 14:58:08 +0300

Crawling pictures on splash with selenium

Because at the beginning of learning crawler, there was a project to crawl a picture of a website, a picture. [website link] (https://www.splash.com/), Today, I thought about this website when I was thinking about the project. Now I want to use selenium to scroll down the page, so as to crawl the content of multiple pag ...

Added by captainplanet17 on Tue, 31 Mar 2020 13:32:07 +0300

Developing the correct posture of function calculation -- using brotli to compress large files

Big file problem Function calculation The size of the uploaded zip code package is limited to 50M. This limit can be exceeded in some scenarios, such as Uncut serverless-chrome , similar to libreoffice, in addition to the common machine learning training model file.At present, there are three ways to solve the problem of large documents Adopt ...

Added by stb74 on Fri, 21 Feb 2020 09:43:32 +0200