Python Selenium.WebDriverWait Browser Startup Parameter Settings "How Edge Uses Startup Parameters"
1. Browser Startup Parameter Settings 🍿
When you create a WebDriver instance, you can configure its startup parameters to make some initial settings that will take effect throughout the lifecycle of the WebDriver
The parameters passed in by WebDriver are different for different browsers, but the main parameters are the same
So what can browser startup parameter settings do, for example:
- Selenium starts the browser as a window by default, maximizes the browser when clicking on a page or performing an action chain operation, and has a built-in method maximize_in Selenium Window() maximizes the browser, but it maximizes it after the browser is launched. So is there any way to start the browser by default to maximize it? The browser startup parameter settings are useful when you start the browser by passing in the parameter value--start-maximized
- Browsers calling Elenium are not difficult to find with an automated prompt bar above the browser
If you don't want your browser to display this line of prompts, you only need to set the value of the incoming parameter in the browser startup parameter--disable-infobars (the new version is no longer valid)
2. WebDriver instantiation parameters 🥑
There are many types of browsers supported by Selenium, here are the Chome "Google" browsers and Edge browsers
1), Chome Browser Section
webdriver.Chrome(executable_path="chromedriver", port=0, options=None, service_args=None, desired_capabilities=None, service_log_path=None, chrome_options=None, keep_alive=True)
The parameters are as follows:
-
executable_path: Browser driver path, if not specified, defaults to path set in environment variable PATH
-
Options: Startup options (Options objects are usually located under the WebDriver module of each browser, such as from selenium.webdriver.browser name.options import Options), and it is recommended that the options parameter be used first to set up the browser (options are based on capabilities, which are automatically written to the browser's preset capabilities when instantiating a browser's Options object)
-
desired_capabilities: similar to the options parameter, used primarily in earlier versions of Elenium, the options parameter is now recommended and only used when instantiating RemoteWebDriver (remote runtime indefinite or browser indefinite)
-
chrome_options: exactly the same as the options parameter, which was used in earlier versions of Selenium and is no longer recommended in current versions
-
Port: port number enabled by the driver, if not filled in, any idle port number will be used automatically, default parameter is 0
-
service_args: The parameters of the browser driver may vary depending on the browser driver used.
Open a command prompt window and run Driver File Name under the path of the driver--help to see the parameters supported by the driver. The following figure shows the driver file named chromedriver 94 in the D drive root directory to see its supporting parametersIt is important to note that the driver file name is best quoted with double quotes and there is an error
-
service_log_path: The location where the driver stores the log files
-
keep_alive: Indicates whether to take the HTTP request header Connection: keep-alive with the link to ChromeDriver, whether to use a long link, Boolean type parameter, and the default value is True
Source code: 👇
def __init__(self, executable_path="chromedriver", port=0,
options=None, service_args=None,
desired_capabilities=None, service_log_path=None,
chrome_options=None, keep_alive=True):
"""
Creates a new instance of the chrome driver.
Starts the service and then creates new instance of chrome driver.
:Args:
- executable_path - path to the executable. If the default is used it assumes the executable is in the $PATH
- port - port you would like the service to run, if left as 0, a free port will be found.
- options - this takes an instance of ChromeOptions
- service_args - List of args to pass to the driver service
- desired_capabilities - Dictionary object with non-browser specific
capabilities only, such as "proxy" or "loggingPref".
- service_log_path - Where to log information from the driver.
- chrome_options - Deprecated argument for options
- keep_alive - Whether to configure ChromeRemoteConnection to use HTTP keep-alive.
"""
Demo code: 👇
Instantiate Browser WebDriver
chrome_path = r"chromedriver.exe" # Relative Path browser = webdriver.Chrome(executable_path=chrome_path)
Incoming Settings Browser Runs Startup Parameters by Default to Maximize Window
edge_path = r"chromedriver.exe" # Relative Path
options = Options()
options.add_argument("--start-maximized") # Add maximize window operation parameters
browser = webdriver.Chrome(executable_path=edge_path, options=options)
2), Edge Browser Section
webdriver.Edge(executable_path='MicrosoftWebDriver.exe', capabilities=None, port=0, verbose=False, service_log_path=None, log_path=None, keep_alive=False)
The parameters are as follows:
- executable_path: Browser driver path, if not specified, defaults to path set in environment variable PATH
- capabilities: a non-browser-specific dictionary object that is required when passing in browser startup parameters
- Port: port number enabled by the driver, if not filled in, any idle port number will be used automatically, default parameter is 0
- verbose: whether to set detailed logging in the service
- service_log_path: The location where the driver stores the log files
- log_path: Unrecommended service_log_path parameter
- keep_alive: Indicates whether to take the HTTP request header Connection: keep-alive with the link to EdgeDriver, whether to use a long link, Boolean type parameter, and the default value is True
Source code: 👇
def __init__(self, executable_path='MicrosoftWebDriver.exe',
capabilities=None, port=0, verbose=False, service_log_path=None,
log_path=None, keep_alive=False):
"""
Creates a new instance of the chrome driver.
Starts the service and then creates new instance of chrome driver.
:Args:
- executable_path - path to the executable. If the default is used it assumes the executable is in the $PATH
- capabilities - Dictionary object with non-browser specific
capabilities only, such as "proxy" or "loggingPref".
- port - port you would like the service to run, if left as 0, a free port will be found.
- verbose - whether to set verbose logging in the service
- service_log_path - Where to log information from the driver.
- log_path: Deprecated argument for service_log_path
- keep_alive - Whether to configure ChromeRemoteConnection to use HTTP keep-alive.
"""
Demo code: 👇
Instantiate Browser WebDriver
edge_path = r"msedgedriver.exe" # Relative Path browser = webdriver.Edge(executable_path=edge_path)
Incoming Settings Browser Runs Startup Parameters by Default to Maximize Window
edge_path = r"chromedriver.exe" # Relative Path
options = {
"browserName": "MicrosoftEdge",
"version": "",
"platform": "WINDOWS",
"ms:edgeOptions": {
"extensions": [], "args": ["--start-maximized"] # Add maximize window operation parameters
}
}
browser = webdriver.Edge(executable_path=edge_path, capabilities=options)
3. Browser Startup Parameters
This article mainly takes Chrome as an example to introduce the function of each startup parameter. It is also recommended that you use Chrome browser first.
Common parameter options| Sequence Number | Parameter Options | describe |
|---|---|---|
| 1 | --user-agent="client proxy type" | User-Agent setting request header, mostly used for responding sites or scenarios where users-Agent determines whether a mobile device is on or off and returns to a different web page |
| 2 | --window-size = width, height | Set the default window size for the browser |
| 3 | --headless | Interface-free (windowless), also known as headless browsers, is commonly used for remote runs and can be used to increase efficiency when running locally |
| 4 | --disable-gpu | Disable GPU acceleration |
| 5 | --disable-gpu-program-cache | Disable GPU program caching |
| 6 | --start-maximized | Set Browser Default to Maximize Window Run |
| 7 | --incognito | Set up the browser to run in invisible mode (Trackless mode) |
| 8 | --disable-javascript | Disable Javascript code running |
| 9 | --disable-infobars | Disable prompts that browsers are being controlled by automated programs (new versions are no longer available) |
| 10 | --enable-automation | Notify users that their browser is controlled by automated testing |
Refer to this article for more browser startup parameters Chrome Browser Startup Parameters
Since the Edge browser is also using the Chromium kernel, the startup parameters for Chrome are mostly generic. I am used to and commonly used the Edge browser, which is one of the reasons why I included an explanation of the use of the browser in this article
4. Edge Browser Use Startup Parameters and Source Analysis
For Edge browsers, in the second, WebDriver instantiation parameters can be found that there are actually fewer options, desired_capabilities, chrome_options three parameters, on the contrary, there is more than one capability, so how to use the browser startup parameters?
(1) First look at the source code in the Chrome browser:
-
Options, desired_capabilities, chrome_ It's easy to be confused between the three options, but once you look at the source code below, you can quickly see the relationship between them
if chrome_options: warnings.warn('use options instead of chrome_options', DeprecationWarning, stacklevel=2) options = chrome_options if options is None: # desired_capabilities stays as passed in if desired_capabilities is None: desired_capabilities = self.create_options().to_capabilities() else: if desired_capabilities is None: desired_capabilities = options.to_capabilities() else: desired_capabilities.update(options.to_capabilities())As you can see, chrome_options has a warning in the source'use options instead of chrome_options', which means use the options parameter instead of chrome_options parameter, and finally chrome_options are assigned to options, which are identical
Then about desired_ Processing of capabilities, in desired_ When capabilities are empty, options are converted to desired_capabilities, desired_capabilities merge options -
For the Options object of Chome browser, we mainly look at these methods and properties in the source code
class Options(object): KEY = "goog:chromeOptions" def __init__(self): self._binary_location = '' self._arguments = [] self._extension_files = [] self._extensions = [] self._experimental_options = {} self._debugger_address = None self._caps = DesiredCapabilities.CHROME.copy() ...... @property def extensions(self): """ Returns a list of encoded extensions that will be loaded into chrome """ encoded_extensions = [] for ext in self._extension_files: file_ = open(ext, 'rb') # Should not use base64.encodestring() which inserts newlines every # 76 characters (per RFC 1521). Chromedriver has to remove those # unnecessary newlines before decoding, causing performance hit. encoded_extensions.append(base64.b64encode(file_.read()).decode('UTF-8')) file_.close() return encoded_extensions + self._extensions def add_extension(self, extension): """ Adds the path to the extension to a list that will be used to extract it to the ChromeDriver :Args: - extension: path to the \*.crx file """ if extension: extension_to_add = os.path.abspath(os.path.expanduser(extension)) if os.path.exists(extension_to_add): self._extension_files.append(extension_to_add) else: raise IOError("Path to the extension doesn't exist") else: raise ValueError("argument can not be null") ...... def to_capabilities(self): """ Creates a capabilities with all the options that have been set and returns a dictionary with everything """ caps = self._caps chrome_options = self.experimental_options.copy() chrome_options["extensions"] = self.extensions if self.binary_location: chrome_options["binary"] = self.binary_location chrome_options["args"] = self.arguments if self.debugger_address: chrome_options["debuggerAddress"] = self.debugger_address caps[self.KEY] = chrome_options return caps1) Add browser parameters by to_capabilities, if you want to add a setup browser to run by default to maximize the window, write:
options = Options() options.add_extension("--start-maximized")2), to_ The extensions method is used internally in the capabilities method code to convert objects into a dictionary that can be recognized when webdriver instantiates in the following format:
options = { "browserName": "MicrosoftEdge", "version": "", "platform": "WINDOWS", "ms:edgeOptions": { "extensions": [], "args": ["--start-maximized"] # Add maximize window operation parameters } }
(2) Edge browser uses startup parameters:
Why do you want to talk about this? Let's first look at the source code for options and combine it with the Edge browser section in the WebDriver instantiation parameter
1) options source under Edge
class Options(object):
def __init__(self):
self._page_load_strategy = "normal"
self._caps = DesiredCapabilities.EDGE.copy()
@property
def page_load_strategy(self):
return self._page_load_strategy
@page_load_strategy.setter
def page_load_strategy(self, value):
if value not in ['normal', 'eager', 'none']:
raise ValueError("Page Load Strategy should be 'normal', 'eager' or 'none'.")
self._page_load_strategy = value
@property
def capabilities(self):
return self._caps
def set_capability(self, name, value):
"""Sets a capability."""
self._caps[name] = value
def to_capabilities(self):
"""
Creates a capabilities with all the options that have been set and
returns a dictionary with everything
"""
caps = self._caps
caps['pageLoadStrategy'] = self._page_load_strategy
return caps
You can see that the option parameter option class under Edge does not add the startup parameter add_ The extension method, and the parameters that the instance Edge driver and the instance Chrome driver can pass in are different, such as the options parameter
The Edge driver instance passes in the startup parameters via the capabilities parameter, but the type passed in must be a dictionary type whose format is recognizable, referring to to to_under chrome.option mentioned above Capabilities method
2) Manually add startup parameters
The Edge startup parameter dictionary is as follows:
options = {
"browserName": "MicrosoftEdge",
"version": "",
"platform": "WINDOWS",
"ms:edgeOptions": {
"extensions": [], "args": []
}
}
Browser startup parameters are written in a list of ms:edgeOptions -> args, such as adding a maximize window operation parameter and passing this dictionary into the Edge Driven Instantiation parameter: 👇
edge_path = r"chromedriver.exe" # Relative Path
options = {
"browserName": "MicrosoftEdge",
"version": "",
"platform": "WINDOWS",
"ms:edgeOptions": {
"extensions": [], "args": ["--start-maximized"] # Add maximize window operation parameters
}
}
browser = webdriver.Edge(executable_path=edge_path, capabilities=options)
3) Writing auxiliary functions
You can write an auxiliary function to help us pass in the startup parameters
def get_options(params):
"""
Edge Browser Add Startup Parameter Auxiliary Function
:param params: startup parameter,Add directly for string type,Add multiple for list or tuple type
:return: Dictionary after adding startup parameters
"""
options = {
"browserName": "MicrosoftEdge",
"version": "",
"platform": "WINDOWS",
"ms:edgeOptions": {
"extensions": [], "args": []
}
}
if isinstance(params, str):
options["ms:edgeOptions"]["args"].append(params)
elif isinstance(params, (list, tuple)):
# De-weighting using sets
params = list(set(params))
options["ms:edgeOptions"]["args"].extend(params)
return options
5. Encapsulate startup parameter options 🍌
Once we know how browser startup parameters work, we can extend and encapsulate the Chrome and Edge parameter option classes Options to make up for some browser startup parameter class incomplete methods or write efficiency-enhancing methods.
-
Chrome Browser Parameter Options Class
For the Options parameter option class of Chrome browser, the function method is already complete. We just need to inherit this class and add a method to add more browser parameters at the same time.
class ChromeOptions(Options): def add_argument(self, argument): """ Add Browser Parameters :param argument: startup parameter """ if argument: if self._is_infobars(argument): self._enable_infobars() self._arguments.append(argument) else: raise ValueError("argument Parameter cannot be empty") def add_arguments(self, arguments: list or tuple): """ Add multiple browser parameters at the same time :param arguments: Start Parameter Set """ if arguments: if isinstance(arguments, str): self.add_argument(arguments) else: for arg in arguments: if self._is_infobars(arg): self._enable_infobars() self.add_argument(arg) else: raise ValueError("argument Parameter cannot be empty") @staticmethod def _is_infobars(string): return string == "--disable-infobars" def _enable_infobars(self): """ Enable'Disable prompts that browsers are being controlled by automated programs'startup parameter """ self._experimental_options["excludeSwitches"] = ["enable-automation"] -
Edge Browser Parameter Options Class
For the Edge browser, the Options parameter option class has few methods, the main add_ The argument method does not exist. Let's write our own Options parameter option class, write down several main properties and methods, and add our own custom methods.
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities class EdgeOptions: KEY = "ms:edgeOptions" def __init__(self): self._arguments = [] self._experimental_options = {} self._caps = DesiredCapabilities.EDGE.copy() @property def arguments(self): return self._arguments @property def experimental_options(self): return self._experimental_options def add_argument(self, argument): """ Add Browser Parameters :param argument: startup parameter """ if argument: if self._is_infobars(argument): self._enable_infobars() else: self._arguments.append(argument) else: raise ValueError("argument Parameter cannot be empty") def add_arguments(self, arguments: list or tuple): """ Add multiple browser parameters at the same time :param arguments: Start Parameter Set """ if arguments: if isinstance(arguments, str): self.add_argument(arguments) else: for arg in arguments: if self._is_infobars(arg): self._enable_infobars() else: self._arguments.append(arg) else: raise ValueError("argument Parameter cannot be empty") @staticmethod def _is_infobars(string): return string == "--disable-infobars" def _enable_infobars(self): """ Enable'Disable prompts that browsers are being controlled by automated programs'startup parameter """ self._experimental_options["excludeSwitches"] = ["enable-automation"] def to_capabilities(self): """ Create functionality with all options set :return: Return a dictionary with all contents """ caps = self._caps edge_options = { "extensions": [], "args": self.arguments } edge_options.update(self.experimental_options) caps[self.KEY] = edge_options return caps
For _ is_infobars, _ Enable_ The role of the infobars method, which is an extension of the knowledge used to enable <Disable browser tips being controlled by automated programs>Startup parameters. Interested friends can see 7. Tips for automated program control "Extension"
6. Headless Browser
The application of headless browser is more important. Headless browser is also called interface-free (windowless) running, that is, it will not pop up the browser window when running the program, but the program is still running. Currently, several mainstream browsers support it.
Benefits of headless browsers:
- This can greatly improve efficiency and save system resources
- Reduce running errors and prevent program errors caused by mistakenly touching browser windows
- Can be used for remote operation
Use a headless browser:
So how do I use a headless browser? It's really simple, just add the--headless startup parameter. Fifth, add the startup parameter by encapsulating the class encapsulated in the startup parameter option.
# Edge Browser options = EdgeOptions() options.add_argument(r"--headless") browser = Edge(executable_path=r"msedgedrive.exe", capabilities=options.to_capabilities()) # ---------------------------------------------------------------- # Chrome Browser options = ChromeOptions() options.add_arguments(r"--headless") browser = Chrome(executable_path=r"chromedriver.exe", options=options)
GPU acceleration:
Usually, using headless browsers also adds a parameter, disable-gpu, which disables GPU acceleration. This is a parameter often used with headless browsers and can be seen in many tutorials. Why add this parameter?
By default, without the--disable-gpu startup parameter, the CPU and memory are dynamic and uncertain. Looking at the GPU, you can see that the headless browser has not successfully used the GPU for hardware acceleration.
With the --disable-gpu boot parameter added, the GPU usage is the same
A comparative summary: 👇
- Disabling GPU acceleration is more stable and less likely to cause program errors than not after a period of comparison. You can try it on your own computer.
- GPU acceleration may result in a black screen or crash in the browser
- Headless browsers are already windowless and GPU acceleration is of little use, so turn them off to improve stability
Use headless browser and turn off GPU acceleration
options = EdgeOptions() options.add_arguments([r"--headless", r"--disable-gpu"]) browser = Edge(executable_path=r"msedgedrive.exe", capabilities=options.to_capabilities()) # --------------------------------------------------------------------------------------------- # Chrome Browser options = ChromeOptions() options.add_arguments([r"--headless", r"--disable-gpu"]) browser = Chrome(executable_path=r"chromedriver.exe", options=options)
7. Prompt for automatic program control "Extension"
In fact, this article is six, headless browser has ended, this part is the problem I encountered while doing tests. As an extension, share the solution here
For continuously updated versions of Chrome browsers, some attributes may be discarded or some attributes may be updated asleep
The--disable-infobars parameter has been abolished in higher versions of browsers to disable the prompt that the browser is being controlled by an automated program. So what else can you do to disable the prompt?
Refer to this article Chrome is being controlled by automated test software" notification Find a way here, but before explaining how to implement the forbidden prompt and how to explain the principles, you need to know this parameter--enable-automation, which tells users that their browser is controlled by automated tests, which can be interpreted as Selenium passing in the parameter automatically when it starts the browser. So just exclude this parameter and you'll be able to get no hints
Source code:
Before we look at the solution, let's see if we want to use add_ Experimental_ The source code for the option method, located in the Options parameter options class. In fact, this method is also simple, that is, to_ The dictionary output by capabilities adds a key-value pair
class Options(object):
KEY = "goog:chromeOptions"
def __init__(self):
self._binary_location = ''
self._arguments = []
self._extension_files = []
self._extensions = []
self._experimental_options = {}
self._debugger_address = None
self._caps = DesiredCapabilities.CHROME.copy()
......
@property
def experimental_options(self):
"""
Returns a dictionary of experimental options for chrome.
"""
return self._experimental_options
def add_experimental_option(self, name, value):
"""
Adds an experimental option which is passed to chrome.
Args:
name: The experimental option name.
value: The option value.
"""
self._experimental_options[name] = value
......
def to_capabilities(self):
"""
Creates a capabilities with all the options that have been set and
returns a dictionary with everything
"""
caps = self._caps
chrome_options = self.experimental_options.copy()
chrome_options["extensions"] = self.extensions
if self.binary_location:
chrome_options["binary"] = self.binary_location
chrome_options["args"] = self.arguments
if self.debugger_address:
chrome_options["debuggerAddress"] = self.debugger_address
caps[self.KEY] = chrome_options
return caps
Solution:
The current solution is to use an option called excludeSwitches and then exclude the enable-automation switch, which is a simple three-line code
options = Options()
options.add_experimental_option("excludeSwitches", ["enable-automation"])
browser = webdriver.Chrome(executable_path="chromedriver.exe", options=options)
Selection of References:
Reference material 💟
- Books:
- Asynchronous book Selenium Automated Testing Complete Guide based on Python
This is a new book published in May 2021. It is written in all respects. It should not be missed by students who are learning crawling or automated testing.
- Asynchronous book Selenium Automated Testing Complete Guide based on Python
- Blog Literature:
- Official Manual:
- Browser driver: