100 Days of Code: Day 49 - Day 53
Notes of 100 Days of Code: The Complete Python Pro Bootcamp.
The content these days is all about Selenium, which is why I’m deciding to skip most of it. The projects only focus on creating non-sense bots for social media (e.g., Facebook, Instagram, Twitter, or Tinder). I think understanding the core concepts and basic syntax of Selenium is enough.
What is Selenium?
Selenium is an open-source umbrella project for a range of tools and libraries that primarily aim to automate web browsers. It’s most commonly used for automated testing of web applications, but it’s also powerful for web scraping and other repetitive web tasks.
It works as a remote control for the web browser. Instead of manually clicking, typing, and navigating, the script tells the browser what to do, and Selenium executes those commands.
Basic Concepts of Selenium
| Concept | Description |
|---|---|
| WebDriver | This is the core of Selenium. It’s a set of language-specific bindings (like Python, Java, C#, etc.) that implements and controls a browser’s native automation support. You need a browser driver (like ChromeDriver, GeckoDriver for Firefox, etc.) to act as a bridge between your code and the browser itself. |
| Locators | This is how Selenium finds elements on a web page (like buttons, text fields, links, etc.) to interact with them. Common locator strategies include: ID, Name, Class Name, Tag Name, Link Text, Partial Link Text, CSS Selector, and XPath. |
| Web Element | Once Selenium finds an element using a Locator, it treats it as a WebElement object. This object has methods to interact with it, such as click(), send_keys(), text, etc. |
| Waits | Websites often load asynchronously. Waits are essential to prevent your script from failing because an element hasn’t loaded yet. There are two main types: Implicit Waits (set a default waiting time) and Explicit Waits (wait for a specific condition to be met). |
Quick Syntax Guide (Python Example)
A glimpse of the basic syntax in Python.
1. Setup and Initialization
This code block sets up the browser (Chrome in this example) and opens a webpage.
from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
from webdriver_manager.chrome import ChromeDriverManager
# 1. Initialize the browser driver
# (Using webdriver_manager makes setup easier)
driver = webdriver.Chrome(service=ChromeService(ChromeDriverManager().install()))
# 2. Open a website
driver.get("https://www.example.com")
2. Finding Elements (Locators)
This shows how to find elements using the most common locator strategies.
from selenium.webdriver.common.by import By
# By ID: The fastest and most reliable
element_by_id = driver.find_element(By.ID, "my-unique-id")
# By Name: Useful for form fields
element_by_name = driver.find_element(By.NAME, "username")
# By CSS Selector: Very powerful and flexible
element_by_css = driver.find_element(By.CSS_SELECTOR, "div.main-content > button.submit")
# By XPath: Another powerful, versatile option (often used when others fail)
element_by_xpath = driver.find_element(By.XPATH, "//a[contains(text(), 'About Us')]")
# To find multiple elements (returns a list)
all_links = driver.find_elements(By.TAG_NAME, "a")
3. Interacting with Elements
Once have a WebElement, then perform actions.
# Assuming 'element_by_id' is an input field
element_by_id.send_keys("Hello World") # Types text into the field
# Assuming 'element_by_css' is a button
element_by_css.click() # Clicks the button
# Get the visible text of an element
element_text = element_by_xpath.text
print(element_text)
# Clear the content of an input field
element_by_id.clear()
4. Closing the Browser
Always clean up after the script is done.
# Closes the currently active window/tab
driver.close()
# Terminates the entire WebDriver session and closes all windows
driver.quit()