Selenium with proxies for automated landing-page QA

QA and marketing-operations teams use Selenium to walk a real browser through pages the way a visitor would. Typical jobs include confirming that a campaign landing page renders in every target market, testing a sign-up or checkout flow by region, and capturing screenshots as evidence for a client. A proxy gives that automated browser a location, so the test sees the regional version of each page.

There is one well-known obstacle. Browsers will not take a proxy username and password from a command-line switch, the way a simple HTTP client does. This guide shows the methods that work in Python, from a quick setup for whitelisted proxies to two approaches that handle a login cleanly.

Where proxied Selenium earns its place

• Regional landing-page QA: load each campaign URL as a visitor in each market and check headlines, prices and consent banners.
• Flow testing: step through sign-up or checkout on your own site to confirm regional rules, currencies and payment options.
• Screenshot evidence: capture what a page showed in a market on a given day, for client reporting or internal sign-off.

Before any code: prove the credentials

Each proxy order has four credentials: host, port, username and password. This guide writes them as HOST, PORT, USER and PASS. Test them outside Selenium first, so you know the proxy is healthy before you debug a browser. Our My IP tool works for this, or run:

curl --proxy http://USER:PASS@HOST:PORT https://api.ipify.org

The proxy's IP should come back, not yours. If the command hangs, sort the credentials out now; otherwise you will spend time investigating the wrong layer.

Installing Selenium

From Selenium 4.6 onwards, Selenium Manager downloads the right browser driver for you, so there is no chromedriver to maintain by hand.

pip install selenium

Three ways to attach the proxy

Method A: a whitelisted proxy through Chrome options

If your office IP is on the order's whitelist, so no login is needed, one argument is enough:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument("--proxy-server=http://HOST:PORT")

driver = webdriver.Chrome(options=options)
driver.get("https://api.ipify.org")
print(driver.page_source)
driver.quit()

Problems start when credentials are added. Put USER:PASS@ in front of the host and Chrome discards it, then opens its own authentication dialog, which Selenium cannot interact with. That dialog is where most automation projects stall, so the next two methods avoid it.

Method B: a generated extension that supplies the login

The dependable way to give Chrome a username and password is a small extension, created by the script itself, that answers the proxy's login request through the webRequest API. The code writes the extension to a temporary folder and loads it:

import zipfile, tempfile, os
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

HOST, PORT, USER, PASS = "HOST", "PORT", "USER", "PASS"

manifest = """
{
  "version": "1.0.0",
  "manifest_version": 3,
  "name": "Proxy Auth",
  "permissions": ["proxy", "webRequest", "webRequestAuthProvider"],
  "host_permissions": ["<all_urls>"],
  "background": { "service_worker": "background.js" }
}
"""

background = f"""
chrome.runtime.onInstalled.addListener(() => {{
  chrome.proxy.settings.set({{
    value: {{
      mode: "fixed_servers",
      rules: {{ singleProxy: {{ scheme: "http", host: "{HOST}", port: {PORT} }} }}
    }},
    scope: "regular"
  }});
}});
chrome.webRequest.onAuthRequired.addListener(
  () => ({{ authCredentials: {{ username: "{USER}", password: "{PASS}" }} }}),
  {{ urls: ["<all_urls>"] }},
  ["blocking"]
);
"""

path = os.path.join(tempfile.mkdtemp(), "proxy_auth.zip")
with zipfile.ZipFile(path, "w") as zf:
    zf.writestr("manifest.json", manifest)
    zf.writestr("background.js", background)

options = Options()
options.add_extension(path)

driver = webdriver.Chrome(options=options)
driver.get("https://api.ipify.org")
print(driver.page_source)
driver.quit()

Since the extension supplies the credentials, Chrome never shows its pop-up, and the setting holds across page navigations. It works with a visible browser window. For headless runs, use the current --headless=new mode, because the older headless mode does not load extensions.

Method C: selenium-wire for Firefox, or Chrome

Firefox relies on profile preferences rather than extensions. The most portable way to give it an authenticated proxy is selenium-wire, which sits between the browser and the network and adds the login for you. Install selenium-wire and point Firefox at the proxy:

pip install selenium-wire blinker==1.7.0

from seleniumwire import webdriver

options = {
    "proxy": {
        "http": "http://USER:PASS@HOST:PORT",
        "https": "http://USER:PASS@HOST:PORT",
        "no_proxy": "localhost,127.0.0.1",
    }
}

driver = webdriver.Firefox(seleniumwire_options=options)
driver.get("https://api.ipify.org")
print(driver.page_source)
driver.quit()

The same selenium-wire configuration also works with Chrome if you would rather not generate an extension. The install line pins a blinker version, because later releases removed something that selenium-wire still depends on.

Matching the proxy type to the test

For consumer-facing pages where the test must receive the regional version local households receive, use residential proxies or residential premium proxies. For your own staging sites, internal tools and permissive public pages, datacenter proxies are quicker and cost less. For mobile-web checks and carrier-specific content, use mobile proxies. All four are compared on the pricing page.

Assert the exit IP inside the test

Make the first step of each run a check that the browser's visible IP belongs to the proxy, not to the office: load a JSON IP endpoint and compare the value, or run our proxy checker with the same credentials beforehand. Suites that open many browser sessions can use a rotating endpoint, so each session gets a new address and the load is spread out.

Being a considerate visitor

Automated browsers can put real load on a website. If a suite visits third-party sites, check each one's robots.txt and Terms of Service before the suite grows, build pauses into every flow, and keep any data you capture to public information your business may use. Never use automation to defeat a login, get round a paywall or create fake accounts. A suite that paces itself and rotates addresses sensibly runs reliably for longer. Our page on choosing a proxy for scraping covers larger data jobs.

Keeping the data bill sensible

A full browser downloads images, scripts and fonts, so each page costs more data than a plain HTTP request, and proxy traffic is billed per gigabyte. Where a test only checks text or layout, Chrome's --blink-settings=imagesEnabled=false argument skips images and trims usage noticeably. With Method B for Chrome or selenium-wire for either browser, the login problem is solved, and the remaining choices are the proxy type and the pace of the suite.