Residential proxies with Selenium

Get your credentials and install selenium-wire

Your username and password are on the Proxy generator page of the dashboard. Use the password exactly as shown there; country, state, city and session options all go in the username.

selenium-wire is no longer actively maintained, and its last release breaks with blinker 1.8 or later, so pin blinker below 1.8. It still works with current Selenium 4 and Chrome.

pip install selenium selenium-wire "blinker<1.8"

Open Chrome through the proxy

Pass the proxy URL, with credentials, in seleniumwire_options. selenium-wire handles the authentication between Chrome and the gateway. Using the sticky port keeps the whole browsing session on one IP.

from selenium.webdriver.common.by import By
from seleniumwire import webdriver

proxy = "http://<USERNAME>-type-residential-country-us-lifetime-10-session-abc12345:<PASSWORD>@gw.nodelighthouse.com:10000"

options = {
    "proxy": {
        "http": proxy,
        "https": proxy,
        "no_proxy": "localhost,127.0.0.1",
    }
}

driver = webdriver.Chrome(seleniumwire_options=options)
driver.get("https://api.ipify.org")
print(driver.find_element(By.TAG_NAME, "body").text)
driver.quit()

Rotate or target a location

Swap the URL for a rotating or geo-targeted one. Rotating suits fetching many unrelated pages; for anything with a login or a cart, keep the sticky port.

# rotating, any country
proxy = "http://<USERNAME>-type-residential:<PASSWORD>@gw.nodelighthouse.com:9000"

# rotating, Los Angeles
proxy = "http://<USERNAME>-type-residential-country-us-state-california-city-losangeles:<PASSWORD>@gw.nodelighthouse.com:9000"

Know what selenium-wire does to HTTPS

selenium-wire decrypts HTTPS locally with its own certificate so it can inspect and modify requests, and it sets up Chrome to accept that certificate. This adds some overhead per request. If you'd rather not depend on an unmaintained package, the alternatives are to run your own local forwarding proxy that adds the credentials, or to use Playwright or Puppeteer, which handle proxy authentication natively.

Questions

Why doesn't --proxy-server=http://user:pass@host:port work in Chrome?

Chrome ignores credentials in that argument and asks for them in an authentication dialog, which WebDriver can't fill in. selenium-wire, a local forwarding proxy, or a different automation library are the ways around it.

Does this work with Firefox?

selenium-wire also supports Firefox through webdriver.Firefox(seleniumwire_options=options), with the same proxy options.

Which port should I use?

Port 10000 (sticky) for browser sessions, so every request behind a page comes from one IP. Port 9000 rotates on every request. See the quick start for the username format.