Residential proxies with Python requests

Get your credentials

Your username and password are on the Proxy generator page of the dashboard. Use the password exactly as shown there; country, state, city and session options all go in the username.

The gateway is gw.nodelighthouse.com. Port 9000 rotates (a new IP per request), port 10000 is sticky (the username carries -lifetime-N-session-xxxxxxxx), and port 11000 is SOCKS5. If you only need a quick test, the $2 test pack on the pricing page gives you 500 MB.

pip install requests

Send a request through a rotating IP

Pass the same proxy URL for both http and https. requests opens an HTTPS tunnel through the gateway, so the target site sees a residential IP. Each call below gets a new IP.

import requests

proxy = "http://<USERNAME>-type-residential:<PASSWORD>@gw.nodelighthouse.com:9000"
proxies = {"http": proxy, "https": proxy}

for _ in range(3):
    r = requests.get("https://api.ipify.org", proxies=proxies, timeout=30)
    print(r.text)  # a different IP each time

Target a country, state or city

Add targeting to the username. Values are lowercase with no spaces or dashes. State and city targeting are included in the flat per-GB price.

# United States
us = "http://<USERNAME>-type-residential-country-us:<PASSWORD>@gw.nodelighthouse.com:9000"

# Los Angeles, California
la = "http://<USERNAME>-type-residential-country-us-state-california-city-losangeles:<PASSWORD>@gw.nodelighthouse.com:9000"

r = requests.get("https://api.ipify.org", proxies={"http": la, "https": la}, timeout=30)
print(r.text)

Keep one IP with a sticky session

Use port 10000 and add -lifetime-N-session-xxxxxxxx to the username. The lifetime is in minutes (1 to 1440) and the session id is 8 characters from a-z and 0-9. Requests that use the same session id share an IP until the lifetime runs out; a new id gets a new IP.

import secrets
import requests

session_id = secrets.token_hex(4)  # 8 chars, a-f0-9
user = f"<USERNAME>-type-residential-country-us-lifetime-10-session-{session_id}"
proxy = f"http://{user}:<PASSWORD>@gw.nodelighthouse.com:10000"
proxies = {"http": proxy, "https": proxy}

with requests.Session() as s:
    s.proxies.update(proxies)
    print(s.get("https://api.ipify.org", timeout=30).text)
    print(s.get("https://api.ipify.org", timeout=30).text)  # same IP

Use SOCKS5 instead

requests needs the PySocks extra for SOCKS. Use the socks5h scheme so DNS lookups happen on the proxy side, not on your machine.

# pip install "requests[socks]"
import requests

proxy = "socks5h://<USERNAME>-type-residential-country-us:<PASSWORD>@gw.nodelighthouse.com:11000"
r = requests.get("https://api.ipify.org", proxies={"http": proxy, "https": proxy}, timeout=30)
print(r.text)

Questions

My password has special characters. Do I need to escape it?

Yes, when it goes inside a URL. Wrap it with urllib.parse.quote(password, safe="") before building the proxy string. Characters such as @, : and / otherwise break URL parsing.

Why do I get the same IP twice on the rotating port?

For HTTPS, the gateway sees one tunnel per connection. A requests.Session reuses connections, so several requests can travel through the same tunnel and exit from the same IP. Use plain requests.get calls, or a fresh Session, when you want a new IP each time. Rotating requests can also land on the same IP by chance.

Which port should I use for scraping?

Port 9000 for independent page fetches, port 10000 when a sequence of requests must come from one IP (a login, a cart, paginated results tied to a session). See the web scraping guide for more on choosing.