Getting eBay competitor prices into a spreadsheet with Python

For resellers, marketplace sellers and pricing analysts, eBay is a deep source of public pricing evidence: what comparable items are listed for, in which condition, and how crowded a category is. Copying that into a spreadsheet by hand eats hours every week. This guide builds a short Python script that collects public eBay search results, turns each listing into a tidy row, and spreads its traffic over proxies, so a few hundred requests do not all come from your office connection and get it throttled.

What the finished script delivers

• One row per listing, holding the title, the price and the link.
• Several result pages per keyword, gathered in one run.
• A CSV file that opens straight in Excel or Google Sheets.

Read the rules, then start

Before the first run, read eBay's robots.txt and Terms of Service and confirm the plan fits them. Stick to what is on show to visitors who are not signed in, and leave authentication, bot challenges and personal information alone. Pause between pages, store results so a page is not fetched twice, and for large-scale or commercial needs, turn to the official APIs that eBay publishes.

Install two libraries

The script needs two packages: requests to make the HTTP calls and beautifulsoup4 to read the HTML.

pip install requests beautifulsoup4

Choosing proxies for the size of the job

eBay shows results according to the visitor's location and watches for unusual traffic patterns, so dozens of requests from one IP often end in CAPTCHAs or temporary blocks. Spread the load, and match the location to the market you are pricing:

All four of our types are compared on the proxies page. Run your proxy through the proxy checker before the first collection.

Credentials and request headers

Each proxy is issued as a host:port address plus a username and password. Put your credentials into a proxies dictionary and send a standard browser User-Agent, so eBay returns its regular desktop results page:

import requests
from bs4 import BeautifulSoup

proxies = {
    "http": "http://USER:PASS@HOST:PORT",
    "https": "http://USER:PASS@HOST:PORT",
}

headers = {
    "User-Agent": (
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
        "AppleWebKit/537.36 (KHTML, like Gecko) "
        "Chrome/124.0 Safari/537.36"
    ),
    "Accept-Language": "en-US,en;q=0.9",
}

How eBay search URLs work

Two URL parameters drive eBay search: _nkw is the search term and _pgn selects the results page. Feed both into one function and you can fetch any page of any search through the proxy:

def fetch_page(keyword, page=1):
    url = "https://www.ebay.com/sch/i.html"
    params = {"_nkw": keyword, "_pgn": page}
    response = requests.get(
        url,
        params=params,
        headers=headers,
        proxies=proxies,
        timeout=20,
    )
    response.raise_for_status()
    return response.text

From listing cards to spreadsheet rows

On a results page, every listing is an s-item list item with a title, a price and a link inside. Layouts drift, so the parser reads a field only after confirming it exists, and it drops the "Shop on eBay" placeholder card that can appear at the top of the list:

def parse_listings(html):
    soup = BeautifulSoup(html, "html.parser")
    items = []
    for card in soup.select("li.s-item"):
        title_el = card.select_one(".s-item__title")
        price_el = card.select_one(".s-item__price")
        link_el = card.select_one("a.s-item__link")

        if not title_el or not price_el:
            continue
        title = title_el.get_text(strip=True)
        if title.lower() in ("shop on ebay", ""):
            continue

        items.append({
            "title": title,
            "price": price_el.get_text(strip=True),
            "url": link_el["href"] if link_el else None,
        })
    return items

Paging politely and saving a CSV

Next, combine the two functions and pause between requests. The time.sleep call is not decoration. It sets the pace of the whole job, and that pace is what the site experiences:

import time
import csv

def scrape(keyword, pages=3):
    results = []
    for page in range(1, pages + 1):
        html = fetch_page(keyword, page)
        page_items = parse_listings(html)
        print(f"Page {page}: {len(page_items)} listings")
        results.extend(page_items)
        time.sleep(4)  # be polite between requests
    return results

if __name__ == "__main__":
    data = scrape("mechanical keyboard", pages=3)
    with open("ebay.csv", "w", newline="", encoding="utf-8") as f:
        writer = csv.DictWriter(f, fieldnames=["title", "price", "url"])
        writer.writeheader()
        writer.writerows(data)
    print(f"Saved {len(data)} rows to ebay.csv")

As written, the example collects three pages for one keyword and saves them to ebay.csv, ready for a pricing sheet.

Rotation for longer keyword lists

A longer keyword list means more requests, and spreading them over more addresses keeps each one's share small. A rotating residential endpoint does this on the proxy side, with no change to your credentials. With a list of static endpoints, let the script pick one at random each time:

import random

proxy_pool = [
    "http://USER:PASS@HOST1:PORT",
    "http://USER:PASS@HOST2:PORT",
    "http://USER:PASS@HOST3:PORT",
]

def get_proxies():
    choice = random.choice(proxy_pool)
    return {"http": choice, "https": choice}

Inside the fetch function, swap the fixed dictionary for proxies=get_proxies() in the requests.get call; every request then picks its own exit from the list.

Empty pages and odd status codes

A page with no listings, or an unexpected status code, is a soft block and a sign to slow down. Pause for longer, retry once or twice at most, and stop for the day if it continues. A small retry loop with exponential backoff around fetch_page keeps the script dependable without hammering the site. To confirm the proxy is really in use, check the outbound IP at any time with our my IP tool.

Sizing the plan

From install to CSV, the script stays well under a hundred lines, so it is quick to trial. Run it on a handful of keywords first, keep within eBay's rules and rate limits, and widen the list only once the data earns a place in your pricing work. When you are ready to size a plan, the pricing page sets out every option side by side.