---
title: "Ecommerce Price Scraping: Build a Price Monitor in Python"
description: "Ecommerce price scraping in Python: read JSON-LD first, fall back to LLM extraction, store price and stock history in SQLite, and alert on drops."
canonical_url: "https://bestaiscraper.com/blog/ecommerce-price-scraping"
---

> Index of every page on this site: [llms.txt](https://bestaiscraper.com/llms.txt)

Ecommerce price scraping sounds simple: fetch a product page, read the price, repeat every day. The hard part is not the fetch. It is getting a **correct number** out of thousands of differently built pages and noticing when your scraper silently starts returning the wrong one.

This guide builds a small price monitor in Python that reads each page the cheapest reliable way, falls back to LLM extraction when that fails, and stores a history you can alert on. It uses only the standard library plus `requests` and `beautifulsoup4`.

## The design: cheapest reliable method first

Product pages give you three sources for a price, in order of how much you can trust them:

1. **JSON-LD structured data.** Many shops embed a `Product` with an `Offer` for search engines. Google's [product structured data docs](https://developers.google.com/search/docs/appearance/structured-data/product-snippet) list `price` as a required property of an offer, with `priceCurrency` as the currency in ISO 4217 format and `availability` as a recommended one. Because shops want rich results, this data is usually kept in step with the visible price.
2. **LLM extraction from the page text.** When a page has no usable JSON-LD, describe the fields you want and let a model read the page. This costs more per page than parsing JSON, so use it as the fallback, not the default.
3. **CSS selectors per shop.** Fastest and free, but each shop needs its own selectors and they break silently on a redesign. Worth it only for the few shops you monitor most. See [the three extraction approaches compared](https://bestaiscraper.com/blog/extract-structured-json-from-any-website).

The script below implements 1 and 2 and records which one produced each row, so you can see how often the fallback fires.

## Reading the price from JSON-LD

Two details trip up naive parsers. JSON-LD often wraps nodes in an `@graph` list, and `offers` can be a single object or a list. The price can also sit inside a nested `priceSpecification` instead of directly on the offer, which Google's docs allow. Availability is a schema.org value such as `https://schema.org/InStock` or `https://schema.org/OutOfStock`, and Google also accepts the short names without the URL prefix, so compare on the ending.

## The full script

```python
import json
import os
import sqlite3
import time

import requests
from bs4 import BeautifulSoup

URLS = [
    "https://example-shop.com/products/espresso-grinder",
]
DB = sqlite3.connect("prices.db")
DB.execute(
    "CREATE TABLE IF NOT EXISTS prices ("
    "url TEXT, ts INTEGER, price REAL, currency TEXT, in_stock INTEGER, source TEXT)"
)


def offers_from_jsonld(html):
    """Return (price, currency, in_stock) from the first Product offer, else None."""
    soup = BeautifulSoup(html, "html.parser")
    for tag in soup.select('script[type="application/ld+json"]'):
        try:
            data = json.loads(tag.string or "")
        except json.JSONDecodeError:
            continue
        nodes = data if isinstance(data, list) else data.get("@graph", [data])
        for node in nodes:
            types = node.get("@type")
            if "Product" not in (types if isinstance(types, list) else [types]):
                continue
            offers = node.get("offers")
            offer = offers[0] if isinstance(offers, list) and offers else offers
            if not isinstance(offer, dict):
                continue
            spec = offer.get("priceSpecification") or {}
            if isinstance(spec, list):
                spec = spec[0] if spec else {}
            price = offer.get("price", spec.get("price"))
            if price in (None, ""):
                continue
            currency = offer.get("priceCurrency", spec.get("priceCurrency"))
            avail = str(offer.get("availability", ""))
            return float(price), currency, avail.endswith(("InStock", "LimitedAvailability"))
    return None


def offers_from_api(url):
    r = requests.post(
        "https://bestaiscraper.com/api/v1/extract",
        headers={"Authorization": f"Bearer {os.environ['BAS_KEY']}"},
        json={
            "url": url,
            "fields": {
                "price": "number — current selling price, without currency symbol",
                "currency": "string — ISO 4217 code such as USD or EUR",
                "in_stock": "boolean",
            },
            "instructions": "Use the price shown for the main product, not related items.",
        },
        timeout=120,
    )
    r.raise_for_status()
    d = r.json()["data"]
    return d["price"], d["currency"], d["in_stock"]


for url in URLS:
    html = requests.get(url, timeout=30, headers={"User-Agent": "price-monitor/1.0"}).text
    found = offers_from_jsonld(html)
    source = "json-ld"
    if found is None:
        found, source = offers_from_api(url), "llm"
    price, currency, in_stock = found
    if price is None:
        continue  # keep the gap rather than storing a guess
    prev = DB.execute(
        "SELECT price FROM prices WHERE url=? ORDER BY ts DESC LIMIT 1", (url,)
    ).fetchone()
    DB.execute(
        "INSERT INTO prices VALUES (?,?,?,?,?,?)",
        (url, int(time.time()), price, currency, int(bool(in_stock)), source),
    )
    DB.commit()
    if prev and price < prev[0] * 0.95:
        print(f"DROP {url}: {prev[0]} -> {price} {currency}")
    time.sleep(2)  # be polite between requests
```

Run it once a day from cron or a scheduled GitHub Action. Each run appends one row per URL to `prices.db`, and prints a line when the new price is more than 5% below the previous one. Change the `0.95` to taste.

Three choices in the script are deliberate:

- **It skips instead of guessing.** If neither method finds a price, nothing is stored. A gap in the history is honest; a wrong number triggers false alerts.
- **It records the source.** `json-ld` or `llm` per row lets you spot a shop whose structured data disappeared.
- **It pauses between requests.** Two seconds is a floor. Slow down further for small shops.

## Making LLM extraction safe for prices

A price is exactly the field where a plausible wrong answer does damage. A few rules help:

- **Ask for a number, not a string.** `"number — current selling price, without currency symbol"` keeps "$1,299.00 (was $1,499.00)" from reaching your database.
- **Allow null.** A good extractor returns nothing when the price isn't on the page, rather than inventing one. Our [extraction API](https://bestaiscraper.com/extract) returns `null` for fields it can't find, and the script above skips those.
- **Sanity-check against history.** Reject a new price that is under, say, 20% or over 300% of the last one, and flag it for review instead of storing it. Wrong picks (a related product, a monthly installment) usually show up as large jumps.
- **Expect no JavaScript.** Our API does not execute JavaScript, so a price that only appears after client-side rendering needs a rendering scraper first. [Firecrawl](https://bestaiscraper.com/tools/firecrawl) and [ScrapingBee](https://bestaiscraper.com/tools/scrapingbee) are two options; each tool page lists what we verified from the vendor's own site, and the [web scraping API rankings](https://bestaiscraper.com/best/web-scraping-apis) compare the rest.

You can try the field definition on a real product page in the [playground](https://bestaiscraper.com/extract) before wiring it into a script.

## Staying on the right side of the site

- **Check robots.txt and the terms.** RFC 9309 says robots.txt rules are not a form of access authorization, so an allowed path is not the same as permission. Many shops restrict automated access in their terms, and that is a separate question from the file. When a retailer offers a feed or an official API, prefer it.
- **Keep volume low.** One request per product per day is enough for most price tracking, and polite traffic is less likely to be blocked.
- **Handle blocks as a signal.** If you start getting challenge pages, slow down first. For the technical side, see [how to scrape without getting blocked](https://bestaiscraper.com/blog/scrape-website-without-getting-blocked).

## From monitor to product

Once you have the history, the useful features are small ones: alert on drops, show the lowest price seen, flag items that go back in stock (`in_stock` flipping from 0 to 1). A single table and a daily job cover all three.

If you would rather not maintain page parsing across dozens of shops, the fallback path in this script is the part our [extraction API](https://bestaiscraper.com/docs) replaces: one request per URL with the fields above, typed JSON back.

## Sitemap

See the full [sitemap](https://bestaiscraper.com/sitemap.md) for all pages.
