Ecommerce Data Scraping Company: How to Choose and Test One
The buyer's side of a page full of vendor sales pages: what an ecommerce data scraping company should deliver, how products get matched, what it costs and a sample test to run before you sign.
By Chirag Jakhariya, Co-founder & CEO · · 13 min read
A good ecommerce data scraping company delivers clean product, price and stock data from the retailers you name, matched to your own products, on a schedule you choose. To choose one, ask how they match products across sites and how they handle site changes. Then test a sample on products you already know before you sign. Prices are usually quoted per project or as a monthly retainer.
Most pages that rank for this search are vendor sales pages. This guide is the buyer's side: what to ask, what it costs, and a test you can run on any provider's sample, including ours.
What should an ecommerce data scraping company deliver?
It should deliver one row per product, per retailer, per collection time, in a format your team already uses. The fields below are the usual core set for pricing and assortment work.
| Field | What it tells you | What to check |
|---|---|---|
| Price and sale price | What a shopper pays today, and the "was" price | Both captured, not just one |
| Stock status | In stock, out of stock, or back-ordered | Out-of-stock rows kept, not dropped |
| Seller | Who is selling on a marketplace listing | One row per seller where several compete |
| Variant | Size, color or pack size the price belongs to | Price tied to the right variant |
| GTIN, UPC or EAN | The barcode number that identifies the item | Present wherever the page shows it |
| Timestamp | When the price was collected | Date and time on every row |
On top of the fields, expect the delivery details in writing:
- The format: CSV, Excel, JSON, or a feed into your database.
- The schedule: once, weekly, daily or hourly.
- Who fixes a broken collection, and how fast.
- The quality checks run before each delivery.
What else can it collect besides prices?
Price is the most common ask, but the same pages hold other data that pricing, brand and category teams use:
- Reviews and star ratings, to spot product problems early.
- Catalog content: titles, images, specs and descriptions.
- Promotions, coupons and bundle offers.
- Search position on a retailer's own site for your key terms.
- Third-party sellers on marketplace listings, and prices below your advertised minimum (MAP, the minimum advertised price).
The questions and the sample test below work the same for any of these.
How should products be matched across retailers?
Matching is the part that decides whether the data is usable. A price for "the same" product is only useful if it really is the same product, in the same size and pack.
Match on the barcode number first
The strongest match is the GTIN, the global barcode number on the package. In North America it is usually the 12-digit UPC; elsewhere, the 13-digit EAN. GS1 US, which issues these numbers, says each size, flavor or packaging style needs its own GTIN. So two listings with the same GTIN are almost always the same item.
What if the retailer doesn't show a barcode?
Many sites don't. Then a provider matches on the brand, title, model number and attributes such as size and color. Ask them to show their matching rules, and how each match is marked as "exact" or "likely".
Why do variants and pack sizes break matching?
Because one listing can hide hundreds of prices. In a recent write-up, Zyte describes one marketplace with 13.3 million variants across 402,000 products, and one listing with 4,900 combinations. Their advice is to agree what counts as a variant up front and to test on difficult pages.
The common errors are easy to miss:
- A 6-pack matched to a single unit, so a competitor looks six times cheaper.
- A 256GB phone matched to the 512GB model.
- The price of the default color applied to every color.
- A bundle treated as the main product.
10 questions to ask before you hire an ecommerce data scraping company
These questions separate a provider who has run price feeds for months from one who has only built a demo. Ask them on the first call and ask for the answers in writing.
- How do you match our products to each retailer's listings, and what share do you match by barcode?
- How do you mark a match you're unsure of, and who reviews it?
- Do you capture the price per variant and per seller?
- Are out-of-stock items kept in the data, with their last price?
- When is each price collected, and is the time on every row?
- What happens when a retailer changes its website, and is the fix included in the price?
- Which quality checks run before each delivery?
- How do you respect each site's rate limits, robots.txt and terms?
- Will you run the free sample on products we choose?
- What exactly is in the price, and what would make it go up?
What does ecommerce data scraping cost?
Self-serve price-monitoring tools start at about $99 a month for 100 products. Managed services rarely publish prices, because the cost depends on how many sites, how often, and how much matching you need. Most quote a fixed price per project or a monthly retainer.
What do price-monitoring tools charge?
Prisync's pricing page is a useful public benchmark. Its URL-based plans cost $99, $199 and $399 a month for up to 100, 1,000 and 5,000 products. Its channel-based plans run from $199 to $599 a month.
With a tool, your team adds the competitor URLs and checks the matches. That's the trade: a lower price for more of your own time.
How do managed services price it?
| Option | How it's priced | Best for | Who maintains it |
|---|---|---|---|
| Price-monitoring tool (e.g. Prisync) | From $99 a month, by product count | A small catalog and a few competitors | You add URLs and check matches |
| Fixed-price project | One price for an agreed scope | A one-off snapshot of a market | The provider, for an agreed window |
| Monthly retainer | A set fee for ongoing collection | Daily or weekly price feeds | The provider, site changes included |
| TheDataHQ | Quoted after a free sample | Many sites, matched data, delivered on schedule | We do, on retainers |
Some providers price per record or per request instead. That looks cheap for a test and can grow fast on a big catalog, so ask for a monthly estimate at your real volume.
What moves the price?
Six things decide most quotes. Use them to compare two offers fairly:
- The number of sites, each with its own layout.
- How often it runs: once, daily or hourly.
- How the site is built: plain pages, or pages that load prices in the browser.
- Volume: thousands of products or millions.
- The fields and the cleaning, including product matching.
- Ongoing maintenance: included in a retainer, or an agreed window on a fixed project.
Our pricing page explains how we set a quote with these six factors.
Price scraping service, scraping tool or in-house?
Choose a price scraping service when you need many sites matched and maintained without your team's time. Choose a tool when the catalog is small and someone can check matches each week. Build in-house only if you have engineers to spare for months of upkeep.
- Tool: fastest start, lowest monthly fee, but your team does the matching and checking.
- Managed service: the provider builds, runs, matches and fixes; you get the data.
- In-house: full control, but site changes become your team's job. If that's your plan, our sister company BinaryBits explains how to build it.
Our price scraping service is the managed option. For a wider comparison of providers, see our guide to competitor price monitoring services.
How do you test the sample before you sign?
Pick 20 products you know well, on 2 or 3 retailers, including your hardest cases. Ask every provider for a sample on those same products. Then check it against the live pages yourself, with the pass marks agreed before you look.
| Test | How to check it | Pass mark (you set it) |
|---|---|---|
| Fill rate | Count empty cells in each field | Agree a minimum per field, such as price and stock |
| Price accuracy | Open the live page for each of your 20 products and compare | The price matches, allowing for changes since the timestamp |
| Freshness | Read the timestamp on every row | Collected within the window you need |
| Variant and pack size | Check sizes, colors and multipacks by hand | No 6-pack matched to a single unit |
| Out of stock | Include a product you know is out of stock | The row is there and marked |
| Currency and tax | Check one retailer in another country, if you track one | Currency named; tax and shipping stated |
How should you score the results?
Score each provider on the same sheet, row by row. Then read their errors, not just their totals:
- A wrong price on a product you chose is a serious fail.
- A missing barcode where the page has none is fine.
- A "likely" match flagged for review is honest; a wrong match marked "exact" is not.
One honest limit: a sample proves the method, not the next six months. Retailers change their sites, so ask how the provider will show you quality over time, such as a fill-rate report with each delivery.
What red flags should you watch for?
Walk away, or at least push back, when you see these:
- An accuracy figure with no explanation of how it was measured.
- Promises of "100% uptime" or "never blocked" on large marketplaces.
- No answer to "how do you match products?"
- A sample built on their own chosen pages, not yours.
- Site-change fixes billed as extras, with no named process.
- No mention of rate limits, terms or data protection.
Scrapers break when sites change, so the fix process matters more than the launch. We explain why in why web scrapers break when site layouts change.
How we run a daily price feed
For one ecommerce client, we track prices across 200+ competitor sites with a daily refresh, daily pricing reports and real-time alerts when a tracked product changes. Before that, the client checked a handful of competitors by hand each week. The project is on our case studies page, anonymized.
The daily run follows the same steps each time:
- Collect the agreed pages from each site, within its rate limits.
- Match each listing to the client's products.
- Run quality checks: empty fields, price jumps, missing products.
- Deliver the report and send alerts on changes.
Across 12 years we've delivered 260+ projects on more than 10,000 websites, with a 99.98% success rate. Those are past results, not a promise for every site.
If your team compares prices by hand today, our post on ecommerce competitor price comparison shows what that costs in time.
Want to run the test above on us? Send us one URL and the fields you want; we'll send back a free sample of the real output. No charge, no commitment. Ask for your free sample.
Frequently asked questions
How much does an ecommerce data scraping service cost?
Self-serve price-monitoring tools start at about $99 a month for 100 products, according to Prisync's public pricing. Managed services usually quote a fixed price per project or a monthly retainer. The quote depends on the number of sites, how often they run, how the sites are built, volume, the fields and matching, and maintenance. We quote after a free sample.
Is it legal to scrape ecommerce websites?
Collecting public product prices is common, but "it's public" isn't the whole test. In the US, the hiQ v. LinkedIn case found collecting public pages likely didn't break the CFAA, the federal anti-hacking law. Yet hiQ was later held to have breached LinkedIn's user agreement. Site terms, database rights and privacy laws all apply. This is not legal advice; ask your lawyer.
How often can product data be refreshed?
Anything from a one-off snapshot to hourly is possible. Most pricing teams use a daily refresh, which shows promotions and stock changes without collecting more than they need. Hourly runs cost more and need more care with each site's rate limits. Choose the slowest schedule that still lets you act in time.
What happens when a retailer changes its website?
The collection for that site can break or, worse, return wrong values without failing. A good provider watches for both, with checks such as empty fields and sudden price jumps, and fixes the collection quickly. Ask whether fixes are included. On our retainers, fixes for site changes come at no extra charge.
Can a scraping company collect Amazon and Walmart prices?
Amazon and Walmart show prices on public product pages, and many providers offer this data. Large marketplaces have many sellers and variants per listing, so ask how the provider captures the price per seller and per variant. Ask how they respect rate limits and terms too. Treat any promise of "never blocked" as a red flag.
What format is the data delivered in?
Most providers deliver CSV, Excel or JSON files, or load the data straight into your database. We deliver in JSON, CSV, Excel, XML or a database, on the schedule you choose. Ask for the sample in the exact format your team will use, so you can test it in your own reports first.
Should I build ecommerce scraping in-house or outsource it?
Build it in-house if you have engineers who can maintain it, because site changes never stop. Outsource it if you want the data rather than the upkeep. A managed provider builds, matches, checks and fixes the collection, and you pay a set fee. A tool sits between the two, with your team doing the matching.