We Built the Tool They Asked For. Most of Them Would Rather We Ran It.

Customers asked to run the collector themselves, so we built them a tool. Most are on a scheduled extract now. What we got wrong about the control they wanted.

By Chirag Jakhariya, Co-founder & CEO · 10 September 2026 · 5 min read

"Can We Run It Ourselves?"

We heard that question often enough to stop treating it as an edge case.

The work behind it was always similar. Someone needs business listings for a set of trades across a set of towns, and the set keeps moving. This month it's one trade in a dozen postcodes. Next month it's three others somewhere else, because a campaign shifted and nobody told the data team until it had.

Waiting on us for every one of those runs felt slow to them. They were right that it was.

So they asked for the collector itself. We said yes and built it. What happened over the following months is the part worth writing down, because it wasn't what we expected.

What We Built, and What It Put in Their Hands

It's a command-line script the customer runs on their own machine.

Two things sit with them: which categories to pull, and which locations to pull them from. They set both, run it when it suits them, and the output lands locally in a clean format they can load into whatever they already use.

That's the shape of it. Nothing exotic, and the plainness is the point - it needed to be usable by someone who isn't an engineer but can follow a runbook.

It's worth being precise about what that control covered, though, because this is where the story turns. They controlled the query. They never controlled the collection.

Three things stay fixed, and none of them are exposed to the customer:

• The rate it runs at
• How it identifies itself
• What it does when a page doesn't come back

We'd have said no to a version that let people turn those up. That's the same position we take on the data extraction work we do generally. A collector that's polite is a collector that keeps working, and handing someone a dial for that would have been doing them a disservice dressed up as a feature.

Then the Source Changes Shape

Sources get reorganised. Fields move, pages get restructured, and a script that matched fine yesterday stops matching today.

That's normal. It isn't a sign that anything went wrong, and it's the single most predictable thing about this work - we say as much in what happens when a site changes its layout.

Here's what changes when the script is on someone else's machine.

When we run collection, we're watching it. We usually spot a break before the customer notices anything. When the customer runs it themselves, that order reverses. They find out first, and they find out as an empty file on the morning they needed the list.

That's the real difference between the two models. It isn't infrastructure, and it isn't cost. It's who's holding the thing when it breaks.

It's also why maintenance sits on its own subscription instead of being folded into the tool. A script is something you buy once. Keeping it working as sources move is ongoing.

Pretending those are the same purchase would just mean the first reorganisation turned into an argument about what was included. We'd rather say it upfront, which is roughly how we handle ongoing collection pricing across the board.

There's a second consequence of running it yourself, and it deserves a mention even though it's an uncomfortable one.

Two US rulings this summer pointed in different directions on what a data collector can be held to. One narrowed the claims against a tooling company, though only partly - some were dismissed outright and others left open to be refiled. In a separate case, a supplier was named alongside the company that used the data, and those claims survived an early attempt to have them thrown out.

Read together, the thing people are taking from those is that running collection draws more attention than owning a copy of some code. If you're running it, you're the one operating it. That's not legal advice and we're not the people to give it - if that distinction matters to your business, ask someone whose job it is.

Where Most of Them Ended Up

Many customers still run the local tool. It does what it was built to do, and there's nothing wrong with it.

But most of them are on a weekly or monthly extract now instead. Same categories, same locations, same output format. We just run it and deliver it.

What surprised us was the direction people moved in. We'd assumed the pull was toward owning the infrastructure, because that's what the ask sounded like from the outside.

It wasn't. What people wanted was to be able to say "these trades, these towns, this week" without queueing behind a request. That's a scheduling problem and a control problem, not an infrastructure one. Once the delivered version gave them the same say over categories and locations, running it themselves stopped being worth the Tuesday morning where the file comes back empty.

Some kept the tool, and for good reasons. A few need to run it at odd hours on their own schedule. A few have a policy about where data lands first and would rather that be their own machine than anyone else's. Those aren't strange requirements - there are just fewer of them than we'd guessed, and they're the cases where local execution genuinely earns its keep.

If your source list is growing, or the data feeds something that can't sit idle for a day, that's usually the point where continuous collection makes more sense than a script and a calendar reminder.

What We'd Do Differently

We heard "we want to run it ourselves" and built for execution control. The requirement underneath it was query control. Those sound like the same thing in a sales call and they aren't the same purchase at all.

If we were quoting this work again, we'd ask which one someone actually means before quoting either. It's a two-minute conversation, and it would have saved a handful of people a detour through a tool they didn't really need.

You can run that conversation on yourself. Ask what would have to be true for you to stop caring who runs it.

If the answer is "nothing, the data has to originate on our hardware" - that's a real constraint, and local execution is the right call. If the answer is "I'd stop caring the moment I can change the categories without emailing anyone" - you wanted query control, and a delivered extract gets you there without inheriting the Tuesday mornings.

Most people, when they sit with it, find they're the second kind. It's worth knowing which you are before anyone quotes you for either.

If you're not sure which of the two you're asking for, send us the categories and locations you need and we'll return a sample against them. You'll see the shape of the data before deciding who runs the collector.