Internal system · Pilot

Marts on top of the data lake

An application with dashboards over two sources: the data lake and the corporate warehouse. It does not compute on the fly - background jobs move aggregates into its own database, and the screen reads a mart whose freshness date is visible.

The pain

Product economics were counted off the marketplace report. Cost price sits on logistics rows too, and there are far more of those - it came out an order of magnitude high.

Now

Two marts: one holds what honestly lands on a product, the other storage, intake, penalties and withholdings that do not spread across items. On screen they are a separate block, not profit that is absent.

What the business gets

The dashboard answers even while the source is unavailable, and it always shows what moment the data is from.
A product with no cost price does not look like a product with a great margin: revenue without cost price travels to the screen as a metric of its own.
Costs that do not land on an item are shown separately rather than smeared across products by an invented rule.
A request for a period deeper than the mart gets an honest refusal instead of a silently truncated series.
The application builds no data collection of its own: it takes the ready lake and keeps a short mart on top of it for the speed of the screen.

From source to result

The data lake
The corporate warehouse
Scheduled background jobs
Marts of our own
Dashboards with a freshness date

Only the background job reaches into the sources - it sits inside the protected perimeter. The interface and the API read nothing but their own database, so where they run does not depend on the source’s network.

How it works

There are two marts on purpose. Storage, intake, penalties and withholdings are zero on product rows, and the withholdings themselves are tied to no item on any row. Spreading them across products takes an invented rule; staying silent means showing profit that is not there.

The source is read from a view rather than the base table: the view normalises the signs on returns and adds the cost price and the link to the accounting system’s item, neither of which the table itself carries.

The source was surveyed before anything was built, and the survey checked not only metadata but freshness, dependencies and schedules. One in seven non-empty tables turned out to carry data older than a week - the catalogue does not tell those from the live ones, because monitoring watches the job status rather than the completeness of the data.

The job queue lives in the same database as the application’s data: enqueueing a job and the business change land in one transaction, so «saved it, lost the job» cannot happen. No separate storage for the queue was needed.

Who maintains it now

An engineer. Both sources are connected read-only; the application can write to its own database and nowhere else.

Other cases for this step

Internal system · In production

Data platform

The number is verified against the source down to a single unit

The pain: Data went through four processing layers and nobody compared the result with what the marketplace shows. Decisions rested on a number with nothing to check it against.
Now: Orders, cancellations and stock reconcile with the marketplace down to a single unit, money down to the kopeck, and the check can be repeated any day with one command.

Verified against
Data marts: sales, stock, presence, P&L, reconciliation

The same system pays off across 2 more steps

Internal system · In production

Advertising management agents

Bids move on data: the DRR plan is ≤ 10 %

The pain: Bids were moved on yesterday’s report and on a hunch. But «yesterday» does not show what actually happened, and a market-wide drop is easily mistaken for your own mistake.
Now: A decision is assembled in order: market regime, product health, the fair target DRR for this item, and only then the bid itself. Metrics come from a matured window; the DRR plan is ≤ 10 %.

DRR
Decision order: market regime → product health → target DRR → bid
Metrics window

The same system pays off on one more step

Own product · In production

Marketplace stores in Claude

An answer about the store - in words, without a dashboard

The pain: To understand what is happening in a store, a seller walks three marketplace accounts and merges reports in a spreadsheet. A dashboard answers only what was built into it.
Now: The seller asks in plain words - «what is running out», «is the advertising paying off», «how much will land on my account» - and Claude goes into the marketplace API with their key.

Coverage
Marketplace keys
Tabular report: downloaded for you and read as rows

measured in August 2026: 945 of 962 operations callable through the API catalogue

The same system pays off on one more step

mcp.hubmarket.ru

Open source · In production

Marketplace knowledge base for agents

The agent answers by the marketplaces’ rules, not from memory

The pain: When an AI agent designs logic against a marketplace’s rules, it leans on memory. Tariffs and limits change, memory does not, and errors surface in money, not at review.
Now: A knowledge base of three marketplaces sits next to the project as ordinary markdown files: one per API method plus the seller help. The agent reads the source, not a retelling.

The base
The build
Updating: one command, the git diff as the change report

GitHub

Type: Internal system
Status: Pilot
Rhythm: the mart refreshes on a schedule
Maintained by: an engineer
Replaced: counting off the raw marketplace report

Python 3.14uvFastAPISQLAlchemy 2.0AlembicPostgreSQL 18procrastinatepython-tdsReact 19TanStack QueryPlaywright