Accuracy and backtesting

The Accuracy page answers two different questions. The scorecard asks how good are our people’s calls? The backtest asks how good is Pipemetry’s model on our data? They are computed differently and worth reading separately.

The accuracy scorecard

For each of the last few closed fiscal periods — plus the current one, flagged as in progress — and for each person who submits a forecast:

Column	What it is
submitted	That person’s latest commit call for the period. Blank if they made no call — never a zero
actual	Realised closed-won attributable to them, reconstructed as of the period’s end date
ai_projected	The selected model’s projection for that owner and period — the same number every other screen shows
error_pct	Signed (submitted − actual) / actual, as a percentage. Positive means they over-called

Actuals are replayed from the event log rather than read from a stored aggregate, so they cannot drift away from what the rest of the product says.

An in-progress period has no final actual yet, so its actual is reported as closed-won to date, it is flagged as open, and its error is excluded from every aggregate.

The per-person aggregates

Computed over the closed periods only:

• MAPE — the mean absolute error percentage. Lower is better.
• Bias — the mean signed error percentage. Positive marks a habitual over-caller; negative marks a sandbagger. This is the number that tells you which direction someone is wrong in, which MAPE alone hides.
• Compliance — the share of considered closed periods in which they actually submitted a call.
• Score — 100 − MAPE, floored at 0, as an at-a-glance grade.

Worth internalising: a rep with 20% MAPE and +20% bias is consistently optimistic, and you can correct for that. A rep with 20% MAPE and 0% bias is just noisy, and you cannot.

Owner vs manager

Grouping by owner scores each individual submitter against their own realised number. Grouping by manager rolls up through the reporting hierarchy: a manager’s own submitted call is compared against the whole team’s realised actual — does their number land where everything beneath them ends up?

Manager roll-up is a Pro capability

Rolling calls up the manager hierarchy — and the view-as that goes with it — is sold on the Pro plan. The hierarchy screens and manager-scoped forecast reads enforce that. This particular scorecard grouping is not itself gated today, so on Starter you may find it works; treat that as something that can be brought in line with the plan, not as a Starter feature to build on.

You can score up to 8 closed periods; the default is 4.

The backtest

The backtest re-runs the projection as it would have been on days that have already passed, and scores it against what actually closed. Because Pipemetry reconstructs pipeline state from an event log rather than accumulating daily snapshots, it can do this the day your backfill lands instead of after months of collecting snapshots.

It appears on the Accuracy page, and Onboarding links to it once setup completes.

How it is computed

For each of the last closed fiscal quarters (up to 4, default 4 — the in-progress quarter is never scored):

• The projection is re-run at three vantage points inside the quarter — quarter start (day 1), mid-quarter (50%), and 80% through — using the same leak-free point-in-time machinery every screen uses. The vantage day is the quarter’s start plus span_days × pct ÷ 100, floored.
• The actual is that quarter’s realised closed-won, taken as a boundary delta: cumulative closed-won at the quarter’s end minus cumulative closed-won the day before it started. It is the quarter’s own number, not an all-time running total.
• Each vantage’s APE is |projected − actual| ÷ actual × 100.
• The summary line is the MAPE per vantage — the mean of the scoreable APEs across quarters.

Reading three vantages rather than one is the useful part: it shows you when in the quarter the forecast becomes trustworthy for your business.

When it refuses to answer

The backtest is built to stay quiet rather than invent a scorecard:

• A quarter with no signal at all — zero actual and zero projection at every vantage — is pre-history and is dropped, not scored as “we said zero and you did zero”.
• If every quarter is dropped, the whole response is insufficient_history with empty lists, and the panel says to come back after your first full quarter. It never renders fabricated zeros.
• A quarter with real pipeline but a zero actual stays visible, with its APE shown as a dash — it is informational and excluded from the MAPE.

The coverage badge

Alongside the scorecard sits a badge derived from your connect-time field-mapping audit, telling you how much real history the replay had to work with:

Badge	Meaning
history-backed	Every audited canonical field is history-tracked — the replay uses real history end to end
partial	Some fields carry history, some do not
current-only	Fields exist but none carry history — the replay leans on current-state anchors
unknown	No mapping audit could be read

If your badge says current-only, turn on field history tracking in your CRM and re-run the backfill. See Connect Salesforce, step 5.

What you see depends on who you are

The backtest is scoped to the deals you can see: a rep backtests their own book, while someone at the top of the hierarchy sees the whole org.

Related

• Forecast models — what is being scored.
• API and webhooks — GET /v1/accuracy returns the scorecard as JSON.