Beta

Signal Accuracy Auditor

Computes a false-positive rate, overall and per signal type, from a log of previously flagged or prioritized accounts and their eventual outcomes, surfacing which triggers actually predict real movement and which are generating expensive false alarms, only judging a signal once it has enough resolved volume to mean something.

Any CRM

The problem

A signal changing an account's priority is a bet, not a fact. Most teams get good at detecting activity and never circle back to check whether the flags it generated actually turned into anything.

How it works

  1. 01Reads a log of previously flagged or prioritized accounts, the signal that triggered each flag, and the eventual outcome
  2. 02Computes a false-positive rate overall and broken down per signal type
  3. 03Only judges a signal type's rate once it has enough resolved volume to mean something
  4. 04Surfaces real rejection reasons from reps so recurring patterns can become negative criteria instead of getting forgotten

Verified on real test data

Signal Accuracy Auditor

27 synthetic flagged-account events across 5 signal types, including two signal types with an identical 66.7% raw false-positive rate, one with enough resolved volume to judge and one deliberately too small, plus 2 still-pending flags used to check they get excluded from every rate.

1/1

High false-positive signal caught

1/1

Identical-rate small sample correctly withheld

2/2

Pending flags correctly excluded

  • Correctly flagged one signal type at a 66.7% false-positive rate across 6 resolved flags, while correctly withholding judgment on a different signal type at the exact same 66.7% rate, because that one only had 3 resolved flags on record.
  • Correctly excluded 2 still-pending flags from every rate calculation rather than letting an unresolved batch drag a signal type's accuracy down artificially.
  • Correctly computed a 40% false-positive rate on a mid-tier signal and left it unflagged against the 50% default threshold, avoiding an overly aggressive call on a genuinely mixed signal.

Two signal types looked identically unreliable by raw rate alone. Only one of them had enough evidence to actually say so.

See the full test run →

Questions about Signal Accuracy Auditor

What counts as a false positive?

A flagged account that outcome-marked Stalled or Rejected. Converted or Progressed count as a true positive. A blank, still-pending outcome is excluded from every rate entirely, it hasn't had time to resolve yet.

Why does a signal need a minimum sample before being flagged?

Because a high raw false-positive rate on 2 or 3 flags isn't evidence of anything. The same discipline Attribution Builder and Enrichment Reliability Auditor apply elsewhere in the toolkit: report the number honestly, but don't judge it until there's enough volume.

Runs against your CRM using your own credentials, through your own AI instance. We don't copy, store, or retain your data.