Use case · Fintech · Marketplace

Move the fraud line without eating losses

The decline threshold is 0.7. Tighter blocks good customers, looser costs money, and nobody wants to be the person who changed it — so it never moves.

chargebacks auto-revert1% canary before anything

The problem

The decline threshold is 0.

Risk thresholds are the clearest example of a number that badly needs experimentation and almost never gets it. The downside of a wrong move is measured in real money and real fraud, so the rational choice for any individual is to leave it alone.

That caution is expensive in the other direction. A threshold tuned for last year's fraud patterns declines customers who would have paid, and nobody attributes lost revenue to a setting nobody touched.

The only way this becomes safe is if the blast radius is small by construction and the exit is automatic — a 1% canary, a chargeback guardrail that reverts rather than warns, and a decision record showing exactly which thresholds were in force.

risk/decision.py
# the number nobody will touch
DECLINE_THRESHOLD = 0.70
REVIEW_THRESHOLD  = 0.45

if
  score > DECLINE_THRESHOLD:
    return decline()

# parameterized, it can be moved
# on 1% of traffic and reverted

What you put under control

Declared once, then governed centrally. This is the only code step.

Parameters
ParameterTypeDefaultRange
risk.decline_thresholdnumber0.700–1
risk.review_thresholdnumber0.450–1
risk.step_up_auth_enabledbooleanfalse
Surfaces that consume them
SurfaceKindRole
Risk servicebackenddecision
Checkoutfrontendrender
Manual review queueotherside_effect

Those bindings are what compute the risk class — critical: backend surface · decision binding · manual override raised.

Variants are parameter value sets
Variantdecline_thresholdstep_up_auth
control0.70false
looser0.76true

Risk class: critical

backend surface · decision binding · manual override raised. A human approves the measurement plan before any traffic, and again before the value becomes the product default.

How it runs

The mechanism is configuration. The application code is already done.

Gradual rollout, canary first
Chosen for a critical-risk change.

Deliberately not an experiment. There is no appetite for a 50/50 split on fraud decisions, so this runs as a one-treatment ramp: 1% canary, then 5, 20, 50, 100 — each step gated on chargeback health.

Adaptive optimization is explicitly the wrong tool here. You never want traffic drifting toward a threshold on its own when the loss function includes fraud.

Phases running
Setup
Canary
Rollout
Ramp
Complete
Guardrails on every phase
chargeback_rate
must not increase > 0.1%
blocking · revert
false_positive_rate
must not increase > 1%
blocking
manual_review_volume
must not increase > 20%
warning

What you get back

A verdict you can defend, and the reasoning kept for next time.

risk-fraud-thresholds / measurement day 4 of ~7
Evidence still accumulating — hold
Leading variant is ahead on the primary metric, but the interval has not cleared zero. All guardrails passing.
primary metric
+2.4%
approval_rate · conversion rate · per authorization
measurement progress
64%
of the sample needed for a verdict
guardrails
3 / 3
passing, none near threshold
approval_rate primary
looser +2.4% · anytime-valid interval still crosses zero
+2.4%
chargeback_rate guardrail
inside the acceptable band on all arms
passing
false_positive_rate guardrail
inside the acceptable band on all arms
passing

A threshold that can finally move — because the worst case is 1% of traffic for twenty minutes and an automatic revert.

When it completes, the winning values become the parameter defaults and the whole thing — question, variants, evidence, verdict — is retained. The next person to propose moving this number starts from the answer instead of the argument.

Retained after this change

The question asked, the values tried, the interval at the moment of the call, which thresholds were in force, and who approved the promotion.

Searchable by parameter, surface, or metric

Keep reading

The mechanics behind this page.

Start with this one number

Parameterize it this week. The first measured change usually pays for the integration.