Confidence & Evidence

Confidence is the trust a piece of learning has earned. It starts where the author set it and is moved by one thing only: the outcomes of runs that relied on it. This is the mechanism that makes the loop self-correcting — learning that keeps leading to good outcomes rises into every briefing, learning that keeps leading to failures sinks out of them, and no one has to decide which is which.

Confidence score

Confidence is a number from 0.0 to 1.0. It starts at the value the author wrote (default 1.0) and is clamped to that range forever after:

entry = mem.write("svc", "pattern", "use connection pooling", confidence=0.85)
print(entry.confidence)        # 0.85
print(entry.evidence_status)   # "untested" — nothing has relied on it yet

Write beliefs below 0.9. A hypothesis that starts at certainty has nowhere to go but down, and the first failure will take it a long way.

Outcomes

When a run finishes, the agent commits an outcome. Four kinds are recognised, and they are not symmetric:

Outcome	Weight	Meaning
`SUCCESS`	1.0	The work held up
`MINOR_FAILURE`	1.5	A setback
`FAILURE`	2.0	The run failed
`CRITICAL_FAILURE`	3.0	The run caused a severe failure

Failures weigh more than successes because trust should be easy to lose and slow to rebuild.

Four older names are still accepted as aliases: clean_deploy (success), regression (minor failure), p2_incident (failure), p1_incident (critical failure).

from amfs import OutcomeType

updated = mem.commit_outcome(
    "INC-1042", OutcomeType.CRITICAL_FAILURE,
    causal_entry_keys=["checkout-service/retry-pattern"],
    task_input="checkout worker restart looping in prod",
    response_text="Rolled back; worker stable.",
)
for entry in updated:
    print(entry.key, entry.confidence, entry.evidence_status)

How an outcome moves confidence

Each outcome adds evidence to every entry it cited, and the entry’s confidence is the balance of that evidence against the author’s prior:

confidence = (2 × prior + evidence_for) / (2 + evidence_for + evidence_against)

The prior counts as two units of evidence, so one confident outcome moves a fresh entry a long way and a handful settle it. Three things decide how much evidence one outcome adds:

Severity. The weight in the table above.
Credit split. The weight is divided across the entries the run relied on. An outcome that cited eight entries cannot hand each of them a full unit of evidence; one that cited a single entry gives it everything.
Surprise. A failure on an entry trusted at 0.95 counts nearly twice as much as one on an entry at 0.5, and a success on an already-trusted entry counts for little. Confirmation of what everyone expected is weak evidence.

Evidence also decays: before each update, both balances are scaled by 0.8, so the last few outcomes dominate and an entry can recover from an old failure once it starts working again. With the defaults, a fresh entry at 0.7 drops to about 0.26 on its first failure and lifts to about 0.82 on its first success.

First-strike tolerance

An entry with three or more successes and no failure yet is treated differently on its first failure: the surprise term is left out. A single failure against a long run of successes is more often the agent’s slip or a noisy environment than a change in the world. The entry is left contested at about 0.62 instead of being discredited outright; a second failure in a row still discredits it (about 0.40). A real change in the world is unlearned in two strikes.

Evidence status

Every entry carries a label derived from its outcome record, and the briefing, retrieve() and the dashboard read the label rather than the raw number:

Status	Meaning
`untested`	No outcome has relied on it yet
`validated`	Outcomes have relied on it and none has failed — or its record is strong enough that an occasional failure does not change the verdict
`contested`	Outcomes disagree, or a long-validated entry has just taken its first failure
`discredited`	A failure pushed it below `0.5`. It is held back from retrieval and listed in the briefing’s `discredited` section, with `replaced_by` once a later run succeeds with something else

Nothing is deleted when an entry is discredited. It stays in history, its versions stay readable, and it comes back if the evidence does.

Regime shift

A regime shift is the record saying that a rule which used to work has just stopped: validated at least three times, latest outcome a failure within the last seven days, label no longer validated. The briefing lists these in regime_shift, and guidance switches to explore for the scope, so agents try something other than the rule that just failed. The flag clears after the window — a rule discredited last month is history, not a reason to keep skipping what has been winning since.

What an outcome is credited to

If causal_entry_keys is not given, the outcome is credited to what the agent read during the session:

Call	Enters the causal set?
`read()` and `read_from()`	Yes, every entry returned
`retrieve()`	Only the top-ranked hit, so an outcome does not reinforce candidates the agent never used
`search()` and `list()`	No — recorded on the trace as query events, not treated as causal

The set is cleared after each commit_outcome(), so the next task starts a fresh causal window.

Mark failed attempts

When a remembered fix did not work and the agent tries something else, it should say so before moving on:

mem.record_attempt(summary="restarted worker per runbook; queue still stuck")

Everything read and done since the previous boundary is attributed to that attempt. At commit time those entries receive the attempt’s failure, and only what was read afterwards is credited with the final success. Without this, an entry that sent the agent down the wrong path would be reinforced by the recovery from it — the opposite of learning.

Name what you acted on

A briefing serves several entries and the agent follows one. Passing causal_entry_keys on the outcome sends the result to that one; left unnamed, every entry read since the last boundary is charged, and a correct lesson that merely shared the context with a wrong one loses confidence alongside it.

Time decay (optional)

Outcomes are the only thing that changes stored confidence. Separately, the SDK can apply time decay on read, so that learning nobody has confirmed in a long time counts for less when compared against min_confidence. It is opt-in:

mem = AgentMemory(agent_id="my-agent", adapter=adapter, decay_half_life_days=30.0)

Four signals set how fast an entry decays:

Signal	Effect
Time	Exponential decay with the configured half-life
Kind of learning	Facts 1×; beliefs decay 2× faster; experiences 1.5× slower; procedures 2× slower
Outcome validation	Entries with any outcome decay 2× slower
Recall	Frequently read entries resist decay (`log1p(recall_count)` extends the half-life)

A fact with a 30-day half-life, 10 reads and one outcome has an effective half-life of about 204 days. Actively used, outcome-validated learning persists; cold, unvalidated beliefs fade.

Filtering by confidence

entry = mem.read("svc", "pattern", min_confidence=0.5)
results = mem.search(min_confidence=0.7)

Prefer reading evidence_status to picking a threshold. An untested 0.9 and a validated 0.9 are different things to act on, and retrieve() already holds discredited entries back.

The feedback loop

This is what separates learning from storage. Storage gives back what was put in. A record with outcomes attached gives back what has been shown to work — and keeps revising the answer as the evidence changes.

SenseLab also measures, per account, how much each kind of outcome should count and how long learning stays useful, from your own history. See ML Layer.

Product

Getting Started

Guides

Self-Hosting

Core Concepts

Pro Features

Confidence & Evidence

Confidence score

Outcomes

How an outcome moves confidence

First-strike tolerance

Evidence status

Regime shift

What an outcome is credited to

Mark failed attempts

Name what you acted on

Time decay (optional)

Filtering by confidence

The feedback loop

​Confidence score

​Outcomes

​How an outcome moves confidence

​First-strike tolerance

​Evidence status

​Regime shift

​What an outcome is credited to

​Mark failed attempts

​Name what you acted on

​Time decay (optional)

​Filtering by confidence

​The feedback loop

Confidence score

Outcomes

How an outcome moves confidence

First-strike tolerance

Evidence status

Regime shift

What an outcome is credited to

Mark failed attempts

Name what you acted on

Time decay (optional)

Filtering by confidence

The feedback loop