When the agent stays the same but its success rate changes

Salesforce says its Winter ’27 update changes several Agentforce Observability metric names and formulas beginning the week of October 12. Under the new definitions, Success Rate includes a completed action step and no step errors; abandonment and deflection use revised eligibility and classification rules; Turn Completion Rate uses a 60-second first-token threshold; and Agent Interaction Duration leaves the dashboard templates. Salesforce documents these changes in its release notes.

That creates a measurement break. A rate can move even when the underlying agent behavior does not. Conversely, stable headline numbers could conceal a behavioral change if the revised formula offsets it. The release note establishes the definition changes, not either outcome for a particular deployment.

For platform and analytics teams, the immediate question is therefore not simply whether the dashboard improved. It is which definition produced each value.

A trend line can cross two measurement systems

A metric name is only part of its identity. Its denominator, eligibility rules, classifications, thresholds, exclusions, and aggregation window determine what the number means.

Consider Turn Completion Rate. Salesforce documents a 60-second first-token threshold in the updated formula. A pre-change value and a post-change value may carry the same label while classifying turns under different rules. Joining them into one uninterrupted series would imply comparability that the source does not establish.

YuniQ’s analysis makes the operational consequence explicit: dashboards can change without corresponding agent-behavior changes. It recommends recording each organization’s actual change date, preserving prior definitions and outputs, and avoiding uninterrupted trend lines unless both periods are reproduced under a common definition. Read YuniQ’s rebaselining analysis.

The actual transition date matters because a release window is not necessarily an organization-specific activation timestamp. Salesforce says the changes begin during the week of October 12; that statement alone does not identify the exact boundary for every environment.

Give every reported value a metric version

Our proposed response is a metric-version ledger: a compact record that binds each KPI value to the semantics that produced it. This is an operating design derived from the documented definition changes and YuniQ’s preservation recommendation, not a Salesforce artifact.

For each metric version, record:

• the displayed metric name and an internal stable identifier;
• the formula, denominator, eligibility rules, classifications, thresholds, and exclusions;
• the effective timestamp observed in the organization;
• the dashboard, export, query, or API that produced the value;
• the reporting window and aggregation grain;
• the retained output from immediately before and after the transition;
• whether historical events can be recalculated under the old and new definitions;
• the owner who approved its use for a particular decision.

A ledger entry might label the two definitions success_rate.pre_winter27 and success_rate.winter27. The internal names are a proposed convention. Their value is that a familiar dashboard label cannot silently erase the semantic boundary.

The same treatment applies when a metric disappears. Because Salesforce says Agent Interaction Duration is removed from dashboard templates, teams can distinguish removal from templates from deletion of all underlying data or interfaces. The supplied evidence establishes only the template change. Continued availability elsewhere remains an environment-specific question.

Rebaseline according to the decision

Not every report calls for the same repair. We would choose among three treatments based on what the comparison will decide.

Split the series. End the old line at the observed transition and begin a new one under the revised definition. This is the clearest presentation when historical events cannot be recalculated.

Reproduce both periods under one definition. If retained event data contains every field used by the selected formula, recompute the earlier and later windows with identical rules. Before treating the result as comparable, verify that eligibility and classification inputs are available for both periods; the supplied sources do not establish that they are.

Show the bridge. During the transition, calculate old and new definitions over the same fixed event set when the data permits it. The gap estimates the effect of the definition change for that sample. It does not establish how the agent would behave under a different deployment or future workload.

A rollout or investment memo can then state its evidence precisely: for example, “The decision uses Success Rate under the Winter ’27 definition for events collected after the organization’s recorded transition.” If the memo spans the boundary, it can state whether the series was split, reproduced, or bridged.

Separate semantic change from behavioral change

A practical investigation can treat a KPI movement as two parallel hypotheses:

1. The measurement system changed.
2. The agent, traffic, configuration, tools, or operating conditions changed.

The metric-version ledger addresses the first hypothesis. Deployment records, configuration history, event traces, and workload slices address the second. Neither source supplied here reports the behavior of a specific production agent, so a changed dashboard value alone cannot resolve the attribution.

This distinction also changes the review question. Instead of asking whether success rose, ask:

• Which metric version produced each side of the comparison?
• Did the eligibility population change?
• Did a threshold or classification rule move?
• Can both windows be recomputed from the same event fields?
• What agent or workload changes occurred across the same boundary?
• Which version supports the pending decision?

Those questions form our recommended rebaseline check, not vendor requirements. They are useful because they force the semantic break and any behavioral change into separate records.

Preserve the claim before scaling the program

The costly failure is not a changed dashboard. It is allowing a changed definition to inherit the authority of an older baseline.

Before using the affected KPIs to justify expansion, freeze the prior outputs, record the organization’s observed change timestamp, version the formulas, and choose a comparison treatment. Then attach that metric version to the decision itself. Teams designing or revising the surrounding agent system can carry the same measurement boundary into the build process.

An agent program can tolerate evolving metrics. It cannot interpret them coherently if the definition disappears behind the number.