The dashboard is green and one of your models is not on it

Amazon Bedrock AgentCore Observability natively covers only agents on AgentCore runtime in AWS. On August 13, 2026 AWS published a walkthrough for wiring up agents running on-premises and on other clouds. https://aws.amazon.com/blogs/machine-learning/monitor-on-premises-and-multi-cloud-ai-agents-with-agentcore-observability/

The part worth reading is in a second post from August 14, 2026, not the first. https://aws.amazon.com/blogs/machine-learning/building-agentic-workflows-with-sagemaker-ai-and-bedrock-agentcore/

Put a managed model and a self-hosted model inside the same agent, and auto-instrumentation covers the managed one while silently dropping token accounting for the other.

Nothing errors.

No alert fires.

The number is simply absent, and an absent number looks exactly like a low number on a cost chart.

The observability failure that matters is not the outage. It is the metric that quietly stops being collected.

Auto-instrumentation is sold as the thing you no longer have to think about, which is precisely why nobody audits what it missed.

Closing the gap takes custom spans on the self-hosted path and a usage flag on the streaming call that most teams have never set.

An agent metric nobody has verified end to end is a guess with a chart attached.

Worth confirming before trusting a cost dashboard:

• Which models in the agent emit token usage, and which are assumed to • Whether a self-hosted endpoint reports usage on streaming responses at all • Whether spend per run comes from spans or is reconstructed from invoices • What the chart shows when a span never arrives, rather than arriving as zero

The green dashboard was never evidence of coverage. It was evidence that whatever is reporting is reporting fine.

Monitoring tells you the agent is running. Instrumentation tells you what it is spending.

#ai #ai-agents #observability #llm