Skip to main content

​
Diagnostics

Diagnostics are operator/admin surfaces. They help explain what the running Gateway is doing after normal setup is complete. They are not the first place to connect models, channels, services, skills, memory, or tasks. Normal setup starts in the selected Agent:
  • Agent > Models for model refs and roles.
  • Agent > Channels for chat app accounts and routes.
  • Agent > Skills for skill install, edit, config, and per-Agent access.
  • Agent > Tools for per-Agent tool allow/deny.
  • Agent > Services for external APIs such as web/search, Gmail, GitHub, and browser/media.
  • Agent > Memory for session-memory and QMD state.

​
Surfaces

SurfaceUI locationUse it for
LogsLogsLive gateway file logs, filter, auto-follow, export, and redacted troubleshooting evidence.
UsageUsageLocal model usage history over the selected window by Agent, provider, model, session, task, channel/source, tokens, cache tokens, and cost when priced.
DebugAdvanced > DebugOrdered status cards, status/health/model snapshots, plugin runtime diagnostics, provider catalog checks, memory repair preview, event log, and raw RPC inspection.
NodesAdvanced > NodesPaired device/runtime clients, pending pairings, command exposure, capabilities, permissions, and node last-seen state.
ConfigAdvanced > ConfigRaw ~/.fased/fased.json escape hatch for settings that do not yet have a focused page.
DashboardDashboardCompact health widgets only. It intentionally does not replace Logs, Usage, or Advanced diagnostics.

​
First Checks

When something looks wrong:
  1. Check the top-bar health dot.
  2. Open Logs and filter for the failing subsystem.
  3. Open the focused page that owns the setting, for example Agent > Services for web/search credentials or Agent > Channels for a Telegram route.
  4. Open Advanced > Debug only when you need raw status snapshots, plugin runtime diagnostics, or Memory Doctor repair preview.
  5. Open Advanced > Nodes only when device pairing or node capabilities are involved.

​
Usage vs Quota

Usage is local accounting for model calls Fased made. It prefers transcript and task run-log usage records, then falls back only when a better source is not available. Provider account quota/status is separate and belongs to provider auth/status surfaces, not local usage history.

​
Diagnostic Flags

Diagnostic flags are targeted log switches. Use them when normal Logs output is not enough, but avoid raising global logging to trace for every subsystem.
  • Configure flags in Advanced > Config under diagnostics.flags, or use the FASED_DIAGNOSTICS environment override for one run.
  • Flags only do something when code checks that exact flag. The current code-backed operator flag is telegram.http; wildcards such as telegram.* match it.
  • Reproduce the issue.
  • Read the output in Logs or with fased logs --follow.
See Diagnostics Flags.

​
Structured Diagnostics

Structured diagnostics are separate from diagnostic flags. Set diagnostics.enabled: true when you need in-process diagnostic events for model usage, webhooks, queues, session state, stuck-session detection, and tool-loop warnings. Advanced > Debug reads the diagnostic stability snapshot through diagnostics.stability. OpenTelemetry export is another layer. It requires:
  • the bundled diagnostics-otel extension enabled and loaded;
  • diagnostics.enabled: true;
  • diagnostics.otel.enabled: true.
The exporter currently uses OTLP/HTTP. See Logging for the full OpenTelemetry setup.

​
Prometheus

Prometheus belongs under Advanced > Debug, not normal setup. Enable diagnostics.prometheus.enabled when a hosted gateway needs a lightweight scrape endpoint. Default endpoint: 
GET /metrics
Remote clients must authenticate by default. Local loopback scrapes are allowed for operator tooling. The endpoint intentionally exposes only low-cardinality runtime and diagnostic gauges, for example gateway uptime, memory usage, readiness, retained diagnostic events, and dropped diagnostic events. Example config: 
{
  diagnostics: {
    enabled: true,
    prometheus: {
      enabled: true,
      path: "/metrics",
      requireAuth: true,
      includeRuntime: true,
    },
  },
}

​
Cache Trace

Cache trace is a temporary, high-detail diagnostic for embedded agent runs. It is separate from normal Logs and from diagnostic flags.
  • Config path: diagnostics.cacheTrace.
  • Env override: FASED_CACHE_TRACE=1.
  • Default file: $FASED_STATE_DIR/logs/cache-trace.jsonl.
Cache trace can include messages, prompt text, and system text. Enable it only while reproducing a cache/context issue, then turn it off.