Monitor checks, incidents, policies, and audit evidence.
Run LatentOps as an operator workflow with live dashboard data, incident review, policy ownership, audit exports, and production readiness checks.
Dashboard
The dashboard reads the same event stream as the API. If it is empty, send a check with log_event: true using a key for the selected workspace.
Decision mix, risk trend, mean risk, intervention count, and onboarding status.
Every pre-tool-call review with prompt, tool, arguments, decision, score, and evidence.
Blocked or escalated checks that need operator review, resolution, or policy follow-up.
Provider, model, framework, environment, latency, token usage, and model-run traces.
Policy activity, custom rule matches, action classes, and audit-ready evidence.
Run monitor comparisons, inspect false positives and false negatives, and review risk-type coverage.
Workspace profile, API keys, provider keys, billing, team, audit, admin, SSO, and RBAC entitlements.
curl "https://api.latentops.space/v1/dashboard/live?limit=120" \ -H "X-API-Key: lo_your_tenant_key_here"
Benchmarks
Compare monitor types on controlled tool-risk datasets. Inspect recall, F1, latency, misses, and risk-type coverage.
Choose AgentRiskBench v0.1, hard negatives, or the mixed preview dataset. Each run is bounded to 10-500 cases.
Run balanced, strict, or permissive policy settings to see how intervention thresholds affect monitor quality.
Compare text-only, action-context, activation-only, and hybrid monitors by accuracy, recall, F1, false positives, false negatives, and latency.
Inspect the first scored cases with labels, expected decisions, hybrid risk scores, decisions, and evidence.
Group results by risk type so teams can see which risky cases are caught and where misses remain.
Use these results for controlled pre-tool-call monitor comparisons, not claims about universal safety or true model intent.
Open Dashboard > Benchmarks, choose dataset, limit, and policy, then run the comparison.
python scripts/run_benchmark_results.py --limit 120 --out reports/benchmark_results.json curl "https://api.latentops.space/v1/benchmarks/datasets" \ -H "X-API-Key: lo_your_tenant_key_here" curl "https://api.latentops.space/v1/benchmarks/results?dataset=agentriskbench_v0_1&limit=120&policy=balanced" \ -H "X-API-Key: lo_your_tenant_key_here"
Incidents
Incidents are created when checks need operator attention. Review the tool, arguments, decision, evidence, routing, and approval state.
Each incident can carry owner, queue, SLA, sensitivity, and tool category metadata.
Escalations start as pending approval. Operators can approve or reject from the dashboard or API.
Open incidents can be resolved after false positive, legitimate block, policy, workflow, or security review.
false positive | legitimate block | policy change | workflow change | security follow-up
Policies
Start with balanced policy, then add rules for destructive shell, protected branches, secrets, infra, deployments, migrations, and repeated failures.
Use built-in templates for engineering, support, finance, cloud, and external messaging workflows.
Keep policy packs in source control, import them as drafts, and export saved versions from the dashboard or API.
Published policies attach matched rule evidence to checks, incidents, and audit exports.
Draft, test, publish, and review policy changes before production enforcement depends on them.
Security and platform teams should review production policy changes.
Audit exports
Export evidence for security reviews, procurement reviews, incident retrospectives, customer trust conversations, and internal AI governance.
curl "https://api.latentops.space/v1/reports/audit.md" \ -H "X-API-Key: lo_your_tenant_key_here"
Deployment
Hosted workspaces use the managed API. Private deployments should configure auth, Postgres, origins, key encryption, SSO/RBAC, retention, alerting, and readiness checks.
LATENTOPS_REQUIRE_API_KEY=true LATENTOPS_ADMIN_API_KEY=change-me LATENTOPS_STORE_BACKEND=postgres DATABASE_URL=postgres://... LATENTOPS_ALLOWED_ORIGINS=https://latentops.space,https://app.your-company.example LATENTOPS_PROVIDER_KEY_ENCRYPTION_SECRET=use-a-long-random-secret NEXT_PUBLIC_APP_URL=https://latentops.space LATENTOPS_API_BASE=https://api.latentops.space
Use /health for liveness and /ready for load balancers.
Troubleshooting
Most issues are key scope, workspace selection, API base, event logging, model IDs, or policy evidence.
Confirm X-API-Key is present, belongs to the selected workspace, log_event is true, and filters are not hiding events.
Check the tenant key, revoked key state, session cookie, workspace membership, and plan entitlement.
For managed live calls, use current OpenAI or Anthropic model IDs, verify provider key access, and read the returned provider status.
Start the FastAPI backend, verify LATENTOPS_API_BASE, and check that the API base matches your deployment.
Inspect policy evidence, protected paths, production flags, failed-test context, and repeated action history.
Store tenant and provider keys in a secret manager. Never commit keys or expose them in frontend code.