Repository navigation
fix: control plane and UI fixes - #1171
Open
santoshkumarradha wants to merge 12 commits into
Open
santoshkumarradha wants to merge 12 commits into
santoshkumarradha wants to merge 12 commits into
Conversation
…und input
Several UI calls use fetch directly instead of the shared client and never send X-API-Key, so with API key auth on they fail with 401: the Integrations and Triggers pages, the trigger sheet, new-trigger dialog and event replay, step approval actions, the workflow graph's step details, execution retry, and the DID resolution bundle download. Two of them read the key from storage slots the app never writes (localStorage agentfield_api_key, sessionStorage apiKey). They now use one authHeaders() helper backed by the key the app signed in with.
Replaying a run in the Playground pre-filled the stored request body, which is the whole {"input": {...}} envelope; Execute wrapped it again, so the reasoner received input.input and rejected the call with missing required fields. The envelope is now removed when the box is filled from a replay or a recent run, and before execute.
The execute response carries both execution_id and run_id; the Playground preferred execution_id, so View run went to /runs/<execution id>, which is a 404. It now prefers run_id.
StoreExecutionVC and StoreExecutionVCRecord wrote created_at with time.Now() in the host's zone. SQLite keeps it as text, and the created_after/created_before filters bind a UTC bound, so on a host west of UTC a VC created a moment ago sorted before a bound in the past and was filtered out; ORDER BY created_at likewise mixed rows written under different offsets. Writers now store UTC, and a one-time startup pass on SQLite rewrites rows an older build stored in the host's zone, keeping each row's instant. Postgres columns are timestamptz and are unaffected.
Every execution event invalidated the runs list, the dashboard summary, the run DAG and the step detail at once, with cancelRefetch on, so under steady traffic the Runs page restarted its list request dozens of times a minute and each cancelled request still ran to completion on the server. Invalidations are now coalesced (at most one flush per two seconds, the first immediate, without cancelling an in-flight fetch), and a running run polls every ten seconds as a safety net while the event stream is connected.
QueryRunSummaries grouped every execution in the table, computing every summary column for every run, and then kept one page. It now chooses the page's run_ids first, computing only the sort key, and aggregates just those runs with the same filters. The default page (no filters, latest activity first) is read from a new expression index on COALESCE(updated_at, started_at) and stops as soon as the page is complete. run_id breaks sort ties so page boundaries no longer move between requests. Tests compare every sort column, direction, filter and page against the previous single grouped query.
SQLite compares started_at as the text the driver wrote, and the driver writes a time in the zone it carries. An execution whose start time was not UTC was filtered and sorted as though it started at a different instant; a since bound given in UTC dropped it. Insert and update now write the UTC instant. Rows already stored are unchanged.
The run page fetched the whole workflow VC chain, one signed credential per step, to show its verifiability warning and issuer DID: about 1.2 MB for a 1500-step run, on every page open. GET /api/ui/v1/workflows/:id/vc-chain accepts view=summary, which returns the same status, step counts, verifiability and workflow issuer without component_vcs, the workflow credential's document or the DID resolution bundle. The default response is unchanged, and the audit download still fetches the full chain.
santoshkumarradha
changed the title
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
📊 Coverage gateThresholds from
✅ Gate passedNo surface regressed past the allowed threshold and the aggregate stayed above the floor. |
Contributor
📐 Patch coverage gateThreshold: 80% on lines this PR touches vs
✅ Patch gate passedEvery surface whose lines were touched by this PR has patch coverage at or above the threshold. |
CodeQL flagged runSummaryPageRunIDs for allocating a slice whose capacity came from the request-supplied limit. The caller clamps it, but CodeQL does not see a reassignment clamp across the call. The slice now grows by append. Assisted-by: CodeAF Co-Authored-By: CodeAF <267109073+agentfield-bot@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
All fixes in one PR, for one review. It replaces #1169, #1170 and #1172, which are closed in favour of this one; their commits are merged here unchanged.
created_atstored in UTC, plus a one-time SQLite rewrite of old rowscreated_atwas written in the host's local zone while the VC filters bind a UTC bound and compare text, so on hosts west of UTC a fresh VC fell out ofcreated_afterand rows written under different offsets sorted out of order; the API returned local-zone timestampsfetchcalls (Integrations, Triggers, trigger sheet/dialog/event replay, step approvals, DAG sidebar, retry panel, DID bundle download)AGENTFIELD_API_KEYset, pages that callfetchdirectly sent noX-API-Keyand got401{"input": …}envelope{"input":{"input":{…}}}, so the reasoner received none of its arguments and ran on defaults/runs/<execution_id>, which is a 404/api/ui/v2/workflow-runsgrouped every execution in the table on each request: ~285 ms with no traffic and ~440 ms p50 under traffic on a 203k-execution database. Now ~25 ms and ~56 mscancelRefetch, so under load the Runs page made ~30 overlapping list requests in 15 svc-chain?view=summarystarted_atstored in UTCsincefilter and sorted by the wrong instantNo conflicts between the three.
Type of change
Evidence
End-to-end run: embedded server built from each branch (
go build -tags embedded), fresh home directory,AGENTFIELD_STORAGE_MODE=local,AGENTFIELD_API_KEYset, host zoneAmerica/Toronto(UTC-5), two agents fromexamples/python_agent_nodes(permission_agent_b,hello_world), calls executed through/api/v1/execute, UI driven in headless Chrome.VC timestamps
Before (
mainbinary, fresh database):After (this branch started on the same database
maincreated):Runs list and run page under traffic
Measured on a copy of a busy local SQLite database: 4.2 GB, about 24,000 runs and 203,000 executions. Live traffic came from a real agent through
/api/v1/execute/async: one 1,500-step run (six steps in parallel) plus a new 5-step run about every 0.66 s. The machine's load average was about 20 from unrelated work, so these are noisy-box numbers.GET /api/ui/v2/workflow-runs?page=1&page_size=25&sort_by=updated_at&sort_order=descQueryRunSummariesalone, default pageCOUNT(DISTINCT run_id)total)QueryRunSummaries, other sort columnsThe per-run lookups in the handler (run metadata and trigger) were measured at about 1–3 ms for a page of 25, so they were left as they are.
Network trace, 15 s on each page under traffic (headless Chrome,
/apirequests only)Runs page:
Run page of the 1,500-step run while it runs:
For a 322-step run, the full chain was 766 KB.
Test plan (runs list, UTC start time, VC summary)
go build ./...go vet ./internal/storage ./internal/handlers/uigo test ./internal/storage/ ./internal/handlers/ui/ ./internal/server/ -count=1POSTGRES_TEST_URL=... go test ./internal/storage -run Postgres -count=1againstpgvector/pgvector:pg17, including the new PostgreSQL page comparisonTestQueryRunSummariesPagesMatchTheSingleGroupedQuerycompares every sort column, both directions, ten filters and five page windows against the previous single grouped query. It fails if either tie-breaker is reversed.TestRecentRunIDsByActivityWidensPastBusyRuns,TestRunActivityIndexServesTheRecentRunsScanTestExecutionStartedAtIsStoredInUTCfails onmain(a non-UTC start drops out of a UTCsincefilter) and passes hereTestUIWorkflowVCChainSummaryViewOmitsPerStepCredentialsweb/client:npx tsc --noEmit -p tsconfig.app.json,npx vitest run(141 files, 840 tests),npm run buildIntegrations page with an API key
/api/v1/sources/api/v1/triggers401401HTTP 401: {"error":"unauthorized",…}200200Run detail → Replay → Execute → View run
{"input": {"limit": 3, "query": "SELECT 3"}}{"input":{"input":{"limit":3,"query":"SELECT 3"}}}"query":"SELECT *","limit":100/ui/runs/exec_…→HTTP error! status: 404{"limit": 3, "query": "SELECT 3"}{"input":{"limit":3,"query":"SELECT 3"}}"query":"SELECT 3","limit":3/ui/runs/run_…→ run page rendersTest plan
cd control-plane && go build ./...go vet ./...go test ./internal/storage/ ./internal/services/ ./internal/handlers/... -count=1(re-run after merging perf(control-plane): fast Runs list and run page under live traffic #1172)cd web/client && npx tsc --noEmit -p tsconfig.app.jsonnpx vitest run(141 files, 840 tests passed, after merging perf(control-plane): fast Runs list and run page under live traffic #1172)npm run buildgo build -tags embedded ./cmd/agentfield-server, run on a fresh home with an API key, two example agents registered, executions via/api/v1/executemain's binary; rows rewritten to UTC once; second restart is a no-opMerge Danger
Door: one-way — #1170 rewrites existing SQLite
execution_vcs.created_atvalues to UTC on first start (same instant, different spelling; not reverted by downgrading). The #1169 web changes are two-way.The #1172 changes are two-way: the new
executionsindex is additive, andview=summaryis opt-in.Blast Radius: low — one column of one table on SQLite only (Postgres untouched), recorded in
schema_migrationsso it runs once; the web changes add a header to direct fetches and fix two Playground code paths. #1172 changes the query behind the Runs list, and on a large PostgreSQLexecutionstable the first start builds one more index.