forge SaaS · AirInfo Solution Back to the demo

Run report: confidential issues for Gitea (#3217)

What forge SaaS did, how long it took, what the team decided at each gate, and what it was worth against the estimate the team approved. Money is shown only in forge tokens (1 forge token = $1).

One real run, on live forge SaaS, on 3 Nov 2026. It was not re-staged or re-run. Northwind Product Co is a fictional company: our own team made each decision as Northwind's people, and a script carried the decision out in the browser. The code was built on a fork, github.com/Gshingala/gitea, branch demo/3217-baseline (commit 1f51d2c). All times are IST (UTC+5:30).

Summary

Two jobs on one estimate. Job 12 built the feature; its review returned FAIL because it found a security leak in forge's own work. Job 13 was the follow-up forge offers for that case ("Fix blocking issues"); its review returned PASS. forge SaaS values a follow-up together with the job it follows, so the figures below cover both.

forge tokens
124.35
121.20 (job 12) + 3.15 (job 13)
Wall clock
6.6 h
09:30 to 16:06
forge working
5.9 h
21,357 s
Team review time
34 min
0.57 h, at 13 decisions
Verdicts
FAIL PASS
job 12, then job 13
Return at $20/h
29.2x
example rate, low estimate
JobStartedFinishedWallforge workingWaiting for teamforge tokensVerdict
12 Confidential issues on public repositories09:30:0615:31:106.0 h5.4 h34 min121.20FAIL
13 Fix: the review's blocking items15:34:2516:06:5232 min29 min3 min3.15PASS
Both09:30:0616:06:526.6 h5.9 h38 min124.35
Job 12: forge working 19,600 s · waiting for the team 2,064 s
Job 13: forge working 1,757 s · waiting for the team 190 s

Delivered as one pull request on the fork, with two commits: https://github.com/Gshingala/gitea/pull/1 (base Gshingala/gitea demo/3217-baseline, never the upstream project).

Timeline of phases

From the start and end events in each job's event log. Blue is forge working; amber is waiting for a decision from the team.

Job 12: the feature

  1. 09:30:08 to 09:51:12 · 21 min
    Discovery of the codebase, then the project's tests on the untouched code: 235 passed, 5 skipped of 240 Go packages, 0 failed.
  2. 09:51:39 to 09:51:51 · 12 s
    Triage
  3. 09:51:51 to 09:55:58 · 4 min
    Brainstorm: 5 open questions for the team.
  4. 09:55:58 to 10:02:38
    Waiting for the answers (gate 34).
  5. 10:02:38 to 10:04:21 · 2 min
    Requirement written with the answers.
  6. 10:04:21 to 10:06:06
    Requirement approved (gate 35).
  7. 10:06:07 to 10:06:43 · 36 s
    Analyst brief
  8. 10:06:43 to 10:08:18
    Brief approved (gate 36).
  9. 10:08:19 to 10:10:37 · 2 min
    PRD with the conventional estimate: 198 / 290 / 428 hours.
  10. 10:10:37 to 10:13:51
    PRD sent back (gate 37).
  11. 10:13:51 to 10:14:33 · 42 s
    PRD revised
  12. 10:14:34 to 10:16:11
    PRD and estimate approved (gate 38).
  13. 10:16:11 to 10:26:51 · 11 min
    Architecture (hit its turn limit once and restarted).
  14. 10:26:51 to 10:32:30
    Architecture sent back (gate 39).
  15. 10:32:31 to 10:35:29 · 3 min
    Architecture revised
  16. 10:35:29 to 10:37:32
    Architecture approved (gate 40).
  17. 10:37:32 to 10:55:13 · 18 min
    Detailed design (hit its turn limit once and restarted).
  18. 10:55:14 to 10:58:16
    Detailed design sent back (gate 41).
  19. 10:58:17 to 11:04:52 · 7 min
    Detailed design revised
  20. 11:04:52 to 11:07:51
    Detailed design approved (gate 42).
  21. 11:07:51 to 11:31:54 · 24 min
    UI design, with 6 mockups rendered in a browser.
  22. 11:31:54 to 11:35:06
    UI design sent back (gate 43).
  23. 11:35:07 to 11:43:53 · 9 min
    UI design revised
  24. 11:43:53 to 11:46:33
    UI design approved (gate 44).
  25. 11:46:34 to 11:51:42 · 5 min
    Stories: 16 planned.
  26. 11:51:42 to 14:19:36 · 2 h 28 min
    Development, stories 1 to 16.
  27. 14:19:36 to 14:42:28 · 23 min
    Tests and project checks. The first run failed (the test command, and the fmt-check, swagger-check, openapi3-check and eslint-json checks). forge fixed them in its fix loop, attempt 1 of 3 (14:26 to 14:37); the re-run passed the tests and 18 project checks.
  28. 14:42:28 to 15:11:49 · 29 min
    UI review: the changed pages opened in Chromium as 7 roles, each page as the roles that matter for it, at 1440 and 390 px: 170 screenshots of 85 page and role pairs. It fixed one cut-off sentence and recorded a hidden issue's title showing in a timeline row as not fixed.
  29. 15:11:49 to 15:22:27 · 11 min
    Review: FAIL, 5 blocking items.
  30. 15:22:27 to 15:31:09 · 9 min
    Handover. After the FAIL verdict it patched the comment filter so the leaking dependency rows are dropped on the web and in the API, added a unit test, and staged the template fix; no review checked that patch. Pull request opened at 15:31, with the patch in its first commit.

Job 13: the follow-up

  1. 15:34:32 to 15:34:39 · 7 s
    Triage; no brainstorm needed.
  2. 15:34:39 to 15:43:34 · 9 min
    Change plan, then the project's tests on the code as it stood (235 passed, 5 skipped of 240 packages, to 15:49).
  3. 15:49:43 to 15:51:16
    Change plan sent back (gate 45).
  4. 15:51:17 to 15:53:08 · 2 min
    Change plan revised
  5. 15:53:27 to 15:55:04
    Change plan approved (gate 46).
  6. 15:55:05 to 16:03:25 · 8 min
    Implement
  7. 16:03:25 to 16:06:52 · 3 min
    Verify: PASS, 0 blocking. Second commit pushed to the same pull request at 16:06.

Decisions log

Every gate in both jobs: 7 gates plus the questions. 2 were approved the first time (the requirement and the analyst brief); 5 were sent back once each and approved on the second round. Active time is the team's review time at the gate, as forge SaaS counts it. The feedback below is the text that was sent back to forge, word for word.

GateJobPhaseRoundDecisionDecided by (Northwind role)Active
3412Brainstorm questions1answeredRequester (security lead)3:00
3512Requirement1approvedAnalyst1:45
3612Analyst brief1approvedAnalyst1:35
3712PRD1sent backProduct Manager3:14
3812PRD and estimate2approvedProduct Manager1:37
3912Architecture1sent backArchitect5:39
4012Architecture2approvedArchitect2:03
4112Detailed design1sent backArchitect3:02
4212Detailed design2approvedArchitect2:59
4312UI design1sent backDesigner3:12
4412UI design2approvedDesigner2:40
4513Change plan1sent backArchitect1:33
4613Change plan2approvedArchitect1:37
Total team review time (job 12: 1,846 s; job 13: 190 s)33:56

The answers to forge's questions (gate 34)

forge offered options and a recommendation for each question. The requester accepted three recommendations and chose differently on two.

#QuestionAnswerforge's recommendation
Q1Which repository roles see confidential issues, apart from the author, assignees and site admins?Everyone with Write or higher (writers, repo admins, org owners)Accepted
Q2May the author change the Confidential setting after the issue is created?Only when creating the issueNot taken: forge suggested the author may make it confidential later, but not public
Q3What happens to webhooks and Gitea Actions for events on a confidential issue?Drop both: no webhook delivery, no Actions triggerAccepted
Q4How does a reference to a confidential issue appear on a public page, to people who may not see it?A placeholder, "referenced a confidential issue", with no title or linkNot taken: forge suggested leaving the event out for them
Q5What do the Issues tab badge and milestone counts show?Per-viewer counts: allowed users see the full count, others see counts without confidential issuesAccepted

Feedback sent back at each gate

Gate 37 · PRDsent backProduct Manager · round 1 · 3:14
Good PRD. Keep every functional requirement, the permission rules and the conventional estimate as they are. Please change only three targets that cannot be checked where this job builds and tests (SQLite, 2 CPUs, no MySQL, PostgreSQL or MSSQL server): 1. NFR-2: drop the 10,000-issue p95 latency target. Keep the checkable parts as they are: SQL statements per list request identical for a page of 1 and a page of 50 issues (query-count test), and permission computed once per repository per request. 2. NFR-7 and the migration requirement (REQ-29): write the migration portably with the existing migration helpers, but run and test it on SQLite only. Do not require MySQL, PostgreSQL or MSSQL runs; those stay in the project's CI. 3. NFR-11: drop the per-test time limits (2 s, 4 s); follow the project's own guidance to keep tests fast. Do not add any size limit (lines, files, bytes) that the requirement does not state.
Gate 39 · Architecturesent backArchitect · round 1 · 5:39
The design is approved as it stands; any ADR marked Proposed is accepted. One change: the test command. `make test-backend` alone never runs the integration tests or the browser test this change is built around, so the checks after dev would not prove the feature. Set the stack's test_command to: make test-backend && GITEA_TEST_DATABASE=sqlite make 'test-integration#<PATTERN>' && make 'migrations.individual.test#v29' && PLAYWRIGHT_BROWSERS=chromium GITEA_TEST_E2E_FLAGS='--project=chromium tests/e2e/issue-confidential.test.ts' make test-e2e where <PATTERN> is a -run regex, joined with |, of (a) Confidential (every new integration test has Confidential in its name) and (b) the anchored top-level test functions (^Name$) of the existing integration test files your Change Impact lists (for example api_issue_test.go, api_comment_test.go, api_comment_attachment_test.go, api_issue_dependency_test.go, api_notification_test.go, api_project_test.go, attachment_test.go, feed_user_test.go, incoming_email_test.go, issue_test.go, project_test.go, repo_activity_test.go, repo_webhook_test.go). Use no dots and no parentheses in the pattern (the Makefile turns dots into slashes; make treats parentheses as archive members). The Playwright browsers are installed: ~/.cache/ms-playwright holds chromium-1248 and chromium_headless_shell-1248, matching the repository's playwright-core 1.64.1. Measured on this infrastructure: a Chromium e2e spec runs in about 12 s and the frontend build in about 15 s. So the e2e step is runnable here. Before you finish this phase, run part (b) of the pattern and the v29 migration test once on the untouched code, and record in the Testing Strategy that they pass (or leave out any test that already fails and say which).
Gate 41 · Detailed designsent backArchitect · round 1 · 3:02
Approved apart from OQ-2. ADR-013 is accepted: tests create their own data. OQ-2: yes, make the REST repository object's open_issues_count per viewer, with the same rule and the same query as the Issues tab badge (D9), so the API never reveals a hidden confidential issue through a count. Add it to the traceability, the contract and the integration matrix; change nothing else.
Gate 43 · UI designsent backDesigner · round 1 · 3:12
Approved, with one change to keep this pull request to the feature. Keep the new .ui.warning.label variant, the dialogs, the helper text, the locale keys and the aria-label additions on the templates this change touches. Revert the three product-wide style changes: the .form .help colour in form.css, and the checkbox/radio border and focus ring in checkbox.css. They change every form and checkbox in Gitea, so they belong in a separate change; list them under 'Findings not fixed here' instead. The helper text may use a token that already passes AA (for example --color-text-light-2) on its own element, without changing .form .help.
Gate 45 · Change plan (job 13)sent backArchitect · round 1 · 1:33
Good plan; two changes before I approve. 1) Make the API guard mandatory, not optional: in services/convert/issue_comment.go, return no dependent_issue (or drop the entry) when the doer cannot see it, mirroring the ref-issue guard at lines 132-146, so the API is safe even if a future caller skips FilterReadableComments. Cover it with an assertion in the integration test. 2) In Verification, say that the pushed branch for the pull request must contain the staged fixes (comments.go, its unit test, the sidebar template), and that the full set of project checks the review ran (fmt-check, lint-json, lint-templates, lint-go, swagger-check and the test command) must pass again.

The other gates (35, 36, 38, 40, 42, 44, 46) were approved with no feedback.

The approved estimate

forge wrote this conventional estimate in the PRD, before design. The Product Manager approved it with the PRD at gate 38, at 10:16:11 IST on 3 Nov 2026 (04:46:11 UTC). It is in hours of effort and person-days (8 h = 1 person-day).

Basis: a conventional team that knows the codebase, estimated before design.

RoleHoursPerson-days
LowLikelyHighLowLikelyHigh
Analysis162436234.5
Design2032482.546
Development9013019011.2516.2523.75
Code review1420301.752.53.75
Testing50721046.25913
Release46100.50.751.25
Documentation46100.50.751.25
Total19829042824.7536.2553.5

Assumptions

Excludes

Job 13 did not get a new estimate: it inherits this one (estimate 2), which is why the two jobs are valued together.

What it was worth, at an example rate

Method. Value is the low end of the estimate approved at the PRD gate, priced at $20 an hour, an example rate (put in your own team's rate on the demo page). Cost is the forge tokens for both jobs, plus the team's active review time at the gates priced at the same rate. forge SaaS gives no value for a job whose review did not pass, so job 12 is counted only together with its passing follow-up, job 13.

Value: 198 h (24.75 person-days) × $20$3,960
forge tokens: 121.20 + 3.15 = 124.35, at 1 forge token = $1$124.35
Team review: 0.57 h (34 min) × $20$11.40
Cost$135.75
Return: value ÷ cost (29.17)29.2x
Net: value − cost$3,824.25

These figures are exactly those of forge SaaS's Report for job 13 at a rate of $20 and the low basis. They compare forge with the team the estimate describes; they are not a quote.

The work

Stories
16 of 16
planned and done
Files
137
13 added, 124 changed
Lines
+4,501
−175
Test functions added
51
50 in job 12's commit (49 at its review, 1 added at handover), 1 in job 13

The pull request now holds two commits, one from each job: https://github.com/Gshingala/gitea/pull/1. The file and line counts are for the whole branch against demo/3217-baseline.

Tests

WhenWhat ranPassedFailedSkippedOf
Before any change (job 12 discovery)make test-backend, Go packages23505240
After job 12The test command set at the architecture gate (unit tests, the SQLite integration tests for the areas touched, the v29 migration test), Go packages23705242
After job 12Playwright, tests/e2e/issue-confidential.test.ts in Chromium1001
After job 13make test-backend, Go packages, plus 5 project checks23505240

How tests are counted. go test run without -v prints one line per package, not per test, so the Go figures are packages; forge does not turn them into a number of tests. The 51 is test functions (50 Go, 1 TypeScript) counted in the diff. Job 13's test command was make test-backend, so it reports the 240 backend packages; job 12's longer command adds the integration and migration packages, hence 242.

The reviews

Job 12: FAIL, 5 blocking items, 8 notes

forge's review would not pass its own work. In plain words, the blocking items were:

  1. A security leak. When a public issue depended on a confidential one, the "added a new dependency" and "removed a dependency" rows in the public issue's timeline still showed the confidential issue's number, title and link to people who may not see it, on the web page and in the REST API (dependent_issue). The reference path was already guarded; this one was not. forge's UI review had seen it in the browser and recorded it as not fixed.
  2. A fix left out of the commit. The UI review's one fix (a sidebar sentence cut off by an ellipsis) was made in the working tree but not staged, so make fmt-check failed.
  3. Scratch data left in the repository. The UI review's scratch Gitea data sat inside the repository tree, so make lint-json failed on one of its files.

Items 4 and 5 restate the two failing checks. Of 20 project checks, 2 failed; one more (shellcheck) was already failing before the change because its tool is missing here, so it was listed and not enforced. After the verdict, job 12's handover stage patched the comment filter (web and API timeline, with the unit test TestFilterReadableCommentsDropsHiddenConfidentialDependency) and staged the sidebar template fix; the scratch data was gone by then. These went into the first commit without being reviewed, so the verdict stayed FAIL. forge delivered the branch and opened the pull request, marked by the FAIL verdict.

Job 13: PASS, 0 blocking, 6 notes

forge drafted the follow-up's requirement from those blocking items. Its change plan found the handover's comment-filter guard and the sidebar fix already staged, and the scratch data already gone. Job 13 added the mandatory guard in the API converter, as the architect asked at gate 45, so the API never returns a hidden dependency even if a caller skips the filter, with a unit test (TestToTimelineCommentConfidentialDependency) and an integration-test assertion; its commit touches only those 3 files. Its review was the first to pass with the fix in place. All 6 project checks passed: the test command, fmt-check, lint-json, lint-templates, lint-go and swagger-check. The review's own notes say what it did not do: it did not start the app for a manual look at the timeline, some unit test results were cached, and it did not run the new tests against the base commit.

Cost of the follow-up: 3.15 forge tokens and 32 minutes of wall clock.