Run report: confidential issues for Gitea (#3217)
What forge SaaS did, how long it took, what the team decided at each gate, and what it was worth against the estimate the team approved. Money is shown only in forge tokens (1 forge token = $1).
One real run, on live forge SaaS, on 3 Nov 2026. It was not re-staged or re-run. Northwind Product Co is a fictional company: our own team made each decision as Northwind's people, and a script carried the decision out in the browser. The code was built on a fork, github.com/Gshingala/gitea, branch demo/3217-baseline (commit 1f51d2c). All times are IST (UTC+5:30).
Summary
Two jobs on one estimate. Job 12 built the feature; its review returned FAIL because it found a security leak in forge's own work. Job 13 was the follow-up forge offers for that case ("Fix blocking issues"); its review returned PASS. forge SaaS values a follow-up together with the job it follows, so the figures below cover both.
| Job | Started | Finished | Wall | forge working | Waiting for team | forge tokens | Verdict |
|---|---|---|---|---|---|---|---|
| 12 Confidential issues on public repositories | 09:30:06 | 15:31:10 | 6.0 h | 5.4 h | 34 min | 121.20 | FAIL |
| 13 Fix: the review's blocking items | 15:34:25 | 16:06:52 | 32 min | 29 min | 3 min | 3.15 | PASS |
| Both | 09:30:06 | 16:06:52 | 6.6 h | 5.9 h | 38 min | 124.35 |
Delivered as one pull request on the fork, with two commits: https://github.com/Gshingala/gitea/pull/1 (base Gshingala/gitea demo/3217-baseline, never the upstream project).
Timeline of phases
From the start and end events in each job's event log. Blue is forge working; amber is waiting for a decision from the team.
Job 12: the feature
- 09:30:08 to 09:51:12 · 21 minDiscovery of the codebase, then the project's tests on the untouched code: 235 passed, 5 skipped of 240 Go packages, 0 failed.
- 09:51:39 to 09:51:51 · 12 sTriage
- 09:51:51 to 09:55:58 · 4 minBrainstorm: 5 open questions for the team.
- 09:55:58 to 10:02:38Waiting for the answers (gate 34).
- 10:02:38 to 10:04:21 · 2 minRequirement written with the answers.
- 10:04:21 to 10:06:06Requirement approved (gate 35).
- 10:06:07 to 10:06:43 · 36 sAnalyst brief
- 10:06:43 to 10:08:18Brief approved (gate 36).
- 10:08:19 to 10:10:37 · 2 minPRD with the conventional estimate: 198 / 290 / 428 hours.
- 10:10:37 to 10:13:51PRD sent back (gate 37).
- 10:13:51 to 10:14:33 · 42 sPRD revised
- 10:14:34 to 10:16:11PRD and estimate approved (gate 38).
- 10:16:11 to 10:26:51 · 11 minArchitecture (hit its turn limit once and restarted).
- 10:26:51 to 10:32:30Architecture sent back (gate 39).
- 10:32:31 to 10:35:29 · 3 minArchitecture revised
- 10:35:29 to 10:37:32Architecture approved (gate 40).
- 10:37:32 to 10:55:13 · 18 minDetailed design (hit its turn limit once and restarted).
- 10:55:14 to 10:58:16Detailed design sent back (gate 41).
- 10:58:17 to 11:04:52 · 7 minDetailed design revised
- 11:04:52 to 11:07:51Detailed design approved (gate 42).
- 11:07:51 to 11:31:54 · 24 minUI design, with 6 mockups rendered in a browser.
- 11:31:54 to 11:35:06UI design sent back (gate 43).
- 11:35:07 to 11:43:53 · 9 minUI design revised
- 11:43:53 to 11:46:33UI design approved (gate 44).
- 11:46:34 to 11:51:42 · 5 minStories: 16 planned.
- 11:51:42 to 14:19:36 · 2 h 28 minDevelopment, stories 1 to 16.
- 14:19:36 to 14:42:28 · 23 minTests and project checks. The first run failed (the test command, and the fmt-check, swagger-check, openapi3-check and eslint-json checks). forge fixed them in its fix loop, attempt 1 of 3 (14:26 to 14:37); the re-run passed the tests and 18 project checks.
- 14:42:28 to 15:11:49 · 29 minUI review: the changed pages opened in Chromium as 7 roles, each page as the roles that matter for it, at 1440 and 390 px: 170 screenshots of 85 page and role pairs. It fixed one cut-off sentence and recorded a hidden issue's title showing in a timeline row as not fixed.
- 15:11:49 to 15:22:27 · 11 minReview: FAIL, 5 blocking items.
- 15:22:27 to 15:31:09 · 9 minHandover. After the FAIL verdict it patched the comment filter so the leaking dependency rows are dropped on the web and in the API, added a unit test, and staged the template fix; no review checked that patch. Pull request opened at 15:31, with the patch in its first commit.
Job 13: the follow-up
- 15:34:32 to 15:34:39 · 7 sTriage; no brainstorm needed.
- 15:34:39 to 15:43:34 · 9 minChange plan, then the project's tests on the code as it stood (235 passed, 5 skipped of 240 packages, to 15:49).
- 15:49:43 to 15:51:16Change plan sent back (gate 45).
- 15:51:17 to 15:53:08 · 2 minChange plan revised
- 15:53:27 to 15:55:04Change plan approved (gate 46).
- 15:55:05 to 16:03:25 · 8 minImplement
- 16:03:25 to 16:06:52 · 3 minVerify: PASS, 0 blocking. Second commit pushed to the same pull request at 16:06.
Decisions log
Every gate in both jobs: 7 gates plus the questions. 2 were approved the first time (the requirement and the analyst brief); 5 were sent back once each and approved on the second round. Active time is the team's review time at the gate, as forge SaaS counts it. The feedback below is the text that was sent back to forge, word for word.
| Gate | Job | Phase | Round | Decision | Decided by (Northwind role) | Active |
|---|---|---|---|---|---|---|
| 34 | 12 | Brainstorm questions | 1 | answered | Requester (security lead) | 3:00 |
| 35 | 12 | Requirement | 1 | approved | Analyst | 1:45 |
| 36 | 12 | Analyst brief | 1 | approved | Analyst | 1:35 |
| 37 | 12 | PRD | 1 | sent back | Product Manager | 3:14 |
| 38 | 12 | PRD and estimate | 2 | approved | Product Manager | 1:37 |
| 39 | 12 | Architecture | 1 | sent back | Architect | 5:39 |
| 40 | 12 | Architecture | 2 | approved | Architect | 2:03 |
| 41 | 12 | Detailed design | 1 | sent back | Architect | 3:02 |
| 42 | 12 | Detailed design | 2 | approved | Architect | 2:59 |
| 43 | 12 | UI design | 1 | sent back | Designer | 3:12 |
| 44 | 12 | UI design | 2 | approved | Designer | 2:40 |
| 45 | 13 | Change plan | 1 | sent back | Architect | 1:33 |
| 46 | 13 | Change plan | 2 | approved | Architect | 1:37 |
| Total team review time (job 12: 1,846 s; job 13: 190 s) | 33:56 | |||||
The answers to forge's questions (gate 34)
forge offered options and a recommendation for each question. The requester accepted three recommendations and chose differently on two.
| # | Question | Answer | forge's recommendation |
|---|---|---|---|
| Q1 | Which repository roles see confidential issues, apart from the author, assignees and site admins? | Everyone with Write or higher (writers, repo admins, org owners) | Accepted |
| Q2 | May the author change the Confidential setting after the issue is created? | Only when creating the issue | Not taken: forge suggested the author may make it confidential later, but not public |
| Q3 | What happens to webhooks and Gitea Actions for events on a confidential issue? | Drop both: no webhook delivery, no Actions trigger | Accepted |
| Q4 | How does a reference to a confidential issue appear on a public page, to people who may not see it? | A placeholder, "referenced a confidential issue", with no title or link | Not taken: forge suggested leaving the event out for them |
| Q5 | What do the Issues tab badge and milestone counts show? | Per-viewer counts: allowed users see the full count, others see counts without confidential issues | Accepted |
Feedback sent back at each gate
Good PRD. Keep every functional requirement, the permission rules and the conventional estimate as they are. Please change only three targets that cannot be checked where this job builds and tests (SQLite, 2 CPUs, no MySQL, PostgreSQL or MSSQL server): 1. NFR-2: drop the 10,000-issue p95 latency target. Keep the checkable parts as they are: SQL statements per list request identical for a page of 1 and a page of 50 issues (query-count test), and permission computed once per repository per request. 2. NFR-7 and the migration requirement (REQ-29): write the migration portably with the existing migration helpers, but run and test it on SQLite only. Do not require MySQL, PostgreSQL or MSSQL runs; those stay in the project's CI. 3. NFR-11: drop the per-test time limits (2 s, 4 s); follow the project's own guidance to keep tests fast. Do not add any size limit (lines, files, bytes) that the requirement does not state.
The design is approved as it stands; any ADR marked Proposed is accepted. One change: the test command. `make test-backend` alone never runs the integration tests or the browser test this change is built around, so the checks after dev would not prove the feature. Set the stack's test_command to: make test-backend && GITEA_TEST_DATABASE=sqlite make 'test-integration#<PATTERN>' && make 'migrations.individual.test#v29' && PLAYWRIGHT_BROWSERS=chromium GITEA_TEST_E2E_FLAGS='--project=chromium tests/e2e/issue-confidential.test.ts' make test-e2e where <PATTERN> is a -run regex, joined with |, of (a) Confidential (every new integration test has Confidential in its name) and (b) the anchored top-level test functions (^Name$) of the existing integration test files your Change Impact lists (for example api_issue_test.go, api_comment_test.go, api_comment_attachment_test.go, api_issue_dependency_test.go, api_notification_test.go, api_project_test.go, attachment_test.go, feed_user_test.go, incoming_email_test.go, issue_test.go, project_test.go, repo_activity_test.go, repo_webhook_test.go). Use no dots and no parentheses in the pattern (the Makefile turns dots into slashes; make treats parentheses as archive members). The Playwright browsers are installed: ~/.cache/ms-playwright holds chromium-1248 and chromium_headless_shell-1248, matching the repository's playwright-core 1.64.1. Measured on this infrastructure: a Chromium e2e spec runs in about 12 s and the frontend build in about 15 s. So the e2e step is runnable here. Before you finish this phase, run part (b) of the pattern and the v29 migration test once on the untouched code, and record in the Testing Strategy that they pass (or leave out any test that already fails and say which).
Approved apart from OQ-2. ADR-013 is accepted: tests create their own data. OQ-2: yes, make the REST repository object's open_issues_count per viewer, with the same rule and the same query as the Issues tab badge (D9), so the API never reveals a hidden confidential issue through a count. Add it to the traceability, the contract and the integration matrix; change nothing else.
Approved, with one change to keep this pull request to the feature. Keep the new .ui.warning.label variant, the dialogs, the helper text, the locale keys and the aria-label additions on the templates this change touches. Revert the three product-wide style changes: the .form .help colour in form.css, and the checkbox/radio border and focus ring in checkbox.css. They change every form and checkbox in Gitea, so they belong in a separate change; list them under 'Findings not fixed here' instead. The helper text may use a token that already passes AA (for example --color-text-light-2) on its own element, without changing .form .help.
Good plan; two changes before I approve. 1) Make the API guard mandatory, not optional: in services/convert/issue_comment.go, return no dependent_issue (or drop the entry) when the doer cannot see it, mirroring the ref-issue guard at lines 132-146, so the API is safe even if a future caller skips FilterReadableComments. Cover it with an assertion in the integration test. 2) In Verification, say that the pushed branch for the pull request must contain the staged fixes (comments.go, its unit test, the sidebar template), and that the full set of project checks the review ran (fmt-check, lint-json, lint-templates, lint-go, swagger-check and the test command) must pass again.
The other gates (35, 36, 38, 40, 42, 44, 46) were approved with no feedback.
The approved estimate
forge wrote this conventional estimate in the PRD, before design. The Product Manager approved it with the PRD at gate 38, at 10:16:11 IST on 3 Nov 2026 (04:46:11 UTC). It is in hours of effort and person-days (8 h = 1 person-day).
Basis: a conventional team that knows the codebase, estimated before design.
| Role | Hours | Person-days | ||||
|---|---|---|---|---|---|---|
| Low | Likely | High | Low | Likely | High | |
| Analysis | 16 | 24 | 36 | 2 | 3 | 4.5 |
| Design | 20 | 32 | 48 | 2.5 | 4 | 6 |
| Development | 90 | 130 | 190 | 11.25 | 16.25 | 23.75 |
| Code review | 14 | 20 | 30 | 1.75 | 2.5 | 3.75 |
| Testing | 50 | 72 | 104 | 6.25 | 9 | 13 |
| Release | 4 | 6 | 10 | 0.5 | 0.75 | 1.25 |
| Documentation | 4 | 6 | 10 | 0.5 | 0.75 | 1.25 |
| Total | 198 | 290 | 428 | 24.75 | 36.25 | 53.5 |
Assumptions
- The team knows the Gitea codebase and the conventions in AGENTS.md.
- The architect confirms the Action table has no issue key and a read-time match is feasible without a data backfill.
- Integration tests run on SQLite only in the estimate; other databases run in the existing CI.
- One round of review changes per change set.
Excludes
- Production deployment and operations.
- GitLab import (REQ-30), the list filter (REQ-34) and the template preset (REQ-35) are included at the low end only; the Should and Could items are about 12 likely hours together.
- Translation of strings beyond English.
- Recalling emails already sent, security advisories, confidential pull requests.
Job 13 did not get a new estimate: it inherits this one (estimate 2), which is why the two jobs are valued together.
What it was worth, at an example rate
Method. Value is the low end of the estimate approved at the PRD gate, priced at $20 an hour, an example rate (put in your own team's rate on the demo page). Cost is the forge tokens for both jobs, plus the team's active review time at the gates priced at the same rate. forge SaaS gives no value for a job whose review did not pass, so job 12 is counted only together with its passing follow-up, job 13.
| Value: 198 h (24.75 person-days) × $20 | $3,960 |
| forge tokens: 121.20 + 3.15 = 124.35, at 1 forge token = $1 | $124.35 |
| Team review: 0.57 h (34 min) × $20 | $11.40 |
| Cost | $135.75 |
| Return: value ÷ cost (29.17) | 29.2x |
| Net: value − cost | $3,824.25 |
These figures are exactly those of forge SaaS's Report for job 13 at a rate of $20 and the low basis. They compare forge with the team the estimate describes; they are not a quote.
The work
The pull request now holds two commits, one from each job: https://github.com/Gshingala/gitea/pull/1. The file and line counts are for the whole branch against demo/3217-baseline.
Tests
| When | What ran | Passed | Failed | Skipped | Of |
|---|---|---|---|---|---|
| Before any change (job 12 discovery) | make test-backend, Go packages | 235 | 0 | 5 | 240 |
| After job 12 | The test command set at the architecture gate (unit tests, the SQLite integration tests for the areas touched, the v29 migration test), Go packages | 237 | 0 | 5 | 242 |
| After job 12 | Playwright, tests/e2e/issue-confidential.test.ts in Chromium | 1 | 0 | 0 | 1 |
| After job 13 | make test-backend, Go packages, plus 5 project checks | 235 | 0 | 5 | 240 |
How tests are counted. go test run without -v prints one line per package, not per test, so the Go figures are packages; forge does not turn them into a number of tests. The 51 is test functions (50 Go, 1 TypeScript) counted in the diff. Job 13's test command was make test-backend, so it reports the 240 backend packages; job 12's longer command adds the integration and migration packages, hence 242.
The reviews
Job 12: FAIL, 5 blocking items, 8 notes
forge's review would not pass its own work. In plain words, the blocking items were:
- A security leak. When a public issue depended on a confidential one, the "added a new dependency" and "removed a dependency" rows in the public issue's timeline still showed the confidential issue's number, title and link to people who may not see it, on the web page and in the REST API (
dependent_issue). The reference path was already guarded; this one was not. forge's UI review had seen it in the browser and recorded it as not fixed. - A fix left out of the commit. The UI review's one fix (a sidebar sentence cut off by an ellipsis) was made in the working tree but not staged, so
make fmt-checkfailed. - Scratch data left in the repository. The UI review's scratch Gitea data sat inside the repository tree, so
make lint-jsonfailed on one of its files.
Items 4 and 5 restate the two failing checks. Of 20 project checks, 2 failed; one more (shellcheck) was already failing before the change because its tool is missing here, so it was listed and not enforced. After the verdict, job 12's handover stage patched the comment filter (web and API timeline, with the unit test TestFilterReadableCommentsDropsHiddenConfidentialDependency) and staged the sidebar template fix; the scratch data was gone by then. These went into the first commit without being reviewed, so the verdict stayed FAIL. forge delivered the branch and opened the pull request, marked by the FAIL verdict.
Job 13: PASS, 0 blocking, 6 notes
forge drafted the follow-up's requirement from those blocking items. Its change plan found the handover's comment-filter guard and the sidebar fix already staged, and the scratch data already gone. Job 13 added the mandatory guard in the API converter, as the architect asked at gate 45, so the API never returns a hidden dependency even if a caller skips the filter, with a unit test (TestToTimelineCommentConfidentialDependency) and an integration-test assertion; its commit touches only those 3 files. Its review was the first to pass with the fix in place. All 6 project checks passed: the test command, fmt-check, lint-json, lint-templates, lint-go and swagger-check. The review's own notes say what it did not do: it did not start the app for a manual look at the timeline, some unit test results were cached, and it did not run the new tests against the base commit.
Cost of the follow-up: 3.15 forge tokens and 32 minutes of wall clock.