starred/App-Store-Connect-CLI

Fork 0

mirror of https://github.com/rudrankriyam/App-Store-Connect-CLI.git synced 2026-04-25 15:45:48 +03:00

[GH-ISSUE #455] CI/CD: deterministic exit codes + optional JUnit reporting (TDD plan) #132

New issue

Closed

opened 2026-02-26 21:33:40 +03:00 by kerem · 1 comment

kerem commented

2026-02-26 21:33:40 +03:00

Owner

Originally created by @rudrankriyam on GitHub (Feb 9, 2026).
Original GitHub issue: https://github.com/rudrankriyam/App-Store-Connect-CLI/issues/455

Originally assigned to: @rudrankriyam on GitHub.

Summary

Add first-class CI/CD support to asc with:

Deterministic, typed exit codes (instead of almost always 1)
Optional JUnit XML report output for pipeline test reporting

This issue is intentionally TDD-first and includes a concrete file-by-file plan.

Current state (re-scan notes)

These are the current anchors this work must integrate with:

Entry-point exit handling is centralized in cmd/run.go and currently returns 0/1 in most cases.
Root/global flags are registered in internal/cli/shared/shared.go via BindRootFlags.
Error hints/classification lives in internal/cli/shared/errfmt/errfmt.go.
ASC API typed errors live in internal/asc/errors.go and are produced by ParseError in internal/asc/client_http.go.
ParseError currently does not carry HTTP status, which limits robust exit-code mapping.
Command-level tests are heavily built around internal/cli/cmdtest/* and captureOutput in internal/cli/cmdtest/commands_test.go.
CI workflows (.github/workflows/pr-checks.yml, .github/workflows/main-branch.yml) currently run tests/builds but do not consume JUnit from CLI commands.

Goals

Make asc failures machine-classifiable in pipelines.
Keep data output behavior unchanged (JSON-first, stdout for data).
Make CI reporting opt-in and explicit.
Implement with incremental, test-driven steps.

Non-goals (for this issue)

Do not add per-resource assertion DSLs yet (e.g. --assert state=...).
Do not change default output format semantics.
Do not add interactive prompts.

Proposed UX

Exit codes

Code	Meaning
`0`	Success
`1`	Generic/unclassified error
`2`	Invalid usage / flags / command invocation
`3`	Auth failure (`missing auth`, `unauthorized`, `forbidden`)
`4`	Resource not found
`5`	Conflict / already exists
`10-59`	Other HTTP 4xx (`10 + (status - 400)`)
`60-99`	HTTP 5xx (`60 + (status - 500)`)

Special behavior:

asc --help and <cmd> --help remain exit 0.
asc with no subcommand is treated as usage error (2).

CI report flags (explicit)

--report junit
--report-file <path>

Behavior:

Keep normal command data on stdout.
Write JUnit XML only to --report-file.
Emit one testcase per invocation (phase 1).

TDD implementation plan (file-by-file)

Phase 1: Exit-code framework

Tests first (RED)

Add cmd/exit_codes_test.go (new)
- flag.ErrHelp as usage context -> 2
- shared.ErrMissingAuth -> 3
- asc.ErrUnauthorized/asc.ErrForbidden -> 3
- asc.ErrNotFound -> 4
- conflict error -> 5
- generic error -> 1
Add/extend cmdtest coverage in internal/cli/cmdtest/exit_codes_test.go (new)
- asc builds info (missing required flag) returns 2
- unknown command returns 2
- invalid flag returns 2
- asc --help returns 0
- asc (no args) returns 2
Update internal/cli/cmdtest/error_hints_test.go
- missing auth now expects exit 3 (still with hint text)

Implementation (GREEN)

Add cmd/exit_codes.go (new)
- constants for public exit-code contract
- mapper function (single source of truth)
Update cmd/run.go
- replace hard-coded return 1 branches with mapped exit codes
- preserve explicit-help 0

Phase 2: Carry HTTP status into ASC errors

Tests first (RED)

Extend internal/asc/client_test.go
- parsed API errors include status-aware classification
Extend internal/asc/client_http_test.go
- status 404 maps not found behavior
- status 409 maps conflict behavior
- status 422 maps to 4xx bucket exit conversion path (indirectly via mapper unit test)
- status 503 maps to 5xx bucket exit conversion path

Implementation (GREEN)

Update internal/asc/errors.go
- add StatusCode on APIError
- add ErrConflict sentinel and matching logic
Update internal/asc/client_http.go
- introduce ParseErrorWithStatus(body []byte, status int)
- keep ParseError wrapper for compatibility (if needed)
- pass real resp.StatusCode at each call site
Update internal/asc/notary.go

pass status code into parse path

Phase 3: JUnit report writer + root flags

Tests first (RED)

Add internal/cli/shared/junit_report_test.go (new)

writes valid XML for success
writes <failure> on error
escapes special chars
preserves deterministic structure

Add internal/cli/cmdtest/ci_report_test.go (new)

successful command with --report junit --report-file ... creates report with failures="0"
failing command creates report with one failure testcase
command stdout remains command stdout (no xml leakage)

Add/extend root-flag parse test in internal/cli/cmdtest/debug_test.go or new dedicated file

accepts --report
accepts --report-file
rejects invalid report value with usage exit code

Implementation (GREEN)

Add internal/cli/shared/ci_flags.go (new)

root-level report flag state + getters

Add internal/cli/shared/junit_report.go (new)

small XML writer for single-invocation report model

Update internal/cli/shared/shared.go

wire report flags in BindRootFlags

Update cmd/run.go

create report payload from invocation result
write JUnit file when --report junit is enabled
if report writing fails:
- command succeeded -> return 1
- command failed -> keep original mapped exit code, print secondary stderr warning

Phase 4: Docs + CI examples

Tests/docs first

Update docs in README.md

add section: CI usage with --report junit + --report-file
add explicit exit code table
include --no-update recommendation for CI determinism

(Optional follow-up) add workflow example snippet in docs (no required workflow changes in this issue)

Acceptance criteria

Exit code contract is implemented as specified above.
Existing help behavior (--help) remains exit 0.
Missing auth exits 3.
Not found exits 4.
Conflict exits 5.
Other 4xx/5xx map into 10-99 buckets.
--report junit --report-file writes valid JUnit XML.
XML is file-only; stdout remains command output.
New tests cover mapper, status propagation, and JUnit output.
make format, make lint, and make test pass.

Example CI usage

asc builds list \
  --app "$ASC_APP_ID" \
  --no-update \
  --report junit \
  --report-file ./artifacts/asc-builds.xml

Suggested PR slicing

PR 1: Exit-code mapper + run integration + tests
PR 2: Status-aware API errors + tests
PR 3: JUnit writer + report flags + tests + README

This keeps review scope manageable and preserves TDD discipline.

Originally created by @rudrankriyam on GitHub (Feb 9, 2026). Original GitHub issue: https://github.com/rudrankriyam/App-Store-Connect-CLI/issues/455 Originally assigned to: @rudrankriyam on GitHub. ## Summary Add first-class CI/CD support to `asc` with: 1. Deterministic, typed exit codes (instead of almost always `1`) 2. Optional JUnit XML report output for pipeline test reporting This issue is intentionally **TDD-first** and includes a concrete file-by-file plan. --- ## Current state (re-scan notes) These are the current anchors this work must integrate with: - Entry-point exit handling is centralized in `cmd/run.go` and currently returns `0`/`1` in most cases. - Root/global flags are registered in `internal/cli/shared/shared.go` via `BindRootFlags`. - Error hints/classification lives in `internal/cli/shared/errfmt/errfmt.go`. - ASC API typed errors live in `internal/asc/errors.go` and are produced by `ParseError` in `internal/asc/client_http.go`. - `ParseError` currently does **not** carry HTTP status, which limits robust exit-code mapping. - Command-level tests are heavily built around `internal/cli/cmdtest/*` and `captureOutput` in `internal/cli/cmdtest/commands_test.go`. - CI workflows (`.github/workflows/pr-checks.yml`, `.github/workflows/main-branch.yml`) currently run tests/builds but do not consume JUnit from CLI commands. --- ## Goals - Make `asc` failures machine-classifiable in pipelines. - Keep data output behavior unchanged (JSON-first, stdout for data). - Make CI reporting opt-in and explicit. - Implement with incremental, test-driven steps. ## Non-goals (for this issue) - Do not add per-resource assertion DSLs yet (e.g. `--assert state=...`). - Do not change default output format semantics. - Do not add interactive prompts. --- ## Proposed UX ### Exit codes | Code | Meaning | |---|---| | `0` | Success | | `1` | Generic/unclassified error | | `2` | Invalid usage / flags / command invocation | | `3` | Auth failure (`missing auth`, `unauthorized`, `forbidden`) | | `4` | Resource not found | | `5` | Conflict / already exists | | `10-59` | Other HTTP 4xx (`10 + (status - 400)`) | | `60-99` | HTTP 5xx (`60 + (status - 500)`) | Special behavior: - `asc --help` and `<cmd> --help` remain exit `0`. - `asc` with no subcommand is treated as usage error (`2`). ### CI report flags (explicit) - `--report junit` - `--report-file <path>` Behavior: - Keep normal command data on stdout. - Write JUnit XML only to `--report-file`. - Emit one testcase per invocation (phase 1). --- ## TDD implementation plan (file-by-file) ## Phase 1: Exit-code framework ### Tests first (RED) 1. Add `cmd/exit_codes_test.go` (new) - `flag.ErrHelp` as usage context -> `2` - `shared.ErrMissingAuth` -> `3` - `asc.ErrUnauthorized`/`asc.ErrForbidden` -> `3` - `asc.ErrNotFound` -> `4` - conflict error -> `5` - generic error -> `1` 2. Add/extend cmdtest coverage in `internal/cli/cmdtest/exit_codes_test.go` (new) - `asc builds info` (missing required flag) returns `2` - unknown command returns `2` - invalid flag returns `2` - `asc --help` returns `0` - `asc` (no args) returns `2` 3. Update `internal/cli/cmdtest/error_hints_test.go` - missing auth now expects exit `3` (still with hint text) ### Implementation (GREEN) 4. Add `cmd/exit_codes.go` (new) - constants for public exit-code contract - mapper function (single source of truth) 5. Update `cmd/run.go` - replace hard-coded `return 1` branches with mapped exit codes - preserve explicit-help `0` --- ## Phase 2: Carry HTTP status into ASC errors ### Tests first (RED) 6. Extend `internal/asc/client_test.go` - parsed API errors include status-aware classification 7. Extend `internal/asc/client_http_test.go` - status `404` maps not found behavior - status `409` maps conflict behavior - status `422` maps to 4xx bucket exit conversion path (indirectly via mapper unit test) - status `503` maps to 5xx bucket exit conversion path ### Implementation (GREEN) 8. Update `internal/asc/errors.go` - add `StatusCode` on `APIError` - add `ErrConflict` sentinel and matching logic 9. Update `internal/asc/client_http.go` - introduce `ParseErrorWithStatus(body []byte, status int)` - keep `ParseError` wrapper for compatibility (if needed) - pass real `resp.StatusCode` at each call site 10. Update `internal/asc/notary.go` - pass status code into parse path --- ## Phase 3: JUnit report writer + root flags ### Tests first (RED) 11. Add `internal/cli/shared/junit_report_test.go` (new) - writes valid XML for success - writes `<failure>` on error - escapes special chars - preserves deterministic structure 12. Add `internal/cli/cmdtest/ci_report_test.go` (new) - successful command with `--report junit --report-file ...` creates report with `failures="0"` - failing command creates report with one failure testcase - command stdout remains command stdout (no xml leakage) 13. Add/extend root-flag parse test in `internal/cli/cmdtest/debug_test.go` or new dedicated file - accepts `--report` - accepts `--report-file` - rejects invalid report value with usage exit code ### Implementation (GREEN) 14. Add `internal/cli/shared/ci_flags.go` (new) - root-level report flag state + getters 15. Add `internal/cli/shared/junit_report.go` (new) - small XML writer for single-invocation report model 16. Update `internal/cli/shared/shared.go` - wire report flags in `BindRootFlags` 17. Update `cmd/run.go` - create report payload from invocation result - write JUnit file when `--report junit` is enabled - if report writing fails: - command succeeded -> return `1` - command failed -> keep original mapped exit code, print secondary stderr warning --- ## Phase 4: Docs + CI examples ### Tests/docs first 18. Update docs in `README.md` - add section: CI usage with `--report junit` + `--report-file` - add explicit exit code table - include `--no-update` recommendation for CI determinism 19. (Optional follow-up) add workflow example snippet in docs (no required workflow changes in this issue) --- ## Acceptance criteria - [ ] Exit code contract is implemented as specified above. - [ ] Existing help behavior (`--help`) remains exit `0`. - [ ] Missing auth exits `3`. - [ ] Not found exits `4`. - [ ] Conflict exits `5`. - [ ] Other 4xx/5xx map into `10-99` buckets. - [ ] `--report junit --report-file` writes valid JUnit XML. - [ ] XML is file-only; stdout remains command output. - [ ] New tests cover mapper, status propagation, and JUnit output. - [ ] `make format`, `make lint`, and `make test` pass. --- ## Example CI usage ```bash asc builds list \ --app "$ASC_APP_ID" \ --no-update \ --report junit \ --report-file ./artifacts/asc-builds.xml ``` --- ## Suggested PR slicing 1. PR 1: Exit-code mapper + run integration + tests 2. PR 2: Status-aware API errors + tests 3. PR 3: JUnit writer + report flags + tests + README This keeps review scope manageable and preserves TDD discipline.