[PR #3815] [MERGED] feat: support secret environment variables in CLI #4532

New issue

Closed

opened 2026-03-17 02:03:40 +03:00 by kerem · 0 comments

kerem commented

2026-03-17 02:03:40 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/hoppscotch/hoppscotch/pull/3815
Author: @jamesgeorge007
Created: 2/8/2024
Status: ✅ Merged
Merged: 2/8/2024
Merged by: @AndrewBastin

Base: release/2023.12.4 ← Head: feat/cli-env-secrets

📝 Commits (2)

19c0a1b feat: support secret environment variables in CLI
c2890c2 test: cleanup

📊 Changes

30 files changed (+785 additions, -139 deletions)

View changed files

📝 packages/hoppscotch-cli/package.json (+1 -0)
📝 packages/hoppscotch-cli/src/__tests__/commands/test.spec.ts (+201 -85)
📝 packages/hoppscotch-cli/src/__tests__/samples/collections/collection-level-headers-auth-coll.json (+0 -0)
📝 packages/hoppscotch-cli/src/__tests__/samples/collections/env-flag-tests-coll.json (+0 -0)
📝 packages/hoppscotch-cli/src/__tests__/samples/collections/fails-coll.json (+0 -0)
📝 packages/hoppscotch-cli/src/__tests__/samples/collections/malformed-coll-2.json (+0 -0)
📝 packages/hoppscotch-cli/src/__tests__/samples/collections/malformed-coll.json (+0 -0)
📝 packages/hoppscotch-cli/src/__tests__/samples/collections/notjson-coll.txt (+0 -0)
📝 packages/hoppscotch-cli/src/__tests__/samples/collections/passes-coll.json (+0 -0)
➕ packages/hoppscotch-cli/src/__tests__/samples/collections/pre-req-script-env-var-persistence-coll.json (+21 -0)
📝 packages/hoppscotch-cli/src/__tests__/samples/collections/req-body-env-vars-coll.json (+0 -0)
➕ packages/hoppscotch-cli/src/__tests__/samples/collections/secret-envs-coll.json (+107 -0)
➕ packages/hoppscotch-cli/src/__tests__/samples/collections/secret-envs-persistence-coll.json (+143 -0)
➕ packages/hoppscotch-cli/src/__tests__/samples/collections/secret-envs-persistence-scripting-coll.json (+30 -0)
➕ packages/hoppscotch-cli/src/__tests__/samples/environments/bulk-envs.json (+32 -0)
📝 packages/hoppscotch-cli/src/__tests__/samples/environments/env-flag-envs.json (+0 -0)
➕ packages/hoppscotch-cli/src/__tests__/samples/environments/malformed-envs.json (+16 -0)
📝 packages/hoppscotch-cli/src/__tests__/samples/environments/req-body-env-vars-envs.json (+2 -1)
➕ packages/hoppscotch-cli/src/__tests__/samples/environments/secret-envs-persistence-scripting-envs.json (+27 -0)
➕ packages/hoppscotch-cli/src/__tests__/samples/environments/secret-envs.json (+40 -0)

...and 10 more files

📄 Description

Description

Supersedes #3780.

This PR adds support for secret variables in environments for the CLI.

There are multiple ways to set values for secret environment variables:

Inject the secret values as variables into the OS environment - When there isn't a value field associated with an environment entry, the corresponding value is fetched from the system environment process.env with a fallback to an empty string.
Edit the environment export file and add the secret values manually - One can specify values directly in the exported environment file for secret environment variables, which will be preferred over values in the system environment if present.
Updating environments from the scripting context - This takes precedence over all the approaches mentioned above.

Secret environment variables referred to in the request endpoint are masked.

Other changes

Updates made to the environment from the pre-request script were not persisted in the post-request script while running a collection from the CLI. This PR includes the changes ensuring the updated environments are persisted throughout the scripting context.
Use the versioned entity for Environment instead of maintaining the schema. verzod is added as a devDependency (gets inlined in the bundle) to achieve the same.
A new displayUrl field is added under the RequestConfig interface & effectiveFinalDisplayURL field under the EffectiveHoppRESTRequest interface. The getEffectiveRESTRequest function now includes a new property, effectiveFinalDisplayURL under effectiveRequest, that includes the secret environment variable values masked. Later, while logging the request endpoint, displayUrl is preferred with a fallback to url ensuring the transformed value is logged (secret environment variables referred to in the request endpoint) if existing or else falling back to the regular URL.
The test suite is reorganized and updated with respective test cases for the new flows along with ensuring environment export files supplied, that are malformed or based on the bulk export format are handled and the support for short flags (--e, --env, -d, --delay).
Sample collection and environment files consumed in the test suite are now organized under collections & environments directories with a consistent naming convention (collection files ending in coll.json & environment files in envs.json).

Depends on #3779

Closes HFE-221.

Steps to verify

While at the branch, run pnpm i so that verzod gets installed under @hoppscotch/cli and the packages are built.

Grab the following collection contents under the name test-secret-envs-coll.json.

{
  "v": 2,
  "name": "test-secret-envs-coll",
  "folders": [],
  "requests": [
    {
      "v": "1",
      "auth": {
        "authType": "none",
        "authActive": true
      },
      "body": {
        "body": null,
        "contentType": null
      },
      "name": "test-secret-envs-req",
      "method": "GET",
      "params": [],
      "headers": [
        {
          "key": "Custom-Header",
          "value": "<<secretKey>>",
          "active": true
        }
      ],
      "endpoint": "https://httpbin.org/get",
      "testScript": "pw.test(\"Successfully picks the value for the secret environment variable\", () => {\n  pw.expect(pw.env.get(\"secretKey\")).toBe(\"secret-value\")\n\n  pw.expect(pw.response.body.headers[\"Custom-Header\"]).toBe(\"secret-value\")\n})",
      "preRequestScript": ""
    }
  ],
  "headers": [],
  "auth": {
    "authType": "inherit",
    "authActive": false
  }
}

A secret environment variable is referred to in the request headers, and relevant tests are added to ensure secret environment variables are picked correctly.

pw.test("Successfully picks the value for the secret environment variable", () => {
  pw.expect(pw.env.get("secretKey")).toBe("secret-value")

  pw.expect(pw.response.body.headers["Custom-Header"]).toBe("secret-value")
})

Grab the following environment export file contents under test-secret-envs-coll.json.

{
  "v": 1,
  "id": "2",
  "name": "secret-envs",
  "variables": [
    {
      "key": "secretKey",
      "secret": true
    } 
  ] 
}

While at the @hoppscotch/cli package path, invoke the CLI supplying secret environment variable entries as key-value pairs along with the above collection and environment files (assuming they are in the same path). Alternately, create a symlink for the CLI and refer to the binary via hopp in any path.
```
  secretKey=secret-value node bin/hopp test test-secret-envs-coll.json -e secret-envs.json
```
Observe all the tests passing ✅
Try omitting the key-value pairs corresponding to the secret environment variables and observe the tests failing, stating the value yield was an empty string. Similarly, try specifying a value for the secret environment variable entry by adding a value field and observe it overrides the one from the system environment. Values set from the scripting context take the highest precedence, observable by setting a value for the same via pre-request script.

Please refer to the test suite for more comprehensive samples.

Checks

My pull request adheres to the code style of this project
All the tests have passed

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/hoppscotch/hoppscotch/pull/3815 **Author:** [@jamesgeorge007](https://github.com/jamesgeorge007) **Created:** 2/8/2024 **Status:** ✅ Merged **Merged:** 2/8/2024 **Merged by:** [@AndrewBastin](https://github.com/AndrewBastin) **Base:** `release/2023.12.4` ← **Head:** `feat/cli-env-secrets` --- ### 📝 Commits (2) - [`19c0a1b`](https://github.com/hoppscotch/hoppscotch/commit/19c0a1bedc369d486187724339de0ef0a1cd3c2a) feat: support secret environment variables in CLI - [`c2890c2`](https://github.com/hoppscotch/hoppscotch/commit/c2890c29b1d1e9440e908d04de9a98973a0c80b2) test: cleanup ### 📊 Changes **30 files changed** (+785 additions, -139 deletions) <details> <summary>View changed files</summary> 📝 `packages/hoppscotch-cli/package.json` (+1 -0) 📝 `packages/hoppscotch-cli/src/__tests__/commands/test.spec.ts` (+201 -85) 📝 `packages/hoppscotch-cli/src/__tests__/samples/collections/collection-level-headers-auth-coll.json` (+0 -0) 📝 `packages/hoppscotch-cli/src/__tests__/samples/collections/env-flag-tests-coll.json` (+0 -0) 📝 `packages/hoppscotch-cli/src/__tests__/samples/collections/fails-coll.json` (+0 -0) 📝 `packages/hoppscotch-cli/src/__tests__/samples/collections/malformed-coll-2.json` (+0 -0) 📝 `packages/hoppscotch-cli/src/__tests__/samples/collections/malformed-coll.json` (+0 -0) 📝 `packages/hoppscotch-cli/src/__tests__/samples/collections/notjson-coll.txt` (+0 -0) 📝 `packages/hoppscotch-cli/src/__tests__/samples/collections/passes-coll.json` (+0 -0) ➕ `packages/hoppscotch-cli/src/__tests__/samples/collections/pre-req-script-env-var-persistence-coll.json` (+21 -0) 📝 `packages/hoppscotch-cli/src/__tests__/samples/collections/req-body-env-vars-coll.json` (+0 -0) ➕ `packages/hoppscotch-cli/src/__tests__/samples/collections/secret-envs-coll.json` (+107 -0) ➕ `packages/hoppscotch-cli/src/__tests__/samples/collections/secret-envs-persistence-coll.json` (+143 -0) ➕ `packages/hoppscotch-cli/src/__tests__/samples/collections/secret-envs-persistence-scripting-coll.json` (+30 -0) ➕ `packages/hoppscotch-cli/src/__tests__/samples/environments/bulk-envs.json` (+32 -0) 📝 `packages/hoppscotch-cli/src/__tests__/samples/environments/env-flag-envs.json` (+0 -0) ➕ `packages/hoppscotch-cli/src/__tests__/samples/environments/malformed-envs.json` (+16 -0) 📝 `packages/hoppscotch-cli/src/__tests__/samples/environments/req-body-env-vars-envs.json` (+2 -1) ➕ `packages/hoppscotch-cli/src/__tests__/samples/environments/secret-envs-persistence-scripting-envs.json` (+27 -0) ➕ `packages/hoppscotch-cli/src/__tests__/samples/environments/secret-envs.json` (+40 -0) _...and 10 more files_ </details> ### 📄 Description ### Description Supersedes #3780. This PR adds support for secret variables in environments for the CLI. > There are multiple ways to set values for secret environment variables: - Inject the secret values as variables into the OS environment - When there isn't a `value` field associated with an environment entry, the corresponding value is fetched from the system environment `process.env` with a fallback to an empty string. - Edit the environment export file and add the secret values manually - One can specify values directly in the exported environment file for secret environment variables, which will be preferred over values in the system environment if present. - Updating environments from the scripting context - This takes precedence over all the approaches mentioned above. > Secret environment variables referred to in the request endpoint are masked. ![image](https://github.com/hoppscotch/hoppscotch/assets/25279263/b25f5a83-6222-4b59-83ec-7936f97d8682) ### Other changes - Updates made to the environment from the pre-request script were not persisted in the post-request script while running a collection from the CLI. This PR includes the changes ensuring the updated environments are persisted throughout the scripting context. - Use the versioned entity for `Environment` instead of maintaining the schema. [verzod](https://www.npmjs.com/package/verzod) is added as a `devDependency` (gets inlined in the bundle) to achieve the same. - A new `displayUrl` field is added under the `RequestConfig` interface & `effectiveFinalDisplayURL` field under the `EffectiveHoppRESTRequest` interface. The `getEffectiveRESTRequest` function now includes a new property, `effectiveFinalDisplayURL` under `effectiveRequest`, that includes the secret environment variable values masked. Later, while logging the request endpoint, `displayUrl` is preferred with a fallback to `url` ensuring the transformed value is logged (secret environment variables referred to in the request endpoint) if existing or else falling back to the regular URL. - The test suite is reorganized and updated with respective test cases for the new flows along with ensuring environment export files supplied, that are malformed or based on the bulk export format are handled and the support for short flags (`--e, --env`, `-d, --delay`). - Sample collection and environment files consumed in the test suite are now organized under `collections` & `environments` directories with a consistent naming convention (collection files ending in `coll.json` & environment files in `envs.json`). Depends on #3779 Closes HFE-221. ### Steps to verify - While at the branch, run `pnpm i` so that `verzod` gets installed under `@hoppscotch/cli` and the packages are built. - Grab the following collection contents under the name `test-secret-envs-coll.json`. ```json { "v": 2, "name": "test-secret-envs-coll", "folders": [], "requests": [ { "v": "1", "auth": { "authType": "none", "authActive": true }, "body": { "body": null, "contentType": null }, "name": "test-secret-envs-req", "method": "GET", "params": [], "headers": [ { "key": "Custom-Header", "value": "<<secretKey>>", "active": true } ], "endpoint": "https://httpbin.org/get", "testScript": "pw.test(\"Successfully picks the value for the secret environment variable\", () => {\n pw.expect(pw.env.get(\"secretKey\")).toBe(\"secret-value\")\n\n pw.expect(pw.response.body.headers[\"Custom-Header\"]).toBe(\"secret-value\")\n})", "preRequestScript": "" } ], "headers": [], "auth": { "authType": "inherit", "authActive": false } } ``` A secret environment variable is referred to in the request headers, and relevant tests are added to ensure secret environment variables are picked correctly. ```js pw.test("Successfully picks the value for the secret environment variable", () => { pw.expect(pw.env.get("secretKey")).toBe("secret-value") pw.expect(pw.response.body.headers["Custom-Header"]).toBe("secret-value") }) ``` - Grab the following environment export file contents under `test-secret-envs-coll.json`. ```json { "v": 1, "id": "2", "name": "secret-envs", "variables": [ { "key": "secretKey", "secret": true } ] } ``` - While at the `@hoppscotch/cli` package path, invoke the CLI supplying secret environment variable entries as key-value pairs along with the above collection and environment files (assuming they are in the same path). Alternately, create a symlink for the CLI and refer to the binary via `hopp` in any path. ```bash secretKey=secret-value node bin/hopp test test-secret-envs-coll.json -e secret-envs.json ``` Observe all the tests passing ✅ - Try omitting the key-value pairs corresponding to the secret environment variables and observe the tests failing, stating the value yield was an empty string. Similarly, try specifying a value for the secret environment variable entry by adding a `value` field and observe it overrides the one from the system environment. Values set from the scripting context take the highest precedence, observable by setting a value for the same via pre-request script. Please refer to the [test suite](https://github.com/hoppscotch/hoppscotch/pull/3780/files#diff-4620ebaebd19183ecd749097a8bae19a95b56017fb749f728b74faee307ee95a) for more comprehensive samples. ### Checks - [x] My pull request adheres to the code style of this project - [x] All the tests have passed --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>