starred/hoppscotch

Fork 0

mirror of https://github.com/hoppscotch/hoppscotch.git synced 2026-04-26 01:06:00 +03:00

[PR #5417] [MERGED] feat(scripting-revamp): chai powered assertions and postman compatibility layer #5200

New issue

Closed

opened 2026-03-17 02:40:07 +03:00 by kerem · 0 comments

kerem commented

2026-03-17 02:40:07 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/hoppscotch/hoppscotch/pull/5417
Author: @jamesgeorge007
Created: 9/30/2025
Status: ✅ Merged
Merged: 10/27/2025
Merged by: @jamesgeorge007

Base: next ← Head: feat/updated-scripting-system

📝 Commits (5)

9ea85b4 feat: chai powered assertions and postman compatibility layer
2a6496e chore: import modal checkbox UI update
ac6f8a1 chore: inform users about postman script import in legacy sandbox
40fb7fd chore: cleanup
4f365bc chore: cleanup

📊 Changes

90 files changed (+30774 additions, -1397 deletions)

View changed files

📝 packages/hoppscotch-cli/src/__tests__/e2e/fixtures/collections/scripting-revamp-coll.json (+548 -4)
📝 packages/hoppscotch-cli/src/utils/pre-request.ts (+19 -6)
📝 packages/hoppscotch-common/locales/en.json (+6 -0)
📝 packages/hoppscotch-common/src/components/collections/ImportExport.vue (+74 -5)
📝 packages/hoppscotch-common/src/components/importExport/Base.vue (+2 -0)
📝 packages/hoppscotch-common/src/components/importExport/ImportExportSteps/FileImport.vue (+49 -2)
📝 packages/hoppscotch-common/src/components/importExport/ImportExportSteps/ImportSummary.vue (+161 -69)
📝 packages/hoppscotch-common/src/helpers/import-export/import/import-sources/FileSource.ts (+7 -2)
📝 packages/hoppscotch-common/src/helpers/import-export/import/postman.ts (+115 -12)
📝 packages/hoppscotch-common/src/types/post-request.d.ts (+701 -37)
📝 packages/hoppscotch-common/src/types/pre-request.d.ts (+517 -32)
📝 packages/hoppscotch-js-sandbox/package.json (+2 -0)
➕ packages/hoppscotch-js-sandbox/src/__tests__/combined/async-await-support.spec.ts (+149 -0)
➕ packages/hoppscotch-js-sandbox/src/__tests__/combined/core-chai-assertions.spec.ts (+607 -0)
➕ packages/hoppscotch-js-sandbox/src/__tests__/combined/deep-include-keys-assertions.spec.ts (+364 -0)
➕ packages/hoppscotch-js-sandbox/src/__tests__/combined/instanceof-assertions.spec.ts (+503 -0)
➕ packages/hoppscotch-js-sandbox/src/__tests__/combined/keys-members-assertions.spec.ts (+443 -0)
➕ packages/hoppscotch-js-sandbox/src/__tests__/combined/length-assertions.spec.ts (+604 -0)
➕ packages/hoppscotch-js-sandbox/src/__tests__/combined/side-effects-assertions.spec.ts (+426 -0)
➕ packages/hoppscotch-js-sandbox/src/__tests__/hopp-namespace/chai-powered-assertions/core-assertions.spec.ts (+616 -0)

...and 70 more files

📄 Description

This PR extends the scripting system with Chai.js-powered assertions for both hopp and pm namespaces, introducing a Postman compatibility layer with script import capability (experimental feature requiring user consent) alongside Postman collection (v2.0/2.1) imports, as proposed in #5221.

Related to FE-319 #2904.

What's changed

Chai.js Assertion Support

Added comprehensive Chai.js BDD assertion support through hopp.expect() for native Hoppscotch scripts and pm.expect() for Postman compatibility, enabling advanced testing patterns with 50+ assertion methods.

Usage Samples

Native Hoppscotch Scripts (`hopp` namespace)

Use hopp.expect() for assertions in native Hoppscotch test scripts:

// Basic equality and type assertions
hopp.test("Status code validation", () => {
  hopp.expect(hopp.response.statusCode).to.equal(200)
  hopp.expect(hopp.response.statusCode).to.be.a('number')
  hopp.expect(hopp.response.statusCode).to.be.above(199).and.below(300)
})

// Property assertions
hopp.test("Response structure", () => {
  const data = hopp.response.body
  hopp.expect(data).to.have.property('userId')
  hopp.expect(data).to.have.own.property('email')
  hopp.expect(data).to.have.nested.property('profile.name')
  hopp.expect(data).to.have.deep.property('settings.notifications.enabled')
})

// Collection and array assertions
hopp.test("Array validation", () => {
  const tags = hopp.response.body.asJSON().tags
  hopp.expect(tags).to.have.lengthOf(3)
  hopp.expect(tags).to.include('nodejs')
  hopp.expect(tags).to.have.members(['nodejs', 'javascript', 'api'])
  hopp.expect(tags).to.have.ordered.members(['api', 'javascript', 'nodejs'])
})

// Object keys and members
hopp.test("Object structure", () => {
  const user = hopp.response.body.asJSON().user
  hopp.expect(user).to.have.all.keys('id', 'name', 'email')
  hopp.expect(user).to.have.any.keys('id', 'name')
  hopp.expect(user).to.include.all.keys('email')
})

// Type checking with instanceof
hopp.test("Type validation", () => {
  hopp.expect(hopp.response.body).to.be.an.instanceOf(Object)
  hopp.expect([1, 2, 3]).to.be.an.instanceOf(Array)
  hopp.expect(new Date()).to.be.an.instanceOf(Date)
  hopp.expect(/regex/).to.be.an.instanceOf(RegExp)
})

// Numeric comparisons
hopp.test("Numeric assertions", () => {
  const responseTime = hopp.response.responseTime
  hopp.expect(responseTime).to.be.below(500)
  hopp.expect(responseTime).to.be.within(0, 1000)
  hopp.expect(3.14159).to.be.closeTo(3.14, 0.01)
})

// String assertions
hopp.test("String validation", () => {
  const contentType = hopp.response.headers['content-type']
  hopp.expect(contentType).to.include('application/json')
  hopp.expect(contentType).to.match(/^application\/json/)
  hopp.expect('hello').to.have.lengthOf(5)
})

// Boolean state assertions
hopp.test("Boolean checks", () => {
  hopp.expect(true).to.be.true
  hopp.expect(false).to.be.false
  hopp.expect(1).to.be.ok
  hopp.expect(null).to.be.null
  hopp.expect(undefined).to.be.undefined
  hopp.expect({}).to.exist
})

// Object state assertions
hopp.test("Object states", () => {
  const sealed = Object.seal({ foo: 'bar' })
  hopp.expect(sealed).to.be.sealed

  const frozen = Object.freeze({ baz: 'qux' })
  hopp.expect(frozen).to.be.frozen

  const extensible = {}
  hopp.expect(extensible).to.be.extensible
})

// Function assertions
hopp.test("Function behavior", () => {
  const throwError = () => { throw new Error('failed') }
  hopp.expect(throwError).to.throw()
  hopp.expect(throwError).to.throw(Error)
  hopp.expect(throwError).to.throw('failed')

  const obj = { getName: function() { return 'test' } }
  hopp.expect(obj).to.respondTo('getName')
})

// Side-effect assertions (change, increase, decrease)
hopp.test("Value changes", () => {
  let counter = 5
  hopp.expect(() => counter++).to.change(() => counter)
  hopp.expect(() => counter++).to.increase(() => counter)
  hopp.expect(() => counter++).to.increase(() => counter).by(1)

  let value = 10
  hopp.expect(() => value -= 2).to.decrease(() => value).by(2)
})

// Negation with .not
hopp.test("Negation assertions", () => {
  hopp.expect(hopp.response.statusCode).to.not.equal(404)
  hopp.expect(hopp.response.body).to.not.be.null
  hopp.expect([1, 2, 3]).to.not.include(4)
  hopp.expect('hello').to.not.match(/world/)
})

// Chaining assertions
hopp.test("Complex chains", () => {
  const data = hopp.response.body
  hopp.expect(data)
    .to.be.an('object')
    .and.have.property('userId')
    .that.is.a('number')
    .and.is.above(0)
})

Postman Compatibility (`pm` namespace)

Use pm.expect() for importing and running Postman collections in Hoppscotch. This provides experimental Postman API compatibility. The supported version range is v2.0/2.1.

Experimental sandbox importer flow
Legacy sandbox import summary nudge

// Basic Postman-style assertions
pm.test("Status code is 200", () => {
  pm.expect(pm.response.code).to.equal(200)
})

pm.test("Response time is acceptable", () => {
  pm.expect(pm.response.responseTime).to.be.below(500)
})

// Response body validation
pm.test("Response structure", () => {
  const jsonData = pm.response.json()
  pm.expect(jsonData).to.have.property('success')
  pm.expect(jsonData.success).to.be.true
  pm.expect(jsonData).to.have.property('data')
  pm.expect(jsonData.data).to.be.an('array')
})

// Array and collection validation
pm.test("Response contains expected items", () => {
  const items = pm.response.json().items
  pm.expect(items).to.have.lengthOf.at.least(1)
  pm.expect(items).to.include('apple')
  pm.expect(items).to.have.members(['apple', 'banana', 'cherry'])
})

// Header validation
pm.test("Headers are correct", () => {
  pm.expect(pm.response.headers.get('content-type')).to.include('application/json')
  pm.expect(pm.response.headers.has('x-api-version')).to.be.true
})

// Postman response assertions (BDD style)
pm.test("Response validation", () => {
  pm.expect(pm.response.to.have.status(200))
  pm.expect(pm.response.to.have.header('content-type'))
  pm.expect(pm.response.to.have.body())
  pm.expect(pm.response.to.have.jsonBody())
  pm.expect(pm.response.to.be.ok) // 2xx status
  pm.expect(pm.response.to.be.success) // alias for ok
  pm.expect(pm.response.to.be.json)
})

// Postman response body assertions
pm.test("JSON body validation", () => {
  pm.expect(pm.response.to.have.jsonBody('userId'))
  pm.expect(pm.response.to.have.jsonBody('profile.name'))
  pm.expect(pm.response.to.have.jsonSchema({
    type: 'object',
    required: ['userId', 'email'],
    properties: {
      userId: { type: 'number' },
      email: { type: 'string' }
    }
  }))
})

// Complex OAuth flow example
pm.test("OAuth token handling", () => {
  const response = pm.response.json()
  const expiresIn = response.expires_in // 3600 seconds
  const expiryTime = Date.now() + (expiresIn * 1000)

  // Auto-converts number to string
  pm.environment.set('token_expiry', expiryTime)
  pm.environment.set('access_token', response.access_token)

  // Verify storage
  pm.expect(pm.environment.get('access_token')).to.equal(response.access_token)
})

// Pre-request script: Set variables
pm.environment.set('requestCount', 0)
pm.environment.set('enableRetry', false)
pm.environment.set('apiKey', null)

Real-World Usage Examples

API Pagination Testing:

hopp.test("Pagination metadata validation", () => {
  const data = hopp.response.body

  // Validate pagination structure
  hopp.expect(data).to.have.all.keys('items', 'page', 'total', 'hasMore')
  hopp.expect(data.items).to.be.an.instanceOf(Array)
  hopp.expect(data.items).to.have.lengthOf.at.most(50) // Max page size

  // Validate each item has required fields
  data.items.forEach(item => {
    hopp.expect(item).to.have.all.keys('id', 'name', 'createdAt')
    hopp.expect(item.id).to.be.a('string')
    hopp.expect(new Date(item.createdAt)).to.be.an.instanceOf(Date)
  })

  // Store next page cursor
  if (data.hasMore) {
    hopp.env.active.set('nextPage', String(data.page + 1))
  }
})

Authentication Flow Testing:

hopp.test("JWT token validation and storage", () => {
  const auth = hopp.response.body

  // Validate token structure
  hopp.expect(auth).to.have.property('accessToken')
  hopp.expect(auth).to.have.property('refreshToken')
  hopp.expect(auth).to.have.property('expiresIn')

  // Validate token format (JWT pattern)
  hopp.expect(auth.accessToken).to.match(/^[A-Za-z0-9-_]+\.[A-Za-z0-9-_]+\.[A-Za-z0-9-_]+$/)

  // Calculate expiry time
  const expiryTime = Date.now() + (auth.expiresIn * 1000)
  hopp.expect(expiryTime).to.be.above(Date.now())

  // Store for subsequent requests
  hopp.env.active.set('authToken', auth.accessToken)
  hopp.env.active.set('tokenExpiry', String(expiryTime))
  hopp.env.active.set('refreshToken', auth.refreshToken)
})

Data Transformation Pipeline:

hopp.test("Product catalog normalization", () => {
  const products = hopp.response.body.products

  // Validate response structure
  hopp.expect(products).to.be.an.instanceOf(Array)
  hopp.expect(products).to.have.lengthOf.at.least(1)

  // Transform and validate each product
  const normalized = products.map(p => ({
    id: p.productId || p.id,
    name: p.title || p.name,
    price: parseFloat(p.price),
    inStock: p.inventory > 0
  }))

  normalized.forEach(product => {
    hopp.expect(product).to.have.all.keys('id', 'name', 'price', 'inStock')
    hopp.expect(product.price).to.be.a('number').and.be.above(0)
    hopp.expect(product.inStock).to.be.a('boolean')
  })

  // Store processed data
  hopp.env.active.set('productCatalog', JSON.stringify(normalized))
})

Rate Limit Handling:

hopp.test("Rate limit headers and retry logic", () => {
  const headers = hopp.response.headers

  // Check rate limit headers exist
  hopp.expect(headers).to.have.property('x-ratelimit-remaining')
  hopp.expect(headers).to.have.property('x-ratelimit-reset')

  const remaining = parseInt(headers['x-ratelimit-remaining'])
  const resetTime = parseInt(headers['x-ratelimit-reset'])

  // Validate rate limit values
  hopp.expect(remaining).to.be.a('number').and.be.at.least(0)
  hopp.expect(resetTime).to.be.above(Date.now() / 1000)

  // Store for request throttling
  hopp.env.active.set('rateLimit', String(remaining))

  if (remaining < 10) {
    const waitTime = (resetTime * 1000) - Date.now()
    hopp.env.active.set('rateLimitWait', String(waitTime))
  }
})

Webhook Signature Verification:

hopp.test("Webhook payload validation", () => {
  const payload = hopp.response.body
  const signature = hopp.response.headers['x-webhook-signature']

  // Validate webhook structure
  hopp.expect(payload).to.have.all.keys('event', 'data', 'timestamp', 'id')
  hopp.expect(payload.event).to.be.a('string').and.have.lengthOf.at.least(1)
  hopp.expect(payload.timestamp).to.be.closeTo(Date.now(), 5000) // Within 5s

  // Validate signature exists
  hopp.expect(signature).to.exist
  hopp.expect(signature).to.match(/^sha256=[a-f0-9]{64}$/)

  // Validate event-specific data
  if (payload.event === 'order.created') {
    hopp.expect(payload.data).to.have.property('orderId')
    hopp.expect(payload.data).to.have.property('total')
    hopp.expect(payload.data.total).to.be.a('number').and.be.above(0)
  }
})

GraphQL Error Handling:

hopp.test("GraphQL response validation", () => {
  const response = hopp.response.body

  // Check for GraphQL errors
  if (response.errors) {
    hopp.expect(response.errors).to.be.an.instanceOf(Array)
    response.errors.forEach(err => {
      hopp.expect(err).to.have.property('message')
      hopp.expect(err).to.have.property('path')
    })
  }

  // Validate data if present
  if (response.data) {
    const user = response.data.user
    hopp.expect(user).to.have.nested.property('profile.email')
    hopp.expect(user.profile.email).to.match(/^[^\s@]+@[^\s@]+\.[^\s@]+$/)

    // Check optional fields
    if (user.profile.avatar) {
      hopp.expect(user.profile.avatar).to.match(/^https?:\/\//)
    }
  } else {
    // If no data, errors must exist
    hopp.expect(response.errors).to.exist
  }
})

Postman Collection Migration Example:

// OAuth flow in Postman compatibility mode
pm.test("OAuth token refresh flow", () => {
  const auth = pm.response.json()

  // Calculate and store expiry
  const expiresIn = auth.expires_in // 3600 seconds
  const expiryTime = Date.now() + (expiresIn * 1000)
  pm.environment.set('token_expiry', expiryTime)
  pm.environment.set('access_token', auth.access_token)

  // Validate token
  pm.expect(pm.response.to.have.status(200))
  pm.expect(pm.response.to.be.json)
  pm.expect(pm.response.to.have.jsonBody('access_token'))
  pm.expect(pm.response.to.have.jsonBody('refresh_token'))
})

Complete Assertion Reference

Type assertions:

.a(type) / .an(type) - Check value type
.instanceof(constructor) - Check instance type
.typeof(type) - Check typeof result

Equality assertions:

.equal(value) / .eq(value) - Strict equality (===)
.eql(value) - Deep equality
.deep.equal(value) - Deep strict equality

Property assertions:

.property(name) - Has property
.own.property(name) - Has own property (not inherited)
.nested.property(path) - Has nested property (e.g., 'a.b.c')
.deep.property(path) - Deep property comparison

Collection assertions:

.include(value) / .contain(value) - Contains value
.members(array) - Has members (order-independent)
.ordered.members(array) - Has members in order
.keys(...keys) - Has keys
.all.keys(...) - Has exactly these keys
.any.keys(...) - Has at least one key
.length(n) / .lengthOf(n) - Length equals n
.lengthOf.at.least(n) - Minimum length
.lengthOf.at.most(n) - Maximum length

Comparison assertions:

.above(n) / .gt(n) - Greater than
.below(n) / .lt(n) - Less than
.at.least(n) / .gte(n) - Greater than or equal
.at.most(n) / .lte(n) - Less than or equal
.within(min, max) - Within range
.closeTo(expected, delta) - Approximately equal

Boolean assertions:

.ok - Truthy
.true - Strictly true
.false - Strictly false
.null - Strictly null
.undefined - Strictly undefined
.exist - Not null or undefined
.empty - Empty (string, array, object)

String assertions:

.match(regex) - Matches regular expression
.string(substring) - Contains substring

Function assertions:

.throw() / .throw(ErrorType) / .throw(message) - Throws error
.respondTo(method) - Has method

Object state assertions:

.extensible - Object.isExtensible()
.sealed - Object.isSealed()
.frozen - Object.isFrozen()

Side-effect assertions:

.change(getter) - Changes value when executed
.increase(getter) - Increases value when executed
.decrease(getter) - Decreases value when executed
.by(delta) - Change amount (chain with change/increase/decrease)

PM-specific response assertions:

pm.response.to.have.status(code) - Status code check
pm.response.to.have.header(name) - Header existence
pm.response.to.have.body() - Has response body
pm.response.to.have.jsonBody() - JSON body exists
pm.response.to.have.jsonBody(path) - JSON property exists
pm.response.to.have.jsonSchema(schema) - Validates JSON schema
pm.response.to.be.ok - 2xx status code
pm.response.to.be.success - Alias for ok
pm.response.to.be.json - JSON content type

Modifiers:

.not - Negation
.deep - Deep comparison
.own - Own properties only
.ordered - Order matters
.nested - Nested property access
.all - All items/keys
.any - Any items/keys
.to / .be / .been / .is / .that / .which / .and / .has / .have / .with / .at / .of / .same - Language chains for readability

Unsupported Features
The following Postman features are not implemented per RFC scope:

pm.sendRequest()
pm.visualizer
pm.collectionVariables
pm.iterationData
pm.execution.setNextRequest()
Legacy patterns like global responseBody variable, require(), etc.

These limitations are documented in error messages when attempting to use unsupported APIs.

Implementation Details

Chai.js runs in the host environment outside the sandbox for security and bundle size optimisation
Pre-check pattern captures object metadata (reference equality, frozen state, own properties) before serialisation
Bootstrap proxy builder in pre-request.js and post-request.js handles method chaining and negation
All assertions recorded to test stack without throwing errors, allowing test execution to continue

Implementation architecture

Serialisation Boundary Handling

The scripting sandbox uses faraday-cage to isolate untrusted code. All data crossing this boundary is serialised, which loses object metadata.

Pre-check pattern:

Inside sandbox: Check object properties (frozen, sealed, reference equality, instanceof)
Pass metadata + values across serialization boundary
Outside sandbox: Use metadata to execute Chai assertions correctly

Example:

// Inside sandbox (bootstrap code)
const isFrozen = Object.isFrozen(myObject)
inputs.chaiFrozen(myObject, modifiers, isFrozen)  // Pass pre-checked result

// Outside sandbox (chai-helpers.ts)
export const chaiFrozen = (value, modifiers, isFrozen) => {
  // Use pre-checked isFrozen rather than checking the serialised value
  const shouldPass = isNegated ? !isFrozen : isFrozen
  // Record result without re-checking
}

This pattern ensures metadata-dependent assertions work correctly despite serialisation.

The __createChaiProxy function appears in both pre-request.js and post-request.js (~1,300 lines each) because each runs in a separate sandbox execution context that cannot share global state.

Notes to reviewers

Use the Postman collection here to test the importer flow, which considers scripts in addition. Failures are expected for the unsupported features.
Use the collection from the CLI test suite for validations.

Chai Integration Rationale

Chai.js is not bundled in the sandbox for several reasons:

Size: Chai is ~300KB; bundling in every execution would be wasteful
Security: Untrusted scripts shouldn't access Chai's internal implementation
Call stack: Complex nested assertions could exceed stack depth in a sandboxed environment
Architecture: Separating the assertion engine from untrusted code improves the security posture

The pre-check pattern trades bandwidth (by passing metadata) for correctness (by preserving object state).

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/hoppscotch/hoppscotch/pull/5417 **Author:** [@jamesgeorge007](https://github.com/jamesgeorge007) **Created:** 9/30/2025 **Status:** ✅ Merged **Merged:** 10/27/2025 **Merged by:** [@jamesgeorge007](https://github.com/jamesgeorge007) **Base:** `next` ← **Head:** `feat/updated-scripting-system` --- ### 📝 Commits (5) - [`9ea85b4`](https://github.com/hoppscotch/hoppscotch/commit/9ea85b49f481af985307048aa685d50a8f451104) feat: chai powered assertions and postman compatibility layer - [`2a6496e`](https://github.com/hoppscotch/hoppscotch/commit/2a6496ef0dc516fb74018b8036a41b8af13c672a) chore: import modal checkbox UI update - [`ac6f8a1`](https://github.com/hoppscotch/hoppscotch/commit/ac6f8a12e9587a01d1bddb6abf44d8476cbcfad5) chore: inform users about postman script import in legacy sandbox - [`40fb7fd`](https://github.com/hoppscotch/hoppscotch/commit/40fb7fde2a7182ec85c3ce469776289a3daabef1) chore: cleanup - [`4f365bc`](https://github.com/hoppscotch/hoppscotch/commit/4f365bc68555b5dddc0e0208ab27247f7fbb569c) chore: cleanup ### 📊 Changes **90 files changed** (+30774 additions, -1397 deletions) <details> <summary>View changed files</summary> 📝 `packages/hoppscotch-cli/src/__tests__/e2e/fixtures/collections/scripting-revamp-coll.json` (+548 -4) 📝 `packages/hoppscotch-cli/src/utils/pre-request.ts` (+19 -6) 📝 `packages/hoppscotch-common/locales/en.json` (+6 -0) 📝 `packages/hoppscotch-common/src/components/collections/ImportExport.vue` (+74 -5) 📝 `packages/hoppscotch-common/src/components/importExport/Base.vue` (+2 -0) 📝 `packages/hoppscotch-common/src/components/importExport/ImportExportSteps/FileImport.vue` (+49 -2) 📝 `packages/hoppscotch-common/src/components/importExport/ImportExportSteps/ImportSummary.vue` (+161 -69) 📝 `packages/hoppscotch-common/src/helpers/import-export/import/import-sources/FileSource.ts` (+7 -2) 📝 `packages/hoppscotch-common/src/helpers/import-export/import/postman.ts` (+115 -12) 📝 `packages/hoppscotch-common/src/types/post-request.d.ts` (+701 -37) 📝 `packages/hoppscotch-common/src/types/pre-request.d.ts` (+517 -32) 📝 `packages/hoppscotch-js-sandbox/package.json` (+2 -0) ➕ `packages/hoppscotch-js-sandbox/src/__tests__/combined/async-await-support.spec.ts` (+149 -0) ➕ `packages/hoppscotch-js-sandbox/src/__tests__/combined/core-chai-assertions.spec.ts` (+607 -0) ➕ `packages/hoppscotch-js-sandbox/src/__tests__/combined/deep-include-keys-assertions.spec.ts` (+364 -0) ➕ `packages/hoppscotch-js-sandbox/src/__tests__/combined/instanceof-assertions.spec.ts` (+503 -0) ➕ `packages/hoppscotch-js-sandbox/src/__tests__/combined/keys-members-assertions.spec.ts` (+443 -0) ➕ `packages/hoppscotch-js-sandbox/src/__tests__/combined/length-assertions.spec.ts` (+604 -0) ➕ `packages/hoppscotch-js-sandbox/src/__tests__/combined/side-effects-assertions.spec.ts` (+426 -0) ➕ `packages/hoppscotch-js-sandbox/src/__tests__/hopp-namespace/chai-powered-assertions/core-assertions.spec.ts` (+616 -0) _...and 70 more files_ </details> ### 📄 Description This PR extends the scripting system with [Chai.js](https://www.chaijs.com/)-powered assertions for both `hopp` and `pm` namespaces, introducing a Postman compatibility layer with script import capability (experimental feature requiring user consent) alongside Postman collection (`v2.0/2.1`) imports, as proposed in #5221. Related to FE-319 #2904. ### What's changed #### Chai.js Assertion Support Added comprehensive Chai.js BDD assertion support through `hopp.expect()` for native Hoppscotch scripts and `pm.expect()` for Postman compatibility, enabling advanced testing patterns with 50+ assertion methods. --- ### Usage Samples #### Native Hoppscotch Scripts (`hopp` namespace) Use `hopp.expect()` for assertions in native Hoppscotch test scripts: ```javascript // Basic equality and type assertions hopp.test("Status code validation", () => { hopp.expect(hopp.response.statusCode).to.equal(200) hopp.expect(hopp.response.statusCode).to.be.a('number') hopp.expect(hopp.response.statusCode).to.be.above(199).and.below(300) }) // Property assertions hopp.test("Response structure", () => { const data = hopp.response.body hopp.expect(data).to.have.property('userId') hopp.expect(data).to.have.own.property('email') hopp.expect(data).to.have.nested.property('profile.name') hopp.expect(data).to.have.deep.property('settings.notifications.enabled') }) // Collection and array assertions hopp.test("Array validation", () => { const tags = hopp.response.body.asJSON().tags hopp.expect(tags).to.have.lengthOf(3) hopp.expect(tags).to.include('nodejs') hopp.expect(tags).to.have.members(['nodejs', 'javascript', 'api']) hopp.expect(tags).to.have.ordered.members(['api', 'javascript', 'nodejs']) }) // Object keys and members hopp.test("Object structure", () => { const user = hopp.response.body.asJSON().user hopp.expect(user).to.have.all.keys('id', 'name', 'email') hopp.expect(user).to.have.any.keys('id', 'name') hopp.expect(user).to.include.all.keys('email') }) // Type checking with instanceof hopp.test("Type validation", () => { hopp.expect(hopp.response.body).to.be.an.instanceOf(Object) hopp.expect([1, 2, 3]).to.be.an.instanceOf(Array) hopp.expect(new Date()).to.be.an.instanceOf(Date) hopp.expect(/regex/).to.be.an.instanceOf(RegExp) }) // Numeric comparisons hopp.test("Numeric assertions", () => { const responseTime = hopp.response.responseTime hopp.expect(responseTime).to.be.below(500) hopp.expect(responseTime).to.be.within(0, 1000) hopp.expect(3.14159).to.be.closeTo(3.14, 0.01) }) // String assertions hopp.test("String validation", () => { const contentType = hopp.response.headers['content-type'] hopp.expect(contentType).to.include('application/json') hopp.expect(contentType).to.match(/^application\/json/) hopp.expect('hello').to.have.lengthOf(5) }) // Boolean state assertions hopp.test("Boolean checks", () => { hopp.expect(true).to.be.true hopp.expect(false).to.be.false hopp.expect(1).to.be.ok hopp.expect(null).to.be.null hopp.expect(undefined).to.be.undefined hopp.expect({}).to.exist }) // Object state assertions hopp.test("Object states", () => { const sealed = Object.seal({ foo: 'bar' }) hopp.expect(sealed).to.be.sealed const frozen = Object.freeze({ baz: 'qux' }) hopp.expect(frozen).to.be.frozen const extensible = {} hopp.expect(extensible).to.be.extensible }) // Function assertions hopp.test("Function behavior", () => { const throwError = () => { throw new Error('failed') } hopp.expect(throwError).to.throw() hopp.expect(throwError).to.throw(Error) hopp.expect(throwError).to.throw('failed') const obj = { getName: function() { return 'test' } } hopp.expect(obj).to.respondTo('getName') }) // Side-effect assertions (change, increase, decrease) hopp.test("Value changes", () => { let counter = 5 hopp.expect(() => counter++).to.change(() => counter) hopp.expect(() => counter++).to.increase(() => counter) hopp.expect(() => counter++).to.increase(() => counter).by(1) let value = 10 hopp.expect(() => value -= 2).to.decrease(() => value).by(2) }) // Negation with .not hopp.test("Negation assertions", () => { hopp.expect(hopp.response.statusCode).to.not.equal(404) hopp.expect(hopp.response.body).to.not.be.null hopp.expect([1, 2, 3]).to.not.include(4) hopp.expect('hello').to.not.match(/world/) }) // Chaining assertions hopp.test("Complex chains", () => { const data = hopp.response.body hopp.expect(data) .to.be.an('object') .and.have.property('userId') .that.is.a('number') .and.is.above(0) }) ``` --- #### Postman Compatibility (`pm` namespace) Use `pm.expect()` for importing and running Postman collections in Hoppscotch. This provides experimental Postman API compatibility. The supported version range is `v2.0/2.1`. - Experimental sandbox importer flow <img width="360" height="264" alt="image" src="https://github.com/user-attachments/assets/5a433054-5138-485b-831d-8a46aeae690b" /> <img width="476" height="454" alt="image" src="https://github.com/user-attachments/assets/2fcc82ad-31d1-4fbe-b9d9-f239b6d05964" /> - Legacy sandbox import summary nudge <img width="379" height="480" alt="image" src="https://github.com/user-attachments/assets/02df13af-356e-46ac-bbaf-a33c71ee1374" /> <img width="380" height="480" alt="image" src="https://github.com/user-attachments/assets/1e409737-54b5-4990-a34d-585afe84e98c" /> ```javascript // Basic Postman-style assertions pm.test("Status code is 200", () => { pm.expect(pm.response.code).to.equal(200) }) pm.test("Response time is acceptable", () => { pm.expect(pm.response.responseTime).to.be.below(500) }) // Response body validation pm.test("Response structure", () => { const jsonData = pm.response.json() pm.expect(jsonData).to.have.property('success') pm.expect(jsonData.success).to.be.true pm.expect(jsonData).to.have.property('data') pm.expect(jsonData.data).to.be.an('array') }) // Array and collection validation pm.test("Response contains expected items", () => { const items = pm.response.json().items pm.expect(items).to.have.lengthOf.at.least(1) pm.expect(items).to.include('apple') pm.expect(items).to.have.members(['apple', 'banana', 'cherry']) }) // Header validation pm.test("Headers are correct", () => { pm.expect(pm.response.headers.get('content-type')).to.include('application/json') pm.expect(pm.response.headers.has('x-api-version')).to.be.true }) // Postman response assertions (BDD style) pm.test("Response validation", () => { pm.expect(pm.response.to.have.status(200)) pm.expect(pm.response.to.have.header('content-type')) pm.expect(pm.response.to.have.body()) pm.expect(pm.response.to.have.jsonBody()) pm.expect(pm.response.to.be.ok) // 2xx status pm.expect(pm.response.to.be.success) // alias for ok pm.expect(pm.response.to.be.json) }) // Postman response body assertions pm.test("JSON body validation", () => { pm.expect(pm.response.to.have.jsonBody('userId')) pm.expect(pm.response.to.have.jsonBody('profile.name')) pm.expect(pm.response.to.have.jsonSchema({ type: 'object', required: ['userId', 'email'], properties: { userId: { type: 'number' }, email: { type: 'string' } } })) }) // Complex OAuth flow example pm.test("OAuth token handling", () => { const response = pm.response.json() const expiresIn = response.expires_in // 3600 seconds const expiryTime = Date.now() + (expiresIn * 1000) // Auto-converts number to string pm.environment.set('token_expiry', expiryTime) pm.environment.set('access_token', response.access_token) // Verify storage pm.expect(pm.environment.get('access_token')).to.equal(response.access_token) }) // Pre-request script: Set variables pm.environment.set('requestCount', 0) pm.environment.set('enableRetry', false) pm.environment.set('apiKey', null) ``` --- #### Real-World Usage Examples **API Pagination Testing**: ```javascript hopp.test("Pagination metadata validation", () => { const data = hopp.response.body // Validate pagination structure hopp.expect(data).to.have.all.keys('items', 'page', 'total', 'hasMore') hopp.expect(data.items).to.be.an.instanceOf(Array) hopp.expect(data.items).to.have.lengthOf.at.most(50) // Max page size // Validate each item has required fields data.items.forEach(item => { hopp.expect(item).to.have.all.keys('id', 'name', 'createdAt') hopp.expect(item.id).to.be.a('string') hopp.expect(new Date(item.createdAt)).to.be.an.instanceOf(Date) }) // Store next page cursor if (data.hasMore) { hopp.env.active.set('nextPage', String(data.page + 1)) } }) ``` **Authentication Flow Testing**: ```javascript hopp.test("JWT token validation and storage", () => { const auth = hopp.response.body // Validate token structure hopp.expect(auth).to.have.property('accessToken') hopp.expect(auth).to.have.property('refreshToken') hopp.expect(auth).to.have.property('expiresIn') // Validate token format (JWT pattern) hopp.expect(auth.accessToken).to.match(/^[A-Za-z0-9-_]+\.[A-Za-z0-9-_]+\.[A-Za-z0-9-_]+$/) // Calculate expiry time const expiryTime = Date.now() + (auth.expiresIn * 1000) hopp.expect(expiryTime).to.be.above(Date.now()) // Store for subsequent requests hopp.env.active.set('authToken', auth.accessToken) hopp.env.active.set('tokenExpiry', String(expiryTime)) hopp.env.active.set('refreshToken', auth.refreshToken) }) ``` **Data Transformation Pipeline**: ```javascript hopp.test("Product catalog normalization", () => { const products = hopp.response.body.products // Validate response structure hopp.expect(products).to.be.an.instanceOf(Array) hopp.expect(products).to.have.lengthOf.at.least(1) // Transform and validate each product const normalized = products.map(p => ({ id: p.productId || p.id, name: p.title || p.name, price: parseFloat(p.price), inStock: p.inventory > 0 })) normalized.forEach(product => { hopp.expect(product).to.have.all.keys('id', 'name', 'price', 'inStock') hopp.expect(product.price).to.be.a('number').and.be.above(0) hopp.expect(product.inStock).to.be.a('boolean') }) // Store processed data hopp.env.active.set('productCatalog', JSON.stringify(normalized)) }) ``` **Rate Limit Handling**: ```javascript hopp.test("Rate limit headers and retry logic", () => { const headers = hopp.response.headers // Check rate limit headers exist hopp.expect(headers).to.have.property('x-ratelimit-remaining') hopp.expect(headers).to.have.property('x-ratelimit-reset') const remaining = parseInt(headers['x-ratelimit-remaining']) const resetTime = parseInt(headers['x-ratelimit-reset']) // Validate rate limit values hopp.expect(remaining).to.be.a('number').and.be.at.least(0) hopp.expect(resetTime).to.be.above(Date.now() / 1000) // Store for request throttling hopp.env.active.set('rateLimit', String(remaining)) if (remaining < 10) { const waitTime = (resetTime * 1000) - Date.now() hopp.env.active.set('rateLimitWait', String(waitTime)) } }) ``` **Webhook Signature Verification**: ```javascript hopp.test("Webhook payload validation", () => { const payload = hopp.response.body const signature = hopp.response.headers['x-webhook-signature'] // Validate webhook structure hopp.expect(payload).to.have.all.keys('event', 'data', 'timestamp', 'id') hopp.expect(payload.event).to.be.a('string').and.have.lengthOf.at.least(1) hopp.expect(payload.timestamp).to.be.closeTo(Date.now(), 5000) // Within 5s // Validate signature exists hopp.expect(signature).to.exist hopp.expect(signature).to.match(/^sha256=[a-f0-9]{64}$/) // Validate event-specific data if (payload.event === 'order.created') { hopp.expect(payload.data).to.have.property('orderId') hopp.expect(payload.data).to.have.property('total') hopp.expect(payload.data.total).to.be.a('number').and.be.above(0) } }) ``` **GraphQL Error Handling**: ```javascript hopp.test("GraphQL response validation", () => { const response = hopp.response.body // Check for GraphQL errors if (response.errors) { hopp.expect(response.errors).to.be.an.instanceOf(Array) response.errors.forEach(err => { hopp.expect(err).to.have.property('message') hopp.expect(err).to.have.property('path') }) } // Validate data if present if (response.data) { const user = response.data.user hopp.expect(user).to.have.nested.property('profile.email') hopp.expect(user.profile.email).to.match(/^[^\s@]+@[^\s@]+\.[^\s@]+$/) // Check optional fields if (user.profile.avatar) { hopp.expect(user.profile.avatar).to.match(/^https?:\/\//) } } else { // If no data, errors must exist hopp.expect(response.errors).to.exist } }) ``` **Postman Collection Migration Example**: ```javascript // OAuth flow in Postman compatibility mode pm.test("OAuth token refresh flow", () => { const auth = pm.response.json() // Calculate and store expiry const expiresIn = auth.expires_in // 3600 seconds const expiryTime = Date.now() + (expiresIn * 1000) pm.environment.set('token_expiry', expiryTime) pm.environment.set('access_token', auth.access_token) // Validate token pm.expect(pm.response.to.have.status(200)) pm.expect(pm.response.to.be.json) pm.expect(pm.response.to.have.jsonBody('access_token')) pm.expect(pm.response.to.have.jsonBody('refresh_token')) }) ``` --- #### Complete Assertion Reference **Type assertions**: - `.a(type)` / `.an(type)` - Check value type - `.instanceof(constructor)` - Check instance type - `.typeof(type)` - Check typeof result **Equality assertions**: - `.equal(value)` / `.eq(value)` - Strict equality (===) - `.eql(value)` - Deep equality - `.deep.equal(value)` - Deep strict equality **Property assertions**: - `.property(name)` - Has property - `.own.property(name)` - Has own property (not inherited) - `.nested.property(path)` - Has nested property (e.g., 'a.b.c') - `.deep.property(path)` - Deep property comparison **Collection assertions**: - `.include(value)` / `.contain(value)` - Contains value - `.members(array)` - Has members (order-independent) - `.ordered.members(array)` - Has members in order - `.keys(...keys)` - Has keys - `.all.keys(...)` - Has exactly these keys - `.any.keys(...)` - Has at least one key - `.length(n)` / `.lengthOf(n)` - Length equals n - `.lengthOf.at.least(n)` - Minimum length - `.lengthOf.at.most(n)` - Maximum length **Comparison assertions**: - `.above(n)` / `.gt(n)` - Greater than - `.below(n)` / `.lt(n)` - Less than - `.at.least(n)` / `.gte(n)` - Greater than or equal - `.at.most(n)` / `.lte(n)` - Less than or equal - `.within(min, max)` - Within range - `.closeTo(expected, delta)` - Approximately equal **Boolean assertions**: - `.ok` - Truthy - `.true` - Strictly true - `.false` - Strictly false - `.null` - Strictly null - `.undefined` - Strictly undefined - `.exist` - Not null or undefined - `.empty` - Empty (string, array, object) **String assertions**: - `.match(regex)` - Matches regular expression - `.string(substring)` - Contains substring **Function assertions**: - `.throw()` / `.throw(ErrorType)` / `.throw(message)` - Throws error - `.respondTo(method)` - Has method **Object state assertions**: - `.extensible` - Object.isExtensible() - `.sealed` - Object.isSealed() - `.frozen` - Object.isFrozen() **Side-effect assertions**: - `.change(getter)` - Changes value when executed - `.increase(getter)` - Increases value when executed - `.decrease(getter)` - Decreases value when executed - `.by(delta)` - Change amount (chain with change/increase/decrease) **PM-specific response assertions**: - `pm.response.to.have.status(code)` - Status code check - `pm.response.to.have.header(name)` - Header existence - `pm.response.to.have.body()` - Has response body - `pm.response.to.have.jsonBody()` - JSON body exists - `pm.response.to.have.jsonBody(path)` - JSON property exists - `pm.response.to.have.jsonSchema(schema)` - Validates JSON schema - `pm.response.to.be.ok` - 2xx status code - `pm.response.to.be.success` - Alias for ok - `pm.response.to.be.json` - JSON content type **Modifiers**: - `.not` - Negation - `.deep` - Deep comparison - `.own` - Own properties only - `.ordered` - Order matters - `.nested` - Nested property access - `.all` - All items/keys - `.any` - Any items/keys - `.to` / `.be` / `.been` / `.is` / `.that` / `.which` / `.and` / `.has` / `.have` / `.with` / `.at` / `.of` / `.same` - Language chains for readability **Unsupported Features** The following Postman features are not implemented per RFC scope: - `pm.sendRequest()` - `pm.visualizer` - `pm.collectionVariables` - `pm.iterationData` - `pm.execution.setNextRequest()` - Legacy patterns like global `responseBody` variable, `require()`, etc. > These limitations are documented in error messages when attempting to use unsupported APIs. --- #### Implementation Details - Chai.js runs in the host environment outside the sandbox for security and bundle size optimisation - Pre-check pattern captures object metadata (reference equality, frozen state, own properties) before serialisation - Bootstrap proxy builder in pre-request.js and post-request.js handles method chaining and negation - All assertions recorded to test stack without throwing errors, allowing test execution to continue ### Implementation architecture #### Serialisation Boundary Handling The scripting sandbox uses [faraday-cage](https://www.npmjs.com/package/faraday-cage) to isolate untrusted code. All data crossing this boundary is serialised, which loses object metadata. **Pre-check pattern**: 1. Inside sandbox: Check object properties (frozen, sealed, reference equality, instanceof) 2. Pass metadata + values across serialization boundary 3. Outside sandbox: Use metadata to execute Chai assertions correctly **Example**: ```javascript // Inside sandbox (bootstrap code) const isFrozen = Object.isFrozen(myObject) inputs.chaiFrozen(myObject, modifiers, isFrozen) // Pass pre-checked result // Outside sandbox (chai-helpers.ts) export const chaiFrozen = (value, modifiers, isFrozen) => { // Use pre-checked isFrozen rather than checking the serialised value const shouldPass = isNegated ? !isFrozen : isFrozen // Record result without re-checking } ``` This pattern ensures metadata-dependent assertions work correctly despite serialisation. The `__createChaiProxy` function appears in both pre-request.js and post-request.js (~1,300 lines each) because each runs in a separate sandbox execution context that cannot share global state. ### Notes to reviewers - Use the Postman collection [here](https://gist.github.com/jamesgeorge007/d443b92945075b50954ecf92f8164d3f) to test the importer flow, which considers scripts in addition. Failures are expected for the unsupported features. - Use the [collection](https://github.com/hoppscotch/hoppscotch/pull/5417/files#diff-918d3794b0c7c8c088f3586a83cd32be359403c1d35b02e89ca10599b625eb35) from the CLI test suite for validations. #### Chai Integration Rationale `Chai.js` is not bundled in the sandbox for several reasons: 1. **Size**: Chai is ~300KB; bundling in every execution would be wasteful 2. **Security**: Untrusted scripts shouldn't access Chai's internal implementation 3. **Call stack**: Complex nested assertions could exceed stack depth in a sandboxed environment 4. **Architecture**: Separating the assertion engine from untrusted code improves the security posture The pre-check pattern trades bandwidth (by passing metadata) for correctness (by preserving object state). --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>