mirror of
https://github.com/hoppscotch/hoppscotch.git
synced 2026-04-26 01:06:00 +03:00
[PR #4945] [MERGED] fix(kernel): deterministic form data processing #4997
Labels
No labels
CodeDay
a11y
browser limited
bug
bug fix
cli
core
critical
design
desktop
discussion
docker
documentation
duplicate
enterprise
feature
feature
fosshack
future
good first issue
hacktoberfest
help wanted
i18n
invalid
major
minor
need information
need testing
not applicable to hoppscotch
not reproducible
pull-request
question
refactor
resolved
sandbox
self-host
spam
stale
testmu
wip
wont fix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/hoppscotch#4997
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
📋 Pull Request Information
Original PR: https://github.com/hoppscotch/hoppscotch/pull/4945
Author: @CuriousCorrelation
Created: 3/28/2025
Status: ✅ Merged
Merged: 4/8/2025
Merged by: @jamesgeorge007
Base:
patch← Head:fix-kernel-request-process-serde📝 Commits (3)
bd7eab5fix(kernel): deterministic form data processingfd7f1b3fix: remove debug logs6908690perf(web): intermediate map for poll optimization📊 Changes
19 files changed (+219 additions, -189 deletions)
View changed files
📝
packages/hoppscotch-agent/src-tauri/Cargo.lock(+1 -2)📝
packages/hoppscotch-common/src/helpers/functional/process-request.ts(+6 -0)📝
packages/hoppscotch-common/src/platform/std/kernel-interceptors/agent/index.ts(+10 -2)📝
packages/hoppscotch-common/src/platform/std/kernel-interceptors/agent/store.ts(+1 -4)📝
packages/hoppscotch-common/src/platform/std/kernel-interceptors/browser/index.ts(+1 -1)📝
packages/hoppscotch-common/src/platform/std/kernel-interceptors/extension/index.ts(+1 -1)📝
packages/hoppscotch-common/src/platform/std/kernel-interceptors/native/index.ts(+146 -112)📝
packages/hoppscotch-common/src/platform/std/kernel-interceptors/proxy/index.ts(+1 -1)📝
packages/hoppscotch-desktop/plugin-workspace/relay/Cargo.lock(+1 -25)📝
packages/hoppscotch-desktop/plugin-workspace/relay/Cargo.toml(+0 -1)📝
packages/hoppscotch-desktop/plugin-workspace/relay/src/content.rs(+2 -3)📝
packages/hoppscotch-desktop/plugin-workspace/relay/src/interop.rs(+1 -2)📝
packages/hoppscotch-desktop/plugin-workspace/tauri-plugin-relay/Cargo.lock(+1 -2)📝
packages/hoppscotch-desktop/plugin-workspace/tauri-plugin-relay/dist-js/index.d.ts(+1 -1)📝
packages/hoppscotch-desktop/plugin-workspace/tauri-plugin-relay/dist-js/index.d.ts.map(+1 -1)📝
packages/hoppscotch-desktop/plugin-workspace/tauri-plugin-relay/guest-js/index.ts(+1 -1)📝
packages/hoppscotch-desktop/src-tauri/Cargo.lock(+2 -3)📝
packages/hoppscotch-kernel/src/relay/v/1.ts(+38 -23)📝
pnpm-lock.yaml(+4 -4)📄 Description
When submitting form data requests, the form fields were being processed in a non-deterministic order due to the use of JavaScript's
Mapstructure for serialization. While this was partially addressed in https://github.com/hoppscotch/hoppscotch/pull/4892, the necessity ofmap -> objflattening as explained in the PR code caused yet another unpredictable ordering.The reason being "The mechanics and order of enumerating the properties ... is not specified." in the ECMAScript spec http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-262.pdf. So while inconsistent request payloads between different runs was fixed, now the order was unintentionally deterministic/sorted, particularly affecting
multipart/form-datarequests where field order would change.This was happening due to all numeric keys being sorted to the top and string keys after that, which is the default behavior for JavaScript objects. For example, if you had form fields with keys like "name", "email", "1", "2", "phone", the JavaScript object would enumerate them as "1", "2", "name", "email", "phone" - sorting all numeric keys first in ascending order, followed by string keys in insertion order. This behavior is problematic for multipart form data where the server might expect fields in a specific order or where the order of fields needs to be preserved exactly as they were entered, see https://github.com/hoppscotch/hoppscotch/issues/4889.
Closes HFE-797
The core issue was the use of
IndexMapin Rust andMapin JavaScript, which while guaranteeing consistent iteration order when serialized across platforms, the inconsistency manifested by Javascript's obj ordering sent different orders at the receiving server's end.This PR implements a deterministic approach to form data processing by replacing map-based structures with array-based representations similar to what
superjsondoes. Added apostProcessRelayRequestwithsuperjson.serializeto make sure we have consistent serialization betweenagentandnativeinterceptors. Also updated form data processing in the kernel interceptors to use this deterministic structure, and removed the dependency onindexmapin the Rust side. ThemakeFormDataSerializablefunction has been refactored to build arrays directly instead of converting maps.Form data now maintains consistent field ordering between requests.
Notes to reviewers
You can verify this fix by running the following curl command and comparing it with
echo.hoppscotch.io's base64 response:Note that copy-pasting this directly will insert it in different order, this is an unrelated issue and can be tackled later, for now reinserting form-data or manually reordering them is a good workaround.
RFC 7578 section 5.2 explicitly states: "Form processors given forms with a well-defined ordering SHOULD send back results in order. Intermediaries MUST NOT reorder the results." but some servers don't follow this explicitly so you may notice non-deterministic ordering. The best way would be to decode the base64 response from
echo.hoppscotch.iosince we do maintain the order.Test Results
Before
Decoded To
After
Decoded To
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.