[GH-ISSUE #1614] [BUG] Export to HAR doesn't include request body #1607

Open
opened 2026-03-03 19:52:52 +03:00 by kerem · 6 comments
Owner

Originally created by @clouedoc on GitHub (Apr 26, 2023).
Original GitHub issue: https://github.com/ProxymanApp/Proxyman/issues/1614

Originally assigned to: @NghiaTranUIT on GitHub.

Description

I am exporting a whole flow to .HAR in order to programatically analyse them.
I've found that a specific request does not includes the request body in the generated .HAR.

Steps to Reproduce

  1. Download and open this Proxyman log file
    proxyman_export_bug_repro.proxymanlogv2.zip

  2. Confirm that the post body of the request is visible
    image

  3. Export to a HAR file

  4. Open the exported HAR file in a text editor

  5. Look at the following path: .log.entries[0].request. There is no body even though it's visible in the Proxyman log.

HAR file
{
  "log": {
    "version": "1.2",
    "creator": { "name": "Proxyman", "version": "4.6.1" },
    "entries": [
      {
        "time": 185.87184143066406,
        "_isHTTPS": true,
        "_webSocketMessages": null,
        "_remoteDeviceIP": "192.168.1.100",
        "timings": {
          "connect": 1,
          "send": -1,
          "dns": -1,
          "ssl": -1,
          "wait": -1,
          "blocked": -1,
          "receive": -1
        },
        "_serverAddress": "35.186.224.25",
        "_isIntercepted": true,
        "_id": "7831",
        "serverIPAddress": "35.186.224.25",
        "_name": "7831",
        "_clientAddress": "192.168.1.100",
        "_clientBundlePath": null,
        "request": {
          "method": "POST",
          "bodySize": 156,
          "headersSize": 248,
          "cookies": [],
          "headers": [
            { "name": "Accept", "value": "application/x-protobuf" },
            {
              "name": "User-Agent",
              "value": "Spotify/8.6.58 Android/33 (sdk_gphone64_arm64)"
            },
            { "name": "Content-Type", "value": "application/x-protobuf" },
            { "name": "Content-Length", "value": "156" },
            { "name": "Host", "value": "clienttoken.spotify.com" },
            { "name": "Connection", "value": "Keep-Alive" },
            { "name": "Accept-Encoding", "value": "gzip" }
          ],
          "queryString": [],
          "httpVersion": "HTTP/1.1",
          "url": "https://clienttoken.spotify.com/v1/clienttoken"
        },
        "_serverPort": 443,
        "_clientName": null,
        "_clientPort": 52078,
        "response": {
          "status": 200,
          "content": {
            "size": 374,
            "mimeType": "application/x-protobuf",
            "encoding": "base64",
            "text": "CAIa8QIKxAJBQUNER0hMWjlPTlpoZ0lOY3ZNOXNtMURGdFQ0Ti9oUUZScHRCZUFHR2JwNWFtRXlaVGRXT3R6M0dla3Q3SnNDS1Q1VHFVdTRtcWJoYmN1d0piYVNyMFZQSGIyMHNnbFRmd1NoZE1sZmJuOUdEaTZGMytJbVJTOTdnS3hWMkhnKzFxbk5lUVQ4NUpmNm5HWGZGTEVwZnJpU2xqRGhkLy8rNWJHbDd2MFFSVndxUG8xN3JpdUovRVhtM1lWSzh0ZVkzWnJJQjR3aWJjeDB4KzZuVGtuVnBiSlk1VXg4WG1KVVVSZ0hHZmhSaVg5aGVIMDIwa0FBcHc1MlltNVIvUk5ES0UrcGNyVzREZ3ZlM3NzY1R0TzFQRXFFS0h3TGhxQldiRHoxdml1OENtbTlYdXdPcHJiREtudXZMMWd2dGtWTGFOdz0SKAgDIiQIFBIgRjlDNTY0QTkyMDRCMTQ2MDY1RjRFN0YyOTYzOEIzODU="
          },
          "bodySize": 374,
          "headersSize": 366,
          "cookies": [],
          "statusText": "OK",
          "headers": [
            { "name": "content-type", "value": "application/x-protobuf" },
            { "name": "cache-control", "value": "private, max-age=0" },
            { "name": "content-encoding", "value": "gzip" },
            {
              "name": "strict-transport-security",
              "value": "max-age=31536000"
            },
            { "name": "x-content-type-options", "value": "nosniff" },
            { "name": "date", "value": "Sat, 22 Apr 2023 14:47:39 GMT" },
            { "name": "server", "value": "envoy" },
            { "name": "Via", "value": "HTTP/2 edgeproxy, 1.1 google" },
            {
              "name": "Alt-Svc",
              "value": "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000"
            },
            { "name": "Transfer-Encoding", "value": "chunked" }
          ],
          "httpVersion": "HTTP/1.1",
          "redirectURL": ""
        },
        "comment": "",
        "startedDateTime": "2023-04-22T15:47:39.910+01:00",
        "cache": {}
      }
    ]
  }
}

Current Behavior

Some POST requests get their body suppressed when exported to HAR

Expected Behavior

This POST request gets their body exported to HAR correctly

Environment

  • App version: Proxyman 4.6.1 (46010)
  • macOS version: macOS Ventura (13.2.1)
Originally created by @clouedoc on GitHub (Apr 26, 2023). Original GitHub issue: https://github.com/ProxymanApp/Proxyman/issues/1614 Originally assigned to: @NghiaTranUIT on GitHub. ## Description I am exporting a whole flow to .HAR in order to programatically analyse them. I've found that a specific request does not includes the request body in the generated .HAR. ## Steps to Reproduce <!-- Add relevant code and/or a live example --> 1. Download and open this Proxyman log file [proxyman_export_bug_repro.proxymanlogv2.zip](https://github.com/ProxymanApp/Proxyman/files/11327911/proxyman_export_bug_repro.proxymanlogv2.zip) 2. Confirm that the post body **of the request** is visible <img width="482" alt="image" src="https://user-images.githubusercontent.com/13921610/234439626-ca6bd5f4-2398-423e-bd0e-6cf4fab830c3.png"> 4. Export to a HAR file 5. Open the exported HAR file in a text editor 6. Look at the following path: `.log.entries[0].request`. There is no body even though it's visible in the Proxyman log. <details> <summary>HAR file</summary> ```json { "log": { "version": "1.2", "creator": { "name": "Proxyman", "version": "4.6.1" }, "entries": [ { "time": 185.87184143066406, "_isHTTPS": true, "_webSocketMessages": null, "_remoteDeviceIP": "192.168.1.100", "timings": { "connect": 1, "send": -1, "dns": -1, "ssl": -1, "wait": -1, "blocked": -1, "receive": -1 }, "_serverAddress": "35.186.224.25", "_isIntercepted": true, "_id": "7831", "serverIPAddress": "35.186.224.25", "_name": "7831", "_clientAddress": "192.168.1.100", "_clientBundlePath": null, "request": { "method": "POST", "bodySize": 156, "headersSize": 248, "cookies": [], "headers": [ { "name": "Accept", "value": "application/x-protobuf" }, { "name": "User-Agent", "value": "Spotify/8.6.58 Android/33 (sdk_gphone64_arm64)" }, { "name": "Content-Type", "value": "application/x-protobuf" }, { "name": "Content-Length", "value": "156" }, { "name": "Host", "value": "clienttoken.spotify.com" }, { "name": "Connection", "value": "Keep-Alive" }, { "name": "Accept-Encoding", "value": "gzip" } ], "queryString": [], "httpVersion": "HTTP/1.1", "url": "https://clienttoken.spotify.com/v1/clienttoken" }, "_serverPort": 443, "_clientName": null, "_clientPort": 52078, "response": { "status": 200, "content": { "size": 374, "mimeType": "application/x-protobuf", "encoding": "base64", "text": "CAIa8QIKxAJBQUNER0hMWjlPTlpoZ0lOY3ZNOXNtMURGdFQ0Ti9oUUZScHRCZUFHR2JwNWFtRXlaVGRXT3R6M0dla3Q3SnNDS1Q1VHFVdTRtcWJoYmN1d0piYVNyMFZQSGIyMHNnbFRmd1NoZE1sZmJuOUdEaTZGMytJbVJTOTdnS3hWMkhnKzFxbk5lUVQ4NUpmNm5HWGZGTEVwZnJpU2xqRGhkLy8rNWJHbDd2MFFSVndxUG8xN3JpdUovRVhtM1lWSzh0ZVkzWnJJQjR3aWJjeDB4KzZuVGtuVnBiSlk1VXg4WG1KVVVSZ0hHZmhSaVg5aGVIMDIwa0FBcHc1MlltNVIvUk5ES0UrcGNyVzREZ3ZlM3NzY1R0TzFQRXFFS0h3TGhxQldiRHoxdml1OENtbTlYdXdPcHJiREtudXZMMWd2dGtWTGFOdz0SKAgDIiQIFBIgRjlDNTY0QTkyMDRCMTQ2MDY1RjRFN0YyOTYzOEIzODU=" }, "bodySize": 374, "headersSize": 366, "cookies": [], "statusText": "OK", "headers": [ { "name": "content-type", "value": "application/x-protobuf" }, { "name": "cache-control", "value": "private, max-age=0" }, { "name": "content-encoding", "value": "gzip" }, { "name": "strict-transport-security", "value": "max-age=31536000" }, { "name": "x-content-type-options", "value": "nosniff" }, { "name": "date", "value": "Sat, 22 Apr 2023 14:47:39 GMT" }, { "name": "server", "value": "envoy" }, { "name": "Via", "value": "HTTP/2 edgeproxy, 1.1 google" }, { "name": "Alt-Svc", "value": "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000" }, { "name": "Transfer-Encoding", "value": "chunked" } ], "httpVersion": "HTTP/1.1", "redirectURL": "" }, "comment": "", "startedDateTime": "2023-04-22T15:47:39.910+01:00", "cache": {} } ] } } ``` </details> ## Current Behavior Some POST requests get their body suppressed when exported to HAR ## Expected Behavior This POST request gets their body exported to HAR correctly ## Environment - App version: Proxyman 4.6.1 (46010) - macOS version: macOS Ventura (13.2.1)
Author
Owner

@NghiaTranUIT commented on GitHub (Apr 26, 2023):

Thanks. I'm able to reproduce it too. I'm fixing it now 👍

<!-- gh-comment-id:1522698321 --> @NghiaTranUIT commented on GitHub (Apr 26, 2023): Thanks. I'm able to reproduce it too. I'm fixing it now 👍
Author
Owner

@NghiaTranUIT commented on GitHub (Apr 26, 2023):

The problem is: HAR 1.2 Spec doesn't specify how to store the Binary data in the Request body. HAR supports Binary Data in the Response Body (in content), but ignore the Request for some reason.

Request Body is stored at postData, but it only supports String or multipart/form-data.

I will double check with other HAR tool, to see how it handles it 👍

Ref:

<!-- gh-comment-id:1522712468 --> @NghiaTranUIT commented on GitHub (Apr 26, 2023): The problem is: HAR 1.2 Spec doesn't specify how to store the Binary data in the Request body. HAR supports Binary Data in the Response Body (in `content`), but ignore the Request for some reason. Request Body is stored at [postData](http://www.softwareishard.com/blog/har-12-spec/#postData), but it only supports String or multipart/form-data. I will double check with other HAR tool, to see how it handles it 👍 Ref: - http://www.softwareishard.com/blog/har-12-spec/ - https://indigo.re/posts/2020-10-09-har-is-clumsy.html
Author
Owner

@NghiaTranUIT commented on GitHub (Apr 26, 2023):

@clouedoc you can try this Beta build: https://download.proxyman.io/beta/Proxyman_4.6.1_Fix_Missing_Request_body_in_HAR_file.dmg

As I mentioned in the previous comment, there is no technical specification on how to store the Binary Request Body in the HAR 1.2. Therefore, I did research on other tools.

  • REST Insomnia: They encode the request body as a Base64 string

=> Proxyman supports it too 👍

CleanShot 2023-04-26 at 11 26 54@2x

  • You can export or import the HAR file. It works well with Text-based, form or binary format in the Request body 👍

Please note that this HAR file could not be parsed by some online HAR tool, because it's not officially supported.

<!-- gh-comment-id:1522764701 --> @NghiaTranUIT commented on GitHub (Apr 26, 2023): @clouedoc you can try this Beta build: https://download.proxyman.io/beta/Proxyman_4.6.1_Fix_Missing_Request_body_in_HAR_file.dmg As I mentioned in the previous comment, there is no technical specification on how to store the Binary Request Body in the HAR 1.2. Therefore, I did research on other tools. - REST Insomnia: They encode the request body as a Base64 string => Proxyman supports it too 👍 ![CleanShot 2023-04-26 at 11 26 54@2x](https://user-images.githubusercontent.com/5878421/234470132-4d219d8f-f9c0-4087-b42e-b8c8ec8d98e6.jpg) - You can export or import the HAR file. It works well with Text-based, form or binary format in the Request body 👍 ------------- Please note that this HAR file could not be parsed by some online HAR tool, because it's not officially supported.
Author
Owner

@clouedoc commented on GitHub (Apr 26, 2023):

Wow, thank you so much! I can resume my work without switching tool, it makes my life easier 🙂

<!-- gh-comment-id:1523651684 --> @clouedoc commented on GitHub (Apr 26, 2023): Wow, thank you so much! I can resume my work without switching tool, it makes my life easier 🙂
Author
Owner

@NghiaTranUIT commented on GitHub (Apr 27, 2023):

Look like there is an bug when parsing the Binary Body from HAR file. It doesn't show the Protobuf data as the proxymanlogv2 does. Fixing it now 👍

<!-- gh-comment-id:1524318746 --> @NghiaTranUIT commented on GitHub (Apr 27, 2023): Look like there is an bug when parsing the Binary Body from HAR file. It doesn't show the Protobuf data as the `proxymanlogv2` does. Fixing it now 👍
Author
Owner
<!-- gh-comment-id:1524529926 --> @NghiaTranUIT commented on GitHub (Apr 27, 2023): Fixed: https://download.proxyman.io/beta/Proxyman_4.6.1_Missing_protobuf_body_in_Response_in_HAR_file.dmg ![CleanShot 2023-04-27 at 08 51 09@2x](https://user-images.githubusercontent.com/5878421/234747169-908f9dfa-18c0-4353-b00d-735e14af8eab.jpg)
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/Proxyman#1607
No description provided.