[GH-ISSUE #828] Parse Error: when https Response enabled #823

Open
opened 2026-03-03 19:22:11 +03:00 by kerem · 10 comments
Owner

Originally created by @TimidLupus on GitHub (Mar 20, 2021).
Original GitHub issue: https://github.com/ProxymanApp/Proxyman/issues/828

Originally assigned to: @NghiaTranUIT on GitHub.

Proxyman version?

(version 2.20.0 (22000)

macOS Version?

(mac 11.2.3)

Steps to reproduce

Unfortunately I can't describe here..this is happening on a particular API request from my android emulator when I enable HTTPS Response.

Following is the list of tools and responses received from them.

Proxy man

Following error is logged:
The operation couldn’t be completed. (NIOHTTP1.HTTPParserError error 15.) (code=15)

HTTP Tool kit

it logs an error stating unable to parse one of the request headers.

Charles

which parses the request without any problem (I don't wanna use charles though 🤦‍♂️)

Expected behaviour

proxyman should parse the response once the request returns 😍😍

Actual behaviour

Response is not parsed when in Enable HTTPS mode 😔

Screenshots (optional)

Screenshot 2021-03-20 at 20 25 07
Originally created by @TimidLupus on GitHub (Mar 20, 2021). Original GitHub issue: https://github.com/ProxymanApp/Proxyman/issues/828 Originally assigned to: @NghiaTranUIT on GitHub. ### Proxyman version? (version 2.20.0 (22000) ### macOS Version? (mac 11.2.3) ### Steps to reproduce Unfortunately I can't describe here..this is happening on a particular API request from my android emulator when I enable HTTPS Response. Following is the list of tools and responses received from them. ### Proxy man Following error is logged: `The operation couldn’t be completed. (NIOHTTP1.HTTPParserError error 15.) (code=15)` ### HTTP Tool kit it logs an error stating unable to parse one of the request headers. ### Charles which parses the request without any problem (I don't wanna use charles though 🤦‍♂️) ### Expected behaviour proxyman should parse the response once the request returns 😍😍 ### Actual behaviour Response is not parsed when in Enable HTTPS mode 😔 ### Screenshots (optional) <img width="550" alt="Screenshot 2021-03-20 at 20 25 07" src="https://user-images.githubusercontent.com/15221681/111884939-a2049b80-89bc-11eb-8df7-3a7820c3fb9e.png">
Author
Owner

@NghiaTranUIT commented on GitHub (Mar 21, 2021):

Hey @TimidLupus it looks like SwiftNIO could parse your request due to malformed HTTP Message.

HTTPParserError error = 15 means invalidHeaderToken

If you don't mind, please use Charles -> capture this request -> go to Raw Tab in Request and Response -> Share with top 5 line via support@proxyman.io

Thank you in advance 🌮

<!-- gh-comment-id:803518051 --> @NghiaTranUIT commented on GitHub (Mar 21, 2021): Hey @TimidLupus it looks like SwiftNIO could parse your request due to malformed HTTP Message. HTTPParserError error = 15 means `invalidHeaderToken` - Ref: https://github.com/apple/swift-nio/blob/8ea768b0b8e52fd11006b88c68f974848094d7e9/Sources/NIOHTTP1/HTTPTypes.swift#L771 If you don't mind, please use Charles -> capture this request -> go to **Raw Tab** in Request and Response -> Share with top 5 line via support@proxyman.io Thank you in advance 🌮
Author
Owner

@NghiaTranUIT commented on GitHub (Mar 21, 2021):

@TimidLupus Please help me to test this scenario:

  • Create this request with Insomnia, Paw or Postman, then using Proxyman to capture the request -> Does it work?
  • Which network library are you using on Android?
  • Does this error request has any special param (Array in Query or weird URL)? 🤔
<!-- gh-comment-id:803518229 --> @NghiaTranUIT commented on GitHub (Mar 21, 2021): @TimidLupus Please help me to test this scenario: - Create this request with Insomnia, Paw or Postman, then using Proxyman to capture the request -> Does it work? - Which network library are you using on Android? - Does this error request has any special param (Array in Query or weird URL)? 🤔
Author
Owner

@TimidLupus commented on GitHub (Mar 21, 2021):

@NghiaTranUIT thanks for your quick response. Yes this works on other tools I mentioned in my original issue comment.
I have sent you the details you wanted via support@proxyman.io

  • we are using OkHttp for networking
  • no there are no array or weirdness in the url. Details of the request are in the email.
<!-- gh-comment-id:803578213 --> @TimidLupus commented on GitHub (Mar 21, 2021): @NghiaTranUIT thanks for your quick response. Yes this works on other tools I mentioned in my original issue comment. I have sent you the details you wanted via support@proxyman.io - we are using OkHttp for networking - no there are no array or weirdness in the url. Details of the request are in the email.
Author
Owner

@TimidLupus commented on GitHub (Mar 22, 2021):

@NghiaTranUIT Hi, I know it's difficult to sort these type of bugs..
Meanwhile can you guide me how to bypass this particular request when HTTPS RESPONSE is enabled?

<!-- gh-comment-id:804228224 --> @TimidLupus commented on GitHub (Mar 22, 2021): @NghiaTranUIT Hi, I know it's difficult to sort these type of bugs.. Meanwhile can you guide me how to bypass this particular request when `HTTPS RESPONSE` is enabled?
Author
Owner

@TimidLupus commented on GitHub (Mar 22, 2021):

@NghiaTranUIT Ok, there is some good news. my team mate actually has pin pointed the problem..
so if the header has brackets "(" ")" then Proxyman is unable to parse and hence that header parsing error.

I believe you have received my email and it has that header with brackets which causes the issue.

Hoping you will get around and smash this bug 😊

Regards,

<!-- gh-comment-id:804255124 --> @TimidLupus commented on GitHub (Mar 22, 2021): @NghiaTranUIT Ok, there is some good news. my team mate actually has pin pointed the problem.. so if the header has brackets "(" ")" then Proxyman is unable to parse and hence that header parsing error. I believe you have received my email and it has that header with brackets which causes the issue. Hoping you will get around and smash this bug 😊 Regards,
Author
Owner

@NghiaTranUIT commented on GitHub (Mar 23, 2021):

@TimidLupus Is there any reason why you use brackets in Header Key Name? From what I google, it doesn't allow to have the brackets.

I tried with curl and Insomnia, it's invalid too.

Screen Shot 2021-03-23 at 08 55 49
<!-- gh-comment-id:804524833 --> @NghiaTranUIT commented on GitHub (Mar 23, 2021): @TimidLupus Is there any reason why you use brackets in Header Key Name? From what I google, it doesn't allow to have the brackets. I tried with curl and Insomnia, it's invalid too. <img width="1085" alt="Screen Shot 2021-03-23 at 08 55 49" src="https://user-images.githubusercontent.com/5878421/112081022-929c7280-8bb5-11eb-94f0-960ee1d5a3ea.png">
Author
Owner

@TimidLupus commented on GitHub (Mar 25, 2021):

@NghiaTranUIT

  • regarding your question TBH we have no idea. We received the spec just like that. (kinda stuck with it)
  • For now we have come up with a solution of url encoding it in the code (which seems to be working)
  • Charles successfully parses these so it's kinda annoying (we are actually kinda moving from Charles actually we have moved)

so probably if you are getting bored in the future and have no big tickets in your way then I would really appreciate resolving this tiny annoyance.

Lastly, I actually wanna thank you for your swift responses and diagnosis.

Regards,

<!-- gh-comment-id:806562952 --> @TimidLupus commented on GitHub (Mar 25, 2021): @NghiaTranUIT - regarding your question TBH we have no idea. We received the spec just like that. (kinda stuck with it) - For now we have come up with a solution of url encoding it in the code (which seems to be working) - Charles successfully parses these so it's kinda annoying (we are actually kinda moving from Charles actually we have moved) so probably if you are getting bored in the future and have no big tickets in your way then I would really appreciate resolving this tiny annoyance. Lastly, I actually wanna thank you for your swift responses and diagnosis. Regards,
Author
Owner

@NghiaTranUIT commented on GitHub (Mar 25, 2021):

Hey @TimidLupus, I'm not sure if I'm able to fix it. Proxyman is using Swift NIO behind the sense and it takes care of the HTTPS Parser too (Source: https://github.com/apple/swift-nio/tree/main/Sources/CNIOHTTPParser). This parser locates deeply inside the source code, so it's really difficult to fix your bug.

Charles works because it ignores the semantic schema of the HTTP Message, but not SwiftNIO.

I highly recommend requesting a change from your Backend or encoded the header to make it works.

<!-- gh-comment-id:806789631 --> @NghiaTranUIT commented on GitHub (Mar 25, 2021): Hey @TimidLupus, I'm not sure if I'm able to fix it. Proxyman is using Swift NIO behind the sense and it takes care of the HTTPS Parser too (Source: https://github.com/apple/swift-nio/tree/main/Sources/CNIOHTTPParser). This parser locates deeply inside the source code, so it's really difficult to fix your bug. Charles works because it ignores the semantic schema of the HTTP Message, but not SwiftNIO. I highly recommend requesting a change from your Backend or encoded the header to make it works.
Author
Owner

@TimidLupus commented on GitHub (Mar 25, 2021):

Fair enough, I am going to close the issue.

  • yes we are encoding the header for now until we get the chance to request a change to our backend.

Thanks for your support.

Regards,

<!-- gh-comment-id:807581088 --> @TimidLupus commented on GitHub (Mar 25, 2021): Fair enough, I am going to close the issue. - yes we are encoding the header for now until we get the chance to request a change to our backend. Thanks for your support. Regards,
Author
Owner

@NghiaTranUIT commented on GitHub (Mar 26, 2021):

I'd like to open this ticket, so I can look back later if SwiftNIO fixes it 👍

<!-- gh-comment-id:807877325 --> @NghiaTranUIT commented on GitHub (Mar 26, 2021): I'd like to open this ticket, so I can look back later if SwiftNIO fixes it 👍
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/Proxyman#823
No description provided.