[GH-ISSUE #357] Some urls are not shown in proxy #354

Closed
opened 2026-03-03 19:17:44 +03:00 by kerem · 14 comments
Owner

Originally created by @Schaefers on GitHub (Dec 20, 2019).
Original GitHub issue: https://github.com/ProxymanApp/Proxyman/issues/357

Originally assigned to: @NghiaTranUIT on GitHub.

Proxyman version? (Ex. Proxyman 1.4.3)

Proxyman 1.11.0

macOS Version? (Ex. mac 10.14)

mac 10.15.2

Steps to reproduce

  1. Call http://customurl/v3/projects?expand=subprojects,comments,partners&filter[projectType][eq]=3&filter[projectStatus][eq]=2 without encoded html entities.
  2. The call is not shown in proxy

Expected behavior

The call should be shown in the list

Additional hints

These calls are performed via mobile apps, if accessing the url using a browser, the special characters are automatically encoded. As a result they are shown in Proxyman.

Originally created by @Schaefers on GitHub (Dec 20, 2019). Original GitHub issue: https://github.com/ProxymanApp/Proxyman/issues/357 Originally assigned to: @NghiaTranUIT on GitHub. ### Proxyman version? (Ex. Proxyman 1.4.3) Proxyman 1.11.0 ### macOS Version? (Ex. mac 10.14) mac 10.15.2 ### Steps to reproduce 1. Call http://*customurl*/v3/projects?expand=subprojects,comments,partners&filter[projectType][eq]=3&filter[projectStatus][eq]=2 without encoded html entities. 2. The call is not shown in proxy ### Expected behavior The call should be shown in the list ### Additional hints These calls are performed via mobile apps, if accessing the url using a browser, the special characters are automatically encoded. As a result they are shown in Proxyman.
kerem 2026-03-03 19:17:44 +03:00
Author
Owner

@NghiaTranUIT commented on GitHub (Dec 20, 2019):

It looks like this request is failed because the URL doesn't properly encode with Percent-Encoding) when going through the Proxyman -> Doesn't appear on the app.

I just tested this URL with curl and Insomnia. It automatically encoded, so it works in Proxyman.

About mobile app, do you use NSURLSession or Alamofire to construct this URL 🤔? I would like to reproduce in iOS app for easier fixing it.

<!-- gh-comment-id:567828835 --> @NghiaTranUIT commented on GitHub (Dec 20, 2019): It looks like this request is failed because the URL doesn't properly encode with Percent-Encoding) when going through the Proxyman -> Doesn't appear on the app. I just tested this URL with curl and Insomnia. It automatically encoded, so it works in Proxyman. About mobile app, do you use NSURLSession or Alamofire to construct this URL 🤔? I would like to reproduce in iOS app for easier fixing it.
Author
Owner

@NghiaTranUIT commented on GitHub (Dec 20, 2019):

On the other hand, I will improve how Proxyman handles the failed requests to make sure that it still visible on the app even though it's failed.

<!-- gh-comment-id:567829056 --> @NghiaTranUIT commented on GitHub (Dec 20, 2019): On the other hand, I will improve how Proxyman handles the failed requests to make sure that it still visible on the app even though it's failed.
Author
Owner

@Schaefers commented on GitHub (Dec 20, 2019):

It looks like this request is failed because the URL doesn't properly encode with Percent-Encoding) when going through the Proxyman -> Doesn't appear on the app.

I just tested this URL with curl and Insomnia. It automatically encoded, so it works in Proxyman.

About mobile app, do you use NSURLSession or Alamofire to construct this URL 🤔? I would like to reproduce in iOS app for easier fixing it.

In the mobile app we're using Alamofire for iOS and Fuel for Android.

Thanks in advance.

<!-- gh-comment-id:567829708 --> @Schaefers commented on GitHub (Dec 20, 2019): > It looks like this request is failed because the URL doesn't properly encode with Percent-Encoding) when going through the Proxyman -> Doesn't appear on the app. > > I just tested this URL with curl and Insomnia. It automatically encoded, so it works in Proxyman. > > About mobile app, do you use NSURLSession or Alamofire to construct this URL 🤔? I would like to reproduce in iOS app for easier fixing it. In the mobile app we're using Alamofire for iOS and Fuel for Android. Thanks in advance.
Author
Owner

@NghiaTranUIT commented on GitHub (Dec 20, 2019):

Thank. Let me try to disable URL-Encoding in Alamofire and investigate it 👍

Question: Why don't we encode the URL before sending? Since percent-encoded is universally accepted in many web-browser and server.

<!-- gh-comment-id:567831041 --> @NghiaTranUIT commented on GitHub (Dec 20, 2019): Thank. Let me try to disable URL-Encoding in Alamofire and investigate it 👍 Question: Why don't we encode the URL before sending? Since percent-encoded is universally accepted in many web-browser and server.
Author
Owner

@Schaefers commented on GitHub (Dec 20, 2019):

I tested again with the iOS App and this request is properly shown, maybe it's related to Fuel on Android..

<!-- gh-comment-id:567833319 --> @Schaefers commented on GitHub (Dec 20, 2019): I tested again with the iOS App and this request is properly shown, maybe it's related to Fuel on Android..
Author
Owner

@NghiaTranUIT commented on GitHub (Dec 20, 2019):

Glad to hear that. Let me know the result on Android build 👍

<!-- gh-comment-id:567838519 --> @NghiaTranUIT commented on GitHub (Dec 20, 2019): Glad to hear that. Let me know the result on Android build 👍
Author
Owner

@Schaefers commented on GitHub (Dec 20, 2019):

I‘m not responsible for the Android app, so it might take some time to test it. But I know from one of our Android developers, that this call works with Charles or mitm-Proxy, so it seems to be a bug on your end.

<!-- gh-comment-id:567840912 --> @Schaefers commented on GitHub (Dec 20, 2019): I‘m not responsible for the Android app, so it might take some time to test it. But I know from one of our Android developers, that this call works with Charles or mitm-Proxy, so it seems to be a bug on your end.
Author
Owner

@NghiaTranUIT commented on GitHub (Dec 20, 2019):

Sure, I will investigate by disabling the Percent-encoding 👍

<!-- gh-comment-id:567847857 --> @NghiaTranUIT commented on GitHub (Dec 20, 2019): Sure, I will investigate by disabling the Percent-encoding 👍
Author
Owner

@ghost commented on GitHub (Dec 29, 2019):

Same issue is with Google Fonts.

This URL is showing in Proxyman and it loads fine:
https://fonts.googleapis.com/css?family=Roboto:300

However this URL does not load and also not showing in Proxyman
https://fonts.googleapis.com/css?family=Roboto:300|Google+Sans

Because of this issue we are unable to debug our web apps. Most of our apps use Google Fonts and when proxy is enabled, the web app does not load.

<!-- gh-comment-id:569497240 --> @ghost commented on GitHub (Dec 29, 2019): Same issue is with Google Fonts. This URL is showing in Proxyman and it loads fine: https://fonts.googleapis.com/css?family=Roboto:300 However this URL does not load and also not showing in Proxyman https://fonts.googleapis.com/css?family=Roboto:300|Google+Sans Because of this issue we are unable to debug our web apps. Most of our apps use Google Fonts and when proxy is enabled, the web app does not load.
Author
Owner

@NghiaTranUIT commented on GitHub (Dec 29, 2019):

Thank for the hint. I suppose that the parser couldn't parse the |, which results the error.

I'm on it now

<!-- gh-comment-id:569508136 --> @NghiaTranUIT commented on GitHub (Dec 29, 2019): Thank for the hint. I suppose that the parser couldn't parse the `|`, which results the error. I'm on it now
Author
Owner

@NghiaTranUIT commented on GitHub (Dec 29, 2019):

@vimalkaul Thank you for the link: https://fonts.googleapis.com/css?family=Roboto:300|Google+Sans

I'm able to reproduce it in my machine. Perfect time to squash it 🐛 💥

<!-- gh-comment-id:569508805 --> @NghiaTranUIT commented on GitHub (Dec 29, 2019): @vimalkaul Thank you for the link: https://fonts.googleapis.com/css?family=Roboto:300|Google+Sans I'm able to reproduce it in my machine. Perfect time to squash it 🐛 💥
Author
Owner

@NghiaTranUIT commented on GitHub (Dec 29, 2019):

Hey @vimalkaul @Schaefers here is the BETA build: https://proxyman.s3.us-east-2.amazonaws.com/beta/Proxyman_1.11.0_Decode_percented_in_invalid_query.dmg

In this build, I would try to do percented-encoding for URLs, which contain invalid characters by following RFC 3986

I tested both Google API URLs and it works well.

Please let me know if it works. If not, please help me to print out the URL. I would investigate more 👍

Thank you guys 🎉

<!-- gh-comment-id:569516341 --> @NghiaTranUIT commented on GitHub (Dec 29, 2019): Hey @vimalkaul @Schaefers here is the BETA build: https://proxyman.s3.us-east-2.amazonaws.com/beta/Proxyman_1.11.0_Decode_percented_in_invalid_query.dmg In this build, I would try to do percented-encoding for URLs, which contain invalid characters by following [RFC 3986](https://www.rfc-editor.org/info/rfc3986) I tested both Google API URLs and it works well. Please let me know if it works. If not, please help me to print out the URL. I would investigate more 👍 Thank you guys 🎉
Author
Owner

@ghost commented on GitHub (Dec 30, 2019):

Hi @NghiaTranUIT checked build with URLs I had shared and with our web apps. This build is working fine. Thanks for looking into it and fixing the issue.

Thanks,
Vimal Kaul

<!-- gh-comment-id:569563257 --> @ghost commented on GitHub (Dec 30, 2019): Hi @NghiaTranUIT checked build with URLs I had shared and with our web apps. This build is working fine. Thanks for looking into it and fixing the issue. Thanks, Vimal Kaul
Author
Owner

@NghiaTranUIT commented on GitHub (Dec 30, 2019):

Cool. Glad to hear that. I will officially release the build in this week 👍

<!-- gh-comment-id:569563891 --> @NghiaTranUIT commented on GitHub (Dec 30, 2019): Cool. Glad to hear that. I will officially release the build in this week 👍
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/Proxyman#354
No description provided.