[GH-ISSUE #518] Memory optimization #433

Open
opened 2026-02-28 01:25:16 +03:00 by kerem · 14 comments
Owner

Originally created by @tablatronix on GitHub (Feb 14, 2018).
Original GitHub issue: https://github.com/tzapu/WiFiManager/issues/518

optimize memory usage, namely RAM

  • store strings in PROGMEM
  • move/declare strings, remove duplicates
  • minimize use of Strings
  • profile memory for leaks
  • optimize html outputting, is it possible to flush to reduce page strings size, does it matter ?
Originally created by @tablatronix on GitHub (Feb 14, 2018). Original GitHub issue: https://github.com/tzapu/WiFiManager/issues/518 optimize memory usage, namely RAM - [x] store strings in PROGMEM - [x] move/declare strings, remove duplicates - [x] minimize use of `Strings` - [ ] profile memory for leaks - [ ] optimize html outputting, is it possible to flush to reduce `page` strings size, does it matter ?
Author
Owner

@tablatronix commented on GitHub (Feb 14, 2018):

Made sure all debug and strings were wrapped in F(), eventually all debug strings will be moved to strings.h with precompiler flags to remove them entirely.

ESP DEBUGGING IS ON

development

BEFORE
Checking program size
text	   data	    bss	    dec	    hex	filename
341992	   9648	  30192	 381832	  5d388	.pioenvs\nodemcuv2_dev\firmware.elf
Uploading 355792 bytes from .pioenvs\nodemcuv2_dev\firmware.bin to flash at 0x00000000

freeheap 37760

data  : 0x3ffe8000 ~ 0x3ffe852c, len: 1324
rodata: 0x3ffe8530 ~ 0x3ffea5b4, len: 8324
bss   : 0x3ffea5b8 ~ 0x3fff1ba8, len: 30192
heap  : 0x3fff1ba8 ~ 0x3fffc000, len: 42072

AFTER

Checking program size
text	   data	    bss	    dec	    hex	filename
345448	   8776	  30200	 384424	  5dda8	.pioenvs\nodemcuv2_dev\firmware.elf
Uploading 358368 bytes from .pioenvs\nodemcuv2_dev\firmware.bin to flash at 0x00000000
+2,576

data  : 0x3ffe8000 ~ 0x3ffe852c, len: 1324
rodata: 0x3ffe8530 ~ 0x3ffea24c, len: 7452 -872
bss   : 0x3ffea250 ~ 0x3fff1848, len: 30200 +8
heap  : 0x3fff1848 ~ 0x3fffc000, len: 42936 +864

free heap 38624 +864
<!-- gh-comment-id:365637487 --> @tablatronix commented on GitHub (Feb 14, 2018): Made sure all debug and strings were wrapped in F(), eventually all debug strings will be moved to strings.h with precompiler flags to remove them entirely. ESP DEBUGGING IS ON ``` development BEFORE Checking program size text data bss dec hex filename 341992 9648 30192 381832 5d388 .pioenvs\nodemcuv2_dev\firmware.elf Uploading 355792 bytes from .pioenvs\nodemcuv2_dev\firmware.bin to flash at 0x00000000 freeheap 37760 data : 0x3ffe8000 ~ 0x3ffe852c, len: 1324 rodata: 0x3ffe8530 ~ 0x3ffea5b4, len: 8324 bss : 0x3ffea5b8 ~ 0x3fff1ba8, len: 30192 heap : 0x3fff1ba8 ~ 0x3fffc000, len: 42072 AFTER Checking program size text data bss dec hex filename 345448 8776 30200 384424 5dda8 .pioenvs\nodemcuv2_dev\firmware.elf Uploading 358368 bytes from .pioenvs\nodemcuv2_dev\firmware.bin to flash at 0x00000000 +2,576 data : 0x3ffe8000 ~ 0x3ffe852c, len: 1324 rodata: 0x3ffe8530 ~ 0x3ffea24c, len: 7452 -872 bss : 0x3ffea250 ~ 0x3fff1848, len: 30200 +8 heap : 0x3fff1848 ~ 0x3fffc000, len: 42936 +864 free heap 38624 +864 ```
Author
Owner

@tablatronix commented on GitHub (Feb 14, 2018):

data : 0x3ffe8000 ~ 0x3ffe852c, len: 1324
rodata: 0x3ffe8530 ~ 0x3ffea23c, len: 7436
bss : 0x3ffea240 ~ 0x3fff1838, len: 30200
heap : 0x3fff1838 ~ 0x3fffc000, len: 42952
*WM: getCoreVersion(): 00000000
*WM: system_get_sdk_version(): 2.1.0(deb1901)
*WM: system_get_boot_version(): 31
*WM: getFreeHeap(): 38640

<!-- gh-comment-id:365719877 --> @tablatronix commented on GitHub (Feb 14, 2018): data : 0x3ffe8000 ~ 0x3ffe852c, len: 1324 rodata: 0x3ffe8530 ~ 0x3ffea23c, len: 7436 bss : 0x3ffea240 ~ 0x3fff1838, len: 30200 heap : 0x3fff1838 ~ 0x3fffc000, len: 42952 *WM: getCoreVersion(): 00000000 *WM: system_get_sdk_version(): 2.1.0(deb1901) *WM: system_get_boot_version(): 31 *WM: getFreeHeap(): 38640
Author
Owner

@horendus commented on GitHub (Feb 15, 2018):

Fantastic :)

<!-- gh-comment-id:365811245 --> @horendus commented on GitHub (Feb 15, 2018): Fantastic :)
Author
Owner

@tablatronix commented on GitHub (Nov 12, 2019):

Investigate fragmentation, suspect is page html and params replace strings, user params

<!-- gh-comment-id:553161095 --> @tablatronix commented on GitHub (Nov 12, 2019): Investigate fragmentation, suspect is page html and params replace strings, user params
Author
Owner

@nikeee commented on GitHub (Aug 24, 2020):

Using the development branch on my ESP32 on Platformio:
With WiFiManager:

RAM:   [===       ]  25.7% (used 84296 bytes from 327680 bytes)
Flash: [======    ]  60.7% (used 1113774 bytes from 1835008 bytes)

And without WiFiManager:

RAM:   [==        ]  19.9% (used 65348 bytes from 327680 bytes)
Flash: [====      ]  39.7% (used 728388 bytes from 1835008 bytes)

(compiled in release mode with optimizations)

Which is 385386 (~385kB) difference in flash.
I remember that the master branch was more lightweight when I used it for an ESP8266.

Is there anything I can do on my side to reduce memory usage?

<!-- gh-comment-id:679263012 --> @nikeee commented on GitHub (Aug 24, 2020): Using the development branch on my ESP32 on Platformio: With WiFiManager: ``` RAM: [=== ] 25.7% (used 84296 bytes from 327680 bytes) Flash: [====== ] 60.7% (used 1113774 bytes from 1835008 bytes) ``` And without WiFiManager: ``` RAM: [== ] 19.9% (used 65348 bytes from 327680 bytes) Flash: [==== ] 39.7% (used 728388 bytes from 1835008 bytes) ``` (compiled in release mode with optimizations) Which is 385386 (~385kB) difference in flash. I remember that the master branch was more lightweight when I used it for an ESP8266. Is there anything I can do on my side to reduce memory usage?
Author
Owner

@tablatronix commented on GitHub (Aug 25, 2020):

hmm, well all debugging is left in, they are not conditional, so removing all debug statements might help. There might be some cruft in dev since its still alpha that can be removed.. also there is a lot of info page stuff that I intend to make optional somehow, compiler directives for now probably.

I would suspect the css additions.
Remove the images, and go back to percentage tokens, no graph

I can play with it, but it might be a while before I have time atm.

Might want to create an issue just for this optimizations so we can select them out easily.
for now defines work fine and we can branch or pio config low mem options.

There are also some very useful wifi helpers, that could be removed, like textual status() strings etc
all the help page docs..

<!-- gh-comment-id:679454615 --> @tablatronix commented on GitHub (Aug 25, 2020): hmm, well all debugging is left in, they are not conditional, so removing all debug statements might help. There might be some cruft in dev since its still alpha that can be removed.. also there is a lot of info page stuff that I intend to make optional somehow, compiler directives for now probably. I would suspect the css additions. Remove the images, and go back to percentage tokens, no graph I can play with it, but it might be a while before I have time atm. Might want to create an issue just for this optimizations so we can select them out easily. for now defines work fine and we can branch or pio config low mem options. There are also some very useful wifi helpers, that could be removed, like textual status() strings etc all the help page docs..
Author
Owner

@nikeee commented on GitHub (Aug 26, 2020):

That seems like a good idea.

What I was also thinking of is offering the user the option to supply his own HTML and CSS (not just the ability to append CSS, but all if the HTML/CSS). For my current use-case that would be easier than overwriting the existing CSS while being more compact.

<!-- gh-comment-id:680826227 --> @nikeee commented on GitHub (Aug 26, 2020): That seems like a good idea. What I was also thinking of is offering the user the option to supply his own HTML and CSS (not just the ability to append CSS, but all if the HTML/CSS). For my current use-case that would be easier than overwriting the existing CSS while being more compact.
Author
Owner

@tablatronix commented on GitHub (Aug 26, 2020):

There IS a define check for the strings, so you can do that at compiler level if you wanted to and replace them elsewhere, problem is its still flash, in order to switch to littlefs/spiffs strings it would require code changes.

<!-- gh-comment-id:680882181 --> @tablatronix commented on GitHub (Aug 26, 2020): There IS a define check for the strings, so you can do that at compiler level if you wanted to and replace them elsewhere, problem is its still flash, in order to switch to littlefs/spiffs strings it would require code changes.
Author
Owner

@nikeee commented on GitHub (Aug 26, 2020):

Do you mean this define: https://github.com/tzapu/WiFiManager/blob/development/strings_en.h#L16?

Despite the comment, does defining an own set of strings work by something like this?

#define WIFI_MANAGER_OVERRIDE_STRINGS
const char HTTP_HEAD_START[] PROGMEM = // ...
// ...

#include <WiFiManager.h>

I feel like I'd have more customization options if this function would be virtual, so the WiFiManager could be inherited and this function could be implemented by the library user:
github.com/tzapu/WiFiManager@31be25d54b/WiFiManager.cpp (L945-L963)

This might not solve the RAM/Flash issue, though.

<!-- gh-comment-id:681134244 --> @nikeee commented on GitHub (Aug 26, 2020): Do you mean this define: https://github.com/tzapu/WiFiManager/blob/development/strings_en.h#L16? Despite the comment, does defining an own set of strings work by something like this? ```c #define WIFI_MANAGER_OVERRIDE_STRINGS const char HTTP_HEAD_START[] PROGMEM = // ... // ... #include <WiFiManager.h> ``` I feel like I'd have more customization options if this function would be `virtual`, so the WiFiManager could be inherited and this function could be implemented by the library user: https://github.com/tzapu/WiFiManager/blob/31be25d54bfa814b7a51ba8fbc8b6019682f6ac8/WiFiManager.cpp#L945-L963 This might not solve the RAM/Flash issue, though.
Author
Owner

@tablatronix commented on GitHub (Aug 27, 2020):

Yeah, this was the first step to abstract and organize the code so it can be refactored better. That func was one of the latest additions with that in mind.

<!-- gh-comment-id:681189243 --> @tablatronix commented on GitHub (Aug 27, 2020): Yeah, this was the first step to abstract and organize the code so it can be refactored better. That func was one of the latest additions with that in mind.
Author
Owner

@dalbert2 commented on GitHub (Apr 4, 2023):

For ESP8266 users, IRAM is a critical resource; the SSL library requires a lot of free heap and anything that reduces it is a problem. The WiFiManager is amazing, but it consumes roughly 2.3K of IRAM even when not instantiated.

Are there ways to reduce WiFiManager IRAM usage when not using (or even instantiating) the WiFiManager?

<!-- gh-comment-id:1496694926 --> @dalbert2 commented on GitHub (Apr 4, 2023): For ESP8266 users, IRAM is a critical resource; the SSL library requires a lot of free heap and anything that reduces it is a problem. The WiFiManager is amazing, but it consumes roughly 2.3K of IRAM _even when not instantiated_. Are there ways to reduce WiFiManager IRAM usage when not using (or even instantiating) the WiFiManager?
Author
Owner

@tablatronix commented on GitHub (Apr 5, 2023):

I am assuming this is all the strings, specifically the html, try removing the HTTP_STYLE css like icons and stuff and see what you find, i made a minimal version to test but it was a long time ago

https://github.com/tzapu/WiFiManager/blob/master/wm_strings_en.h#L90

<!-- gh-comment-id:1496749670 --> @tablatronix commented on GitHub (Apr 5, 2023): I am assuming this is all the strings, specifically the html, try removing the HTTP_STYLE css like icons and stuff and see what you find, i made a minimal version to test but it was a long time ago https://github.com/tzapu/WiFiManager/blob/master/wm_strings_en.h#L90
Author
Owner

@dalbert2 commented on GitHub (Apr 14, 2023):

I submitted a separate issue #1587 addressing at least two problems:

  1. string arrays in wm_consts_en.h aren't properly placed in PROGMEM
    I provided examples of how to do it properly and can submit a fix if desired, but the more serious issue is:
  2. all of the strings are declared in header files (which are themselves nested)
    so any code that includes WiFiManager.h also creates instances of those strings.
    Why aren't those strings declared in a compilation unit and declared extern in header files (if needed)?
<!-- gh-comment-id:1509402063 --> @dalbert2 commented on GitHub (Apr 14, 2023): I submitted a separate issue #1587 addressing at least two problems: 1. string arrays in wm_consts_en.h aren't properly placed in PROGMEM I provided examples of how to do it properly and can submit a fix if desired, but the more serious issue is: 2. all of the strings are declared in header files (which are themselves nested) so any code that includes WiFiManager.h also creates instances of those strings. Why aren't those strings declared in a compilation unit and declared extern in header files (if needed)?
Author
Owner

@tablatronix commented on GitHub (Apr 15, 2023):

.2 but there are include guards..?

<!-- gh-comment-id:1509468735 --> @tablatronix commented on GitHub (Apr 15, 2023): .2 but there are include guards..?
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/WiFiManager#433
No description provided.