Skip to content

Move all PSTRs to own section, allow string dedup #6565

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Sep 30, 2019

Conversation

earlephilhower
Copy link
Collaborator

@earlephilhower earlephilhower commented Sep 28, 2019

GNU ld can deduplicate strings, and does so for most normal char *s
automatically. However, for PSTRs we were using a unique section per
string and the section was not flagged as containing dedupable
0-terminated strings.

Modify the PSTR macro to emit assembly, which lets us set the section
flags required (SM) for the variables, and use inline assembly to get
the asm-block defined address.

Should result in smaller compiled binaries if any strings are
duplicated.

GNU ld can deduplicate strings, and does so for most normal char *s
automatically.  However, for PSTRs we were using a unique section per
string *and* the section was not flagges as containing dedupable
0-terminated strings.

Modify the PSTR macro to emit assembly, which lets us set the section
flags required (SM) for the variables, and use inline assembly to get
the asm-block defined address.

Should result in smaller compiled binaries if any strings are
duplicated.
Add the "a" section flag to allow the linker to throw away unneeded
strings.  Without this flag, the linker will not discard unreferenced
strings and ROMs will increase in size, possibly dramatically.
@earlephilhower
Copy link
Collaborator Author

earlephilhower commented Sep 29, 2019

With the latest push ROM sizes look to have decreased between 32 bytes to 2.2KB, depending on the example, so it's saving some flash even in our own code.

@earlephilhower earlephilhower added this to the 2.6.0 milestone Sep 30, 2019
@d-a-v d-a-v merged commit 97c926e into esp8266:master Sep 30, 2019
@TD-er
Copy link
Contributor

TD-er commented Oct 1, 2019

And now for some real example :)

Some build I made last night and built again this morning with this PR included:

  • Before this merge: 821 KB (841,696 bytes)
  • With this merged: 800 KB (819,472 bytes)

So I would say it does indeed make some difference :)

@TD-er
Copy link
Contributor

TD-er commented Oct 1, 2019

Only problem is, my unit does constantly crash reboot.

 ets Jan  8 2013,rst cause:2, boot mode:(3,2)

load 0x4010f000, len 1384, room 16
tail 8
chksum 0x2d
csum 0x2d
v244dbd77
~ld
�U67 : 

INIT : Booting version:  (ESP82xx Core 2.6.0-dev stage, NONOS SDK 2.2.2-dev(38a443e), LWIP: 2.1.2 PUYA support)
67 : INIT : Free RAM:32560
68 : INIT : Warm boot #120 Last Task: Background Task - Restart Reason: Unknown
70 : FS   : Mounting...
95 : FS   : Mount successful, used 77810 bytes of 957314
111 : CRC  : program checksum       ...FAIL
134 : CRC  : SecuritySettings CRC   ...OK
160 : INIT : Free RAM:29416
161 : INIT : I2C
161 : INIT : SPI not enabled
312 : INFO : Plugins: 15 [Normal] (ESP82xx Core 2.6.0-dev stage, NONOS SDK 2.2.2-dev(38a443e), LWIP: 2.1.2 PUYA support)
314 : EVENT: System#Wa
Exception (28):
epc1=0x4000e140 epc2=0x00000000 epc3=0x00000000 excvaddr=0x00000000 depc=0x00000000

>>>stack>>>

ctx: cont
sp: 3fff20b0 end: 3fff2480 offset: 01a0
3fff2250:  00000000 0000000b 3fff22b0 402636f9
3fff2260:  3fff4f7c 3fff238f 3fff22c8 4026354d  
3fff2270:  0000000a 3fff238f 3fff236c 40264495
3fff2280:  00000020 3fff238f 3fff22b0 0000000b
3fff2290:  3fff2354 3fff238f 3fff2354 402636aa
3fff22a0:  3fff2354 3fff238f 3fff236c 4023ab9e
3fff22b0:  00000000 00000000 03ff236c 3fff4214
3fff22c0:  000b000f 00000002 3fff50b4 0011001f
3fff22d0:  3fff4f7c 402b49e0 3fff2354 40263928
3fff22e0:  3fff2390 00000001 3fff2430 4026328d  
3fff22f0:  00000011 3fff2354 00000000 3fff238d
3fff2300:  3fff2330 3fff23e0 3fff238e 4024412f
3fff2310:  3fff238e 3fff2388 3fff2384 3fff238d
3fff2320:  3fff238c 7fffffff 3fff047c 402639b1
3fff2330:  40270e88 00000000 000003e8 40263427
3fff2340:  3fff3384 3fff4174 00000000 00000000
3fff2350:  3fff27f8 00000000 00000000 80263ac0  
3fff2360:  3fff4fd4 3fff5014 3fff5014 3fff4f7c
3fff2370:  000e004f 00000005 00000000 00000000
3fff2380:  80000008 00000001 3fff23ec 00010000
3fff2390:  00000000 3fff23ec 00000001 0000000a
3fff23a0:  00000012 3fff2430 3fff2390 00000040
3fff23b0:  3fff2388 3fff2384 3fff238c 40263b22  
3fff23c0:  402b0031 3fff03e9 3fff0e0c 3fff23ec
3fff23d0:  3fff2430 00000001 3fff23ec 40233159  
3fff23e0:  00000000 00000000 07ff2430 656c7572
3fff23f0:  742e3173 8a007478 3fff2430 40263928
3fff2400:  0004c83b 3ffe8540 3fff2430 40263958
3fff2410:  402bd840 3ffe8540 00000002 3fff1348
3fff2420:  3fff03e9 3ffe8540 3fff0e0c 4025a55b
3fff2430:  3fff5f94 000b000f 0926005d 4026f6c4
3fff2440:  3fff406c 0072007f 7eefeffe feefeffe  
3fff2450:  feefeffe feefeffe feefeffe 3fff24c8
3fff2460:  3fffdad0 00000000 3fff2488 40264cf8
3fff2470:  feefeffe feefeffe 3ffe86d4 401003d1
<<<stack<<<

Edit:
Tested with last night's build again using the same code and settings to make sure it is just the core lib changes and then the node works fine again.
So there's definitely something wrong here with this change.

Edit2:
Just in case you may want to peek into the .elf file, I captured it here: ESPEasyBuild_Arduino_PR6565.rar
Included are a "custom" build and "normal" build.

  • custom: core 2.6.0 feature/stage (the one from the dump above)
  • normal: core 2.5.2 (different set of plugins, just built to revive the node again)

There are also some .bin builds in the 'custom' dir which I made yesterday.

@TD-er TD-er mentioned this pull request Oct 1, 2019
6 tasks
@earlephilhower
Copy link
Collaborator Author

@TD-er, please do a stack decode on the crash.

@TD-er
Copy link
Contributor

TD-er commented Oct 1, 2019

Is that already possible on PlatformIO builds?
I know you added that as a new issue a while ago, but not sure what the status is.

@earlephilhower
Copy link
Collaborator Author

Try https://github.com./me21/EspArduinoExceptionDecoder , I was in process of adding it to the repo to do CLI based decodes.

@TD-er
Copy link
Contributor

TD-er commented Oct 1, 2019

Has it been tested on Windows?

PS C:\GitHub\TD-er\ESPEasy> python .\tools\decoder.py -t C:\Users\gijs\.platformio\packages\toolchain-xtensa\ -e .\.pio\build\custom_ESP8266_4M1M\firmware.elf .\stacktrace.txt
ERROR: Parser not complete!

@TD-er
Copy link
Contributor

TD-er commented Oct 1, 2019

Ah bug found in decoder.

You have to move the >>>stack>>> line to right where the stack begins.

Like this:

Exception (28):
epc1=0x4000e140 epc2=0x00000000 epc3=0x00000000 excvaddr=0x00000000 depc=0x00000000

ctx: cont
sp: 3fff20b0 end: 3fff2480 offset: 01a0

>>>stack>>>
3fff2250:  00000000 0000000b 3fff22b0 402636f9 

Decoded result:

PS C:\GitHub\TD-er\ESPEasy> python .\tools\decoder.py -t C:\Users\gijs\.platformio\packages\toolchain-xtensa\ -e .\.pio\build\custom_ESP8266_4M1M\firmware.elf .\stacktrace.txt
Exception: 28 (LoadProhibited: A load referenced a page mapped with an attribute that does not permit loads)

epc1:     0x4000e140
epc2:     0x00000000
epc3:     0x00000000
excvaddr: 0x00000000
depc:     0x00000000

ctx: cont

sp:       0x3fff20b0
end:      0x3fff2480
offset:   0x000001a0

stack:
0x402636f9: String::operator=(char const*) at ??:?
0x4026354d: String::move(String&) at ??:?
0x40264495: String::substring(unsigned int, unsigned int) const at ??:?
0x402636aa: String::operator=(String&&) at ??:?
0x4023ab9e: parseCompleteNonCommentLine(String&, String&, String&, String&, bool&, bool&, bool&, bool*, bool*, unsigned char&, unsigned char&) at ??:?
0x402b49e0: chip_v6_unset_chanfreq at ??:?
0x40263928: String::operator=(__FlashStringHelper const*) at ??:?
0x4026328d: String::invalidate() at ??:?
0x4024412f: rulesProcessingFile(String const&, String&) at ??:?
0x402639b1: String::concat(char const*, unsigned int) at ??:?
0x40270e88: spiffs_cache_page_get_by_fd at ??:?
0x40263427: String::reserve(unsigned int) at ??:?
0x40263b22: String::concat(int) at ??:?
0x402b0031: sleep_reset_analog_rtcreg_8266 at ??:?
0x40233159: rulesProcessing(String&) at ??:?
0x40263928: String::operator=(__FlashStringHelper const*) at ??:?
0x40263958: String::String(__FlashStringHelper const*) at ??:?
0x402bd840: chip_v6_unset_chanfreq at ??:?
0x4025a55b: setup at ??:?
0x4026f6c4: std::_Function_handler<void (WiFiEventSoftAPModeStationDisconnected const&), void (*)(WiFiEventSoftAPModeStationDisconnected const&)>::_M_invoke(std::_Any_data const&, WiFiEventSoftAPModeStationDisconnected const&) at ??:?
0x40264cf8: loop_wrapper() at core_esp8266_main.cpp:?
0x401003d1: cont_wrapper at ??:?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants