Different pool memory consumption in sim and bare metal

Bug (archived)
#1

I run the same firmware with minimal differences on PC (with sim drivers) and bare metal. It has “monitor”, printing lvgl memory use (on bare metal - wia retargeted stdout to USB CDC).

Problem is, memory use difers significantly, while it’s expected to be platform-independent. Here are real value with displayed lists of 6 and 16 elements (with custom drawers):

hw 6 items:   [Memory] total: 6144 bytes, free: 4088 bytes, use: 34% (2056)
hw 16 items:  [Memory] total: 6144 bytes, free: 3088 bytes, use: 50% (3056)

sim 6 items:  [Memory] total: 6144 bytes, free: 2832 bytes, use: 54% (3312)
sim 16 items: [Memory] total: 6144 bytes, free: 1072 bytes, use: 83% (5072)

On bare metal each item adds 100 bytes. On sim - 170 bytes. WTF?

Of cause, PC platform is x64, while stm32 - 32 bits. Can this cause so big difference and if yes, is it possible to workaround? I use sim intensive, and would like it reproduce lvgl pool use more close to real hardware.

#2
  • What are the differences between the platforms’ lv_conf.h files? Do you have some features enabled on the PC simulator that aren’t enabled on the real hardware? Are the color depths the same?
  • You should definitely compile the PC simulator as a 32-bit application (-m32) if you want the results to be similar, as there are a number of pointers stored in the heap.
#3

There are no difference in lvgl config. Since i’m not interested in pitfails, application has hal layer to keep differences as minimal as possible.

As far as i understand, lvgl has internal allocator implementation for it’s heap, and in theory it’s possible to force it work the same way everywhere. No?

I tried to use -m32 and got huge number of different errors. What i did:

sudo apt-get install gcc-multilib g++multilib
# sdl packages conflict, remove native
sudo apt-get remove libsdl2-dev
sudo apt-get install libsdl2-dev:i386

Now reached linking stage, and has errors about not found/incompatible libs, no idea how to fix. Also tried -Wl,-march_i386 in global build flags.

Could you help? I use Ubuntu 18.04LTS.

IMO, this issue needs more care in doc, because it’s importand to have “similar” memory behaviour while develop most of project on PC. Should i create issue in tracker or nobody except me interested in?

#4

The internal allocator works the same way, but pointers are twice the size on 64-bit computers (no way to change that), so it’s impossible to get identical behavior if one platform is 64-bit and another is 32-bit.

I haven’t heard of anyone else trying to do what you’re trying to do, so I have no idea if anyone requires this.

Personally, I would do the math and see if the memory consumption increased by a similar amount on both platforms. For 6 items, the ratio between hw and sim is 1:1.61. For 16 items, it’s 1:1.66. Those seem similar to me, so if you just divided all numbers you got from the simulator by 1.6, that should be good enough for your use case.

#5

It’s not a problem to just define 1.7x more space for sim build. But it’s not convenient to keep such things in mind. If it’s possible to make heap behave the same way as on stm32, that would be convenient.

I need that for PlatformIO, but working Makefile example would be enougth. Also, worth to mention in std2 docs about different heap requirements.

#6

I don’t think it requires mentioning in the documentation, mainly because it’s just an inherent feature of C on different platforms. A 64-bit program will always need more memory than a 32-bit one, regardless of whether LittlevGL is involved.

#7

To know this thing, user must keep in mind difference between platforms and know lvgl heap store pointers (because data structs can be designed to not store those, as binary font format).

I could avoid some optimizations, if know about heap in advance. I found this difference by accident at all, because decided to add USB CDC just for fun.

#8

On 64 bit systems, lvgl uses 8-byte alignments instead of 4 byte. I’ve added it intentionally once because there were issues due to the misaligned memories.

I agree to some extent because now you can’t simulate your project precisely. However, e.g. lv_style_t * style in lv_obj_t will also use 8 bytes instead of 4. So the size of every object will be different regardless of the memory allocator.

#9

If you plan to resolve that, i see 2 possibilities:

  • Styles can be refered via index id instead of direct pointers
  • Create manual how to install/build project with sdl2 in i386 mode.
#10

Or, simply document the relative memory size difference, which involves the least changes to the code and still lets you accurately measure how much memory will be used.

#11

For clarity, it should be really mentioned somewhere.

I was able to compile it like this:

  • Install 32 bit SDL: sudo apt-get install libsdl2-dev:i386
  • GCC Multilib: sudo apt-get install gcc-multilib (if not working install g++-multilib too )
  • Add -m32 to compiler and linker flags too

If you can confirm it works we can add a few lines of description about it in the README.

#12

See Different pool memory consumption in sim and bare metal. I could not compile, -m32 was not enough.

Could you provide working example with Makefile? I will try to port flags to pio then.

#13

I’ve only added -m32 to CFLAGS and LDFLAGS in Eclipse SDL project. (And installed 32 bit SDL as menitioned above)


CFLAGS ?=  -m32 -O3 -g0 -I$(LVGL_DIR)/ -Wall -Wshadow -Wundef -Wmaybe-uninitialized -Wmissing-prototypes -Wno-discarded-qualifiers -Wall -Wextra -Wno-unused-function -Wundef -Wno-error=strict-prototypes -Wpointer-arith -fno-strict-aliasing -Wno-error=cpp -Wuninitialized -Wmaybe-uninitialized -Wno-unused-parameter -Wno-missing-field-initializers -Wtype-limits -Wsizeof-pointer-memaccess -Wno-format-nonliteral -Wno-cast-qual -Wunreachable-code -Wno-switch-default -Wno-switch-enum -Wreturn-type -Wmultichar -Wformat-security -Wno-ignored-qualifiers -Wno-error=pedantic -Wno-sign-compare -Wno-error=missing-prototypes -Wdouble-promotion -Wclobbered -Wdeprecated -Wempty-body -Wtype-limits -Wshift-negative-value -Wstack-usage=1024 -Wno-unused-value -Wno-unused-parameter -Wno-missing-field-initializers -Wuninitialized -Wmaybe-uninitialized -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wtype-limits -Wsizeof-pointer-memaccess -Wno-format-nonliteral -Wpointer-arith -Wno-cast-qual -Wmissing-prototypes -Wunreachable-code -Wno-switch-default -Wswitch-enum -Wreturn-type -Wmultichar -Wno-discarded-qualifiers -Wformat-security -Wno-ignored-qualifiers -Wno-sign-compare
LDFLAGS ?=  -lSDL2 -lm -m32

The memory usage of a demo was changed to 10724 from 17864.

What error message do you get?

#14

After addimg -m32 to linker without -Wl,-m32, build works, but executable crashes

dbus[30505]: arguments to dbus_message_new_method_call() were incorrect, assertion "path != NULL" failed in file ../../../dbus/dbus-message.c line 1362.
This is normally a bug in some application using the D-Bus library.

  D-Bus not built with -rdynamic so unable to print a backtrace

How to reproduce (ubuntu 18.04):

# remove libsdl2-dev and deps if installed, libsdl2-dev:i386 conflicts with some deps
sudo apt-get install gcc-multilib g++-multilib libsdl2-dev:i386

Checkout master branch https://github.com/puzrin/dispenser, open in platformio (vscode), uncomment -m32 flag and build “emulator”.

Then run .pio/build/emulator/program from built-in terminal and see error.

Builder verbose output (linker only):

> Executing task in folder dispenser: platformio run --verbose --environment emulator <

Processing emulator (platform: native; extra_scripts: support/sdl2_build_extra.py; build_flags: -D LV_CONF_INCLUDE_SIMPLE, -I src, -DLV_MEM_SIZE=8000, -I hal/sdl2, -lSDL2, -m32, -D LV_LVGL_H_INCLUDE_SIMPLE, -D LV_DRV_NO_CONF, -D USE_MONITOR, -D MONITOR_ZOOM=2, -D USE_MOUSE, -D USE_MOUSEWHEEL, -D USE_KEYBOARD; src_build_flags: -Wall, -Wdouble-promotion, -Wextra, -Wclobbered, -Wdeprecated, -Wempty-body, -Wignored-qualifiers, -Wimplicit-fallthrough=3, -Wmissing-field-initializers, -Wsign-compare, -Wredundant-decls, -Wtype-limits, -Wuninitialized, -Wshift-negative-value, -Wunused-parameter, -Wunused-but-set-parameter; lib_deps: lvgl=https://github.com/littlevgl/lvgl/archive/7afd70a00564bea0150f2f96648b3cba7983d091.zip, Embedded Template Library@~15.5.0, lv_drivers@^6.0.2; src_filter: +<*>, +<../hal/sdl2>)
-------------------------------------------------------------------------------------------------
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 4 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <lvgl> 6.1.1 [https://github.com/littlevgl/lvgl/archive/7afd70a00564bea0150f2f96648b3cba7983d091.zip] (/home/vitaly/Coding/dispenser/.pio/libdeps/emulator/lvgl)
|-- <Embedded Template Library> 15.5.0 (/home/vitaly/Coding/dispenser/.pio/libdeps/emulator/Embedded Template Library_ID930)
|-- <lv_drivers> 6.1.1 (/home/vitaly/Coding/dispenser/.pio/libdeps/emulator/lv_drivers_ID6613)
|   |-- <lvgl> 6.1.1 [https://github.com/littlevgl/lvgl/archive/7afd70a00564bea0150f2f96648b3cba7983d091.zip] (/home/vitaly/Coding/dispenser/.pio/libdeps/emulator/lvgl)
Building in release mode
g++ -o .pio/build/emulator/program -m32 .pio/build/emulator/hal/sdl2/app_hal.o .pio/build/emulator/src/app.o .pio/build/emulator/src/doses.o .pio/build/emulator/src/fonts/my_font_icons_18.o .pio/build/emulator/src/fonts/my_font_roboto_12.o .pio/build/emulator/src/fonts/my_font_roboto_14.o .pio/build/emulator/src/fonts/my_font_roboto_num_18.o .pio/build/emulator/src/screen_dose.o .pio/build/emulator/src/screen_flow.o .pio/build/emulator/src/screen_settings.o -L.pio/build/emulator -Wl,--start-group .pio/build/emulator/lib070/liblvgl.a ".pio/build/emulator/lib26d/libEmbedded Template Library_ID930.a" .pio/build/emulator/lib4d5/liblv_drivers_ID6613.a -lSDL2 -Wl,--end-group
================================== [SUCCESS] Took 1.83 seconds ==================================

Environment      Status    Duration
---------------  --------  ------------
emulator         SUCCESS   00:00:01.831
hardware_stlink  IGNORED
hardware_usb     IGNORED
================================== 1 succeeded in 00:00:01.831 ==================================

Terminal will be reused by tasks, press any key to close it.

Src

Could anybody explain, what is wrong now?

#15

I’ve just found it in Google. Hope it helps:
https://forum.odroid.com/viewtopic.php?t=30652

#16

Does not help :(. Still the same error. Had anyone succeeded with exec of 32-bit build on linux (may be, not ubuntu)?

For future experiments, merged to master -m32 fixes:

Now you can quick-test both builds.

#17

https://bugs.launchpad.net/ubuntu/+source/libsdl2/+bug/1775067/comments/7

Found workaround. DBUS_FATAL_WARNINGS=0 .pio/build/emulator/program shows message in console, but works.

No idea, is this bug specific for ununtu 18.04 or not.

1 Like
#18

I’m using Linux Mint 19 which is based on Ubuntu 18.04 but didn’t see this issue.

Anyway glad to hear that you have found this.

#19

I’ve solved usability issue in pio:

  • No more boring manual run. Added “task” to exec sim build from vscode menu
  • Added extra option to fix crash transparently
  • Updated docs

Can not test in OSX, so don’t know if it worth copy all to pio demo.

1 Like
#20

Got nice advice for non-pio env, with manual run. In code, add before init:

setenv("DBUS_FATAL_WARNINGS","0",1);

Probably worth add to docs.