Micropython v1.19.1 and LVGL v8.3

Hi everyone. A quick update:

lv_micropython was aligned to Micropython v1.19.1 and LVGL v8.3.

lv_micropython master branch will now start tracking LVGL v9 development, and may include breaking changes.

For Micropython + LVGL v8, please use the lv_micropython release/v8 branch.

Please let us know if there are any questions or issues!

1 Like

Hello Armigon,

i thought maybe this would solve the compilation problem with esp32s2 / c2 / c3 / h2 but it´s not the case. The GENERIC_SPIRAM target builds fine, but not the other variants of ESP32.
Build fails with:

fatal error: esp32/clk.h: No such file or directory

it seems the variable CONFIG_IDF_TARGET_* is not set when lv_micropython/lib/lv_bindings/driver/esp32/espidf.h is preprocessed.

I use esp-idf 4.3.3 / Micropython v1.19.1 / and LVGL v8.3
but the problem is same with other esp-idf releases (although i did not test with older esp-idf 3)

The boards GENERIC_SPIRAM and GENERIC_D2WD build fine.

Currently the ESP32 port of lv_micropython “officially” supports only the GENERIC / GENERIC_SPI and derivatives (such as the M5CORE2).

I can only support the GENERIC / GENERIC_SPI boards.
Support for other boards need to come as community contributions.

Feel free to open a PR and I’ll do my best to review it and help where I can.

See also:

Thank you for the feedback, I totally get your point and if I was able to quickly fix it, I would have immediately, and would have issued a PR.
For the moment I´m trying to understand how everything is tied together, and where the problem is.
The link you gave me is already putting me on the right track, thank you for that !

Hmmm, this doesn’t look right:

MicroPython v1.18-1492-g5e76417ef on 2022-07-19; Raspberry Pi Pico with RP2040
Type "help()" for more information.
>>> import os
>>> os.uname()
(sysname='rp2', nodename='rp2', release='1.19.1', version='v1.18-1492-g5e76417ef on 2022-07-19 (GNU 10.3.1 MinSizeRel)', machine='Raspberry Pi Pico with RP2040')

A straight build of 1.19.1 using same host shows:

MicroPython v1.19.1 on 2022-07-19; Raspberry Pi Pico with RP2040
Type "help()" for more information.
>>> import os
>>> os.uname()
(sysname='rp2', nodename='rp2', release='1.19.1', version='v1.19.1 on 2022-07-19 (GNU 10.3.1 MinSizeRel)', machine='Raspberry Pi Pico with RP2040')

Note the version strings. Also, help(‘modules’) doesn’t work for me.

Right. I always forget to push the Micropython tags into lv_micropython.
Please pull lv_micropython again and rebuild, it should be ok now.

I’m not sure about this one.
help(‘modules’) works fine for me on unix and esp32 ports.
Maybe it’s an rp2 specific issue? Does it work on upstream Micropython?

Yep. Sorted. Thanks.

Yes, it works normally on 1.19.1.

I’ve narrowed down the difference in behaviour to py/builtinhelp.c line 126:

if (obj == MP_OBJ_NEW_QSTR(MP_QSTR_modules))

These values match in 1.19.1 mpy and do not in lv_mpy.

I’m not in a position to try a different port, unfortunately.

I notice makeqstrdata.py has significant changes upstream, with the result that qstrdefs.generated.h now looks different (the values for ‘modules’ are logically the same, but the order has changed.)

@Campbell see this related PR:

It’s not merged into Micropython (yet), but I’ve already merged it into lv_micropython because of the very significant improvement it provides (due to the size of the LVGL API and the very large number of strings).

You can try reverting this patch and see if this problem still happens (b912b45)

Did you experience any other anomalies? Did you try running some non-trivial LVGL code?

Yes, I recall you mentioned that enhancement in the past.

git revert b912b45 -m 1

Restores expected behaviour.

I can see how that QSTR handling change might cause the regression, However, I not understanding why the esp32 and unix ports are unaffected.

I have not yet tried an application, I usually like to make sure everything is working before building anything on top of it. I’ll go back to release/v8, press ahead and see if I see any other regressions.

Very interesting!
Perhaps, by some rare chance, only the rp2 port experience some hash collision that is not handled correctly? Just a wild guess.

Could you please open an issue on lv_binding_micropython describing this, how to reproduce it and all we know about it so far?
That would help tracking this in the future, because at the moment I’m a bit too busy to try reproducing and exploring it on my side.

@Campbell We identified a bug in py: Faster qstr search.
It would be interesting to know if it also fixes this issue you are experiencing with rp2.

I cloned and built the latest repository.
As a result, help(‘modules’), os.listdir(). work on RP2040 ports.

>>>
>>> import os
>>> os.uname()
(sysname='rp2', nodename='rp2', release='1.19.1', version='v1.19.1-635-g4111f1b5f-dirty on 2022-08-06 (GNU 8.3.1 MinSizeRel)', machine='Raspberry Pi Pico with RP2040')
>>>
>>> help('modules')
__main__          gc                uasyncio/funcs    umachine
_boot             lodepng           uasyncio/lock     uos
_boot_fat         lvgl              uasyncio/stream   urandom
_onewire          math              ubinascii         ure
_rp2              micropython       ucollections      uselect
_thread           neopixel          ucryptolib        ustruct
_uasyncio         onewire           uctypes           usys
builtins          rp2               uerrno            utime
cmath             uarray            uhashlib          uzlib
dht               uasyncio/__init__ uheapq
ds18x20           uasyncio/core     uio
framebuf          uasyncio/event    ujson
Plus any modules on the filesystem
>>> os.listdir('.')
['advanced_demo.py', 'display_driver.py', 'display_driver_utils.py', 'fs_driver.py', 'generic-st77xx-with-xpt2046.py', 'imagetools.py', 'lv_colors.py', 'lv_utils.py', 'rp2_dma-test.py', 'rp2_dma.py', 'st77xx.py', 'test.py', 'tpcal.py', 'xpt2046.py']
>>>
>>>
>>>

Yep confirmed. Thank you Amir for keeping this rp2 issue in mind.

It looks like the rp2 port was going to be more susceptible to that bug because it defines

MICROPY_QSTR_BYTES_IN_HASH (1)

Which increases the probability of hash collisions.

How is that value determined? It seems to me that the necessarily large set of QSTRs LVGL requires might change the performance tradeoff of the hash variable size ?

I’m a little surprised that RP2 defines MICROPY_QSTR_BYTES_IN_HASH (1).
I would expect that only on the lower-end MCUs, the 16/8 bit ones etc.
Smaller hash size would definitely cause many hash collisions, so this optimization would be slightly less effective than 16 bit hash. On lv_micropython we have around ~3500 qstrs so with an ideal hash function we would get 256 buckets of ~14 qstr with the same hash which will be scanned sequentially. It’s still much better than scanning ~3500 qstrs sequently…

You can try profiling this, by measuring qstr performance when building with MICROPY_QSTR_BYTES_IN_HASH set to 1 or to 2. My guess would be that you’d see some improvement when setting it to 2, but it won’t be huge.

Confirming:

MICROPY_QSTR_BYTES_IN_HASH (1)

is in the rp2 port upstream too.

I’ll experiment with a larger hash and report back.

I take it you no longer need me to raise an issue on lv_binding_micropython? I mean the fix is now committed, right? (sorry, I was somewhat slow in raising this due to a case of covid)

Finally, should this fix not also be applied to the release/v8 branch?

Correct.
Feel well!

Correct, thanks for pointing this out.
Done.