MicroPython Display Drivers part 2

I’m starting a new thread because this thread is so long and we lost quite a few watchers!

I ported the SPI bus driver from lcd_bus written in C to MicroPython as lcd_bus.py. This version is platform agnostic and has been tested on RP2040 and STM32 outside of LVGL! The performance is very good. This means we can use @kdschlosser’s lv_binding_micropython with platforms other than ESP32!!! We still need platform specific bus drivers written in C, BUT, they aren’t required. This opens up the new binding and new drivers to all platforms the platform will compile on, at least for SPI devices. I’m going to tackle the i80 bus next. I probably won’t try to create a MicroPython version of the RGB bus driver.

Check out mpdisplay on GitHub for more details.

@kdschlosser What is the status of the binding compiling for other platforms? I get errors when I try to compile for RP2040 using make.py, and haven’t dug into how to do it with standard make yet.

@kisvegabor Please take a look at the link and let me know what you think.

I know it compiles for unix. I have not yet tested it with other platforms. I am still dinking about with the display I have so I can test the framework stuff better. Once I have that nailed down then I will move into getting the other platforms to compile properly. Once that is done then it is up to @kisvegabor to decide on what he would like to do from there.

I have also been working on an addition to LVGL that will spit out the entire API in a JSON file. This will really simplify things for binding developers as that can be used to generate the code for a binding. I am working on bringing it together with the documentation build system to the documentation gets pulled into the JSON file as well this way a binding developer will be able to easily generate documentation and stub files (if needed). Trying to make it easier for binding developers to maintain their code or to create new bindings.

I have been messing about with adding the JSON functionality to the gen_mpy script and it has be a pretty large headache to do. I showed @kisvegabor what I had so far and he asked if something like that could be added to LVGL itself. It would be a better thing to do to add it to LVGL so it would be accessible to ll of the binding developers. That is what I have been working on. Just trying to nail down a nice API for it that makes sense.

That’s an awesome project. I’m looking forward to the documentation that can be generated from it!

@kdschlosser, the firmware is being compiled with an error. After the last commit, the directory driver/esp32/heap_caps/include disappeared

I was just told this by someone else. I will fix it shortly.

@kdschlosser Do you know if anyone has been able to use RGBBus yet? I’m trying on an Adafruit Qualia and it reboots when calling init(). Here’s the cut down code that can duplicate the issue:

from lcd_bus import RGBBus

display_bus = RGBBus(
    hsync=41,
    vsync=42,
    de=2,
    disp=-1,
    pclk=1,
    data0=11,  #r0
    data1=10,  #r1
    data2=9,  #r2
    data3=46,  #r3
    data4=3,  #r4
    data5=48,  #g0
    data6=47,  #g1
    data7=21,  #g2
    data8=14,  #g3
    data9=13,  #g4
    data10=12,  #g5
    data11=40,  #b0
    data12=39,  #b1
    data13=38,  #b2
    data14=0,  #b3
    data15=45,  #b4
    freq=16000000,
    hsync_pulse_width=2,
    hsync_front_porch=46,
    hsync_back_porch=44,
    hsync_idle_low=False,
    vsync_pulse_width=2,
    vsync_front_porch=16,
    vsync_back_porch=18,
    vsync_idle_low=False,
    pclk_active_neg=False,
    pclk_idle_high=False,
    de_idle_high=False,
    num_fbs=2,
    bb_size_px=0,
    bb_inval_cache=False,
    fb_in_psram=False,
    no_fb=False,
    disp_active_low=False,
    refresh_on_demand=False,
)

display_bus.init(720, 720, 16, buffer_size=1036800, rgb565_byte_swap=False)

I tried different values for buffer_size to see if that made a difference, but it didn’t.

I got all the settings from here.

Your memory use is going to be massive having a display size of 720 x 720 x 16bpp. You have the number of frame buffers set to 2 so you are going to need to have 2,073,600 bytes of free contiguous RAM. The rgb565 doesn’t do anything and neither does the buffer_size. those are only there to keep the API identical across all of the drivers.

Now I had forgotten that the callback for the trans done is set up differently for the RGBBus and I never added the new ISR code to that callback. I will do that now.

Well scratch that. I did add it to that callback.

from lcd_bus import RGBBus

display_bus = RGBBus(
    hsync=41,
    vsync=42,
    de=2,
    disp=-1,
    pclk=1,
    data0=11,  #r0
    data1=10,  #r1
    data2=9,  #r2
    data3=46,  #r3
    data4=3,  #r4
    data5=48,  #g0
    data6=47,  #g1
    data7=21,  #g2
    data8=14,  #g3
    data9=13,  #g4
    data10=12,  #g5
    data11=40,  #b0
    data12=39,  #b1
    data13=38,  #b2
    data14=0,  #b3
    data15=45,  #b4
    freq=25000000,
    hsync_pulse_width=2,
    hsync_front_porch=46,
    hsync_back_porch=44,
    hsync_idle_low=False,
    vsync_pulse_width=2,
    vsync_front_porch=16,
    vsync_back_porch=18,
    vsync_idle_low=False,
    pclk_active_neg=False,
    pclk_idle_high=False,
    de_idle_high=False,
    num_fbs=1,
    bb_size_px=0,
    bb_inval_cache=False,
    fb_in_psram=True,
    no_fb=False,
    disp_active_low=False,
    refresh_on_demand=False,
)

disp = lv.display_create(720, 720)
disp.set_color_format(lv.COLOR_FORMAT.RGB565)

display_bus.init(720, 720, 16, buffer_size=720 * 720 * 2, rgb565_byte_swap=False)

buf_1 = display_bus.get_frame_buffer(1)
disp.set_draw_buffers(buf1, None, len(buf1), lv.DISPLAY_RENDER_MODE.FULL)

def tx_done():
    disp.flush_ready()

display_bus.register_callback(tx_done)

def flush(_, area, color_p):
    x1 = area.x1
    x2 = area.x2

    y1 = area.y1
    y2 = area.y2
    size = (x2 - x1 + 1) * (y2 - y1 + 1) * 2
    data_view = color_p.__dereference__(size)
    data_bus.tx_color(0x00, data_view, x1, y1, x2, y2)


data_bus.register_callback(flush)

The reason why it was failing is you were trying to allocate 2mb of memory into the SRAM which is only 350K or so. That’s not going to work. You should be able to get it running using the example above. Or something close to it anyhow. Get it running using a single buffer first and then try 2 frame buffers. The buffers are going to have to be created in psram as there is simply not going to be enough space in SRAM.

I m setting the pixel clock to 25Mhz because that is what should be achievable. The SPIFLASH and the SPIRAM both share the same SPI bus. That bus is going to be moving along at 80mhz. but since you have to share it between the flash and the ram you could end up with problems if you don’t trim back the speed in which the data is going to be read from the memory especially if you end up collecting something from the flash memory while a dma transfer is going on.

It’s all simple math and it is going to depend on the number of lanes you have attached to the display and also the number of lanes being used for the spiram. If you have octal spiram there are 8 lanes available. if you have quad flash only 4 of those 8 lanes are shared. if you have octal flash than all 8 are shared. So account for that when setting the pixel clock frequency. It has to be balanced against the number of lanes being used for the display. You don’t want to be trying to collect bytes from memory faster than it can be done. that will lead to tearing of the display data. so set the clock accordingly. The 25Mhz is a good number for a 16 lane RGB connection using octal spiram and quad flash. If you have 24 lanes connected then you will want to lower that pixel clock to probably 20mhz maybe lower.

so if you have 24 lanes of display requesting 24 bytes of memory all at the same time from an octal SPI bus you will not be able to have the pixel clock set to 80mhz. you will overrun the ability to collect the bytes from memory.

You probably already know the above information and it is more for the folks that don’t know. I know you had yours set at a safe 16mhz and you can leave it there and once you get it running dial it up until you get pixel anomalies.

I didn’t add the byte swapping for RGB565 to the RGB bus because I was not sure if the byte swapping needed to be done for the RGB displays I had thought it was only an issue on the i8080 and SPI displays. Lemme know if it needs to be added to the RGB bus as well.

OK so it is compiling for Unix, RP2 and ESP32. have to check out STM32 and Webassembly. I know those are the other 2 that get used more often than not. I will also poke about the rest of the ports but I do not think they are going to be too much of an issue to get running.

EDIT

STM32 is now compiling.

EDIT
NRF compiles but the board I tries gets a QSTR overflow. so the firmware won’t fit onto the board.

EDIT
Renesas-ra is now compiling

Looking to be in really good shape with this. I am pretty happy the boards are compiling without too much issue.

In testing RGBBus, I’m no longer getting reboots, but display_bus.get_frame_buffer(1) returns None

from lcd_bus import RGBBus

display_bus = RGBBus(
    hsync=41,
    vsync=42,
    de=2,
    disp=-1,
    pclk=1,
    data0=11,  #r0
    data1=10,  #r1
    data2=9,  #r2
    data3=46,  #r3
    data4=3,  #r4
    data5=48,  #g0
    data6=47,  #g1
    data7=21,  #g2
    data8=14,  #g3
    data9=13,  #g4
    data10=12,  #g5
    data11=40,  #b0
    data12=39,  #b1
    data13=38,  #b2
    data14=0,  #b3
    data15=45,  #b4
    freq=25000000,
    hsync_pulse_width=2,
    hsync_front_porch=46,
    hsync_back_porch=44,
    hsync_idle_low=False,
    vsync_pulse_width=2,
    vsync_front_porch=16,
    vsync_back_porch=18,
    vsync_idle_low=False,
    pclk_active_neg=False,
    pclk_idle_high=False,
    de_idle_high=False,
    num_fbs=1,
    bb_size_px=0,
    bb_inval_cache=False,
    fb_in_psram=True,
    no_fb=False,
    disp_active_low=False,
    refresh_on_demand=False,
)

display_bus.init(720, 720, 16, buffer_size=720 * 720 * 2, rgb565_byte_swap=False)
buf_1 = display_bus.get_frame_buffer(1)

print(f"buf_1 is None:  {buf_1 is None}")

prints buf_1 is None: True

Compiling for RP2 with:

python make.py rp2  BOARD=ADAFRUIT_QTPY_RP2040

I get the following:

make -C ports/rp2 LV_PORT=rp2 USER_C_MODULES==/home/brad/gh/lv_binding_micropython/make_build BOARD=ADAFRUIT_QTPY_RP2040
make: Entering directory '/home/brad/gh/lv_binding_micropython/micropython/ports/rp2'
[ -e build-ADAFRUIT_QTPY_RP2040/Makefile ] || cmake -S . -B build-ADAFRUIT_QTPY_RP2040 -DPICO_BUILD_DOCS=0 -DMICROPY_BOARD=ADAFRUIT_QTPY_RP2040 -DMICROPY_BOARD_DIR=/home/brad/gh/lv_binding_micropython/micropython/ports/rp2/boards/ADAFRUIT_QTPY_RP2040 -DUSER_C_MODULES==/home/brad/gh/lv_binding_micropython/make_build
-- Configuring incomplete, errors occurred!
See also "/home/brad/gh/lv_binding_micropython/micropython/ports/rp2/build-ADAFRUIT_QTPY_RP2040/CMakeFiles/CMakeOutput.log".
See also "/home/brad/gh/lv_binding_micropython/micropython/ports/rp2/build-ADAFRUIT_QTPY_RP2040/CMakeFiles/CMakeError.log".
make: Leaving directory '/home/brad/gh/lv_binding_micropython/micropython/ports/rp2'
PICO_SDK_PATH is /home/brad/gh/lv_binding_micropython/micropython/lib/pico-sdk
PICO platform is rp2040.
Build type is MinSizeRel
PICO target board is adafruit_qtpy_rp2040.
Using board configuration from /home/brad/gh/lv_binding_micropython/micropython/lib/pico-sdk/src/boards/include/boards/adafruit_qtpy_rp2040.h
TinyUSB available at /home/brad/gh/lv_binding_micropython/micropython/lib/tinyusb/src/portable/raspberrypi/rp2040; enabling build support for USB.
BTstack available at /home/brad/gh/lv_binding_micropython/micropython/lib/btstack
cyw43-driver available at /home/brad/gh/lv_binding_micropython/micropython/lib/pico-sdk/lib/cyw43-driver
Pico W Bluetooth build support available.
lwIP available at /home/brad/gh/lv_binding_micropython/micropython/lib/lwip
mbedtls available at /home/brad/gh/lv_binding_micropython/micropython/lib/pico-sdk/lib/mbedtls
Including User C Module(s) from =/home/brad/gh/lv_binding_micropython/make_build
CMake Error at /home/brad/gh/lv_binding_micropython/micropython/py/usermod.cmake:42 (include):
  include could not find requested file:

    =/home/brad/gh/lv_binding_micropython/make_build
Call Stack (most recent call first):
  CMakeLists.txt:80 (include)


Found User C Module(s):
make: *** [Makefile:54: all] Error 1

I haven’t committed and pushed the latest changes to get the compiling working for the other boards yet.

Strange about the display buffer. Now sure why it is not populating properly…

for some reason I believe this is a bug in the ESPIDF. I remember having a problem collecting the display buffer last time I messed around with writing an RGB bus driver for the binding. I ended up having to jump through hoops to get a valid display buffer that could be used.

I have to go and see if I can locate the code for it. I don’t remember where I may have stashed it I might have a branch for it. Not sure tho.

OK I removed the creation of the bus drivers from the RGB Bus. Now this is handled exactly the same way as it is done with the other busses. The no_fb, fb_in_psram, and the num_fbs parameters from the constructor of the RGB Bus. The display driver framework has handling added to it to set the buffer size correctly when using the RGB bus. That’s if no buffers are supplied to the display driver. The get_frame_buffer method has also been removed from the RGB bus driver as this is no longer needed.

I improved the handling of the frame buffer creation. In the event that double buffering is not possible if a single frame buffer is possible the best location for the buffer is used. Without being able to allocate both buffers in DMA space double buffering is a waste and doesn’t do anything and using DMA space for a single buffer is a waste to do as well. Everything is handled in order to get the best possible performance when letting the driver figure out the buffers.

Give that a shot and see if it corrects the RGB bus driver issues.

OH I also pushed the commit that fixes the build for the other MCU’s as well. There are specific things that have to be done to get those builds to work and I recommend using the make.py for handling this. The user doesn’t need to know anything in order to get it to work properly just run the make command and anything that needs to be done gets done.

Can you post the repository you pused hese changes to?
I think I read it at some point but cannot find it anymore.
Thanks

I ported I80Bus from lcd_bus to MicroPython and uploaded it along with the helper gpio_registers.py here. It works, but is VERY slow. It will serve as a good reference for anyone writing non-ESP32 bus drivers in C. It uses lookup tables whose size is determined by the bus_width, so a bus_width of 8 needs lookup tables sized 2^8, or 256 entries. Each entry is 32 bits. The reason they are 32 bits is that is the most number of pins that may be set (or reset) at a time using machine.mem32. There has to be a lookup table for each set of 32 pins, so if all the data pins are in the range 0 to 31 or 32 to 63, then only one lookup table is needed. The WT32SC01-plus has 7 of the data pins between 0 and 31, but one data pin is 46, so it requires 2 lookup tables. It would be more memory efficient to implement the lookup tables as byte arrays, but this is a proof of concept and reference for porting to C, so I didn’t take the time to do that.

A bus width of 16 would require 65,536 entries in each lookup table. Ignoring the overhead for a list of ints instead of a byte array, that’s still 256k per lookup table, so this lookup table method isn’t practical for larger bus widths. I put a line in the code that checks to make sure bus_width = 8 for that reason. Hopefully platform-specific versions won’t need lookup tables at all.

Another limitation is it depends on using pin numbers as opposed to pin names. That means it should work on ESP32 (which is what I tested it on), RP2, SAMD and NRF, but it won’t work on STM32, MIMXRT or Renesas-RA without a major rewrite of the GPIO_SET_CLR_REGISTERS class in gpio_registers.py.

I’m going to go back to testing RGBBus in lcd_bus next but have no intention of trying to port it to MicroPython because it relies heavily on perfect timing.