Micropython Display Drivers

Micropython
21

memory use I believe. I do know that it should be set to the size of the buffer when DMA memory is being used. Also if double buffering is not being used there are no benefits to be had from using DMA memory.

trans_queue_depth should be set to 1. There is no reason to have it set to 10. again this is more memory use when it doesn’t need to be used. you are using pre allocated buffers and those buffers are what is always use. LVGL has the flush ready function that must be called so LVGL known when it is able to fill the buffer. Because of that mechanism there will never be more than a single buffer that gets queued. Even when using double buffering LVGL waits until the first buffer has finished sending before calling flush with the second buffer. So the mechanics of LVGL keep things aligned properly so only 1 thing can be queued at any point in time.

This is the reason why I have things coded like I do. so that if the user wants to supply the buffer size they can when creating the bus. This will ensure that a buffer gets created in DMA memory is SRAM if that is where they want it placed. There is not a large amount of DMA memory available in SRAM so capturing that memory first thing is important otherwise it might not be available if other things load beforehand. If the user doesn’t supply a buffer size that is OK also. The display driver calls the init method for the bus passing the width, height and pits per pixel and if no buffers have already been made then it will create them. The init command is where the bus is actually initialized because that is where the last few missing pieces get passed by the driver. Passing the width and height of a display to the bus constructor didn’t seem right to do to me. The display size is something the display driver should know. For the purposes of creating a correct sized buffer that information can be passed to the bus driver but it is not something that gets stored in the bu driver.

22

I have thought about separating my bus drivers so they are in different modules. No reason to have parts of the SPI, I8080 and I2C busses loaded if only RGB is getting used.

I don’t know how much memory could be saved by doing that. Considering a single pointer uses 4 bytes of memory it might be worth doing. Classes would be 4 bytes used and then another 4 bytes for each of the methods. That adds up to 108 bytes… It would however increase the code footprint so more storage would end up being used.

23

Circuitpython has their buses in separate modules, probably for the same reason. In fact, nearly all of their display components are separate modules. Their graphics engine DisplayIO is much different than LVGL, but the structure of their modules is something to consider. With version 9, BusDisplay, EPaperDisplay, ParallelBus for i8080, Fourwire for SPI with CS and DC, and I2CDisplay for just the I2C bus are all separate modules. Their display drivers are subclasses of BusDisplay or EPaperDisplay. I’m a fan of the way their Fourwire bus is setup. You create the SPI object first and pass that as one of the parameters of the Fourwire class. That makes it easy to share the SPI bus with SD card readers or SPI touchscreens, something I’ve seen lots of complaints about on this forum. Sharing the display bus isn’t desirable, but if that’s how the hardware is wired, we need to account for it. I really think I will be able to make my driver work that way for SPI as well. I’ve seen some of what you have written about that, but it looks to me like its worth trying.

Thanks for the pointers on saving memory. I’ve got some other priorities first, but I plan to get to those within the week.

24

This is because the users do not know how to do it. pins and the host being passed into the SDCard class in MicroPython need to be identical to what is used for the display. All pins except the CS pin. If this is not done then it will not work. If the display does not have a broken out CS pin then guess what? it cannot be done. when calling the constructor of SDCard the “slot” is both the slot number and also the SPI host.

0 = slot zero
1 = slot one
2 = spi host 0
3 = spi host 1
4 = spi host 2

etc…

so when construction the SDCard class you want to use the same host that is used for the display and add 2 to it. Then and only then will it initialize properly.

25

correction from that last post…

// Slots 0 and 1 are native SD/MMC, slots 2 and 3 are SPI
    bool is_spi = (slot_num >= 2);
    if (is_spi) {
        slot_num -= 2;
    }

0 = slot zero
1 = slot one
2 = spi host 0
3 = spi host 1

It makes you believe that spi host 3 is not going to work but it should.

SPI1_HOST=0
SPI2_HOST=1
SPI3_HOST=2

in MicroPython the SPI class has the parameter id which is the SPI host number. numbers 1, 2 and 3 are what designate to use SPI1_HOST, SPI2_HOST or SPI3_HOST which translates into id - 1. The use of different parameter names and the oddball way or handling them doesn’t make linking them together easily understood. So if a user has say a display and a touch screen connected to the same bus then they would initilize the display using 0, 1 or 2 for the Display SPI bus but when initilizing the SPI bus built into micropython for use with the touch screen they would need to add 1 to whatever host number they passed to the display constructor and if using an SDCard reader they would need to add 2 to the number.

Make sense?

circuit python does make this easier by being able to create a bus instance and then passing that bus instance to SD card or to a display but that ease of use comes at a cost. That cost is additional resources. This is because each of these things use different structures to hold the information so you are actually only using the SPI bus instance to carry setting that end up getting copied which uses more memory.

26

Makes sense. I don’t have a use for SDCards, but I’ll get one to try since a couple of my boards have slots.

27

@bdbarnett I’ve just added you to the Google Calendar event as well!

See you soon guys!

1 Like
28

found it

1 Like
29

I have a naive question: if LVGL is already a complete standalone C project, it must have its own display drivers which are already maintained by LVGL contributors. Why is it that we need additional “MicroPython” drivers developed by MicroPython contributors? Why couldn’t we just reuse the LVGL drivers?

One possible reason is that MicroPython users don’t like to compile their own firmware and it is infeasible to compile every LVGL driver into the standard MicroPython firmware. If that’s the only major problem, has anyone considered dynamic linking as a solution? I.e., build individual LVGL drivers like they were shared libraries and create an on-device dynamic linker that allows MicroPython users to load and link only the specific drivers they need at runtime.

30

That pretty much sums up what we’ve been discussing. The drivers LVGL includes for use in C code must have all the display configuration details specified at compile time. If you are writing your code in C, changing from one display type to another (ILI9341 to ST7789), changing from one family of microcontroller to another (ESP32 to RP2), changing to a different board with the same microcontroller (feather to QT Py), or even pin numbers for custom wiring requires recompiling your code. That’s not practical in Micropython for reasons you alluded to, so those drivers aren’t the solution. @kdschlosser is addressing the issue with code written specifically for LVGL and included in the lv_binding. I am addressing it with code written for Micropython, not specifically for LVGL, that is compiled as an external C module. My goal is to have my drivers compile as a native .mpy module that can be added to an already running Micropython file system like any other .py file, and will work with any graphics library that needs to show something on a display, not just LVGL. The two projects have different goals even though from the perspective of LVGL on Micropython they accomplish the same thing.

31

One possible reason is that MicroPython users don’t like to compile their own firmware and it is infeasible to compile every LVGL driver into the standard MicroPython firmware

The reason for choosing MicroPython instead of C is to have fast prototyping for our project, given its substantial project size. Opting for quick prototyping over performance trade-offs, The code freezing and compile own firmware is not a option for me, it is must to do item to enhance performance, and it’s not that every MicroPython user dislikes building firmware. At least, that’s the case for my situation.

1 Like
32

You cannot do it that way without having to compile again if a change is going to be made to the display. The entire purpose to MicroPython is not having to do that. Plus you also have the different boards and the user must be able to select the pins they are using to attach to the display. It has to be made so it exposes the interface to MicroPython there is no way around it.

The display drivers are easy to write. It takes all of about 10 minutes to do if there is a framework behind the scenes and the bus drivers all have the exact same API. Those are the 2 things that are needed to make it easy.

It’s the bus drivers that are going to be different per MCU/Board Manufacturer. Currently the drivers that are available also have the bus driver in with the display driver. The 2 need to be separated in order to be able to use display drivers with any board.

1 Like
33

ahem, My code is not specific to LVGL BTW. It is attached to the same build system that compiles LVGL because that is the ONLY way you can compile multiple user C modules. You can only specify a single user c module at the command line so both modules have to be built by the same build system. The code I wrote for the bus drivers is not reliant on LVGL in any way shape or form.

34

compiling as an MPY file consumes more RAM when the code is executed. The user also has to know what to compile as an MPY based on the board being used. Where as compiling as a user C module handles all of that internally in the build system. The user should only have to know the board they are compiling for and not have to know where source code is located.

I don’t know if LVGL would even be able to run if it is compiled as an MPY file. The additional memory use and also the import time of the module may not make it a viable thing to do. The import time slows down a lot because of the dynamic loading of the Q Strings and there is a really large number of them that get made for LVGL.

35

I just found this on creating a mpy file from C code. It doesn’t work across all boards. The compiled file can only be used on boards that are the same. so moving the file form an ESP32 to an STM32 is not going to work. There are all kinds of limitations for doing it as well.

https://docs.micropython.org/en/latest/develop/natmod.html

36

@kdschlosser

in your lv_binding_micropython/mkrules.cmake

    if(ESP_PLATFORM)
        file(GLOB_RECURSE LV_ESPIDF_HEADERS ${IDF_PATH}/components/*.h ${LV_BINDINGS_DIR}/driver/esp32/*.h)
        lv_bindings(
            OUTPUT
                ${LV_ESPIDF}
            INPUT
                ${LV_BINDINGS_DIR}/driver/esp32/espidf.h

and

if(ESP_PLATFORM)
    LIST(APPEND LV_SRC
        ${LV_BINDINGS_DIR}/driver/esp32/espidf.c

See How to build MicroPython with multiple external user C modules

I didn’t suggest compiling LVGL as an MPY. It certainly would be too big and slow to load.

You linked to the same link I did:

Agreed, native .mpy modules are platform specific, Micropython version specific, and currently only work on these platforms:

  • x86 (32 bit)
  • x64 (64 bit x86)
  • armv6m (ARM Thumb, eg Cortex-M0)
  • armv7m (ARM Thumb 2, eg Cortex-M3)
  • armv7emsp (ARM Thumb 2, single precision float, eg Cortex-M4F, Cortex-M7)
  • armv7emdp (ARM Thumb 2, double precision float, eg Cortex-M7)
  • xtensa (non-windowed, eg ESP8266)
  • xtensawin (windowed with window size 8, eg ESP32)

So, on a project running on a Cortex-M7 for example, the developer could have the benefit of not having to compile their own drivers at all. The same way Micropython offers precompiled binaries on their website, Github CI could be used to provide precompiled display driver binaries for that handful of platforms, and those binaries could be available to anyone using one of those platforms. No matter how the user got their firmware, all it would take for them to add display drivers would be to add a file to the microcontroller’s file system. Remember, my drivers will work with other graphics libraries as well, so maybe all they need is the firmware from micropython.org and the precompiled display driver MPY that I hope to provide, and they never have to run make. For other platforms, say RP2, they would still have the option to compile their own as a user c module. Having native .mpy modules available might even sway the person making the hardware decisions to use an ESP32 or Cortex-M7 in their build for that very reason. Plus, I haven’t been able to find anything that says the above list will never grow.

If those limitations don’t impede the ability of the driver to compile as an MPY and don’t impede it’s ability to work, who cares? To be honest with you, I don’t know if they will or not, but that’s no reason for me not to try.

At this point I want to apologize to you for picking apart what you posted and coming across argumentative. That is not my nature at all. I did it to support what I’m about to say.

I am 50 years old and made my first forum post only 2 months. First ever, not just on LVGL’s forums. I’m a smart man, but prefer to take in what others share and keep to myself. I feel strongly about the topic at hand and also feel like I’ve got something to contribute, which is why I’ve been so vocal all of a sudden. I try to go out of my way to not talk down to anyone. In fact, when I could, I’ve praised you for your work and the contributions you are making to the community. That is very sincere. With @amirgon away from the project, you appear to be the future of lv_binding_micropython and your work will likely benefit us all. There’s no doubt in my mind you have the skills and are up for the task. Yet every time I post anything, I write and rewrite, trying to rephrase things so you won’t pick them apart. For instance, in the previous paragraph, I didn’t mention that x86 and x64 drivers are implemented differently than drivers for microcontrollers. I went out of my way to phrase things the way I did so you won’t focus on those or point that out. It’s not just me you do these things to. Here’s one example from another thread on this forum:

It looks like this person has a valid reason, but since they didn’t explain it to you first, you basically told them what they were doing is invalid. I ask you to step back and realize we all have have our own goals and use-case scenarios. You may not see the reasoning or circumstances surrounding what someone says. That doesn’t make what they say invalid. You may not agree with what someone says, and you may be right. No need to say “That won’t work…” There are ways to make your point without being adversarial, argumentative or belittling.

I have said all I am going to say. From here forward, I’ll go back to lurking. I may chime in from time to time if I think I can help someone with a problem or if I have some accomplishment I want to share with the community. I may try the Micropython forums instead. At any rate, I don’t want to argue with you publicly and have no intentions on responding to whatever you may post next. I’ll see it for certain, but if you are interested in a reply from me, you should message me directly. One last thing… I still respect your knowledge and skills and appreciate everything you’ve done to help me, both in your posts addressing me directly and in your posts for others that I’ve benefited from, even before I created a forum account.

37

Hi @bdbarnett, I hope you don’t go back to lurking as I think your contributions to this discussion have been positive and productive. I am new to this forum too, but not to forums in general and it seems like a typical forum to me. Forums are inherently an impersonal form of communication. I find it best to focus on what people are saying rather than how they are saying it, as people are often unaware of their tone or how it affects others. Anyway back to business…

The display drivers are easy to write. It takes all of about 10 minutes to do if there is a framework behind the scenes and the bus drivers all have the exact same API. Those are the 2 things that are needed to make it easy.

Is this a goal for C LVGL too, or just for MicroPython LVGL?

You cannot do it that way without having to compile again if a change is going to be made to the display.

This is the problem we’re trying to solve. I think I have a similar proposal to bdbarnett which I will attempt to re-iterate here.

  • The builder of MicroPython distros builds a firmware binary that contains the core LVGL but no drivers.
  • In addition to that they also build every LVGL driver as separate binaries.
  • MicroPython users download and flash the MicroPython firmware as usual.
  • Then depending on what display hardware they have, they also download the binaries for those drivers, which were pre-built by the distro builder.
  • The way these driver binaries are “flashed” to the device will be a bit different, but it is essentially the same user workflow as copying a py or mpy file to the device.

One way where my proposal differs from bdbarnett’s is in the use of native mpy files. In my opinion the native mpy support in MicroPython is not scalable because it doesn’t allow the driver writer to link to arbitrary functions in the firmware. Instead I would create a new dynamic linking system which allows that.

Plus you also have the different boards and the user must be able to select the pins they are using to attach to the display.

Agreed that that’s also a problem. But let’s come back to that when we’ve solved the separate compilation problem.

38

I am not sure what you are looking at but it sure isn’t the code I have written.

https://github.com/kdschlosser/lv_binding_micropython/tree/MicroPython_1.21.0_Update

exactly. that user is creating a build file and including the 2 user c modules build files into it. That is because only a single user_c_module path is able to be supplied via the command line. As I stated previously. only a single user c module can be compiled unless you join the build systems for each user c module.

So using mpy files is not portable.

so what would be the point to compiling the bus drivers or the display drivers as MPY files? The only thing it is going to do is consume more memory. There is not a huge hit as far as storage is concerned for just the drivers. There is zero in terms of performance gains VS leaving the display drivers as python source files. Why do the additional work if there is no benefit? Portability went out the window if compiling C code and zero performance benefits for compiling python source into MPY files.

I could see spending the additional time and energy if there was some benefit to be had, I am not coming up with any

I’m not trying to be argumentative at all. I am trying to understand why it is a good idea to do. It seems like it is a lot of additional work with no return other then the ability to say that you did it. From the type of person I have figured you were I didn’t take you as a person that would spend the time and energy for really no reason. That is the only reason why I am trying to point out why it wouldn’t be worth the time and energy.

I am not the future of lv_binding_micropython nor do I want to be. My only objective is to thumb up different ideas on how to go about making it work and in such a way that it is easier to maintain. If someone wants to use the code I wrote that’s fine. If they don’t, that’s fine too. I did what I did because I could see in my head that it would work but trying to explain it would be hard to do. Easier to provide a proof of concept.

39

@gneverov

There are 2 types of drivers that are being worked with. You have the display drivers and then you have the driver for the transport of data. A lot of chips made for displays have support for more than on kind of transport or “bus”. So you ideally want to separate a display driver into 2 pieces, one if the display driver and the other is the bus driver. The bus drivers is the low level protocol for how to transfer the data where as the display driver is the format of the data being sent. The bus specifications are a standardized thing that is not unique to a single manufacturer, that means it can be reused across all displays that support that type of communications.

The bus drivers are something that is typically provided by the manufacturer of the MCU. It is apart of the SDK. so the only thing that has to be done is a layer needs to be made to expose that portion of the SDK to the user that is using MicroPython. since access to the SDK is needed it makes sense to compile it in with the firmware. The bus driver is what does the transmitting of the data to the display.

The display driver only has 2 tasks. pick the data that needs to be sent to the display to initialize it. This is a one shot and done and it takes typically < 1/2 a second to perform and the majority of that time is waiting for the display to do it’s thing (nothing sent when waiting).

The other thing it is responsible for doing is flushing the frame buffer. What I mean by that is the display driver hands off a memory address for the frame buffer to the bus driver. The display driver does no real work, definitely nothing that would impact the speed in any way.

Uploading a python source file to a board is exactly the same as uploading an MPY compiled binary. No difference there. compiling into the binary add more complexity to the build system and if API’s change in C code then changes might need to be made for it to compile. Where if it is written in python those changes would not need to be done.

Now if you include display drivers with LVGL you would still need to write a layer to expose those drivers to the users. The bus drivers would not be able to be done because LVGL has no clue what it is being installed on. so those would still have to be written for MicroPython specifically as well. You also loose the runtime aspect because you would have to specify the display driver you want to use at compile time because the bits and pieces need to exist in LVGL in order for the layer to expose it to the user.

40

Just to be concrete:

Is this right?

My fundamental question is: even if you’re running with MicroPython, why can’t you keep using the same driver code in lvgl_esp32_drivers (which I’ll generically refer to as C (display) drivers)?

Two answers I’ve heard:

  1. The C drivers use #define for configuration settings.
  2. MicroPython users can’t compile their own C driver and MicroPython distributers can’t include every C driver in the firmware.

I think a solution to #1 will be easy. For #2, I’m proposing separate compilation of the drivers and firmware as a solution. Are there any more issues?

The bus drivers are already in the SDK. Yes, MicroPython provides its own bus APIs so that users can directly use those buses from Python, but LVGL doesn’t have to go through that. LVGL can keep using the SDK bus driver like it does without MicroPython. MicroPython internally uses the SDK for bus access, so under the hood it is all the same.