How to port LVGL to the RA6M3 processor?

Tinus · March 29, 2023, 8:11am

Description

Hello, I would like to try to run LVGL on the Renesas RA6M3G Evaluation kit but I’m not sure where to begin.

What MCU/Processor/Board and compiler are you using?

I am using a Renesas RA6M3G Evaluation kit with D\ave2D hardware acceleration. For compiling I would like to use the GNU ARM GCC toolchain version 10.3.1 (or older).

What do you want to achieve?

I want to have a minimal working project with some sort of simple LVGL program (just some text on a screen) running on the Renesas RA6M3G Evaluation kit using FreeRTOS. With this I would like to use the included D\ave2D hardware acceleration for LVGL drawing functions as this is probably faster than software rendering.

If I understand correctly I would have to implement the following:

Add LVGL to a Renesas e2 Studio project (e2 studio is based on eclipse)
Somehow make LVGL use D/ave2D hardware acceleration
Initialize LVGL
Make LVGL use the screen on the evaluation kit as display device (WITH hardware acceleration)
Use the screen on the evaluation kit as touch input (touchscreen input will be recieved via I2C and read on a seperate thread)
Call lv_tick_inc() via FreeRTOS
Call lv_task_handler() in some other FreeRTOS Thread

I think this will get quite complicated quickly because I of the different FreeRTOS Threads and needing to handle hardware acceleration somehow.

What have you tried so far?

Not much, I supposed asking this question first would allow me to work quicker, after this post is up I will start working down the list above. I have of course read the porting guide, but information about LVGL on Renesas specifically is not easy to come by.

I think the most difficult part will be configuring the display, I would like some pointers if anybody here has managed to make hardware acceleration work (maybe even Dave/2D specifically).
Making the touchscreen input work might also be difficult via a seperate RTOS thread as it is expected to use an LVGL callback. Any tips would be very helpful!

I will be keeping this post or thread up to date if I get any further.

Kind regards,
Tinus

robekras · March 29, 2023, 5:21pm

Your approach looks reasonable.

On the first step just don’t use the hardware acceleration. Just to see if everything works without that.
On my project there is one simple FreeRTOS thread for reading the touch.
This thread is triggered by the touch interrupt. The thread reads the touch controller and stores the readings into a small array.
The array is read from lvgl task by an indev driver call back.

Tinus · March 30, 2023, 12:16pm

Thanks for the reply.
Yesterday I did not get very far with hardware acceleration so indeed trying software rendering first is a better approach.

Following the porting guide, at one point I came across the custom flush callback function and I’m not sure how this works exactly. Could you perhaps explain the following:

When is this lv_disp_drv_t.flush_cb callback supposed to be called? Setting a breakpoint to some arbitrary function inside the callback never seems to trigger.
What am I supposed to inside this function? I thought LVGL filled the framebuffer and then this framebuffer would have to be passed to the display inside this callback. But now I’m no longer sure.

EDIT: I suppose something would have to be drawn to the framebuffer in order for the callback to trigger.

I’ve now done the following:

#include "app_thread.h"

#include "lvgl.h"

#define TICK_MS_TIME 5

#define DISP_HOR_RES 480
#define DISP_VER_RES 272

//DRAW BUFFER
static lv_disp_draw_buf_t draw_buf;
static lv_color_t buf1[DISP_HOR_RES];

//DISPLAY DRIVER
static lv_disp_drv_t disp_drv;

void my_flush_cb();

const display_status_t* displayStatus;

/* Application Thread entry function */
/* pvParameters contains TaskHandle_t */
void app_thread_entry(void *pvParameters)
{
    FSP_PARAMETER_NOT_USED (pvParameters);

//    SETUP TIMER RELATED FUNCTIONALITY
    xTimerChangePeriod(g_timer_lvgl, pdMS_TO_TICKS(TICK_MS_TIME), 0 );
    xTimerStart(g_timer_lvgl, 0);

    //START LVGL
    lv_init();

//    INIT DRAW BUFFER
    lv_disp_draw_buf_init(&draw_buf, buf1, NULL, DISP_HOR_RES);

    //INIT DISPLAY DRIVER
    lv_disp_drv_init(&disp_drv);
    disp_drv.draw_buf = &draw_buf;
    disp_drv.hor_res = DISP_HOR_RES;
    disp_drv.ver_res = DISP_VER_RES;
    disp_drv.flush_cb = my_flush_cb;

    lv_obj_t* label = lv_label_create(lv_scr_act());
    lv_label_set_text(label, "TEST");
    lv_obj_align(label, LV_ALIGN_CENTER, 0, 0);

    while (1)
    {
        vTaskDelay (1);
    }
}

void g_timer_lvgl_callback(TimerHandle_t xTimer)
{
    FSP_PARAMETER_NOT_USED(xTimer);

    lv_tick_inc(TICK_MS_TIME);
}

void my_flush_cb()
{
    //DO SOMETHING WITH LCD.... (r_gcldc)
    lv_disp_flush_ready(&disp_drv);
}

However, this causes the program to crash immediately, I’ve seen this specific error before when there was not enough memory available for the thread. However I’ve already allocated 0x4000 bytes (16 KB) just to be sure and that doesn’t fix it. Any idea what I could be doing wrong? It runs fine until I try to create the label to draw it to the buffer with lv_obj_t* label = lv_label_create(lv_scr_act());.

Kind regards,
Tinus

pete-pjb · March 30, 2023, 2:36pm

Hi @Tinus ,

It looks like your main while loop in the app_thread needs checking, you could try:

	while(1) {
		lv_task_handler();
		vTaskDelay(pdMS_TO_TICKS(5));
	}

Hopefully that will get you going…

Cheers,

Pete

robekras · March 30, 2023, 2:45pm

Two things after a quick view:

Your

static lv_color_t buf1[DISP_HOR_RES];

should be:

static lv_color_t buf1[DISP_HOR_RES * 20];  // 20 horizontal lines (more or less)

This should also be changed:

lv_disp_draw_buf_init (&draw_buf, buf1, NULL, DISP_HOR_RES);

to

lv_disp_draw_buf_init (&draw_buf, buf1, NULL, DISP_HOR_RES * 20);

And yes, your while loop should be changed as pete recommends.
The lv_task_handler is responsible for let lvgl do the work. And now my_flush_cb should be called.
Fill my_flush_cb with live. The correct function definition is:

void my_flush_cb (lv_disp_drv_t* disp_drv, const lv_area_t* area, lv_color_t* color_p)

Tinus · March 31, 2023, 10:03am

Thanks both of you for your answers.
I changed the code to what you suggested, sadly to no avail.

In lv_label_create() the application gets all the way to calling lv_obj_class_init_obj(obj). Here the obj can still be found via the debugger, see below:

Then it goes on with this function properly. However, in lv_obj_mark_layout_as_dirty(obj), the function lv_obj_get_disp(obj) calls lv_obj_get_screen(obj). This very function causes the application to crash. Somehow the object pointer here is no longer valid, even though it was earlier and just gets passed normally everywhere, as far as I can see.

What happens here? Did I define the pointer to the label incorrectly somewhere?

EDIT: I found that this even happens when running one of the examples, I have tried switching from version 8.3 to 8.0 but even that did not help. Then I guess I iniated LVGL incorrectly somewhere?

robekras · March 31, 2023, 2:32pm

What did you setup/change within the lv_conf.h?
Especially for using malloc.

What is g_timer_lvgl good for? I don’t see any declaration for it.

I miss the lv_disp_drv_register (&disp_drv);
It should be right after your

disp_drv.flush_cb = my_flush_cb;

kdschlosser · March 31, 2023, 7:06pm

you guys are missing one tidbit of information…

lv_disp_flush_ready must be called when the data is finished being written to the display. Read that carefully…

If you are using DMA memory you are not going to have that call inside of the flush function. You will have that call in another callback that is attached to the IO object instead. The IO object will call the callback when it is finished sending the data and at that time you would calllv_disp_flush_ready. If you are not using DMA memory then you can call lv_disp_flush_ready at the end of the flush callback function.

When using DMA memory the transfer of the buffer doesn’t use CPU time so the flush function is able to return and LVGL would be able to start filling the buffer. The problem here is the buffer may not be empty. LVGL needs to be told when the buffer has emptied so it knows when it can start filling ti again. By using double buffering you are able to get the benefits of using a DMA memory buffer. While the first buffer is being written to the display LVGL will be filling the second buffer. If you have the ability to use DMA memory I do recommend using it and also enabling double buffering, even if you have to shrink the buffer size down to be able to do it. How much DMA memory is available depends on the MCU. I recommend setting up the buffers as soon as possible when the program starts. This way you are able to secure the memory you need for the buffers.

The code below is an example of how to do it for the ESP32. You would have to find out how to set the buffers up to use DMA memory for your MCU. I am not all that proficient in C code and below is an example and has not been tested. I always mess up on the pointers so there is a good chance I got the declarations for the buffers wrong.

uint32_t buf_size = DISP_HOR_RES * 10 * sizeof(lv_color_t);
uint8_t  * buf1 = esp_heap_caps_malloc(buf_size, MALLOC_CAP_DMA);
uint8_t  * buf2 = esp_heap_caps_malloc(buf_size, MALLOC_CAP_DMA);

lv_disp_draw_buf_init(&draw_buf, buf1, buf2, DISP_HOR_RES * 10);

Tinus · April 3, 2023, 9:14am

Hello,

Believe it or not, using lv_disp_drv_register() fixed my issues so far! I can’t believe I looked over this detail.

Furthermore:
I have not touched any of the settings in lv_conf.h. The color depth was already correct, I have simply set the #if 0 to if 1 at the top of the file. I have disabled all demos for now.

g_timer_lvgl is a (FreeRTOS) timer I have setup via Renesas e2 Studio’s “FSP”, which allows me to setup (among other things) FreeRTOS via the IDE. This is an interrupt timer that calls the corresponding callback every TICK_MS_TIME milliseconds (5ms). I use this to call lv_tick_inc() on a regular basis.

Thanks a lot for your answer,
I can now continue with porting LVGL by somehow implementing the flushing functionality.

Tinus · April 3, 2023, 9:30am

Hello,

thanks for your answer, I had not heard of DMA before and it seems really useful for this use case.
I see the processor I’m using right now does indeed support it and it would be really worth looking into.

I will definitely keep your answer in mind, once I get LVGL to work without DMA I will try implementing it.
It seems DMA on Renesas processors works with callbacks to notify the user when x data is transferred, this could be very useful for knowing when to call lv_disp_flush_ready.

Best regards

kdschlosser · April 3, 2023, 5:43pm

This is exactly right. That is where you would let LVGL know the flush buffer has been written.

Using DMA memory gives a pretty good performance boost when running LVGL. this is because it doesn’t stop LVGL from doing it’s thing when data is being written to the display.

How to allocate DMA memory and how to write the data to the display is going to be dependent on the MCU being used.

kdschlosser · April 3, 2023, 5:44pm

make sure that when you call lv_tick_inc you are passing a 5 in that call. The 5 indicates how many milliseconds has passed between calls so the timer can update. I would not call lv_task_handler from inside of the interrupt. if your main program loop is not something that can take a long time then you can also set a flag that would tell your main program loop to run lv_task_handler. If your main program loop takes more than 1-2 milliseconds to run you should consider placing both the lv_tick_inc and lv_task_handler calls into the main loop and record how much time passes between the calls and that is what you would pass to lv_tick_inc instead.

There are many ways of going about doing this. the easiest one to use and will be trouble free is going to be doing it in your main program loop.

Tinus · April 4, 2023, 1:00pm

Hello all,

I have finally gotten to the point of being able to run all of the examples, even the benchmark is surprisingly quick. No hardware acceleration and user inputs yet, nor DMA but I will open seperate threads for that when the time comes.

Thanks a lot to everyone who helped me out in this thread! Have a happy easter.

Kind regards,
Tinus

kdschlosser · April 4, 2023, 7:05pm

Fantastic that you got it up and running. please do reach out if you run into any issues getting DMA working. Also let us know if you got DMA working and if you noticed a difference in performance.

EagleLiao · April 11, 2023, 5:25am

You can start with running LVGL simulator on Windows.

PJ_Besora · October 11, 2023, 10:04am

Hi Tinus,
I’ve reached a similar scenario than you where I’m able to display basic things.
But when I try to use a little more complex example ( lv_example_label_1(); that is using a scrolling label) I end up with BSP_CFG_HANDLE_UNRECOVERABLE_ERROR(0);
My feeling is that maybe as I’m using a basic approach to fill the fb_background buffer (pixel by pixel) maybe this leads to error when some animation (even is very basic) is needed…

void my_flush_cb (lv_disp_drv_t* displaydrv, const lv_area_t* area, lv_color_t* color_p)
{
int32_t x = 0;
int32_t y = 0;
uint32_t drawPos = 0;
for(y = area->y1; y <= area->y2; y++) {
for(x = area->x1; x <= area->x2; x++) {
drawPos = (uint32_t)(y * disp_drv.hor_res + x);

        uint8_t col1 = (uint8_t)(color_p->full & 0xff);
        uint8_t col2 = (uint8_t)(color_p->full >> 8);

        frameBuffer[drawPos * 2] = col1;
        frameBuffer[(drawPos * 2)+1] = col2;

        color_p++;
    }
}

lv_disp_flush_ready(displaydrv);

}

Could you please show me how your flush:callback looks like ?

BTW I’ve already set stack heap size to 0x2000 or also to 0x4000 with no luck either

Thnks

Tinus · October 11, 2023, 12:51pm

Hello,

If you are on version 8.3.9 or higher, there is actually support for the RA6M3’s GPU driver (r_drw, Dave2D).

See the corresponding option in lv_conf.h: #define LV_USE_GPU_RA6M3_G2D

If you enable this and then follow the instructions here: Renesas — LVGL documentation
it should not be to hard to get working. Make sure to enable the driver in the corresponding thread you use for the graphical interface, then put the header file of that thread into the #define below this lv_conf option.

My Dave2D version of flushing and display driver setup now looks like this:

//DISP. DRIVER SETUP
    lv_disp_draw_buf_init(drawBuffer, buffer1, buffer2, horRes * bufLines);

    lv_disp_drv_init(displayDriver);
    displayDriver->draw_buf = drawBuffer;
    displayDriver->hor_res = (lv_coord_t)horRes;
    displayDriver->ver_res = (lv_coord_t)verRes;
    displayDriver->flush_cb = simple_buffer_flush_cb;

    lv_draw_ra6m3_g2d_init();

    lv_disp_drv_register(displayDriver);

//FLUSH
void simple_buffer_flush_cb(lv_disp_drv_t* disp_drv, const lv_area_t* area, lv_color_t* color_p)
{
    lv_port_gpu_blit(area->x1, area->y1, color_p, area);
    lv_disp_flush_ready(disp_drv);
}

That is all! Let me know if it works. There’s also the possibility to use DMA (Direct memory access) via the DMAC. I have a topic on that here: Using DMA to transfer to the screen's framebuffer. Through testing I have found DMA to be quicker than using the special graphical processor, however the GPU can be expanded upon much more and I believe it may be much quicker if properly implemented in conjuction with DMA.

PJ_Besora · October 11, 2023, 2:00pm

Hi,
Thanks for your quick feedback.
I’ve tested with Dave2D and I end up with the same BSP_CFG_HANDLE_UNRECOVERABLE_ERROR(0);, so I’m missing something more, or something is wrong with my code…
Note that I can see the 2 labels from the lv_example_label_1
But he 2n one, that should be scrolling horitzontally is fixed

Since I cannot upload 7z files I attach the main module
app_thread_entry.c (2.9 KB)

I also attach some images from project configuration

Appreciated

Tinus · October 11, 2023, 2:12pm

Hello, something is wrong indeed.

It seems you are calling lv_timer_handler() twice. You should only call it in the while loop, whereas lv_tick_inc() should be called in the lvgl_timer callback function.

Furthermore, frameBuffer= (uint8_t *)fb_background; should be changed to frameBuffer = (uint8_t*)fb_background[0]. fb_background is a 2D array.

PJ_Besora · October 11, 2023, 2:21pm

OMG
You are totally right, the issue was calling the timer twice !!! Now it’s working.
(The frameBuffer definition is also wrong as you say, but it was not used at this moment for Dave2D, in any case I will also correct it)
I’ve spent some hours checking line by line and I missed an obvious part…

In next days I will start with DMA tests and touch implementations , so most probably I will come back with more doubts…

Thanks a lot for your help…