Screen not updating inside loop

Description

I have a little dictionary / crossword clue application that reads from a sqlite database on an SD card and outputs results. I’m using a T-Deck, which is an ESP32 with Display and Keyboard attached.
When I search my clues database, it can take a a minute or more to find all the results.
This involves looping over the sqlite rows, which get fetched one-by-one, and then update the label that shows the results.

Ideally each loop, the display will update, and scroll to show the bottom of the text in the label.

Right now the display only updates at the end of the loop. I’m calling lv_task_handler(); each iteration of the loop.

The screen doesn’t have any trouble updating elsewhere in my code. It seems to be very fast to update, when I usually update it I can’t see it redrawing by eye.

What MCU/Processor/Board and compiler are you using?

ESP32 (T-Deck)

What LVGL version are you using?

V8.3.0

What do you want to achieve?

What have you tried so far?

I tried adding lv_obj_invalidate(definition_ui) before the lv_task_handler() call

I tried setting #define LV_DISP_DEF_REFR_PERIOD 1 or to other values, like 3 or 5

I tried adding lv_refr_now(NULL) before lv_task_handler() call

Here is the loop, and some of the output when I enabled logging

            Serial.print("SQL: ");
            Serial.println(sqlQuery);

            Serial.println("Starting SQL portion");
            sqlite3_stmt *res;
            long startSqlPrepare = micros();
            int db_rc = sqlite3_prepare_v2(db, sqlQuery, -1, &res, NULL);
            Serial.print(F("Time taken:"));
            Serial.print(micros()-startSqlPrepare);
            Serial.println(F(" us"));
            Serial.println("Finished executing SQL");
            if (db_rc != SQLITE_OK) {
              Serial.println("error: db_rc != SQLITE_OK");
              return;
            } else {
              Serial.println("SQLITE_OK");
            }

            Serial.print("deviceMode: ");
            Serial.println(deviceMode);
            if (deviceMode == "DictionaryMode") {
              ..........
              ..........
              ..........

            } else if (deviceMode == "ClueMode") {
              Serial.println("Clue Mode");
              int clueCount = 0;
              Serial.println("Looping over rows");
              int rowLoopTimer = millis();
              while (sqlite3_step(res) == SQLITE_ROW) {
                clue = (const char *) sqlite3_column_text(res, 0);
                Serial.print("clue variable set: ");
                Serial.println(clue);
                answer = (const char *) sqlite3_column_text(res, 1);
                Serial.print("answer variable set: ");
                Serial.println(answer);
                Serial.println("Add to the clueOutput variable");
                clueOutput += answer;
                clueOutput += "\n";
                clueOutput += clue;
                clueOutput += "\n---------------------------------------\n";
                Serial.println("Update the definition_ui label");
                lv_label_set_text(definition_ui, clueOutput.c_str());
                if (lv_obj_get_scroll_bottom(definition_ui) > 0) {
                  Serial.println("Text is off the bottom of the display, so scroll to keep the most recent text on the screen");
                  lv_obj_scroll_to_y(definition_ui, lv_obj_get_scroll_bottom(definition_ui), LV_ANIM_OFF);
                }
                Serial.println("Update display");
                lv_task_handler();
                clueCount++;
              }

I add a bunch of Serial.print() and I can see the code running in a loop, but the display never updates until the loop finishes. Very occasionally it will update half way through.

I turned on some logging:

11:22:15.495 -> SQL: select clue, answer from clues_fts where clue like '%pig';
11:22:15.495 -> Starting SQL portion
11:22:15.495 -> Time taken:13887 us
11:22:15.495 -> Finished executing SQL
11:22:15.495 -> SQLITE_OK
11:22:15.495 -> deviceMode: ClueMode
11:22:15.495 -> Clue Mode
11:22:15.495 -> Looping over rows
11:22:15.594 -> clue variable set: Blade beheaded wild pig
11:22:15.594 -> answer variable set: OAR
11:22:15.594 -> Add to the clueOutput variable
11:22:15.594 -> Update the definition_ui label
11:22:15.594 -> Update display
11:22:15.594 -> [Trace]	(760.094, +169)	 lv_timer_handler: begin 	(in lv_timer.c line #69)
11:22:15.626 -> [Trace]	(760.094, +0)	 lv_timer_handler: already running, concurrent calls are not allow, returning 	(in lv_timer.c line #74)
11:22:16.220 -> clue variable set: Flower attached to tail of giant guinea pig
11:22:16.220 -> answer variable set: TASTER
11:22:16.220 -> Add to the clueOutput variable
11:22:16.220 -> Update the definition_ui label
11:22:16.220 -> Update display
11:22:16.220 -> [Trace]	(760.706, +612)	 lv_timer_handler: begin 	(in lv_timer.c line #69)
11:22:16.220 -> [Trace]	(760.707, +1)	 lv_timer_handler: already running, concurrent calls are not allow, returning 	(in lv_timer.c line #74)
11:22:16.515 -> clue variable set: African mammal found in book, a pig
11:22:16.515 -> answer variable set: OKAPI
11:22:16.515 -> Add to the clueOutput variable
11:22:16.515 -> Update the definition_ui label
11:22:16.515 -> Update display
11:22:16.515 -> [Trace]	(761.012, +305)	 lv_timer_handler: begin 	(in lv_timer.c line #69)
11:22:16.547 -> [Trace]	(761.012, +0)	 lv_timer_handler: already running, concurrent calls are not allow, returning 	(in lv_timer.c line #74)
11:22:17.666 -> clue variable set: English army officer can be a pig

Here are some logs from when I added the lv_refr_now(NULL) but still no screen updates during the loop

11:39:37.055 -> SQLITE_OK
11:39:37.055 -> deviceMode: ClueMode
11:39:37.055 -> Clue Mode
11:39:37.055 -> Looping over rows
11:39:37.153 -> clue variable set: Blade beheaded wild pig
11:39:37.153 -> answer variable set: OAR
11:39:37.153 -> Add to the clueOutput variable
11:39:37.153 -> Update the definition_ui label
11:39:37.153 -> Update display
11:39:37.153 -> [Trace]	(34.034, +169)	 _lv_disp_refr_timer: begin 	(in lv_refr.c line #290)
11:39:37.153 -> [Trace]	(34.044, +10)	 call_flush_cb: Calling flush_cb on (0;0)(319;239) area with 0x3d800908 image pointer 	(in lv_refr.c line #1060)
11:39:37.220 -> [Trace]	(34.080, +36)	 _lv_disp_refr_timer: finished 	(in lv_refr.c line #442)
11:39:37.220 -> [Trace]	(34.081, +1)	 lv_timer_handler: begin 	(in lv_timer.c line #69)
11:39:37.220 -> [Trace]	(34.083, +2)	 lv_timer_handler: already running, concurrent calls are not allow, returning 	(in lv_timer.c line #74)
11:39:37.814 -> clue variable set: Flower attached to tail of giant guinea pig
11:39:37.814 -> answer variable set: TASTER
11:39:37.814 -> Add to the clueOutput variable
11:39:37.814 -> Update the definition_ui label
11:39:37.814 -> Update display
11:39:37.814 -> [Trace]	(34.698, +615)	 _lv_disp_refr_timer: begin 	(in lv_refr.c line #290)
11:39:37.847 -> [Trace]	(34.710, +12)	 call_flush_cb: Calling flush_cb on (0;0)(319;239) area with 0x3d800908 image pointer 	(in lv_refr.c line #1060)
11:39:37.880 -> [Trace]	(34.747, +37)	 _lv_disp_refr_timer: finished 	(in lv_refr.c line #442)
11:39:37.880 -> [Trace]	(34.747, +0)	 lv_timer_handler: begin 	(in lv_timer.c line #69)
11:39:37.880 -> [Trace]	(34.749, +2)	 lv_timer_handler: already running, concurrent calls are not allow, returning 	(in lv_timer.c line #74)
11:39:38.177 -> clue variable set: African mammal found in book, a pig

Here is my disp_flush() function which I can see getting called when I add lv_refr_now(NULL)

static void disp_flush( lv_disp_drv_t *disp, const lv_area_t *area, lv_color_t *color_p )
{
    Serial.println("-----disp_flush() triggered-----");
    uint32_t w = ( area->x2 - area->x1 + 1 );
    uint32_t h = ( area->y2 - area->y1 + 1 );
    if ( xSemaphoreTake( xSemaphore, portMAX_DELAY ) == pdTRUE ) {
        tft.startWrite();
        tft.setAddrWindow( area->x1, area->y1, w, h );
        tft.pushColors( ( uint16_t * )&color_p->full, w * h, false );
        tft.endWrite();
        lv_disp_flush_ready( disp );
        xSemaphoreGive( xSemaphore );
    }
}

Playing around with it more, when I entry a query like %pig I get the behavior above. However when I enter a query like %o the screen updates much more often. However I don’t see anything different in the logs.