Screen orientation change in runtime, like a smartphone

Description

Display driver for a small OLED of 0.96" (64*128) monochrome developed. Orientation change in compile time is working depicted in a photo below. The essential code to do this is shown at the end of this post.

What MCU/Processor/Board and compiler are you using?

STM32L432

What LVGL version are you using?

Version 7.2.0

What do you want to achieve?

Want to go one step further to change the orientation in runtime just like a smartphone. Will install an accelerometer for this feature.

Code to reproduce

In lv_conf.h

#define LV_SCREEN_ROTATE_270 //change this for different orientation

#if defined (LV_SCREEN_ROTATE_90) || defined (LV_SCREEN_ROTATE_270)
#define LV_HOR_RES_MAX          (128)
#define LV_VER_RES_MAX          (64)
#else
#define LV_HOR_RES_MAX          (64)
#define LV_VER_RES_MAX          (128)
#endif

my_flush_cb() is like:

void my_flush_cb(lv_disp_drv_t * disp_drv, const lv_area_t * area, lv_color_t * color_p)
{
	/**
	 * variables dev_y1 and dev_y2 in device space, in the sense that it always follow the
	 * scanning direction of the OLED
	 */
	lv_coord_t dev_y1, dev_y2;

#if defined (LV_SCREEN_ROTATE_270)
	dev_y1 = (LV_HOR_RES_MAX-1)-(area->x1);
	dev_y2 = (LV_HOR_RES_MAX-1)-(area->x2);
#elif defined (LV_SCREEN_ROTATE_180)
	dev_y1 = (LV_VER_RES_MAX-1)-(area->y1);
	dev_y2 = (LV_VER_RES_MAX-1)-(area->y2);
#elif defined (LV_SCREEN_ROTATE_90)
	dev_y1 = area->x1;
	dev_y2 = area->x2;
#else
	dev_y1 = area->y1; //in native orientation of the OLED, dev_y = area->y always
	dev_y2 = area->y2;
#endif

  	//flush display by DMA
	//dev_x1/_x2 is not relevant because we are updating the entire OLED_HOR_RES
  	framebuf_fill_area(0, dev_y1, OLED_HOR_RES-1, dev_y2, (const color_t *)color_p, 0);
}

void my_set_px_cb(lv_disp_drv_t * disp_drv, uint8_t * buf, lv_coord_t buf_w, lv_coord_t x, lv_coord_t y, lv_color_t color, lv_opa_t opa) {
  (void) disp_drv;
  (void) buf_w;
  (void) opa;

  lv_coord_t dev_x, dev_y;

  /**
   * @note: dev_x or _y always on device space, following the same scanning direction of the OLED
   * (x,y) in virtual space, following the screen orientation defined in LVGL.
   * So we need to think like this: what is the value of dev_x in device space if we have a pixel at x in VDB?
   */

//now only in compile time
#if defined (LV_SCREEN_ROTATE_270)
  dev_x = y;
  dev_y = (LV_HOR_RES_MAX-1)-x;
#elif defined (LV_SCREEN_ROTATE_180)
  dev_x = (LV_HOR_RES_MAX-1)-x;
  dev_y = (LV_VER_RES_MAX-1)-y;
#elif defined (LV_SCREEN_ROTATE_90)
  dev_x = (LV_VER_RES_MAX-1)-y;
  dev_y = x;
#else
  dev_x = x;
  dev_y = y;
#endif

  if (lv_color_to1(color) == 1) {
    buf[BUFIDX(dev_x, dev_y)] |=  PIXIDX(dev_x);  //Set VDB pixel bit to 1 for other colors than BLACK
  } else {
    buf[BUFIDX(dev_x, dev_y)] &= ~PIXIDX(dev_x);  //Set VDB pixel bit to 0 for BLACK color
  }
}

What is the difficulty?

There is a member rotated in lv_disp_drv_t. Setting lv_disp_drv.rotated to a different value will swap hor_res vs ver_res but there is no callback function to let LVGL know that the screen orientation has changed. I tried to call lv_refr_now(lv_disp) in the main loop to force screen update but there is no change in screen orientation, until there is a touch event that calls my_flush_cb().

Need an interface (similar to input devices like an encoder) for the accelerometer sensor to pump messages containing screen orientation, so that LVGL will refresh the screen with new screen orientation.

Any suggestion is welcome.

Try invalidating the screen: lv_obj_invalidate(lv_scr_act()).

Calling lv_refr_now does not force every object to be redrawn; it just forces rendering to happen right then (as opposed to the next refresh period).

1 Like

@embeddedt

Try invalidating the screen: lv_obj_invalidate(lv_scr_act())

Thank you. It is the solution. Snippet to do it:

static disp_orientation_t _disp_orientation;
void display_orientation_set(disp_orientation_t disp_orientation) {_disp_orientation = disp_orientation;}
disp_orientation_t display_orientation_get(void) {return _disp_orientation;}
int main(void)
{
  //...lv_init(), hal_init() skipped for simplicity
  int rotate=0;
  while(1){
	  if(rotate++ == 1000) //there is no accelerometer yet, use a simple 1sec delay loop as simulation for now
	  {
		  if (display_orientation_get()==DISP_270_DEG)
		  {
			  display_orientation_set(DISP_0_DEG);
			  lv_disp_drv.rotated = 0;
		  }
		  else if (display_orientation_get()==DISP_0_DEG)
		  {
			  display_orientation_set(DISP_90_DEG);
			  lv_disp_drv.rotated = 1;
		  }
		  else if (display_orientation_get()==DISP_90_DEG)
		  {
			  display_orientation_set(DISP_180_DEG);
			  lv_disp_drv.rotated = 0;
		  }
		  else if (display_orientation_get()==DISP_180_DEG)
		  {
			  display_orientation_set(DISP_270_DEG);
			  lv_disp_drv.rotated = 1;
		  }
		  rotate = 0;
		  lv_obj_invalidate(lv_scr_act());
	  }
  }
}

Video to show the result

But now there is a problem of screen truncated to only half the screen when rotated.
It is not a simple LV_HOR_RES_MAX to lv_disp_get_hor_res(lv_disp) to make runtime screen rotation work. Still working on the problem…

The flush callback is updated with the snippet:

void my_flush_cb(lv_disp_drv_t * disp_drv, const lv_area_t * area, lv_color_t * color_p)
{
    lv_coord_t dev_y1, dev_y2;
	disp_orientation_t curr_disp_orientation = display_orientation_get();

	printf("New orientation is %d:, rotated flag is :%d, hor_res is: %d, ver_res is: %d\r\n", (int)curr_disp_orientation, \
			lv_disp_drv.rotated,\
			lv_disp_get_hor_res(lv_disp), lv_disp_get_ver_res(lv_disp));

		switch(curr_disp_orientation)
		{
		case DISP_270_DEG:
			dev_y1 = (lv_disp_get_hor_res(lv_disp)-1)-(area->x1);
			dev_y2 = (lv_disp_get_hor_res(lv_disp)-1)-(area->x2);
			break;
		case DISP_180_DEG:
			dev_y1 = (lv_disp_get_ver_res(lv_disp)-1)-(area->y1);
			dev_y2 = (lv_disp_get_ver_res(lv_disp)-1)-(area->y2);
			break;
		case DISP_90_DEG:
			dev_y1 = area->x1;
			dev_y2 = area->x2;
			break;
		default:
			dev_y1 = area->y1; //in native orientation of the OLED, dev_y = area->y always
			dev_y2 = area->y2;
		}
}

The problem is that, lv_disp_get_hor_res() always return 64 and lv_disp_get_ver_res returns 128 from debug printf above. That means they are not changed even with lv_disp_drv.rotated set 1. Why lv_disp_get_hor/ver_res() not working?

Solution:

Now I have it working! I missed the critical function to update the driver with lv_disp_drv_update(lv_disp, &lv_disp_drv). Snippet to rotate the frame is shown below

if(rotate++ == 1000)
	  {
		  if (display_orientation_get()==DISP_270_DEG)
		  {
			  display_orientation_set(DISP_0_DEG);
			  lv_disp_drv.rotated = 0;
		  }
		  else if (display_orientation_get()==DISP_0_DEG)
		  {
			  display_orientation_set(DISP_90_DEG);
			  lv_disp_drv.rotated = 1;
		  }
		  else if (display_orientation_get()==DISP_90_DEG)
		  {
			  display_orientation_set(DISP_180_DEG);
			  lv_disp_drv.rotated = 0;
		  }
		  else if (display_orientation_get()==DISP_180_DEG)
		  {
			  display_orientation_set(DISP_270_DEG);
			  lv_disp_drv.rotated = 1;
		  }
		  lv_disp_drv_update(lv_disp, &lv_disp_drv); //this is critical!
		  rotate = 0;
		  lv_obj_invalidate(lv_scr_act());
	  }

Video to show the final result:

With this prototype, it would be possible to make a small display with orientation changed in runtime to follow the G-value returned from an accelerometer like a smartphone does.

1 Like

If you are interested in run-time changing of screen and touch controller orientation, it might be worth to look at this issue.
It has a different approach, it is handling orientation at the driver level.

Resolution change (horizontal <-> vertical switch) is not yet addressed in lvgl, though.
It would be nice if lvgl would initiate screen redraw if screen resolution changes as a result of orientation change.

It has been taken care of with two functions lv_disp_drv_update(lv_disp, &lv_disp_drv) & lv_obj_invalidate(lv_scr_act()). There is also a dummy function in my code display_orientation_get() that accelerometer readings is returned for run time angle information. But there is a problem. I posted a follow-up thread at Screen rotation like a smartphone in runtime

My OLED does not have driver support for orientation change. That is why I am seeking a solution in sw level.

I understood.
I am into pushing orientation handling (including the TC) into the driver level.
If you look at the issue I was referring to, you can see that support for this is there for the color TFTs and a couple of touch controllers on the ESP-IDF platform.
OLED (1 bit depth) screens are not yet supported.

I want the drivers to be platform agnostic (or at least available on multiple platforms).
What framework are you using for STM32L432?

I am interested in your code. Is it possible to see it in its entirety?

STM32L is programmed in STM32CubeIDE with its HAL.
The code will be available in GitHub but this moment it is still working in progress. The snippets I have posted in the thread Screen rotation like a smartphone in runtime exposes all relevant code for rotation related.

Thanks.
The approach how you handle orientation is seen from the snippets.
Mine is different. flush() is plain (does not bother with orientation). Upon orientation change, two things happen:

  • display controller is reprogrammed for new screen orientation
  • TC conversion matrix is recalculated (a conversion matrix is used to convert between TC and display coordinates)

I am interested in the drivers you are using, in the transport implementation (e.g. SPI) especially.
I came up with a new driver architecture. If you are interested, see the issue for details.
That is currently for ESP32 only. I want to extend it to other platforms. The next candidate is STM32. This is why I asked what framework you are using.

Can I have a look at the driver code somewhere?

oled_flush:

void oled_flush(int16_t y1, int16_t y2, const color_t* color)
{
	/**
	 * Simple SPI tx without FR synchronization or DMA
	 */
	uint8_t *buf_p = (uint8_t *)color;
	uint16_t buf_size = (y2-y1+1)*(OLED_HOR_RES>>3); //sending the whole line so that no x position required. 

	uint8_t cmd[3] = {0x21, (uint8_t)y1, (uint8_t)y2}; //segment range defined
	SPI_Write_Command((const uint8_t *)&cmd, 3);
	SPI_Write_Data(buf_p, buf_size);
}

Thanks. I see. The comment summarizes what is otherwise seen from the SPI function invocations. SPI traffic is fully synchronous with flush operation (flush returns only when all bits have already been sent on the wire).

My ESP32 drivers do this in the “background” with DMA. flush() returns right after the whole series of SPI transactions necessary for screen refresh are queued. lvgl can render the next screen while the previous is still in flight.

I want the same concept to be carried over to the STM32 drivers. For this reason, that will look quite different from the above.

Because it is a small screen of only 68*128 pixels in 1-bit depth, I can afford to use an array of 68x128/8 bytes as the buffer for drawing. The consequence is, only a single SPI transaction is required to update a widget.

At the end, I find the bug comes from the wrong assumption of coordinate transformation in lv_point for different screen angles. The function orientation_change() assumed LVGL know the screen orientation has changed and updated its (x,y) coordinates accordingly but this is not correct.

I think it is better to use a second buffer (or call it back buffer) to store the transformed draw buffer and get the back buffer updated to OLED by SPI.

The buffer for the whole screen in lvgl is 68x128 = 8704 bytes (bytes, because even at 1 bit color depth, lvgl uses 1 byte per pixel)
You need to send only 1/8 of this on the wire. It is 1088 bytes. At 10 MHz SPI clock rate, this takes 870 us (plus overhead).
Let’s see a case with a color TFT with 480*240 resolution at RGB888. Let’s assume that the buffer only takes 20 screen lines (as opposed to your case when the whole screen fits). In this case, you need to transfer 28.8 kbytes. At 10 MHz SPI clock, it takes more than 23 ms. And this is only 1/16 of the whole screen.

The above is just a comparison of monochrome OLED and color TFT.

This is not right. You need at least two SPI transactions (SPI_Write_command, SPI_Write_data) for one flush operation. But, this is not the point.
The point is if you do it synchronously (flush returns only after SPI transfer is complete), rendering is blocked by the transfer, lvgl can not start rendering the next screen.
If you do it asynchronously (no matter how many SPI transactions are needed), lvgl can start rendering the next screen almost right away while the SPI transport is still pumping bits on the wire using DMA. Screen rendering and pushing (a previously) rendered image to the display can be done in parallel.

I am sorry, but I can not comment on this and the rest. I have not played with this.

But I am using a buffer of 1kByte (not 8704kByte) as static lv_color_t buf1[(OLED_VER_RES)*(OLED_HOR_RES>>3)] to initialize the hall. OLED_VER_RES=128, OLED_HOR_RES=64.

See code below for initialization

void my_hal_init(void)
{
	my_init();

	static lv_disp_buf_t lv_disp_buf;

	/* @note: It is possible to use a smaller buf1[] but the side effect
	 * is a higher number of screen flushing is required
	 * */
	static lv_color_t buf1[(OLED_VER_RES)*(OLED_HOR_RES>>3)];

	lv_disp_buf_init(&lv_disp_buf, buf1, NULL, (LV_HOR_RES_MAX)*(LV_VER_RES_MAX));

	lv_disp_drv_init(&lv_disp_drv);

	if((display_orientation_get()==DISP_90_DEG) || (display_orientation_get()==DISP_270_DEG))
		lv_disp_drv.rotated = 1;
	else
		lv_disp_drv.rotated=0;

	lv_disp_drv.buffer = &lv_disp_buf;
	lv_disp_drv.flush_cb = my_flush_cb;
	lv_disp_drv.set_px_cb = my_set_px_cb;
	lv_disp_drv.rounder_cb = my_rounder_cb;

	lv_disp = lv_disp_drv_register(&lv_disp_drv);
}

You are right, it is 2x SPI transactions for command and data.

set_px_cb can bypass that limitation and make it use 1 bit per pixel.

Yes. It is clearly seen from the 3 bit shift to the right.
If LV_COLOR_DEPTH is set to 1, lv_color_t becomes a c union which has a size of one byte. This is the basic scenario. I was talking about this.
As @embeddedt highlighted, this can be optimized using set_px_cb, but I could see this only after you sent another snippet.

Anyways, this is not what I was trying to say. I was comparing the synchronous and the asynchronous transport methods and the benefits of the latter. I was citing the color TFT scenario, because the gain is more significant at that.

@techtoys

In the video, you are touching the screen and the display reacts. Apart from the OLED display, does that panel have a touch panel and a touch controller associated with that?

I am asking, because I have not seen lv_indev_drv_register() in your code.

What is the name of the whole board or the display module?
Can you give a link to that?

There is nothing special with the touch part so I have not mentioned it. Its initialization is the same as others except that the small oled doesn’t feedback with touch coordinate. Only gesture and a single touch response are available that is why I assigned it as an encoder instead of a pointer.

    lv_indev_drv_t touch_drv;
	lv_indev_drv_init(&touch_drv);
	touch_drv.type = LV_INDEV_TYPE_ENCODER; // LV_INDEV_TYPE_POINTER;
	touch_drv.read_cb = my_touch_cb;
	touch_indev = lv_indev_drv_register(&touch_drv);

The board is a custom board built in-house. Before the project is exposed on GitHub I really cannot say much about it. Sorry.

And yes, this OLED has touch and graphics combined in a single IC, so-called an incell OLED.