Best method to display a moving waveform?

How-to
1

Description

I’m working on an Audio project with a Teensy 4.1 and an ILI9341 that can playback WAV files and manipulate pitch and playback speed just like a Pioneer CDJ1000
I’ve got most of the audio parts working, and now starting a bit on the visuals

I’d like to display a two colored moving waveform on the display (currently on the ILI, later on, on a bigger display)
A video of what I want to display can be found here

What would be the best way to do this on v9.1? Use a chart? Can I layer one on top of the other for displaying peak & RMS value?
Or should I use a canvas + use an optimized draw function specifically for this?

If anyone has done something similar in the past, would love to see how it’s been implemented.

What MCU/Processor/Board and compiler are you using?

Teensy 4.x

What LVGL version are you using?

v9.1

2

I am not into audio, but could you show which part is needed?

3

Sorry, I should have been clearer in my description:
The moving waveform of the audio track
image

4

Try chart, it might work.

5

Question is, is the chart going to be responsive enough?
I know some users use an empty canvas object and use their own logic to draw, which in some cases would be faster.

I’ve used the chart before to display raw ADC & processed data, but never as a moving window

6

There are several factors limiting the real-time performance. Certainly, chart is a rather capable, but therefore a bit overcomplicated widget. Study the code in lv_chart.c, and you will see that there are many possibilities to optimize it if you only want to display an audio waveform. I’d suggest to copy the source, and remove all unnecessary code.

The second problem is the update speed, which means the time to transfer the pixels from memory to the display controller. What is the resolution of your display? Obviously it takes (much) longer to update a high-resolution screen. Do you use a parallel (i8080), or a serial (SPI) interface, or eventually RGB? Depending on the resolution SPI may not be fast enough for real-time update. Parallel may be OK, especially with 16 bits. In any case, you should use DMA for transferring the data. The best is to use the integrated LCD controller of the Teensy with either a 16-bit parallel, or an RGB interface.

The third problem is tearing. If the screen update is not synchronized with the hardware refresh (e.g., using the TE output of the controller, or reading the scanline), then the screen will not be updated completely in one refresh period, which results in visible tearing. This is especially bad if your display has a native portrait orientation, while you want to use it in landscape mode. This configuration causes a diagonal tearing, which is even more noticeable. This can be solved only by rotating the screen in memory (simply reconfiguring the controller is not enough), and updating the display in its native direction. However, this needs additional CPU time, thus it makes the update even slower. In general, if you have enough RAM, you should use direct mode with double buffering for the best result for animated display.

The fourth problem – somewhat related to this – is that if you use partial rendering in order to save memory, then you need to be aware that you should only update the screen when the whole screen has been refreshed, otherwise you will almost certainly run into tearing problems. This can be checked in the flush callback using lv_display_flush_is_last().

The fifth problem is the overall frame rate. To have smooth scrolling you’d need a rather high (>30 FPS) frame rate. This not only requires a fast processor (possibly using hardware graphics acceleration), but you need to configure LVGL for a low refresh period.

As you see, there’s more to it than just efficiently rendering the waveform to the frame buffer. Depending on your hardware, it may not be possible to get a smooth animation. But even with a capable hardware it need some consideration to configure your system.

The Teensy 4.1 is rather powerful, but I’m not sure how well the i.MXRT1062 is supported by LVGL. In any case, try to use a parallel interface instead of SPI.

Meanwhile check out the TGX demos for some smooth 3D animation on the Teensy. It would be an interesting project to integrate TGX into LVGL. Unfortunately, the LGPL license of TGX makes it difficult.

7

@zjazz thank you for the detailed response!

I’ve been a Teensy user for quite a few years now, even developed an automotive product based on a Teensy 4.x with an 8080 DMA enabled display driver I wrote a few years ago.

Right now I am just doing some initial testing on a Teensy 4.1, but I do have custom Teensies with 32MB SDRAM, eLCDIF pins exposed and more goodies

I think for now I will go with a 800x480 NE35510 display driven using a 16 bit 8080 bus with DMA for transfers - can easily get 30+ FPS out of that and don’t need full screen sized buffers - so updating only dirty areas will be fast.

As I do have large SDRAM available, I might just load the entire track’s waveform into a large buffer and just display a moving window rather than using a moving buffer.

I also have a small 320X240 display connected over SPI that is just showing the jog wheel/vinyl position:
image

8

Been playing with the lv_chart trying to display the waveform, but not having any luck!
The simplest (and I think most efficient way) would be to have some form of function that would allow to draw a vertical line, where all I have to do is give it X and Y coordinates for the start point, and a length parameter to determine it’s length, as well as the color of the line.
Something similar to this:
void ForceDrawVLine(uint16_t Xpos, uint16_t Ypos, uint16_t Length, uint32_t color)

I can draw this into a canvas or directly into the frame buffer - can someone guide me on how to achieve the above?

9

Just wanted to update that for now, I’ve gone with an lv_canvas and wrote a simple draw_vline function that will just loop over lv_canvas_set_px_color function.
This will draw a vertical line top to bottom, based on x,y coordinates and the length of the line

void lv_draw_vline(uint16_t x, uint16_t y, uint8_t length, lv_color_t color){
    // Loop from y downwards by `length` pixels
    for(int i = y; i < y + length; i++){
        lv_canvas_set_px(waveform_canvas, x, i, color, LV_OPA_100);
    }
}

It can be optimized, but for now it does the job.

EDIT - my function above needed a correction on the loop condition - works a charm now!

1 Like
10

image
image1024×768 62.4 KB

Here is the latest :slight_smile:
Blue lines displays amplitude, white displays RMS.

1 Like