Glad that we managed to get the jpeg library working for doing your encoding at least.
Yea, it certainly will be a time saver in the long run, since generation now really is superfast and that makes it easier to create multiple dataset versions with minor changes.
Hi,
I would be happy to take loo, but could you summarize the issue and send a code snippet for the last state and an image to test with? Do you see the problem in C too?
We tried a pure python jpeg encoder and also libjpeg-turbo and they both ended up having the same issue with the data collected from lv_screenshot.
this is the code that runs in python when a screen shot is takenā¦
import jpeg
import lvgl as lv
def take_screenshot(output_file: str, quality:int = 100):
"""Take a screenshot of a container using the LVGL snapshot API and save it to a JPG file."""
disp = lv.display_get_default()
width = disp.get_horizontal_resolution()
height = disp.get_vertical_resolution()
scrn = lv.screen_active()
lv.timer_handler()
snapshot = lv.snapshot_take(scrn, lv.COLOR_FORMAT.RGB888)
print(f"Snapshot: {snapshot} ({type(snapshot)}, {snapshot.data_size} bytes)")
data = snapshot.data.__dereference__(snapshot.data_size)
try:
jpeg.encode(data, output_file, width, height, quality)
except MemoryError as e:
print(e)
finally:
snapshot.destroy()
and this is the code code that runs to encode the jpeg.
#include <stddef.h>
#include <stdio.h>
#include <stdlib.h>
#include "jpeg_encoder.h"
#include "jpeglib.h"
#include "jerror.h"
void jpeg_encode(unsigned char *image, char *filename, int width, int height, int quality) {
struct jpeg_compress_struct cinfo;
struct jpeg_error_mgr jerr;
JSAMPROW row_pointer[1];
int row_stride;
cinfo.err = jpeg_std_error(&jerr);
jpeg_create_compress(&cinfo);
FILE * outfile; /* target file */
if ((outfile = fopen(filename, "wb")) == NULL) {
fprintf(stderr, "can't open %s\n", filename);
exit(1);
}
jpeg_stdio_dest(&cinfo, outfile);
cinfo.image_width = width;
cinfo.image_height = height;
cinfo.input_components = 3;
cinfo.in_color_space = JCS_RGB;
jpeg_set_defaults(&cinfo);
jpeg_set_quality(&cinfo, quality, TRUE);
jpeg_start_compress(&cinfo, TRUE);
// 1 BPP
row_stride = width * 3;
// Encode
while (cinfo.next_scanline < cinfo.image_height) {
row_pointer[0] = &image[cinfo.next_scanline * row_stride];
jpeg_write_scanlines(&cinfo, row_pointer, 1);
}
jpeg_finish_compress(&cinfo);
fclose(outfile);
jpeg_destroy_compress(&cinfo);
}
The c code along with libjpeg gets compiled as a user c module into MicroPython It is being compiled for the unix port.
Something is not being done properly in lv_snapshot that is causing the issue. We know itās not being caused by the jpeg encoder because 2 different ones were tried and end up with the same result. The problem is not every time either so I am not sure what is causing it.
As the image is generated via a design file, I can only provide that to recreate.
Here is a design file which causes the error on my system:
{
"ui": {
"window": {
"width": 640,
"height": 640,
"title": "Board Game Explorer"
},
"root": {
"type": "container",
"id": "mainFrame",
"width": 640,
"height": 640,
"options": {
"layout_type": "flex",
"layout_options": {
"flow": "column"
}
},
"style": ["WoodPanelStyle"],
"children": [
{
"type": "label",
"id": "titleLabel",
"width": 600,
"height": 50,
"options": {
"text": "Explore Board Games"
},
"style": ["GamePieceStyle"]
},
{
"type": "buttonmatrix",
"id": "gameSelection",
"width": 600,
"height": 150,
"options": {
"map": [
["Chess", "Monopoly", "Go"]
]
},
"style": ["CardStyle"]
},
{
"type": "textarea",
"id": "gameDescription",
"width": 600,
"height": 100,
"options": {
"text": "Select a game to learn more about its history and rules."
},
"style": ["CardStyle"]
},
{
"type": "button",
"id": "learnMoreBtn",
"width": 200,
"height": 50,
"options": {
"text": "Learn More"
},
"style": ["HighlightStyle"]
},
{
"type": "image",
"id": "gameBoardImg",
"width": 200,
"height": 200,
"options": {
"source": "path/to/image.jpg"
},
"style": []
}
]
},
"styles": {
"WoodPanelStyle": {
"bg_color": "#8B4513",
"text_color": "#FFFFFF",
"border_color": "#654321",
"border_width": 2,
"shadow_width": 4,
"shadow_color": "#000000",
"shadow_opa": 50
},
"GamePieceStyle": {
"bg_color": "#FFD700",
"border_color": "#B8860B",
"border_width": 1,
"shadow_width": 3,
"shadow_color": "#8B4513",
"shadow_opa": 75
},
"CardStyle": {
"bg_color": "#FFFFFF",
"text_color": "#000000",
"border_color": "#000000",
"border_width": 1,
"line_dash_width": 1,
"line_dash_gap": 2
},
"HighlightStyle": {
"bg_color": "#32CD32",
"text_color": "#FFFFFF",
"border_color": "#228B22",
"border_width": 1
}
}
}
}
The design file is loaded into the generator via the following command:
poetry run inv generate-design --design-file <path_to_design_file>/design.json
Or alternatively, by directly calling the script with the compiled micropython:
./lv_micropython/ports/unix/build-standard/micropython src/main.py -m design -n -o screenshot.jpg -f <path_to_design_file>/design.json
The file can be created under the tmp directory, which is ignored by git, but generally it doesnāt matter where it resides.
The repository contains the latest commits with the C module added, however I havenāt updated the build script to work properly, which is why it needs to be compiled regularly:
cd lv_micropython
make -C mpy-cross
make -C ports/unix submodules
make -j4 -C ports/unix USER_C_MODULES="$(pwd)/custom_mod" V=1
The relevant code for generating from the design file is in design_parser.py (for parsing the JSON) and widget.py (for creating the individual widgets)
Thank you for the summary.
Finally what is the actual problem with the jpeg output?
Can you try encoding a normal image instead of a snapshot?
The problem is, the output image is sheared/distorted, but hints of what it might look like are there, suggesting some kind of data misalignment or similar:
I donāt have a normal encoded image, although there are working examples of snapshots:
Iāll try to provide one later when I get to it.
this doesnāt happen all the time right?
It happens all the time for specific examples I have noticed:
I noticed, that the distortion error is not entirely random.
It can be reproduced given certain JSON design files, and it only occurs in design mode. I would not go directly to the conclusion that it happens entirely due to the snapshot API or LVGL internally. But I cannot verify it either, as from a theoretical standpoint, the snapshot should work regardless since the corresponding UI from the JSON can be displayed.
The code for lv_snapshot_take_to_draw_buf really doesnāt make any sense in what it is doing.
It creates a draw buffer that has the correct size taking into account the value returned from _lv_obj_get_ext_draw_size. that value is then applied to the draw buffers width and height which has already has that value factored in. This would skew the output like what is being seen.
1 Like
Itās funky how the code is reading. The same work is being performed multiple times so there ends up being entry for something to be off. I have to write up a flow of what is happening with the data to see what is exactly going on.
It seems the width of the result image is not set correctly. My guess is that LVGL is configured the LV_COLOR_DEPTH 32 but the JPEG is 24 bit (RGB888).
The lv_conf.h is in the repository and color_depth is set to 24:
#define LV_COLOR_DEPTH 24
But it might be that something went wrong in compilation, since in my build script I am overwriting the lv_conf.h in the lv_micropython submodule.
Let me retry.
I donāt believe that LV_COLOR_DEPTH has anything to do with the snapshot widget.
I do have a question tho.
in lv_snapshot.c in the lv_snapshot_take_to_buf function there is this codeā¦
layer.buf = buf;
layer.buf_area.x1 = snapshot_area.x1;
layer.buf_area.y1 = snapshot_area.y1;
layer.buf_area.x2 = snapshot_area.x1 + w - 1;
layer.buf_area.y2 = snapshot_area.y1 + h - 1;
layer.color_format = cf;
layer._clip_area = snapshot_area;
Why is this done like it is? It would be better if it was
layer.buf = buf;
layer.buf_area = snapshot_area;
layer.color_format = cf;
layer._clip_area = snapshot_area;
I am not sure of the comments are incorrect or not but here is another location that could be causing the issue.
in the lv_layer_t structure there is a comment for the buf_area field that states /** The absolute coordinates of the buffer */
Absolute would be x1=0, y1=0, x2=buffer_width, y2=buffer_height
but what is getting fed into this field is a relitive location of the object to itās parent. That is seen in this code
lv_area_t snapshot_area;
lv_obj_get_coords(obj, &snapshot_area); <=== object coords are relative to it's parent
lv_area_increase(&snapshot_area, ext_size, ext_size);
lv_memzero(buf, buf_size);
dsc->data = buf;
dsc->data_size = buf_size_needed;
/*Keep header flags unchanged, because we don't know if it's allocated or not.*/
dsc->header.w = w;
dsc->header.h = h;
dsc->header.cf = cf;
dsc->header.magic = LV_IMAGE_HEADER_MAGIC;
lv_layer_t layer;
lv_memzero(&layer, sizeof(layer));
layer.buf = buf;
layer.buf_area.x1 = snapshot_area.x1; <==== relative coords are set
layer.buf_area.y1 = snapshot_area.y1;
layer.buf_area.x2 = snapshot_area.x1 + w - 1;
layer.buf_area.y2 = snapshot_area.y1 + h - 1;
This is in the lv_snapshot_take_to_buf function in lv_snapshot.c
The other thing is in the lv_layer_t structure you have the _clip_area field and that field has a specific comment that states always the same or smaller than buf_area. Now maybe I am wrong in this but it would appear that buf_area is smaller than _clip_area by one pixel horizontally and one pixel vertically.
say the object returns this for itās width and height.
width = 200
height = 200
and say the object has this set for itās position
x = 150
y = 150
that would mean the lv_area_t as populated by lv_obj_get_coords would be
x1=150
y1=150
x2=350
y2=350
and buf_areaās x2 and y2 are being set to 150 + 200 - 1 = 349 which is smaller than what _clip_area has set for x2 and y2 which is 350.
Maybe that is where the issue is coming from?