I could be wrong, but I believe MicroPython can manipulate the stack, so standard C calling conventions get thrown out the window. 
That’s stack unwinding. Same idea as C++ exceptions, implemented with setjmp etc.
You can catch and handle Micropython exceptions in C code.
See an example on cb_isr on espidf.c.
Another option, as mentioned above, is to reimplement modlvesp32 in Python (like done for stm32) and handle the exceptions in Python code.
The whole setup most of the time works but often locks up and sometimes I have even weirder effects like the advance_demo suddenly complaining that list indices are out of range or that some object in the touch driver has no append member.
Sounds like something is messing with memory or the stack. I’ll see if I can increase the stack size …
If it’s working then it’s working really great with great download speeds while the Chart demo runs. It’s IMHO really worth to have this running stable …
You can try changing MP_TASK_STACK_SIZE.
I’ve already increased it in lv_micropython to 32KB, which is a lot, but you can try increasing it even more.
There’s also a 4k stack passed to the httpd as a config parameter. I’ll change that first.
Attached is a patch of the current state. It can be applied to the current https://github.com/lvgl/lv_micropython
It runs the attached python script which expects a previous network setup in boot.py
It serves /index.html from python and will serve all files stored in /www under the uri /fs.
http_server.patch.txt (11 KB)
http_server.py.txt (3.6 KB)
This also includes the latest patch of gen_mpy.py which hasn’t made it into lv_micropython, yet.
Edit: This does not include any patch for modlvesp32.c and will thus not work at all under high graphics load.
For your entertainment, here are some of the error messages I see when running MP+LVGL with the httpd. The httpd itself seems to be fine as nothing fails if i request non-existent urls which httpd handles internally without callback to python.
Traceback (most recent call last):
File "xpt2046.py", line 209, in read
File "xpt2046.py", line 181, in get_coords
File "xpt2046.py", line 174, in get_med_coords
TypeError: 'httpd_req_t' object isn't iterable
SyntaxError: Can't convert httpd_req_t to lv_anim_t!
Traceback (most recent call last):
File "xpt2046.py", line 209, in read
File "xpt2046.py", line 181, in get_coords
File "xpt2046.py", line 170, in get_med_coords
File "xpt2046.py", line 162, in xpt_cmds
AttributeError: 'httpd_req_t' object has no attribute 'append'
Very funny how httpd_req_t sneaks into things that are totally unrelated.
Interestingly all of these involve httpd_req_t which is the object created by the httpd and which is then passed to MP through the callback …
How sure are we that m_new_obj() is ok to be called from a different context like the httpd?
When is it called from different context?
If it is called inside a scheduled function, then it is always called in the same context.
It’s called here
if(!mp_sched_schedule((mp_obj_t)&http_server_handler_cb_obj, NEW_PTR_OBJ(httpd_req_t, req))) { ... }
inside NEW_PTR_OBJ as this is needed to create the object that can then be passed to the scheduler.
If that is really the problem then I could try to store the req structure somewhere, create an empty httpd_req_t in the python callback and request this to be filled by native code from the stored req structure …
Edit: But I need some minimal context in the scheduled C function. How do I get that if this needs to be a python object and if i cannot create python objects from the httpd context?
Right.
I think it’s safe.
m_new_obj calls m_malloc which calls gc_alloc.
gc_alloc is protected with a mutex. (GC_ENTER)
So my guess is that it’s ok to call m_new_obj() from other threads.
EDIT - I’m assuming MP_STATE_MEM is the same for both contexts, so it’s worth making sure this is the case.
As you can see from my error messages, httpd_req_t shows up in places where it shouldn’t. So something is seriously being messed up somewhere … and it really feels like the python wrapper surrounding httpd_req_t overwrites things it shouldn’t. For some odd reason it likes to show up in the xpt driver. But that may just be because i did these tests at low load and the touch driver cobstantly being polled may just be the part most used in this case.
Can I somehow pass a Python object to the c side? I could avoid having to create the object then.
Yes, both sides seem to agree on the memory location of that and its contents.
The malloc is definitely related to this. This runs forever:
esp_err_t http_server_internal_handler(httpd_req_t *req) {
printf("req\n");
httpd_resp_sendstr(req, "<h1>Micropython test</h1>");
return 0;
}
This crashes after some time or triggers those weird python error messages:
esp_err_t http_server_internal_handler(httpd_req_t *req) {
printf("req\n");
httpd_resp_sendstr(req, "<h1>Micropython test</h1>");
void *p = NEW_PTR_OBJ(httpd_req_t, req);
printf("p = %p\n", p);
return 0;
}
Nothing else involved, no scheduler, no semaphores, no python callback … just the pure object creation …
Sure, but under the hoods the Python object you are passing is also created with m_new_obj so I don’t see how that would help.
On the C side a Python object is represented by mp_obj_t.
Maybe the object is garbage collected?
In such case its memory is allocated to another object. So when you write to it you overwrite some other Python object.
To prevent this, a reference to your object must be preserved somewhere on the Python side.
You can verify whether gc is related by trying to disable garbage collection by gc.disable(), and see if the problem still happens when gc is disabled.
Maybe. That would not hurt. I am not touching the object ever again. I am just creating it and then forget about it.
I am testing a very ugly solution which so far looks pretty good … I am not generating a new object for every callback. Instead I create it once to re-use it. So I keep a reference and replace the embedded pointer on every callback invocation,
But now I likely have a gc problem since I’ll never know when gc will destroy this object … I can of course keep a reference on python side but that will look confusing as I keep a reference to this object for no apparent reason.
You can keep it as another member of handler_data_t next to user_data.
The user won’t care about your object in the same way he doesn’t care about user_data, but gc will not collect it as long as the user holds handler_data_t.
Still… I wonder what we are missing here that is causing this issue.
I am doing exactly that. But how should gc know that this void pointer actually points to one of the objects its about to delete? MP IMO does not know anything about pointers I store inside handler_data_t.