SystemError AST constructor recursion depth mismatch error on Python 3.11

[mikeckennedy]

Hey there. Love Chameleon. But since upgrading our site to run on 3.11, we've been getting a fair number of errors like this:

SystemError: AST constructor recursion depth mismatch (before=396, after=241)

on chameleon/astutil.py in parse at line 46. Any idea what's happening? Most of the time the pages run fine, but some of the time, we get this.

[malthe]

It might be that you've got the same template which is being rendered from different call sites such that the stack depth could vary between runs.

What happens if you use the environment variable CHAMELEON_EAGER=1 – ?

[mikeckennedy]

Thanks @malthe Here's what I get:

pyramid.httpexceptions.HTTPNotFound: The resource could not be found.

During handling of the above exception, another exception occurred:
    d['__loader'] = self._loader
AttributeError: 'PyramidPageTemplateFile' object has no attribute '_loader'

Does that tell you what's happening? I don't know what to make of this. Obviously, without this setting there is no HTTPNotFound error.

This code has been mostly unchanged for a long time and this error only started appearing on Python 3.11 (though it may have been hiding in there somewhere before).

[malthe]

@mikeckennedy I have fixed this issue; are you able to apply this patch to see if we're getting any result out of CHAMELEON_EAGER=1 now?

[mikeckennedy]

H @malthe i. I tried it out. I think it fixes it. Certainly no more errors that I could find. Thank you very much!

[malthe]

I'm closing this one out because I think the root cause is that the stack is too deep which is out of our control. Eager parsing is a remedy in this case (which has now been fixed).

[mikeckennedy]

Awesome, thanks. Do I need to set CHAMELEON_EAGER=1 on my servers? It sounds like I should.

[malthe]

@mikeckennedy released now as https://pypi.org/project/Chameleon/3.10.2/.

Yes – you would have to use that environment variables on your servers, simply because of how your app apparently works. Well, that's our "working theory" about it at least ...

[mikeckennedy]

Great, thanks for the info. That’s fine by me. Just want to make sure I’m “doing it right.” :)