413 when dealing with large array and infinite recursion errors...

[dragonpaw]

We recently had a ML process start crashing with no logs in Sentry, and were very worried...

Here's the end of the stack trace:

      File "/app/redacted/ml/trie/utils.py", line 80, in f
        return f(clst, idx + 1, acc)
      [Previous line repeated 974 more times]
      File "/app/redacted/ml/trie/utils.py", line 76, in f
        if len(clst) == 0 or idx == len(clst): return acc
    RecursionError: maximum recursion depth exceeded while calling a Python object

It seems like there's a bug in our ML that's making for a list that's huge and infinitely recursing, but this is what Sentry does with it:

[D 201023 00:58:30 connectionpool:939] Starting new HTTPS connection (1): [REDACTED].ingest.sentry.io:443
[D 201023 00:58:30 connectionpool:433] https://[REDACTED].ingest.sentry.io:443 "POST /api/[REDACTED]/store/ HTTP/1.1" 413 84