Security fixe reported by github

[paulocoutinhox]

Hi,

I received this warning message:

[paulocoutinhox]

I think that it was fixed when use 4.2b2.

Thanks.

[lfaraone]

Hi! This means you need to update your project to the newer, fixed version of pyyaml.

[joekohlsdorf]

So what is the security issue? I don't see anything in the changelog and there are only beta releases of 4.1.
Maybe the fix should be backported to the 3.x branch.

[RealOrangeOne]

There's a writeup in #193 about why the issue has come to light.

TL;DR:

Prior to #74, yaml.load could run arbitrary python when parsing yaml, by design. #74 changed this so that by default, load didnt run python, and instead was an alias to safe_load. In this release, this was reverted, which means it's now a known security issue, hence the release of a security notice (CVE-2017-18342).

How to fix:

If you don't want people running arbitrary python (you probably don't), then change uses of yaml.load to yaml.safe_load.

#193 discusses the idea that an API change should be made to make pyyaml safe by default, which will be a new major version.

[joekohlsdorf]

I think you should work with the Github security team to pull this security notice as long as there is no released version of a fix.

[RealOrangeOne]

@joekohlsdorf but there is a security issue? It's not so much that there's no fix, it's just you use the library in a different way. That's still a fix

[joekohlsdorf]

Except that I have no way of knowing which code path external libraries in my project take and no direct way of fixing it except monkey-patching.

[RealOrangeOne]

Then it's not your job to fix it, it's the job of your dependencies. Then when they release a new version, you update to that. I agree GitHub notifications are kinda annoying like this, but it's the right way to do it. Otherwise you end up not being told of issues in your software, which you may need to be aware of.

[umeboshi2]

I spent some time last year in correspondence with github explaining that reporting security issues that were actually false positives limits the effectiveness of this new service. It seems to be the original intent of the authors to provide the .safe_load when the developer knows they will be dealing with remote or arbitrary yaml files. In fact, projects like ansible have been using the .safe_load for a very long time, since the developers know that the yaml configs people use would be potentially coming from many sources. I still think that when a project developer knows that they will be handling potentially dubious input, they should be actively searching for input validation features in their solution. This issue is really just an informative advisory, unless the user of the library expects it to perform input validation without actually checking for this feature before they use it. The lxdock/lxdock#160 issue above really helps describe the usefulness of the alert, and a claim of the alert being spurious is made here marshmallow-code/apispec#278. I had also been notified that bootstrap had a vulnerability where the recommendation was to upgrade to the latest bootstrap4, yet the existence of the vulnerability was not verified in bootstrap-3.3.7, so those repos also received spurious alerts. It seems that github will recommend upgrading to a version that would break functionality before actually making a determination that a vulnerability even exists in the used version. It seems that github eventually removed the alerts for bootstrap3 only after the security provider narrowed the affected versions to > 4.0.

[sacha-senchuk]

I feel it's pretty normal Github notifies with an alert about these kind of issues.

I don't see how developers are supposed to know that yaml.load is unsafe. Intuitively it should work in the same way as json.loads does, so I guess a lot of developers might completely miss the security issue.

[RealOrangeOne]

Simple, the manual!

Warning: It is not safe to call yaml.load with any data received from an untrusted source! yaml.load is as powerful as pickle.load and so may call any Python function. Check the yaml.safe_load function though.

https://pyyaml.org/wiki/PyYAMLDocumentation

[sacha-senchuk]

Sure, if you go beyong data = load(stream, Loader=Loader) and I didn't…

And if you have time to read comments on stackoverflow…

Anyway, excellent news that this is safe by default now. This is how it should have been!

[stavmeir]

Is there an expected release date to a stable 4.2 version?
The latest pre-release version, 4.2b4, was released half a year ago and a stable version would stop the GitHub security notifications and save people from refactoring every yaml.load to yaml.safe_load without using a pre-release versions.