YAML 1.2 support

[kislyuk]

I maintain https://github.com/kislyuk/yq, and as seen in kislyuk/yq#10, the lack of YAML 1.2 support in this library is a hindrance to parsing YAML in a consistent "least surprise" manner. Recognizing the time constraints that the maintainers face, is there any roadmap or guidance on how YAML 1.2 support may be introduced into PyYAML (either from ruamel.yaml or otherwise)? Any implementation guidance for potential contributors who might work on a PR?

[Athanasius]

Having just started a project which is also my first use of YAML, and reading the official documentation, I've been wondering why "%YAML 1.2" at the start of the file was throwing errors. I take it there's still no support for this ?

[kislyuk]

Any updates on this issue?

[perlpunk]

I have some ideas on how to implement it.
I think the syntax is less important; most 1.2 syntax rules are already supported.

1. The resolving of booleans and numbers is easy, it's just a bit of work and needs good testing.

I created a temporary repo with test data for that: yaml-test-schema
It might get part of the yaml/yaml-test-suite at some point.

Apart from that we need:
2. Add a yaml_version option
3. Accept %YAML 1.2 and change the resolving rules depending on the %YAML directive

I could start with creating the resolvers/constructors, so that they can already be loaded explicitly.
Point 2 and especially 3 depend on if @ingydotnet wants that automatic behaviour.
Point 3 could be complicated because I would have to store a complete new set of resolvers and constructors somewhere.

In YAML::PP I implemented it like this. I added an option for the yaml version that can take the following values:

• 1.1 - Only support 1.1, ignore %YAML directive
• 1.2 - Only support 1.2, ignore %YAML directive
• [1.1, 1.2] - Support both and default to 1.1
• [1.2, 1.1] - Support both and default to 1.2

[ingydotnet]

@perlpunk what would be the default if no yaml_version is specified?

I think it needs to be 1.1, with a deprecation period.
The deprecation would look for things that change between 1.1 and 1.2 (like yes for true) and warn to either switch to 1.2 style or use yaml_version.
We'll want to do this same style deprecation for 1.3.

I'd like the default to eventually be the most recent version.

I like this plan to move to 1.2 support overall and suggest we move towards making it happen.

[perlpunk]

@perlpunk what would be the default if no yaml_version is specified?

1.1 for now

I think it needs to be 1.1, with a deprecation period.
The deprecation would look for things that change between 1.1 and 1.2 (like yes for true) and warn to either switch to 1.2 style or use yaml_version.

This could be problematic, and a lot of work. What should we do if we encounter 010, which is octal 8 in 1.1 and simply 10 in 1.2?
We would have to double check all input with both sets of regexes for each schema.

I like this plan to move to 1.2 support overall and suggest we move towards making it happen.

I experimented already a bit (with implementing point 1) and can try to push my branch the next days. It includes a lot of moving functions around, but isn't complicated.

[dHannasch]

I have some ideas on how to implement it.

Apart from that we need:
2. Add a yaml_version option
3. Accept %YAML 1.2 and change the resolving rules depending on the %YAML directive

I could start with creating the resolvers/constructors, so that they can already be loaded explicitly.
Point 2 and especially 3 depend on if @ingydotnet wants that automatic behaviour.
Point 3 could be complicated because I would have to store a complete new set of resolvers and constructors somewhere.

In YAML::PP I implemented it like this. I added an option for the yaml version that can take the following values:

• 1.1 - Only support 1.1, ignore %YAML directive
• 1.2 - Only support 1.2, ignore %YAML directive
• [1.1, 1.2] - Support both and default to 1.1
• [1.2, 1.1] - Support both and default to 1.2

When you're talking about adding a yaml_version option, you're talking about adding it as a kwarg to the likes of safe_load(), right? https://github.com/yaml/pyyaml/blob/master/lib3/yaml/__init__.py#L154

As far as how to implement it, couldn't we just take the existing YAML 1.2 parser from ruyaml/ruamel.yaml? That seems easier than translating the Perl version.

Indeed, in the short term we could probably enable the easiest case (adding a yaml_version kwarg) in five minutes by adding ruyaml as a dependency and just kicking it over to the 1.2 parser in that case. And then in the long term re-integrate them back into a single codebase.