show diff of files which will be modified with --dry

[khimaros]

Is your feature request related to a problem? Please describe

it would be helpful to see specifically what changes will be made to a file before doing it. the only tool i've seen do this well is slack, but i miss it every time.

Describe the solution you'd like

when running in --dry mode (or perhaps with a new flag --diff), show the files which would be changed by a deploy and the differences between the old and new file.

[Fizzadar]

Love this idea. As part of tackling #805 I am going to overhaul the diff output to be more verbose (think terraform).

[xvello]

I managed to hack a prototype by wrapping files.put and running the diff there. It's inefficient because it needs to always pull the remote file (even if the sha1 matches), implementing it within files.put would remove this issue.

Here's my current code, feel free to reuse under MIT (click to unfurl). I'd be happy to try cleaning it up and turning it into a PR after we decide how the output should look and how to turn it on.

 

import difflib
from io import BytesIO
from typing import IO, Any, Generator, Optional

import click
from pyinfra import host, logger
from pyinfra.api.operation import OperationMeta
from pyinfra.api.util import get_file_io
from pyinfra.facts.server import Hostname
from pyinfra.operations import files


def put(
    src: str | IO[Any], dest: str, name: Optional[str] = None, present: bool = True, **kwargs: Any
) -> OperationMeta:
    if not present:
        return files.file(path=dest, present=False, name=(name or f"Delete {dest}"))

    current_contents = BytesIO()
    current_lines: list[str] = []
    try:
        if host.get_file(dest, current_contents):
            current_lines = current_contents.getvalue().decode("utf-8").splitlines(keepends=True)
    except FileNotFoundError:
        pass

    result = files.put(src, dest, name=name, **kwargs)

    if result.changed:
        hostname = host.get_fact(Hostname)
        if current_lines:
            logger.info(f"\n    Will modify {click.style(dest, bold=True)} on {hostname}:")
        else:
            logger.info(f"\n    Will create {click.style(dest, bold=True)} on {hostname}:")

        with get_file_io(src, "r") as f:
            desired_lines = f.readlines()
        for line in generate_color_diff(current_lines, desired_lines):
            logger.info(f"  {line}")
        logger.info("")

    return result


# Customized copy of difflib.unified_diff, added color, removed diff header
def generate_color_diff(current_lines: list[str], desired_lines: list[str]) -> Generator[str, None, None]:
    def _format_range_unified(start: int, stop: int) -> str:
        beginning = start + 1  # lines start numbering with one
        length = stop - start
        if length == 1:
            return "{}".format(beginning)
        if not length:
            beginning -= 1  # empty ranges begin at line just before the range
        return "{},{}".format(beginning, length)

    for group in difflib.SequenceMatcher(None, current_lines, desired_lines).get_grouped_opcodes(2):
        first, last = group[0], group[-1]
        file1_range = _format_range_unified(first[1], last[2])
        file2_range = _format_range_unified(first[3], last[4])
        yield "@@ -{} +{} @@".format(file1_range, file2_range)

        for tag, i1, i2, j1, j2 in group:
            if tag == "equal":
                for line in current_lines[i1:i2]:
                    yield "  " + line.rstrip()
                continue
            if tag in {"replace", "delete"}:
                for line in current_lines[i1:i2]:
                    yield click.style("- " + line.rstrip(), "red")
            if tag in {"replace", "insert"}:
                for line in desired_lines[j1:j2]:
                    yield click.style("+ " + line.rstrip(), "green")

[olfway]

I think it's pretty important feature, in ansible I always run it with --check first to review changes
It's dangerous to overwrite sshd_config without checking the difference first

Also I think it should be a standard way to report differences from operations, so for example postgresql.role operation can report what changes it will do to existing role

[spacez320]

Just started using Pyinfra and am loving it, but quickly came looking for this in the issue backlog when I realized there was no way to diff files. I'd be happy to help with contributions if this is still something that seems like a good idea to introduce.

[khimaros]

@Fizzadar is this still on your radar? i see that #805 was closed. does that make this feature easier to implement?

[mpolden]

I started migrating a rather large Ansible playbook to Pyinfra, and I really like how it works so far.

However, the ability to show diffs in dry-run mode is essential, and would simplify my migration process as I can tell if I ported a given role correctly based on the diff. It is also generally useful to verify files that are dynamically built (templates).

[jgelens]

Implemented @xvello's solution here: #1385