Add pylint and typing across pyttb

[ntjohnson1]

This will probably need to be split across multiple sub-tasks/prs. The proposal and initial support landed here #54. They are not automatically enforced yet

[ntjohnson1]

I think for improving readability and extensibility starting with the tensor implementations is the way to go.
For that scoping I will track to this initial layout. For most of the files I'll type and lint in one go.

Linting

• tensor Tensor docs plus Linting and Typing and Black oh my #54
• sptensor Lint pyttb_utils and lint/type sptensor #77
• ktensor Ktensor lint type #186
• ttensor
• cp_als
• cp_apr
• hosvd
• tucker_als
• khatrirao
• pyttb_utils Lint pyttb_utils and lint/type sptensor #77

Typing

• tensor Tensor docs plus Linting and Typing and Black oh my #54
• sptensor Lint pyttb_utils and lint/type sptensor #77
• ktensor Ktensor lint type #186
• ttensor (Waiting on this one until the documentation refactor lands)
• pyttb_utils (This will probably wait until the tensor typing is complete)
• cp_als
• cp_apr
• hosvd
• tucker_als
• khatrirao

[dmdunla]

Is the linting working on the main branch at this point? When I tried the following using python 3.9.16, pytest 7.1.2, pylint 2.9.6, and mypy 0.981 on Windows:

pytest . --packaging

I got the following error:

================================================================ FAILURES ================================================================ 
______________________________________________________________ test_linting ______________________________________________________________ 

    @pytest.mark.packaging
    def test_linting():
        """Confirm linting of the project is enforce"""

        enforced_files = [
            os.path.join(os.path.dirname(ttb.__file__), f"{ttb.tensor.__name__}.py"),
        ]
        # TODO pylint fails to import pyttb in tests
        # add mypy check
        root_dir = os.path.dirname(os.path.dirname(__file__))
        toml_file = os.path.join(root_dir, "pyproject.toml")
        for a_file in enforced_files:
>           subprocess.run(f"pylint {a_file} --rcfile {toml_file}", check=True)

tests\test_package.py:32:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _  

input = None, capture_output = False, timeout = None, check = True
popenargs = ('pylint <ABS_PATH>\\pyttb\\pyttb\\tensor.py --rcfile <ABS_PATH>\\pyttb\\pyproject.toml',)
kwargs = {}, process = <Popen: returncode: 1 args: 'pylint <ABS_PATH>\\pyt...>, stdout = None, stderr = None      
retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command 'pylint <ABS_PATH>\pyttb\pyttb\tensor.py --rcfile <ABS_PATH>\pyttb\pyproject.toml' returned non-zero exit status 1.

..\..\..\Anaconda3\lib\subprocess.py:528: CalledProcessError
---------------------------------------------------------- Captured stderr call ---------------------------------------------------------- 
Traceback (most recent call last):
  File "<ABS_PATH>\Anaconda3\Scripts\pylint-script.py", line 10, in <module>
    sys.exit(run_pylint())
  File "<ABS_PATH>\Anaconda3\lib\site-packages\pylint\__init__.py", line 24, in run_pylint
    PylintRun(sys.argv[1:])
  File "<ABS_PATH>\Anaconda3\lib\site-packages\pylint\lint\run.py", line 316, in __init__
    linter.read_config_file(verbose=self.verbose)
  File "<ABS_PATH>\Anaconda3\lib\site-packages\pylint\config\option_manager_mixin.py", line 281, in read_config_file
    for option, value in values.items():
AttributeError: 'int' object has no attribute 'items'

[ntjohnson1]

I wasn't able to reproduce on my pre_commit branch where I've been simplifying some of the dev experience (still WIP) https://github.com/ntjohnson1/pyttb/tree/pre_commit

From the root of the project on Windows

$ conda create -n py39 python=3.9
$ conda activate py39
$ pip install -e ".[dev]"
$ pytest . --packaging

Which passes

[dmdunla]

OK, I'll wait for this and bypass linting tests for now. From: Nick ***@***.***> Sent: Monday, April 3, 2023 12:01 PM To: sandialabs/pyttb ***@***.***> Cc: Dunlavy, Daniel M ***@***.***>; Comment ***@***.***> Subject: [EXTERNAL] Re: [sandialabs/pyttb] Add pylint and typing across pyttb (Issue #63) I wasn't able to reproduce on my pre_commit branch where I've been simplifying some of the dev experience (still WIP) https://github.com/ntjohnson1/pyttb/tree/pre_commit<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fntjohnson1%2Fpyttb%2Ftree%2Fpre_commit&data=05%7C01%7Cdmdunla%40sandia.gov%7Ca71974d0db6f43d8d7aa08db346d8003%7C7ccb5a20a303498cb0c129007381b574%7C1%7C0%7C638161417124730283%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=xVH4ylaT9lxNjcg7reMyC%2BsGkL7BpQ%2FfAJg%2BtxgKkGE%3D&reserved=0> From the root of the project on Windows $ conda create -n py39 python=3.9 $ conda activate py39 $ pip install -e ".[dev]" $ pytest . --packaging Which passes - Reply to this email directly, view it on GitHub<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsandialabs%2Fpyttb%2Fissues%2F63%23issuecomment-1494748837&data=05%7C01%7Cdmdunla%40sandia.gov%7Ca71974d0db6f43d8d7aa08db346d8003%7C7ccb5a20a303498cb0c129007381b574%7C1%7C0%7C638161417124730283%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=PxoIl4wU5yDUvQyijzTMCDsjTocSLK%2FDtbCEyoRyj38%3D&reserved=0>, or unsubscribe<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAHIY2IT6Z557JUZJ6H2LHJDW7MF7LANCNFSM6AAAAAAVHPM5JY&data=05%7C01%7Cdmdunla%40sandia.gov%7Ca71974d0db6f43d8d7aa08db346d8003%7C7ccb5a20a303498cb0c129007381b574%7C1%7C0%7C638161417124730283%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=bKIJoHmYHulI9xzqm9PRjSj%2BxhS%2B%2FecRBUkd1UsZ4WE%3D&reserved=0>. You are receiving this because you commented.Message ID: ***@***.******@***.***>>

[ntjohnson1]

For clarification my branch shouldn't have any functional changes. You error looks like pylint itself is erroring out internally. I was just pointing out a fresh install doesn't see an issue (it was just easier to share the fresh install steps from my branch).