Description
Convert some Makefiles to use Python's Invoke project
Most of the Makefiles for StackStorm projects don't actually build any artifacts (literally the only thing make was designed to do) so they function more as a developer interface to do common tasks. In the st2 repo, we use make to do things like generate files, run lint tests, run unit tests (Python 2 or Python 3?), run integration tests (Python 2 or Python 3?), run packs tests (Python 2 or Python 3?), generate coverage results, and even push new versions to PyPI.
But treating make as a glorified task runner is problematic at best. How many ways can you run tests? Currently, we have ten different user-friendly make targets for running tests:
pytestsunit-testsitestsmistral-itestsorquesta-itestspacks-testsrunners-testsrunners-itestsci-py3-packs-testsci-packs-tests
That's kind of a lot. And if we include intermediate targets as well (the ones starting with dots), the list gets very large:
testspytests.pytests.pytests-coverageunit-tests.unit-tests.run-unit-tests-coverage.combine-unit-tests-coverage.report-unit-tests-coverage.unit-tests-coverage-htmlitests.itests.run-integration-tests-coverage.combine-integration-tests-coverage.report-integration-tests-coverage.integration-tests-coverage-html.itests-coverage-htmlmistral-itests.mistral-itests.run-mistral-itests-coverage.mistral-itests-coverage-htmlorquesta-itests.orquesta-itests.orquesta-itests-coverage-htmlpacks-tests.packs-testsrunners-tests.runners-testsrunners-itests.runners-itests.runners-itests-coverage-htmlci-py3-packs-testsci-packs-tests
Here's a slightly outdated visualization of the st2 Makefile:
(it has only become more complicated since I created this)
A great deal of that complication is frankly, my fault, because I added make targets to run tests with coverage, combine disparate coverage results, and generate coverage reports, and every one of those was at least one make target. And I say at least one make target because I had to duplicate make targets for some of the user-friendly targets that run tests, since make does not allow you to pass in arguments to targets.
Ideally, "run tests with coverage" should run tests the exact same was as "run tests without coverage", which means that the all of the targets to "run tests" should accept options determining whether or not they should run with coverage - they shouldn't be duplicate make targets.
The same is true for running with Python 2 and Python 3. How exactly those tests are run shouldn't differ greatly, so that, too, should be an option passed to the make targets.
Enter Python's Invoke project, which is just a task runner - it was split out of Fabric along with paramiko. All it does is run tasks. Some of those tasks have prerequisites that must be run before, like creating a virtualenv before installing all of the packages in the test requirements, or running tests with the coverage option turned on before generating coverage reports, and Invoke handles all of that perfectly fine.
Furthermore, you can break up your Invoke task modules and put it under the tasks/ directory. That lets us keep file sizes small and focused. To see the tests that handle installing build, development, and test requirements, check out the modules in tasks/requirements/. The tasks.travis module handles all Travis-specific configuration. Just trying to build the project? tasks.build is where it's at. Decomposing tasks into logical modules makes it much easier to see, at a glance, what is going on.
Invoke also auto-generates a list of targets to display, and also uses function docstrings to autogenerate help text for tasks when passed with the --help flag. You can get make to do that, but it requires a lot of hand holding and manually listing the targets and their purposes. With Invoke you get that for free.
Finally, we don't have to completely trash all of our Makefiles in all of their glory complexity. On the contrary, they are still used by things like Travis-CI and fellow developers. But we can move most of their "meat" into Invoke, and then cleverly use the .DEFAULT target to check for and install the virtualenv with Invoke in it, pass in the entire target string into Invoke, and let Invoke handle the rest.
Here's is an excerpt from the Makefile in my PR:
virtualenv-components:
virtualenv --python=$(PYTHON_VERSION) --no-site-packages $@ --no-download
virtualenv-st2client:
virtualenv --python=$(PYTHON_VERSION) --no-site-packages $@ --no-download
$(VIRTUALENV_DIR)/bin/invoke: $(VIRTUALENV_DIR)
. $(VIRTUALENV_DIR)/bin/activate && pip install invoke
.PHONY: invoke
invoke: $(VIRTUALENV_DIR)/bin/invoke
.DEFAULT:
@# Manually make virtualenv target
if [ ! -d $(VIRTUALENV_DIR) ]; then make virtualenv; fi
@# Manually make invoke target
if [ ! -e $(VIRTUALENV_DIR)/bin/invoke ]; then make invoke; fi
. $(VIRTUALENV_DIR)/bin/activate && invoke $@
@#. $(VIRTUALENV_DIR)/bin/activate && echo $$PYTHONPATHIt's a little dirty (explicitly recursively calling make has been "considered harmful"), but every Makefile is a little dirty, and this Makefile is a lot more clean than it used to be. With this excerpt, make will create the virtualenv if it doesn't exist, install Invoke into it, and then call invoke with all of the make targets (can somebody say command line "options")!?
I'm not suggesting that we immediately replace all Makefiles with Invoke, and I'm not even suggesting that we entirely get rid of all of our Makefiles, but I do think we can dramatically reduce the size and complexity of our task runners if we convert the bulk of make targets over to Invoke tasks. Here are a few candidate repositories with relatively small or straightforward Makefiles:
Invoke is not difficult or complex to learn - it's actually pretty well designed. And one thing it certainly does not have is arbitrary (and somewhat ungoogleable!) special variables ($@???, $*???) or arbitrary special task/target names (.PHONY? .DEFAULT?). So each new developer, and especially newbie developers coming to StackStorm will have to learn a single well designed, straightforward, boringly simple task runner framework (Invoke), or they will have to learn the old fashioned, inscrutable, syntax of a Makefile and dig through our overly complex Makefile to figure out what it's doing. I get it - make builds on top of Bash, and Bash is a great task runner, but the additional layer of make dramatically complicates what should be simple tasks. Invoke is pure Python, and makes running tasks a lot better.