Documentation and helpers for using galaxy-selenium package within Jupyter

[jmchilton]

Second crack at #10784, removes some of the more "magical" aspects of that PR (including notebooks as test cases) and replaces it with detailed documentation developed writing real-world test cases that have been merged recently.

Some things included in the detailed new pydocs:

• The new Makefile-based command make serve-selenium-notebooks will serve example notebooks in lib/galaxy_test/selenium/jupyter.
• The notebooks will target a Galaxy host running on port 8080 - but are parameterized to be run with papermill (https://papermill.readthedocs.io/) for automate targeting different Galaxy servers, remote Selenium servers, etc..
• Screenshots will be embedded right into the notebook.
• Example notebooks setup to target just the library code in galaxy-selenium as well as one appropriate to leverage the Galaxy testing framework.

This PR also includes two smaller commits - one that drops the deprecated, unused variant of the navigation data from casperjs and one that links some recent pydocs into the docs.

[OlegZharkov]

Thanks a lot John, this is really great and extremely useful! It's really convenient and dramatically speeds-up selenium test development.
There are just few minor details, I'd like to address.

If we're using jupyter, why don't we put it in requirements? Also would be useful to start jupyter already with activated .venv like we do in run_test.sh. (Just happen to me and I was wondering why does it produce "missing modules" error)

Is there a way to automatically deploy testing server with this? Already with mounted test_data, configured temp folders and triggered @classmethod from selenium file. So what I ended up doing, was running a test using run_test.sh with

  for i in range(99999):
            self.sleep_for(self.wait_types.UX_RENDER)

and then running Jupyter on the server deployed by run_test.sh

Is there an easier/less-hacky way to do this?

We of course can use functions from navigates_galaxy, populators and etc. However is there a way to run functions from framework and util functions from regular selenium tests? Like, if I work on annon_history and I want to run util function. Can I do it without making it global?

In general, this works great, thanks a lot for your work!

[jmchilton]

If we're using jupyter, why don't we put it in requirements?

I'd like to avoid putting it in the requirements for now I think, it is a big dependency and this is still a relatively niche use case. I will update the docs to mention how to install it though.

Is there a way to automatically deploy testing server with this? Already with mounted test_data, configured temp folders and triggered @classmethod from selenium file.

There is not, my tests didn't use classmethods like that. It is a good question though. You could put _upload_file_anonymous_then_register_user into navigates_galaxy.py as like a helper you can call in the classmethod and from the Selenium context. But this is effectively making it global like you said.

One thing that would make your approach a little less hacky is to set GALAXY_TEST_PORT=8080. Then you wouldn't have to change that top cell.

We could should consider a GALAXY_TEST_SERVE environment variable to do that. I wonder how best to implement that 🤔.

[OlegZharkov]

thanks a lot :)

[OlegZharkov]

One thing that would make your approach a little less hacky is to set GALAXY_TEST_PORT=8080. Then you wouldn't have to change that top cell.

I wish, I knew it earlier. What about framework.py? Is it somehow possible to use it from jupyter ?

[jmchilton]

This (https://github.com/galaxyproject/galaxy/pull/11156/files#diff-9a67f84d90f9581ba096f5fab3b42e8a554feb8fc38fbf7570e6bcdabf3f164dR185) added a bunch of important things into that context available to Jupyter.

If there are other things you want in that context I think it could be added like this, but most of the rest of that stuff is very test case specific.

diff --git a/lib/galaxy_test/selenium/framework.py b/lib/galaxy_test/selenium/framework.py
index 9067c18d32..6905e228c4 100644
--- a/lib/galaxy_test/selenium/framework.py
+++ b/lib/galaxy_test/selenium/framework.py
@@ -200,6 +200,27 @@ class GalaxyTestSeleniumContext(GalaxySeleniumContext):
         """A workflow populator connected to the Galaxy session described by Selenium context."""
         return SeleniumSessionWorkflowPopulator(self)

+    def workflow_upload_yaml_with_random_name(self, content, **kwds):
+        workflow_populator = self.workflow_populator
+        name = self._get_random_name()
+        workflow_populator.upload_yaml_workflow(content, name=name, **kwds)
+        return name
+
+    def assert_initial_history_panel_state_correct(self):
+        # Move into a TestsHistoryPanel mixin
+        unnamed_name = self.components.history_panel.new_name.text
+
+        name_element = self.history_panel_name_element()
+        assert name_element.is_displayed()
+        assert unnamed_name in name_element.text
+
+        initial_size_str = self.components.history_panel.new_size.text
+        size_selector = self.components.history_panel.size
+        size_text = size_selector.wait_for_text()
+        assert initial_size_str in size_text, f"{initial_size_str} not in {size_text}"
+
+        self.components.history_panel.empty_message.wait_for_visible()
+

 class TestWithSeleniumMixin(GalaxyTestSeleniumContext, UsesApiTestCaseMixin):
     # If run one-off via nosetests, the next line ensures test
@@ -347,21 +368,6 @@ class TestWithSeleniumMixin(GalaxyTestSeleniumContext, UsesApiTestCaseMixin):
     def timeout_multiplier(self):
         return TIMEOUT_MULTIPLIER

-    def assert_initial_history_panel_state_correct(self):
-        # Move into a TestsHistoryPanel mixin
-        unnamed_name = self.components.history_panel.new_name.text
-
-        name_element = self.history_panel_name_element()
-        assert name_element.is_displayed()
-        assert unnamed_name in name_element.text
-
-        initial_size_str = self.components.history_panel.new_size.text
-        size_selector = self.components.history_panel.size
-        size_text = size_selector.wait_for_text()
-        assert initial_size_str in size_text, f"{initial_size_str} not in {size_text}"
-
-        self.components.history_panel.empty_message.wait_for_visible()
-
     def admin_login(self):
         self.home()
         self.submit_login(
@@ -371,12 +377,6 @@ class TestWithSeleniumMixin(GalaxyTestSeleniumContext, UsesApiTestCaseMixin):
         with self.main_panel():
             self.assert_no_error_message()

-    def workflow_upload_yaml_with_random_name(self, content, **kwds):
-        workflow_populator = self.workflow_populator
-        name = self._get_random_name()
-        workflow_populator.upload_yaml_workflow(content, name=name, **kwds)
-        return name
-
     def ensure_visualization_available(self, hid, visualization_name):
         """Skip or fail a test if visualization for file doesn't appear.

We could also potentially add UsesHistoryItemAssertions to the JupyterTestContextImpl if that'd be useful.

[OlegZharkov]

Thanks a lot! This is really great!

[bgruening]

Pretty cool and a great idea!