Skip to content

vdk-jupyter: introduce Task Runner, a polling mechanism that runs tasks in the background and tracks their status#2869

Merged
yonitoo merged 39 commits intomainfrom
topic/ysalambashev/vdk-jupyter-task-runner
Nov 16, 2023
Merged

vdk-jupyter: introduce Task Runner, a polling mechanism that runs tasks in the background and tracks their status#2869
yonitoo merged 39 commits intomainfrom
topic/ysalambashev/vdk-jupyter-task-runner

Conversation

@yonitoo
Copy link
Copy Markdown
Contributor

@yonitoo yonitoo commented Nov 3, 2023 •
edited
Loading

What: Introduce Task Runner, a polling mechanism that runs tasks in the background and tracks their status.
Implements #2806 and #2807.

Why: Address the problem described in #2789.

Testing Done: CI/CD is passing. Introduced relevant tests

yonitoo added 10 commits October 24, 2023 17:40
@yonitoo
vdk-jupyter: introduce task runner class and update handlers
17c967b 
Signed-off-by: Yoan Salambashev <ysalambashev@vmware.com>
@yonitoo
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
b4d48a4
@yonitoo
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
d9174ce
@yonitoo
vdk-jupyter: adapt the polling mechanism to the frontend
c4d9765 
Signed-off-by: Yoan Salambashev <ysalambashev@vmware.com>
@yonitoo
vdk-jupyter: fix docstring of convert job
1bee478 
Signed-off-by: Yoan Salambashev <ysalambashev@vmware.com>
@yonitoo
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
3430f1c
@yonitoo
vdk-jupyter: fix Task Runner API-related problems
27de36a 
Signed-off-by: Yoan Salambashev <ysalambashev@vmware.com>
@yonitoo
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
9e25e32
@yonitoo
vdk-jupyter:
657fea0 
Signed-off-by: Yoan Salambashev <ysalambashev@vmware.com>
@yonitoo
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
3ed8caa
@yonitoo yonitoo marked this pull request as ready for review November 9, 2023 11:15
@murphp15
Copy link
Copy Markdown
Contributor

murphp15 commented Nov 10, 2023

I would much prefer if you pushed the formatting in a separate PR.

@dependabot @yonitoo
build(deps): Bump org.mockito:mockito-core
4bb6b40 
Bumps [org.mockito:mockito-core](https://github.com/mockito/mockito) from 5.6.0 to 5.7.0.
- [Release notes](https://github.com/mockito/mockito/releases)
- [Commits](mockito/mockito@v5.6.0...v5.7.0)

---
updated-dependencies:
- dependency-name: org.mockito:mockito-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Momchil Z and others added 3 commits November 13, 2023 08:51
@yonitoo
control-service: integration test for async job deploy (#2829)
5913607 
what: Add integration test that checks the status of the database when
using the new data jobs deployment synchronizer.

why: part of the
https://github.com/vmware/versatile-data-kit/tree/main/specs/vep-2272-complete-data-job-configuration-persistence
VEP

testing: n/a

---------

Signed-off-by: mrMoZ1 <mzhivkov@vmware.com>
Co-authored-by: github-actions <>
@dependabot @murphp15 @yonitoo
build(deps): Bump org.junit.jupiter:junit-jupiter-engine from 5.9.3 t…
01fa502 
…o 5.10.1 in /projects/control-service/projects/pipelines_control_service (#2881)

Bumps
[org.junit.jupiter:junit-jupiter-engine](https://github.com/junit-team/junit5)
from 5.9.3 to 5.10.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/junit-team/junit5/releases">org.junit.jupiter:junit-jupiter-engine's
releases</a>.</em></p>
<blockquote>
<p>JUnit 5.10.1 = Platform 1.10.1 + Jupiter 5.10.1 + Vintage 5.10.1</p>
<p>See <a
href="http://junit.org/junit5/docs/5.10.1/release-notes/">Release
Notes</a>.</p>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/junit-team/junit5/compare/r5.10.0...r5.10.1">https://github.com/junit-team/junit5/compare/r5.10.0...r5.10.1</a></p>
<p>JUnit 5.10.0 = Platform 1.10.0 + Jupiter 5.10.0 + Vintage 5.10.0</p>
<p>See <a
href="http://junit.org/junit5/docs/5.10.0/release-notes/">Release
Notes</a>.</p>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/junit-team/junit5/compare/r5.10.0-RC2...r5.10.0">https://github.com/junit-team/junit5/compare/r5.10.0-RC2...r5.10.0</a></p>
<p>JUnit 5.10.0-RC2 = Platform 1.10.0-RC2+ Jupiter 5.10.0-RC2 + Vintage
5.10.0-RC2</p>
<p>See <a
href="http://junit.org/junit5/docs/5.10.0-RC2/release-notes/">Release
Notes</a>.</p>
<p>JUnit 5.10.0-RC1 = Platform 1.10.0-RC1 + Jupiter 5.10.0-RC1 + Vintage
5.10.0-RC1</p>
<p>See <a
href="http://junit.org/junit5/docs/5.10.0-RC1/release-notes/">Release
Notes</a>.</p>
<p>JUnit 5.10.0-M1 = Platform 1.10.0-M1 + Jupiter 5.10.0-M1 + Vintage
5.10.0-M1</p>
<p>See <a
href="http://junit.org/junit5/docs/5.10.0-M1/release-notes/">Release
Notes</a>.</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/junit-team/junit5/commit/e5f50d8720741623915979ac154b65bcbcd6a577"><code>e5f50d8</code></a>
Release 5.10.1</li>
<li><a
href="https://github.com/junit-team/junit5/commit/ac86d18e9b15dbebe046e82743ac7f9534a17582"><code>ac86d18</code></a>
Fix typo in AfterAll documentation</li>
<li><a
href="https://github.com/junit-team/junit5/commit/388c5beaf42944961ab5b455c900d958a6e15588"><code>388c5be</code></a>
Harmonize application of method and field filters in search
algorithms</li>
<li><a
href="https://github.com/junit-team/junit5/commit/f82dd1e716f8717e012152b1d1d5cc0da10d33cd"><code>f82dd1e</code></a>
Apply field predicate before searching type hierarchy</li>
<li><a
href="https://github.com/junit-team/junit5/commit/1d1eb8571552bbf28e578241965010de6c8ee779"><code>1d1eb85</code></a>
Polishing</li>
<li><a
href="https://github.com/junit-team/junit5/commit/5ce280eff69b43759a3cb0c176207abe0a41b579"><code>5ce280e</code></a>
Update picocli to 4.7.5 and enable help width computation</li>
<li><a
href="https://github.com/junit-team/junit5/commit/fea05c3aa80de76686f326b5ce26ddf7f153ff5a"><code>fea05c3</code></a>
Fix ConsoleLauncherTests and StandaloneTests</li>
<li><a
href="https://github.com/junit-team/junit5/commit/c5567354c2556e772f8a0035ef7647161011d1c0"><code>c556735</code></a>
Use same expected files for all JDK versions</li>
<li><a
href="https://github.com/junit-team/junit5/commit/808493ab09b30970b506a48fda3d616ac1ba4fff"><code>808493a</code></a>
Run StandaloneTests for Java 8 under Java 8</li>
<li><a
href="https://github.com/junit-team/junit5/commit/9ec57661c78c3889db004ab6a89416e56a2fb2e0"><code>9ec5766</code></a>
Unify messages about exit codes in StandaloneTests</li>
<li>Additional commits viewable in <a
href="https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.1">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=org.junit.jupiter:junit-jupiter-engine&package-manager=gradle&previous-version=5.9.3&new-version=5.10.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: murphp15 <murphp15@tcd.ie>
@yonitoo
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
a8d9cf6
@yonitoo
Copy link
Copy Markdown
Contributor Author

yonitoo commented Nov 13, 2023

I would much prefer if you pushed the formatting in a separate PR.

I didn't realise it became such big part of the change. I might move it to a separate PR, yes.

yonitoo and others added 10 commits November 13, 2023 13:00
@yonitoo
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
eb00f46
@yonitoo
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
5ad5401
@yonitoo
vdk-jupyter: add Task Runner docstring and address review comments
a7b3c63 
Signed-off-by: Yoan Salambashev <ysalambashev@vmware.com>
@yonitoo
vdk-jupyter: add a comment about the polling default values
3c6c81b 
Signed-off-by: Yoan Salambashev <ysalambashev@vmware.com>
@yonitoo
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
a119d57
@yonitoo
vdk-jupyter: modify failing tests accordingly
5642ff6 
Signed-off-by: Yoan Salambashev <ysalambashev@vmware.com>
@yonitoo
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
3046f1b
@yonitoo
vdk-jupyter: make task ids unique and remove polling error threshold
b499fc9 
Signed-off-by: Yoan Salambashev <ysalambashev@vmware.com>
@yonitoo
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
945869e
@yonitoo
vdk-jupyter: pin react-icons dependency version to 4.11.0
26ffefa 
Signed-off-by: Yoan Salambashev <ysalambashev@vmware.com>
gabrielgeorgiev1
gabrielgeorgiev1 approved these changes Nov 14, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@gabrielgeorgiev1 gabrielgeorgiev1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, left 1 comment

@yonitoo yonitoo merged commit 23fd3cf into main Nov 16, 2023
@yonitoo yonitoo deleted the topic/ysalambashev/vdk-jupyter-task-runner branch November 16, 2023 10:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gabrielgeorgiev1 gabrielgeorgiev1 Awaiting requested review from gabrielgeorgiev1

@duyguHsnHsn duyguHsnHsn Awaiting requested review from duyguHsnHsn

@DeltaMichael DeltaMichael Awaiting requested review from DeltaMichael

1 more reviewer

@murphp15 murphp15 murphp15 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

cla-not-required

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@yonitoo @murphp15 @gabrielgeorgiev1 @duyguHsnHsn @DeltaMichael @vmwclabot