Time to test is too high for backend integration tests

[JanCizmar]

Heads up: This task requires significant knowledge of Gradle and Spring Boot. If you have never worked with these technologies, this is probably not an issue for you, and attempting it might be a waste of your time.

When I run integration test e.g. (io.tolgee.api.v2.controllers.v2ProjectsController.ProjectsControllerTest) it takes too much time on my high end MacBook Pro M3.

• 10.7 seconds for subsequent build without any change to codebase
• 11.7 seconds to start the spring boot context
  Started ProjectsControllerTest in 10.551 seconds (process running for 11.711)

Lower time test is crucial for our developer experience. We need to get time to test for integration tests under 10 seconds (build + context load).

• Tune up gradle build, so subsequent builds are immediate (or fast as possible)
• Tune up Spring Integration Tests, so it doesn't take so long to start the context. We need to get under 5 seconds.

To setup the project, follow this guide.

[JanCizmar]

/bounty $300

[algora-pbc]

💎 $1,000 bounty • Tolgee

## 💎 $100 bounty • Tolgee

Steps to solve:

1. Start working: Comment /attempt #2443 with your implementation plan
2. Submit work: Create a pull request including /claim #2443 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to tolgee/tolgee-platform!

Add a bounty • Share on socials

Attempt	Started (GMT+0)	Solution	Info
🟢 @Vayras	Aug 26, 2024, 2:07:59 PM	WIP
🔴 @c0d33ngr	Sep 2, 2024, 6:29:25 AM	WIP
🟢 @exi	Sep 6, 2024, 7:29:12 AM	#2443	Link

[Vayras]

@JanCizmar can I get assigned to this I have almost one year of testing experience now, and have worked testing for wiremock, stakwork and nabers

[Vayras]

/attempt #2443

Options

• Cancel my attempt

[JanCizmar]

Hey! Thanks for attempting. I don't know if wiremock experience will help here much. This task will involve a lot of Spring Boot and Gradle. I want to keep the task unassigned. I want to keep space for other contributors. Please update me if you proceed in any way with this, and then I will assign you. 🚀

[c0d33ngr]

/attempt #2443

Options

• Cancel my attempt

[exi]

/attempt #2443

Algora profile	Completed bounties	Tech	Active attempts	Options
@exi	1 bounty from 1 project	JavaScript, Nix, Clojure & more		Cancel attempt

[exi]

I spent roughly a day on this and here are my findings, although none of them will achieve the goal you set since this is a "death by 1000 paper cuts" situation.

Container startup is very slow, for a couple of reasons, which leads to very slow test times even for completely empty test classes.

Suggestions first

My proper long-term suggestion if you really want faster dev times, is to split your monolith up into maybe 2-3 separate service with a lot less stuff going on in each since you have reached the practical limits of JPA and Hibernate who are generally not very fast to startup.

A lighter variant would be to refactor your code to make spring test slices like @WebMvcTest work again. They currently do not work because of some dependency issues. This would however not be a complete integration test.

Another much more toilsome approach would be to drop hibernate/jpa and migrate to plain Spring Data JDBC, which is much simpler and faster.

One potential idea would be to upgrade to the bleeding edge spring version because it will support parallel bean initialization, which could speed things up a lot: spring-projects/spring-framework#13410

In anycase, I documented some potential suggestions to improve the performance in the next section that might help a bit.

Findings

All these profiles numbers are just for the context startup. They were taken from an empty test class with a single @SpringBootTest annotation.

JPA Repositories

21% of the startup time is spent creating JPA repositories, a large section of that on @Query queries.

You can see the repository creation impact here:

Also the impact of just the HQL Query parsing is visible here, with the hibernate and JPA HQL parsers highlighted:

Suggestion

A potential optimization here would be to use native queries, which will be parsed by postgres but a bit harder to write.

@Query parsing alone accounts for 11% of the total startup time.

JPA Entities and EntityManager

17.6% of the startup is furthermore used to create the JPA EntityManagerFactory, which ties the Repositories, Entities and their relations together:

This is mostly a function of how many entities and relations you have and I don't have a good suggestions on how to reduce this apart from having less relations or columns or fields, which is impractical.

ConditionalOn

6.8% of the startup time is spent evaluating @ConditionalOn annotations, however I cannot tell for certain whether these are your own annotations or the spring internal ones.
What I can say however, is that the language choice of kotlin makes this a lot slower because it triggers a lot of additional kotlin reflection to resolve the names in addition to the normal java class reflection.

Kotlin reflection alone accounts for 4% of total startup time.

Suggestion

None, since converting kotlin to plain java is probably no viable, but it would be interesting to measure the impact of removing the few @ConditionalOn you have.

[JanCizmar]

Hey! Thanks for the suggestions. I created this PR to prevent the startup validation of queries. All is lazy now and it saved around 1 second. So now it starts in 6 to 8 seconds on my M3 MacBook pro. I've also updated spring to 3.3.3, hibernate to 6.5.2 and Kotlin to 2.0.20. Looks like the kotlin update saved also second or 2 on the subsequent build time.

#2457

[exi]

@JanCizmar Cool, glad I could help a bit. I attempted a version upgrade myself but quickly abandoned the idea because I already spent a lot of time just profiling and did not want to deal with all the resulting test failures since I was not sure if my time investment would pay off.

[JanCizmar]

I am happy you helped. Your suggestions lead me to the right path. We got close to 10 seconds in build + context load. There is still lot of space on the context load path, but I would like to give you at least part from this bounty.

/tip $100 @exi

[algora-pbc]

👉 @JanCizmar: Navigate to your dashboard to proceed