Threads blocking on getSingleton method on DefaultSingletonBeanRegistry class [SPR-9190]

[spring-projects-issues]

ggn opened SPR-9190 and commented

We have been facing numerous threads being blocked waiting for getSingleton method on DefaultSingletonBeanRegistry class and because of this our servers have been hanging which requires a restart whenever it happens.

We have been collecting threaddumps whenever this happens and we are consistently seeing as below for hundreds of threads:

"catalina-exec-567" daemon prio=10 tid=0x00002aaabc5eb000 nid=0xc54 waiting for monitor entry [0x00002aaaf6320000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:180)

• waiting to lock <0x00000005a2a58f70> (a java.util.concurrent.ConcurrentHashMap)
  at org.springframework.beans.factory.support.AbstractBeanFactory.isTypeMatch(AbstractBeanFactory.java:466)
  at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:319)
  at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:298)
  at org.springframework.context.support.AbstractApplicationContext.getBeanNamesForType(AbstractApplicationContext.java:1132)

java.lang.Thread.State: BLOCKED (on object monitor)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:180)

• waiting to lock <0x00000005a2a58f70> (a java.util.concurrent.ConcurrentHashMap)
  at org.springframework.beans.factory.support.AbstractBeanFactory.isTypeMatch(AbstractBeanFactory.java:466)
  at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:319)
  at org.springframework.beans.factory.BeanFactoryUtils.beanNamesForTypeIncludingAncestors(BeanFactoryUtils.java:185)
  at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:833)
  at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:790)
  at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:707)
  at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor.autowireResource(CommonAnnotationBeanPostProcessor.java:433)
  at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor.getResource(CommonAnnotationBeanPostProcessor.java:411)
  at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor$ResourceElement.getResourceToInject(CommonAnnotationBeanPostProcessor.java:543)
  at org.springframework.beans.factory.annotation.InjectionMetadata$InjectedElement.inject(InjectionMetadata.java:147)
  at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:84)
  at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor.postProcessPropertyValues(CommonAnnotationBeanPostProcessor.java:298)
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1074)
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517)
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
  at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:312)
  at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200)
  at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1087)

I have already commented requesting for help on the related problem https://jira.springsource.org/browse/SPR-7949 but haven't gotten any response.

https://jira.springsource.org/browse/SPR-7949 has some information about some patch but i haven't seen the patch being applied to 3.0.6 or 3.0.7 or 3.1.

Please help with this issue as this is causing some major issue with our application.

Thanks for help.
ggn

---

Affects: 3.0.6

Reference URL: https://jira.springsource.org/browse/SPR-7949

Issue Links:

• Improve performance of DefaultSingletonBeanRegistry.getSingletonNames(..) for high-concurrent-load applications [SPR-7949] #12604 Improve performance of DefaultSingletonBeanRegistry.getSingletonNames(..) for high-concurrent-load applications ("duplicates")

[spring-projects-issues]

Stevo Slavić commented

This issue duplicates #12604. From my experience creating duplicates will not get issues fixed sooner, on the contrary.

[spring-projects-issues]

ggn commented

Hi Stevo,

Yes this issue i opened is related to to #12604, but as i said the status of #12604 seems misleading about the patch and also the following versions of spring.

That is why i opened a new issue requesting for someone to address this or update #12604 with correct information. Please check my notes that i initially added above regarding #12152 to this #13828 description.

I hope the spring team understands this and it gets prioritized accordingly.

Thanks
GGN

[spring-projects-issues]

ggn commented

Sorry for the typo in above comment. I meant #12604 not #12152. I realized i cannot edit the posted comment.

[spring-projects-issues]

Stevo Slavić commented

Go through #12604 issue thoroughly, you'll notice it is related to #11536, and in #11536 there is a comment leading to a blog entry with a solution/workaround, and you can apply that workaround without Spring being changed/patched.

This issue is a duplicate of #12604, and #12604 is duplicate of #11536.

[spring-projects-issues]

Chris Beams commented

Resolving as duplicate per Stevo's comments. @ggn, we've slated #12604 for 3.2 M1 and will do what we can, thanks.

[spring-projects-issues]

ggn commented

Thank you Chris and Stevo. I will check #11536 and see if we can get patch or wait for the actual fix as part of stock.

-ggn