Open
Description
Expected Behavior
ActivityStubs should be created for each workflow instance / execution separately.
But if somebody uses Dependency Injection frameworks with Temporal it's very easy to misconfigure the wiring (for example, by making them lazily initialized on first usage singletons) in a way that activity stubs will be leaking between workflow instances.
We should detect or provide better protection or diagnostic if this happens that actually helps to understand the problem.
Actual Behavior
Cryptic Event Loop error that makes to look in a completely wrong direction.
Caused by:
io.temporal.failure.ApplicationFailure: message='Operation allowed only while eventLoop is running', type='java.lang.IllegalStateException', nonRetryable=false
at io.temporal.internal.statemachines.WorkflowStateMachines.checkEventLoopExecuting(WorkflowStateMachines.java:915)
at io.temporal.internal.statemachines.WorkflowStateMachines.randomUUID(WorkflowStateMachines.java:611)
at io.temporal.internal.replay.ReplayWorkflowContextImpl.randomUUID(ReplayWorkflowContextImpl.java:93)
at io.temporal.internal.sync.SyncWorkflowContext.constructExecuteLocalActivityParameters(SyncWorkflowContext.java:351)
at io.temporal.internal.sync.SyncWorkflowContext.executeLocalActivityOnce(SyncWorkflowContext.java:292)
at io.temporal.internal.sync.SyncWorkflowContext.executeLocalActivityOnce(SyncWorkflowContext.java:275)
at io.temporal.internal.sync.SyncWorkflowContext.lambda$executeLocalActivity$a6925f48$1(SyncWorkflowContext.java:269)
at io.temporal.internal.sync.WorkflowRetryerInternal.retryAsync(WorkflowRetryerInternal.java:232)
at io.temporal.internal.sync.SyncWorkflowContext.executeLocalActivity(SyncWorkflowContext.java:268)
at io.temporal.common.interceptors.WorkflowOutboundCallsInterceptorBase.executeLocalActivity(WorkflowOutboundCallsInterceptorBase.java:48)
at io.temporal.opentracing.internal.OpenTracingWorkflowOutboundCallsInterceptor.executeLocalActivity(OpenTracingWorkflowOutboundCallsInterceptor.java:70)
at io.temporal.internal.sync.LocalActivityStubImpl.executeAsync(LocalActivityStubImpl.java:50)
at io.temporal.internal.sync.ActivityStubBase.execute(ActivityStubBase.java:38)
at io.temporal.internal.sync.LocalActivityStubImpl.execute(LocalActivityStubImpl.java:29)
at io.temporal.internal.sync.LocalActivityInvocationHandler.lambda$getActivityFunc$0(LocalActivityInvocationHandler.java:72)
at io.temporal.internal.sync.ActivityInvocationHandlerBase.invoke(ActivityInvocationHandlerBase.java:70)
at com.sun.proxy.$Proxy158.execute(Unknown Source:0)
<!-- a project specific workflow code -->
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method:0)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at io.temporal.internal.sync.POJOWorkflowImplementationFactory$POJOWorkflowImplementation$RootWorkflowInboundCallsInterceptor.execute(POJOWorkflowImplementationFactory.java:321)
at io.temporal.common.interceptors.WorkflowInboundCallsInterceptorBase.execute(WorkflowInboundCallsInterceptorBase.java:37)
at io.temporal.opentracing.internal.OpenTracingWorkflowInboundCallsInterceptor.execute(OpenTracingWorkflowInboundCallsInterceptor.java:72)
at io.temporal.internal.sync.POJOWorkflowImplementationFactory$POJOWorkflowImplementation.execute(POJOWorkflowImplementationFactory.java:295)
at io.temporal.internal.sync.WorkflowExecuteRunnable.run(WorkflowExecuteRunnable.java:53)
at io.temporal.internal.sync.SyncWorkflow.lambda$start$0(SyncWorkflow.java:131)
at io.temporal.internal.sync.CancellationScopeImpl.run(CancellationScopeImpl.java:101)
at io.temporal.internal.sync.WorkflowThreadImpl$RunnableWrapper.run(WorkflowThreadImpl.java:110)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
This specific issue happened with DI wiring using Dagger
Alternative
There is no reason for ActivityStub that is a pair of ActivityType, ActivityOptions to don't be workflow context-independent. We can make internal changes to make them safely shareable, which is probably a better user experience.