Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dapr Sidecar fails to publish message to azure service bus #3559

Open
MattCosturos opened this issue Oct 2, 2024 · 7 comments
Open

Dapr Sidecar fails to publish message to azure service bus #3559

MattCosturos opened this issue Oct 2, 2024 · 7 comments
Labels
kind/bug Something isn't working

Comments

@MattCosturos
Copy link

Expected Behavior

Dapr service bus component should be able to publish messages to the azure service bus.

Actual Behavior

After a period of running, the Dapr Sidecar contains logs indicating a failure.

time="2024-10-01T20:37:57.486086085Z" level=error msg="Too many failed attempts while publishing Service Bus message (nil): context canceled" app_id=actor-microservice component="status-pubsub (pubsub.azure.servicebus/v1)" instance=actor-microservice-8649476968-28bzt scope=dapr.contrib type=log ver=1.13.5
time="2024-10-01T20:37:59.447828639Z" level=error msg="Too many failed attempts while publishing Service Bus message (nil): context canceled" app_id=actor-microservice component="status-pubsub (pubsub.azure.servicebus/v1)" instance=actor-microservice-8649476968-28bzt scope=dapr.contrib type=log ver=1.13.5
time="2024-10-01T20:38:02.723978748Z" level=error msg="Too many failed attempts while publishing Service Bus message (nil): context canceled" app_id=actor-microservice component="status-pubsub (pubsub.azure.servicebus/v1)" instance=actor-microservice-8649476968-28bzt scope=dapr.contrib type=log ver=1.13.5

Using Dapr 1.13.5 in AKS.

Steps to Reproduce the Problem

Unable to repo on demand. Pods run for a while then errors appear.

Possibly related to #2617 ?
@MattCosturos MattCosturos added the kind/bug Something isn't working label Oct 2, 2024
@yaron2
Copy link
Member

yaron2 commented Oct 2, 2024

Can you try this with Dapr 1.14.4? The SDK for ASB was updated and it includes some fixes that may be relevant to what you're seeing.

@MattCosturos
Copy link
Author

Yes I will update to 1.14.4 and test that out.

@MattCosturos
Copy link
Author

Now running 1.14.4. Haven't been running long enough to be confident the Service bus error is gone, will keep monitoring.

I am now seeing this new type of message in the logs.
level=warning msg="Error processing operation DaprBuiltInActorRetries. Retrying in 1s…" app_id=actor-microservice instance=actor-microservice-7cc7fb6b4b-hjs76 scope=dapr.runtime type=log ver=1.14.4

Is there any way to figure out what operation failed from the daprd logs?

@MattCosturos
Copy link
Author

time="2024-10-11T14:58:19.041817468Z" level=error msg="Too many failed attempts while publishing Service Bus message (nil): context canceled" app_id=actor-microservice component="status-pubsub (pubsub.azure.servicebus/v1)" instance=actor-microservice-54bb48b7bc-cz5pr scope=dapr.contrib type=log ver=1.14.4

Still seeing this error with 1.14.4

@MattCosturos
Copy link
Author

Looking at our logs in more detail, from our code, we catch the timeout ex. We then build a new dapr client and try again. We will try 3 times before giving up. The curious thing is that our logs all have the same timestamp. I assumed context cancelled meant timeout, but with all 3 attempts having the same timestamp, I feel like there couldn't have been a timeout since it didn't wait any time.

I am not sure, but I think the next call we make to publish the next message to the service bus succeeds. I think this since our logs are not full of errors publishing to the service bus.

daprd error

2024-10-11T10:58:19-04:00 time="2024-10-11T14:58:19.041817468Z" level=error msg="Too many failed attempts while publishing Service Bus message (nil): context canceled" app_id=actor-microservice component="status-pubsub (pubsub.azure.servicebus/v1)" instance=actor-microservice-54bb48b7bc-cz5pr scope=dapr.contrib type=log ver=1.14.4

C# logs.


2024-10-11T10:58:19-04:00 fail: MyNamespace.Proxies.StatusChange.MessagePublishProxy[0]
2024-10-11T10:58:19-04:00       Exception in PublishMessage.
2024-10-11T10:58:19-04:00       System.OperationCanceledException: The operation was canceled.
2024-10-11T10:58:19-04:00        ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
2024-10-11T10:58:19-04:00          at System.Net.Http.Http2Connection.Http2Stream.<>c.<WaitForDataAsync>b__88_0(Object s, CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at System.Threading.CancellationTokenSource.ExecuteCallbackHandlers(Boolean throwOnFirstException)
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Internal.GrpcCall`2.CancelCall(Status status)
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Internal.GrpcCall`2.CancelCallFromCancellationToken(CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Internal.GrpcCall`2.<>c.<RegisterCancellation>b__70_0(Object state, CancellationToken ct)
2024-10-11T10:58:19-04:00          at System.Threading.CancellationTokenSource.Invoke(Delegate d, Object state, CancellationTokenSource source)
2024-10-11T10:58:19-04:00          at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
2024-10-11T10:58:19-04:00          at System.Threading.CancellationTokenSource.ExecuteCallbackHandlers(Boolean throwOnFirstException)
2024-10-11T10:58:19-04:00          at System.Threading.TimerQueueTimer.Fire(Boolean isThreadPool)
2024-10-11T10:58:19-04:00          at System.Threading.TimerQueue.FireNextTimers()
2024-10-11T10:58:19-04:00          at System.Threading.ThreadPoolWorkQueue.Dispatch()
2024-10-11T10:58:19-04:00          at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
2024-10-11T10:58:19-04:00       --- End of stack trace from previous location ---
2024-10-11T10:58:19-04:00          at System.Net.Http.Http2Connection.Http2Stream.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
2024-10-11T10:58:19-04:00          at System.Net.Http.Http2Connection.Http2Stream.ReadResponseHeadersAsync(CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at System.Net.Http.Http2Connection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout)
2024-10-11T10:58:19-04:00          --- End of inner exception stack trace ---
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Internal.GrpcCall`2.ResolveException(String summary, Exception ex, Nullable`1& status, Exception& resolvedException)
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout)
2024-10-11T10:58:19-04:00          at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread threadPoolThread)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox box, Boolean allowInlining)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.Task.RunContinuations(Object continuationObject)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.Task.TrySetCanceled(CancellationToken tokenToRecord, Object cancellationException)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetException(Exception exception, Task`1& taskField)
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread threadPoolThread)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox box, Boolean allowInlining)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.Task.RunContinuations(Object continuationObject)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.Task.TrySetCanceled(CancellationToken tokenToRecord, Object cancellationException)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetException(Exception exception, Task`1& taskField)
2024-10-11T10:58:19-04:00          at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread threadPoolThread)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox box, Boolean allowInlining)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.Task.RunContinuations(Object continuationObject)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.Task.TrySetCanceled(CancellationToken tokenToRecord, Object cancellationException)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetException(Exception exception, Task`1& taskField)
2024-10-11T10:58:19-04:00          at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread threadPoolThread)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox box, Boolean allowInlining)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.Task.RunContinuations(Object continuationObject)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.Task.TrySetCanceled(CancellationToken tokenToRecord, Object cancellationException)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetException(Exception exception, Task`1& taskField)
2024-10-11T10:58:19-04:00          at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread threadPoolThread)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox box, Boolean allowInlining)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.Task.RunContinuations(Object continuationObject)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.Task.TrySetCanceled(CancellationToken tokenToRecord, Object cancellationException)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetException(Exception exception, Task`1& taskField)
2024-10-11T10:58:19-04:00          at System.Net.Http.Http2Connection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread threadPoolThread)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox box, Boolean allowInlining)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.Task.RunContinuations(Object continuationObject)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetException(Exception exception, Task`1& taskField)
2024-10-11T10:58:19-04:00          at System.Net.Http.Http2Connection.Http2Stream.ReadResponseHeadersAsync(CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread threadPoolThread)
2024-10-11T10:58:19-04:00          at System.Threading.ThreadPoolWorkQueue.Dispatch()
2024-10-11T10:58:19-04:00          at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
2024-10-11T10:58:19-04:00       --- End of stack trace from previous location ---
2024-10-11T10:58:19-04:00          at Dapr.Client.DaprClientGrpc.MakePublishRequest(String pubsubName, String topicName, ByteString content, Dictionary`2 metadata, String dataContentType, CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at MyNamespace.Proxies.StatusChange.MessagePublishProxy.PublishMessage(Object message, String pubsubName, String topicName) in /src/Shared/MyNamespace/Proxies/MessagePublish/MessagePublishProxy.cs:line 109
2024-10-11T10:58:19-04:00 warn: MyNamespace.Proxies.StatusChange.MessagePublishProxy[0]
2024-10-11T10:58:19-04:00       Disposing DaprClient.
2024-10-11T10:58:19-04:00 warn: MyNamespace.Proxies.StatusChange.MessagePublishProxy[0]
2024-10-11T10:58:19-04:00       Building new DaprClient.
2024-10-11T10:58:19-04:00 fail: MyNamespace.Proxies.StatusChange.MessagePublishProxy[0]
2024-10-11T10:58:19-04:00       Exception in PublishMessage.
2024-10-11T10:58:19-04:00       System.OperationCanceledException: The operation was canceled.
2024-10-11T10:58:19-04:00        ---> System.OperationCanceledException: The operation was canceled.
2024-10-11T10:58:19-04:00          at System.Threading.CancellationToken.ThrowOperationCanceledException()
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout)
2024-10-11T10:58:19-04:00          --- End of inner exception stack trace ---
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Internal.GrpcCall`2.ResolveException(String summary, Exception ex, Nullable`1& status, Exception& resolvedException)
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout)
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Internal.GrpcCall`2.StartUnaryCore(HttpContent content)
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Internal.HttpClientCallInvoker.AsyncUnaryCall[TRequest,TResponse](Method`2 method, String host, CallOptions options, TRequest request)
2024-10-11T10:58:19-04:00          at Grpc.Core.Interceptors.InterceptingCallInvoker.<AsyncUnaryCall>b__4_0[TRequest,TResponse](TRequest req, ClientInterceptorContext`2 ctx)
2024-10-11T10:58:19-04:00          at Grpc.Core.Interceptors.InterceptingCallInvoker.AsyncUnaryCall[TRequest,TResponse](Method`2 method, String host, CallOptions options, TRequest request)
2024-10-11T10:58:19-04:00          at Dapr.Client.DaprClientGrpc.MakePublishRequest(String pubsubName, String topicName, ByteString content, Dictionary`2 metadata, String dataContentType, CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
2024-10-11T10:58:19-04:00          at Dapr.Client.DaprClientGrpc.PublishEventAsync[TData](String pubsubName, String topicName, TData data, CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at MyNamespace.Proxies.StatusChange.MessagePublishProxy.PublishMessage(Object message, String pubsubName, String topicName) in /src/Shared/MyNamespace/Proxies/MessagePublish/MessagePublishProxy.cs:line 109
2024-10-11T10:58:19-04:00          at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread threadPoolThread)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox box, Boolean allowInlining)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.Task.RunContinuations(Object continuationObject)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.Task.DelayPromise.CompleteTimedOut()
2024-10-11T10:58:19-04:00          at System.Threading.TimerQueueTimer.Fire(Boolean isThreadPool)
2024-10-11T10:58:19-04:00          at System.Threading.TimerQueue.FireNextTimers()
2024-10-11T10:58:19-04:00          at System.Threading.ThreadPoolWorkQueue.Dispatch()
2024-10-11T10:58:19-04:00          at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
2024-10-11T10:58:19-04:00       --- End of stack trace from previous location ---
2024-10-11T10:58:19-04:00          at Dapr.Client.DaprClientGrpc.MakePublishRequest(String pubsubName, String topicName, ByteString content, Dictionary`2 metadata, String dataContentType, CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at MyNamespace.Proxies.StatusChange.MessagePublishProxy.PublishMessage(Object message, String pubsubName, String topicName) in /src/Shared/MyNamespace/Proxies/MessagePublish/MessagePublishProxy.cs:line 109
2024-10-11T10:58:19-04:00 warn: MyNamespace.Proxies.StatusChange.MessagePublishProxy[0]
2024-10-11T10:58:19-04:00       Disposing DaprClient.
2024-10-11T10:58:19-04:00 warn: MyNamespace.Proxies.StatusChange.MessagePublishProxy[0]
2024-10-11T10:58:19-04:00       Building new DaprClient.
2024-10-11T10:58:19-04:00 fail: MyNamespace.Proxies.StatusChange.MessagePublishProxy[0]
2024-10-11T10:58:19-04:00       Exception in PublishMessage.
2024-10-11T10:58:19-04:00       System.OperationCanceledException: The operation was canceled.
2024-10-11T10:58:19-04:00        ---> System.OperationCanceledException: The operation was canceled.
2024-10-11T10:58:19-04:00          at System.Threading.CancellationToken.ThrowOperationCanceledException()
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout)
2024-10-11T10:58:19-04:00          --- End of inner exception stack trace ---
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Internal.GrpcCall`2.ResolveException(String summary, Exception ex, Nullable`1& status, Exception& resolvedException)
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout)
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Internal.GrpcCall`2.StartUnaryCore(HttpContent content)
2024-10-11T10:58:19-04:00          at Grpc.Net.Client.Internal.HttpClientCallInvoker.AsyncUnaryCall[TRequest,TResponse](Method`2 method, String host, CallOptions options, TRequest request)
2024-10-11T10:58:19-04:00          at Grpc.Core.Interceptors.InterceptingCallInvoker.<AsyncUnaryCall>b__4_0[TRequest,TResponse](TRequest req, ClientInterceptorContext`2 ctx)
2024-10-11T10:58:19-04:00          at Grpc.Core.Interceptors.InterceptingCallInvoker.AsyncUnaryCall[TRequest,TResponse](Method`2 method, String host, CallOptions options, TRequest request)
2024-10-11T10:58:19-04:00          at Dapr.Client.DaprClientGrpc.MakePublishRequest(String pubsubName, String topicName, ByteString content, Dictionary`2 metadata, String dataContentType, CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
2024-10-11T10:58:19-04:00          at Dapr.Client.DaprClientGrpc.PublishEventAsync[TData](String pubsubName, String topicName, TData data, CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at MyNamespace.Proxies.StatusChange.MessagePublishProxy.PublishMessage(Object message, String pubsubName, String topicName) in /src/Shared/MyNamespace/Proxies/MessagePublish/MessagePublishProxy.cs:line 109
2024-10-11T10:58:19-04:00          at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
2024-10-11T10:58:19-04:00          at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread threadPoolThread)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox box, Boolean allowInlining)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.Task.RunContinuations(Object continuationObject)
2024-10-11T10:58:19-04:00          at System.Threading.Tasks.Task.DelayPromise.CompleteTimedOut()
2024-10-11T10:58:19-04:00          at System.Threading.TimerQueueTimer.Fire(Boolean isThreadPool)
2024-10-11T10:58:19-04:00          at System.Threading.TimerQueue.FireNextTimers()
2024-10-11T10:58:19-04:00          at System.Threading.ThreadPoolWorkQueue.Dispatch()
2024-10-11T10:58:19-04:00          at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
2024-10-11T10:58:19-04:00       --- End of stack trace from previous location ---
2024-10-11T10:58:19-04:00          at Dapr.Client.DaprClientGrpc.MakePublishRequest(String pubsubName, String topicName, ByteString content, Dictionary`2 metadata, String dataContentType, CancellationToken cancellationToken)
2024-10-11T10:58:19-04:00          at MyNamespace.Proxies.StatusChange.MessagePublishProxy.PublishMessage(Object message, String pubsubName, String topicName) in /src/Shared/MyNamespace/Proxies/MessagePublish/MessagePublishProxy.cs:line 109
2024-10-11T10:58:19-04:00 fail: MyNamespace.Proxies.StatusChange.MessagePublishProxy[0]
2024-10-11T10:58:19-04:00       PublishEventAsync failed 3 times. This message will be lost forever.

@github-actions GitHub Actions
Copy link

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale label Nov 10, 2024
@MattCosturos
Copy link
Author

Not stale

@github-actions github-actions bot removed the stale label Nov 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@yaron2 @MattCosturos