Skip to content

fix: accessing null object (cache) after shutdown#9678

Closed
egil wants to merge 1 commit intodotnet:mainfrom
egil:fix/null-exception-after-shutdown
Closed

fix: accessing null object (cache) after shutdown#9678
egil wants to merge 1 commit intodotnet:mainfrom
egil:fix/null-exception-after-shutdown

Conversation

@egil
Copy link
Copy Markdown
Contributor

@egil egil commented Sep 11, 2025

This fixes a NullReferenceException, that I think happens in GetQueueMessagesAsync when it tries to acces this.cache after EventHubAdapterReceiver.Shutdown() has been called.

This only happens under high load.

Here is the stack trace I have seen in production multiple times.

System.NullReferenceException:
   at Orleans.Streaming.EventHubs.EventHubAdapterReceiver+<GetQueueMessagesAsync>d__22.MoveNext (Orleans.Streaming.EventHubs, Version=9.0.0.0, Culture=neutral, PublicKeyToken=null: /_/src/Azure/Orleans.Streaming.EventHubs/Providers/Streams/EventHub/EventHubAdapterReceiver.cs:181)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Orleans.Streams.PersistentStreamPullingAgent+<ReadFromQueue>d__34.MoveNext (Orleans.Streaming, Version=9.0.0.0, Culture=neutral, PublicKeyToken=null: /_/src/Orleans.Streaming/PersistentStreams/PersistentStreamPullingAgent.cs:434)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Orleans.Internal.AsyncExecutorWithRetries+<ExecuteWithRetriesHelper>d__4`1.MoveNext (Orleans.Core, Version=9.0.0.0, Culture=neutral, PublicKeyToken=null: /_/src/Orleans.Core/Async/AsyncExecutorWithRetries.cs:241)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Orleans.Internal.AsyncExecutorWithRetries+<ExecuteWithRetriesHelper>d__4`1.MoveNext (Orleans.Core, Version=9.0.0.0, Culture=neutral, PublicKeyToken=null: /_/src/Orleans.Core/Async/AsyncExecutorWithRetries.cs:272)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Orleans.Streams.PersistentStreamPullingAgent+<PumpQueue>d__33.MoveNext (Orleans.Streaming, Version=9.0.0.0, Culture=neutral, PublicKeyToken=null: /_/src/Orleans.Streaming/PersistentStreams/PersistentStreamPullingAgent.cs:363)
Microsoft Reviewers: Open in CodeFlow

Copilot AI review requested due to automatic review settings September 11, 2025 08:52
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes a NullReferenceException that occurs when accessing the cache field after EventHubAdapterReceiver.Shutdown() has been called, particularly under high load conditions.

  • Adds null-conditional operator when accessing this.cache in GetQueueMessagesAsync
  • Uses null-coalescing operator to handle potential null messageStreamPositions
  • Includes minor formatting improvements with additional newlines
Comments suppressed due to low confidence (1)

src/Azure/Orleans.Streaming.EventHubs/Providers/Streams/EventHub/EventHubAdapterReceiver.cs:1

  • The GetCacheCursor method still accesses this.cache directly without null checking, which could cause the same NullReferenceException after shutdown. Consider adding null safety here as well.
using System;

@egil egil force-pushed the fix/null-exception-after-shutdown branch from c96f0b3 to aafbea6 Compare September 11, 2025 09:20
@egil egil force-pushed the fix/null-exception-after-shutdown branch from aafbea6 to c4ba371 Compare September 30, 2025 16:38
Copy link
Copy Markdown

@shacal shacal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cleaner, I like it :)

@ReubenBond ReubenBond force-pushed the fix/null-exception-after-shutdown branch from c4ba371 to c1afcd8 Compare October 1, 2025 17:55
@github-actions github-actions bot locked and limited conversation to collaborators Nov 2, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants