[Bug] [Zeta] CheckpointMonitorService causes NullPointerException during cluster startup, blocking Jetty and other services initialization

[kafeikk98]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

Error while initializing proxy: IMap{name='engine_checkpoint_monitor'}
java.lang.NullPointerException: null

SeaTunnel Version

2.3.13

SeaTunnel Config

hazelcast:
  cluster-name: seatunnel-cluster
  network:
    rest-api:
      enabled: true
      endpoint-groups:
        CLUSTER_WRITE:
          enabled: true
        DATA:
          enabled: true
    port:
      auto-increment: false
      port: 5801
    join:
      kubernetes:
        enabled: true
        service-dns: seatunnel-cluster.ns-name.svc.cluster.local
        service-port: 5801
  map:
    engine*:
      map-store:
        enabled: true
        initial-mode: EAGER
        factory-class-name: org.apache.seatunnel.engine.server.persistence.FileMapStoreFactory
        properties:
          type: hdfs
          namespace: /seatunnel/imap/
          clusterName: seatunnel-cluster
          storage.type: oss
          oss.bucket: oss://bucket-name
          fs.oss.endpoint: xxxxxxx
          fs.oss.accessKeyId: xxxxxx
          fs.oss.accessKeySecret: xxxxxxxx
  properties:
    hazelcast.invocation.max.retry.count: 20
    hazelcast.tcp.join.port.try.count: 30
    hazelcast.logging.type: log4j2
    hazelcast.phone.home.enabled: false
    hazelcast.operation.generic.thread.count: 50
    hazelcast.heartbeat.failuredetector.type: phi-accrual
    hazelcast.heartbeat.interval.seconds: 2
    hazelcast.max.no.heartbeat.seconds: 60
    hazelcast.heartbeat.phiaccrual.failuredetector.threshold: 10
    hazelcast.heartbeat.phiaccrual.failuredetector.sample.size: 200
    hazelcast.heartbeat.phiaccrual.failuredetector.min.std.dev.millis: 100

Running Command

SeaTunnel Zeta cluster startup (K8s deployment)

Error Exception

[] 2026-03-06 16:20:16,245 WARN  [o.a.h.u.NativeCodeLoader      ] [main] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[] 2026-03-06 16:20:16,774 WARN  [c.h.s.i.p.ProxyService        ] [main] - [10.244.5.90]:5801 [seatunnel-cluster] [5.1] Error while initializing proxy: IMap{name='engine_checkpoint_monitor'}
java.lang.NullPointerException: null
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.<init>(Invocation.java:209) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.operationservice.impl.PartitionInvocation.<init>(PartitionInvocation.java:51) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.operationservice.impl.PartitionInvocation.<init>(PartitionInvocation.java:62) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.operationservice.impl.OperationServiceImpl.invokeOnPartition(OperationServiceImpl.java:328) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.map.impl.proxy.MapProxySupport.waitUntilLoaded(MapProxySupport.java:760) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.map.impl.proxy.MapProxyImpl.waitUntilLoaded(MapProxyImpl.java:112) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.map.impl.proxy.MapProxySupport.initializeMapStoreLoad(MapProxySupport.java:330) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.map.impl.proxy.MapProxySupport.initialize(MapProxySupport.java:265) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.map.impl.proxy.MapProxyImpl.initialize(MapProxyImpl.java:112) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.proxyservice.impl.ProxyRegistry.doCreateProxy(ProxyRegistry.java:266) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.proxyservice.impl.ProxyRegistry.createProxy(ProxyRegistry.java:235) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.proxyservice.impl.ProxyRegistry.getOrCreateProxyFuture(ProxyRegistry.java:202) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.proxyservice.impl.ProxyRegistry.getOrCreateProxy(ProxyRegistry.java:182) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.proxyservice.impl.ProxyServiceImpl.getDistributedObject(ProxyServiceImpl.java:153) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.instance.impl.HazelcastInstanceImpl.getDistributedObject(HazelcastInstanceImpl.java:355) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.instance.impl.HazelcastInstanceImpl.getMap(HazelcastInstanceImpl.java:174) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.engine.server.checkpoint.monitor.CheckpointMonitorService.<init>(CheckpointMonitorService.java:55) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.engine.server.SeaTunnelServer.startMaster(SeaTunnelServer.java:191) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.engine.server.SeaTunnelServer.init(SeaTunnelServer.java:159) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.servicemanager.impl.ServiceManagerImpl.initService(ServiceManagerImpl.java:235) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.servicemanager.impl.ServiceManagerImpl.initServices(ServiceManagerImpl.java:211) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.servicemanager.impl.ServiceManagerImpl.start(ServiceManagerImpl.java:103) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.NodeEngineImpl.start(NodeEngineImpl.java:250) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.instance.impl.Node.start(Node.java:458) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.instance.impl.HazelcastInstanceImpl.<init>(HazelcastInstanceImpl.java:124) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.instance.impl.HazelcastInstanceFactory.constructHazelcastInstance(HazelcastInstanceFactory.java:217) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.instance.impl.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:196) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.engine.server.SeaTunnelServerStarter.initializeHazelcastInstance(SeaTunnelServerStarter.java:70) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.engine.server.SeaTunnelServerStarter.createHazelcastInstance(SeaTunnelServerStarter.java:52) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.core.starter.seatunnel.command.ServerExecuteCommand.execute(ServerExecuteCommand.java:83) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelServer.main(SeaTunnelServer.java:34) ~[seatunnel-starter.jar:2.3.13]
[] 2026-03-06 16:20:16,774 ERROR [c.h.s.i.s.i.ServiceManagerImpl] [main] - [10.244.5.90]:5801 [seatunnel-cluster] [5.1] Error while initializing service: null
java.lang.NullPointerException: null
	at com.hazelcast.spi.impl.operationservice.impl.Invocation.<init>(Invocation.java:209) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.operationservice.impl.PartitionInvocation.<init>(PartitionInvocation.java:51) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.operationservice.impl.PartitionInvocation.<init>(PartitionInvocation.java:62) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.operationservice.impl.OperationServiceImpl.invokeOnPartition(OperationServiceImpl.java:328) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.map.impl.proxy.MapProxySupport.waitUntilLoaded(MapProxySupport.java:760) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.map.impl.proxy.MapProxyImpl.waitUntilLoaded(MapProxyImpl.java:112) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.map.impl.proxy.MapProxySupport.initializeMapStoreLoad(MapProxySupport.java:330) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.map.impl.proxy.MapProxySupport.initialize(MapProxySupport.java:265) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.map.impl.proxy.MapProxyImpl.initialize(MapProxyImpl.java:112) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.proxyservice.impl.ProxyRegistry.doCreateProxy(ProxyRegistry.java:266) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.proxyservice.impl.ProxyRegistry.createProxy(ProxyRegistry.java:235) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.proxyservice.impl.ProxyRegistry.getOrCreateProxyFuture(ProxyRegistry.java:202) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.proxyservice.impl.ProxyRegistry.getOrCreateProxy(ProxyRegistry.java:182) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.proxyservice.impl.ProxyServiceImpl.getDistributedObject(ProxyServiceImpl.java:153) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.instance.impl.HazelcastInstanceImpl.getDistributedObject(HazelcastInstanceImpl.java:355) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.instance.impl.HazelcastInstanceImpl.getMap(HazelcastInstanceImpl.java:174) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.engine.server.checkpoint.monitor.CheckpointMonitorService.<init>(CheckpointMonitorService.java:55) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.engine.server.SeaTunnelServer.startMaster(SeaTunnelServer.java:191) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.engine.server.SeaTunnelServer.init(SeaTunnelServer.java:159) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.servicemanager.impl.ServiceManagerImpl.initService(ServiceManagerImpl.java:235) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.servicemanager.impl.ServiceManagerImpl.initServices(ServiceManagerImpl.java:211) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.servicemanager.impl.ServiceManagerImpl.start(ServiceManagerImpl.java:103) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.spi.impl.NodeEngineImpl.start(NodeEngineImpl.java:250) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.instance.impl.Node.start(Node.java:458) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.instance.impl.HazelcastInstanceImpl.<init>(HazelcastInstanceImpl.java:124) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.instance.impl.HazelcastInstanceFactory.constructHazelcastInstance(HazelcastInstanceFactory.java:217) ~[seatunnel-starter.jar:2.3.13]
	at com.hazelcast.instance.impl.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:196) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.engine.server.SeaTunnelServerStarter.initializeHazelcastInstance(SeaTunnelServerStarter.java:70) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.engine.server.SeaTunnelServerStarter.createHazelcastInstance(SeaTunnelServerStarter.java:52) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.core.starter.seatunnel.command.ServerExecuteCommand.execute(ServerExecuteCommand.java:83) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) ~[seatunnel-starter.jar:2.3.13]
	at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelServer.main(SeaTunnelServer.java:34) ~[seatunnel-starter.jar:2.3.13]

Zeta or Flink or Spark Version

Zeta (Hazelcast 5.1)

Java or Scala Version

1.8

Screenshots

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[DanielCarter-stack]

Thanks for reporting this. This is a valid initialization order bug.

Root cause: CheckpointMonitorService retrieves the engine_checkpoint_monitor IMap in its constructor, which is called during SeaTunnelServer.startMaster() (line 186-191). Since startMaster() is invoked within SeaTunnelServer.init()—a Hazelcast service initialization callback—this triggers MapStore's initial-mode: EAGER loading before PartitionService is fully initialized, causing the NPE at Invocation:209.

Evidence:

• CheckpointMonitorService:53-56 - eagerly gets IMap in constructor
• SeaTunnelServer:186-191 - creates CheckpointMonitorService during Hazelcast init
• CoordinatorService:404-413 - demonstrates correct deferred IMap initialization via initCoordinatorService() called later

Workaround: Exclude engine_checkpoint_monitor from EAGER mode or switch to LAZY mode:

hazelcast:
  map:
    engine_checkpoint_monitor:
      map-store:
        enabled: false

Fix: Refactor CheckpointMonitorService to defer IMap retrieval (similar to CoordinatorService.initCoordinatorService()), or lazy-initialize on first access.

Could you confirm if engine_checkpoint_monitor requires persistence?

[kafeikk98]

Thank you for your reply. Persistence for engine_checkpoint_monitor is not required. I have excluded it from EAGER mode.

[DanielLeens]

I checked the current code status for this one as well. The root cause is consistent with the earlier analysis: CheckpointMonitorService accesses engine_checkpoint_monitor too early during server startup, and that becomes fatal when the map is configured with eager persistence loading.

Two important status points: the workaround shared in the thread is valid for older versions, and the actual fix has already landed on dev via PR #10610.