[bug fix] set finish before notifying load in LanceArrowWriter setFinished method

[xx789633]

… method

Purpose

Linked issue: close #xxx

This commit is copied from lance-format/lance-spark#92.

I found that the Lance writer has a certain probability of hanging. After some troubleshooting, I discovered this is related to the LanceArrowWriter.setFinished method.

The original code appears to have a bug where it sets the finished status after notifying loadNextBatch, which could cause loadNextBatch to hang.

Root Cause

The ideal flow should be:

(thread 1) loadToken.release
(thread 1) finished = true
(thread 2) loadNextBatch
(thread 2) finished is true and count is 0 so return false
However, there's a chance it becomes:

(thread 1) loadToken.release
(thread 2) loadNextBatch
(thread 2) finished is false so return true and waiting
(thread 1) finished = false
If the second scenario occurs, thread 2 will hang indefinitely and cannot receive new notifications. jstack will show stacks hanging in LanceDataWriter.commit.

Brief change log

Tests

API and Format

Documentation

[Copilot]

Pull Request Overview

This PR fixes a critical race condition bug in LanceArrowWriter.setFinished() that could cause loadNextBatch() to hang indefinitely. The fix reorders operations to set the finished flag before releasing the semaphore, ensuring proper thread synchronization.

Key Changes:

• Reordered finished = true to execute before loadToken.release() in setFinished() method to prevent race condition
• Added explicit false initialization to the finished field

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

[nitpick] The explicit initialization = false is redundant in Java. Boolean instance fields are automatically initialized to false by default. While this doesn't cause any issues, it's unnecessary and can be removed for cleaner code.

Suggested change

	private volatile boolean finished = false;
	private volatile boolean finished;

[luoyuxia]

+1