Hi — I think I found a bug in TMap.remove and TMap.removeAll.
Filing per the contributing guide; happy to open a PR if helpful.
Summary
When two or more keys hash to the same bucket, calling TMap.remove(key) clears the entire bucket while only decrementing tSize by 1.
The remaining keys become unreachable via TMap.get even though TMap.size reports them as present.
Reproducer
Effect 3.21.2 (current latest on npm).
Forces a hash collision via a custom Hash so the bug is deterministic regardless of environment.
import { Effect, Equal, Hash, Option, STM, TMap } from "effect"
class Key implements Equal.Equal {
constructor(readonly id: string) {}
[Hash.symbol](): number {
return 1 // all keys collide into the same bucket
}
[Equal.symbol](that: unknown): boolean {
return that instanceof Key && that.id === this.id
}
}
const program = Effect.gen(function* () {
const m = yield* STM.commit(TMap.empty<Key, number>())
yield* STM.commit(TMap.set(m, new Key("a"), 1))
yield* STM.commit(TMap.set(m, new Key("b"), 2))
yield* STM.commit(TMap.set(m, new Key("c"), 3))
yield* STM.commit(TMap.remove(m, new Key("b")))
const size = yield* STM.commit(TMap.size(m))
const a = yield* STM.commit(TMap.get(m, new Key("a")))
const b = yield* STM.commit(TMap.get(m, new Key("b")))
const c = yield* STM.commit(TMap.get(m, new Key("c")))
console.log({
size,
a: Option.isSome(a),
b: Option.isSome(b),
c: Option.isSome(c)
})
})
Effect.runPromise(program)
Expected: { size: 2, a: true, b: false, c: true }
Observed: { size: 2, a: false, b: false, c: false }
With keys whose hashes happen to land in different buckets the bug does not surface, which is why it is easy to miss.
Current code
packages/effect/src/internal/stm/tMap.ts (current main):
// remove
const [toRemove, toRetain] = Chunk.partition(bucket,
(entry) => Equal.equals(entry[1], key))
if (Chunk.isNonEmpty(toRemove)) {
const currentSize = tRef.unsafeGet(self.tSize, journal)
tRef.unsafeSet(buckets.chunk[index], toRetain, journal)
tRef.unsafeSet(self.tSize, currentSize - 1, journal)
}
// removeAll
const [toRemove, toRetain] = Chunk.partition(bucket,
(entry) => Equal.equals(next.value)(entry[0]))
if (Chunk.isNonEmpty(toRemove)) {
const currentSize = tRef.unsafeGet(self.tSize, journal)
tRef.unsafeSet(buckets.chunk[index], toRetain, journal)
tRef.unsafeSet(self.tSize, currentSize - 1, journal)
}
Chunk.partition returns [excluded, satisfying] (predicate=false first, predicate=true second). The destructuring [toRemove, toRetain] is reversed relative to that contract, so toRetain actually holds the entries that should be deleted, and the bucket gets replaced with that subset. In remove the predicate also compares entry[1] (the value) instead of entry[0] (the key), which makes it nearly always false and therefore clears the bucket entirely.
For comparison, set uses the expected shape:
const shouldUpdate = Chunk.some(bucket, (entry) => Equal.equals(key)(entry[0]))
Suggested fix
// remove
const [toRetain, toRemove] = Chunk.partition(bucket,
(entry) => Equal.equals(entry[0], key))
if (Chunk.isNonEmpty(toRemove)) {
const currentSize = tRef.unsafeGet(self.tSize, journal)
tRef.unsafeSet(buckets.chunk[index], toRetain, journal)
tRef.unsafeSet(self.tSize, currentSize - 1, journal)
}
// removeAll
const [toRetain, toRemove] = Chunk.partition(bucket,
(entry) => Equal.equals(next.value)(entry[0]))
if (Chunk.isNonEmpty(toRemove)) {
const currentSize = tRef.unsafeGet(self.tSize, journal)
tRef.unsafeSet(buckets.chunk[index], toRetain, journal)
tRef.unsafeSet(self.tSize, currentSize - 1, journal)
}
A regression test that intentionally collides two keys in the same bucket would also be useful.
Thanks for Effect.
Hi — I think I found a bug in
TMap.removeandTMap.removeAll.Filing per the contributing guide; happy to open a PR if helpful.
Summary
When two or more keys hash to the same bucket, calling
TMap.remove(key)clears the entire bucket while only decrementingtSizeby 1.The remaining keys become unreachable via
TMap.geteven thoughTMap.sizereports them as present.Reproducer
Effect
3.21.2(currentlateston npm).Forces a hash collision via a custom
Hashso the bug is deterministic regardless of environment.With keys whose hashes happen to land in different buckets the bug does not surface, which is why it is easy to miss.
Current code
packages/effect/src/internal/stm/tMap.ts(currentmain):Chunk.partitionreturns[excluded, satisfying](predicate=false first, predicate=true second). The destructuring[toRemove, toRetain]is reversed relative to that contract, sotoRetainactually holds the entries that should be deleted, and the bucket gets replaced with that subset. Inremovethe predicate also comparesentry[1](the value) instead ofentry[0](the key), which makes it nearly alwaysfalseand therefore clears the bucket entirely.For comparison,
setuses the expected shape:Suggested fix
A regression test that intentionally collides two keys in the same bucket would also be useful.
Thanks for Effect.