enhancement
major
minor
major
minor
major
#29604
ElementSecurityUpdateManager: quadratic collection diff on commit (AbstractSet.removeAll over a List) makes appending to a large reference take seconds
Problem
Appending one object to a multiple reference that already holds many elements makes the commit take seconds. Observed in the issue tracker tl-dev (engine snapshot of 2026-09-16): creating a ticket in a project whose Project#tickets (composite, ordered, ~30 000 elements) it is appended to logs
WARN com.top_logic.knowledge.service.db2.DefaultDBContext - Long running commit (revision 2657 lasting 00:05,932)
Thread dumps taken every 0.5 s during the commit show the request thread in the same place in every sample:
at com.top_logic.model.AbstractTLObject.equals(AbstractTLObject.java:28) at java.util.ArrayList.indexOfRange(ArrayList.java:299) at java.util.ArrayList.indexOf(ArrayList.java:286) at java.util.ArrayList.contains(ArrayList.java:275) at java.util.AbstractSet.removeAll(AbstractSet.java:175) at com.top_logic.element.boundsec.manager.ElementSecurityUpdateManager.handleSecurityUpdate(ElementSecurityUpdateManager.java:253) at com.top_logic.element.boundsec.manager.StorageAccessManager.doHandleSecurityUpdate(StorageAccessManager.java:481) at com.top_logic.element.boundsec.manager.StorageAccessManager.handleSecurityUpdate(StorageAccessManager.java:474) at com.top_logic.knowledge.service.db2.DefaultDBContext.handleSecurityUpdate(DefaultDBContext.java:1203) at com.top_logic.knowledge.service.db2.DefaultDBContext.commitTransaction(DefaultDBContext.java:788)
ElementSecurityUpdateManager.handleSecurityUpdate computes, for every updated collection reference, the added and the removed elements:
{{{#!java HashSet<?> addedValues = new HashSet<>(newColValue); addedValues.removeAll(oldColValue); ... HashSet<?> removedValues = new HashSet<>(oldColValue); removedValues.removeAll(newColValue); }}}
oldColValue and newColValue are the List values of the reference. AbstractSet.removeAll(Collection) iterates over the argument and removes from the set only while the set is larger than the argument; otherwise it iterates over the set and calls contains on the argument - a linear scan of an ArrayList per element (JDK-6394757). After an append the old set (30 000) is not larger than the new list (30 001), so the second removeAll performs 30 000 × 30 001 equals calls: the whole 6 s. The first removeAll takes the fast path only by luck of the sizes; after a removal the roles swap and the first one degrades instead.
The append itself is O(1) (LiveOrderedAssociationsList.updateOrderOnAppend); nothing else in the commit shows up in the samples.
Solution
Compute both differences with hash lookups regardless of the sizes: build the two sets and remove element-wise by iterating the other collection (or wrap the argument in a HashSet before removeAll), so the diff is O(|old| + |new|). The same pattern should be checked wherever removeAll / retainAll is called with a list argument on a hot path.
Verification: creating a ticket in the 30 000-ticket project commits without the "Long running commit" warning; a unit test over handleSecurityUpdate with a large collection update stays linear.