iterators and fail-fast
modCount and the ConcurrentModificationException you got in a single thread.
- chapter
- chapter 06.10
- reading time
- 7 min read
every structural change to a collection bumps a counter called modCount. an iterator remembers that number when it's created and checks it on every next(). change the collection behind the iterator's back, even in a single thread, even inside a for-each, and the next next() throws ConcurrentModificationException.
the fix: remove through the iterator (it.remove()), or let the collection do it (removeIf).
the picture
you know how you remove one element in a for-each and get an exception called ConcurrentModificationException, in a program with exactly one thread? kinda cursed. the name is about the iterator's point of view: something modified the list concurrently with it, meaning while it was in the middle of walking. that something was you.
think of a tour guide walking a group through a museum with a printed map. if a room gets closed mid-tour, the map is wrong, and a good guide stops instead of walking people into a wall. the map has a version number (the iterator's expectedModCount). the museum has the current one (modCount). the guide checks them at every doorway.
what a for-each really is
the compiler turns a for-each over a collection into an iterator loop:
for (String s : list) {
if (s.equals("b")) list.remove(s);
}Iterator<String> it = list.iterator();
while (it.hasNext()) {
String s = it.next();
if (s.equals("b")) list.remove(s);
}(the left side isn't wrong because it's a for-each. it's wrong because it calls list.remove while an iterator is walking list, and the for-each hides that iterator from you.)
ArrayList's iterator is three ints: cursor (the next index to return), lastRet (the index it returned last) and expectedModCount. hasNext() is just cursor != size. next() first checks modCount == expectedModCount, then returns elementData[cursor++].
- 1Iterator<String> it = list.iterator();
- 2while (it.hasNext()) {
- 3 String s = it.next();
- 4 if (s.equals("b")) list.remove(s);
- 5}
- 6IO.println(list);
the for-each asks the list for an iterator. it remembers modCount (0) as expectedModCount.
list.remove(s), removing "a", "b" or "d" ends in ConcurrentModificationException. removing "c" doesn't throw at all, and that's worse: watch which element never gets visited. then switch to it.remove().void main() {
List<String> list = new ArrayList<>(List.of("a", "b", "c", "d"));
IO.println(list);
for (String s : list) {
if (s.equals("b")) list.remove(s);
}
}[a, b, c, d] Exception in thread "main" java.util.ConcurrentModificationException
List<String> list = new ArrayList<>(List.of("a", "b", "c", "d"));
for (String s : list) {
if (s.equals("c")) list.remove(s);
}
IO.println(list);the fixes
for (String s : list) {
if (s.isBlank()) list.remove(s);
}list.removeIf(String::isBlank);removeIf(predicate)java 8+: the clean answer for "remove everything that matches".ArrayList's version is also faster than removing one by one: it marks the doomed elements first, then compacts the array in a single pass, instead of shifting the tail for every removal.iterator.remove(): removes the elementnext()just returned and resynchronises the iterator (cursor = lastRet,expectedModCount = modCount). use it when the decision needs more than a predicate.ListIterator:setandaddthrough the iterator, for lists.- maps:
map.entrySet().removeIf(...),map.keySet().removeIf(...), andreplaceAll/computeIfPresentfor updating values.
void main() {
List<String> a = new ArrayList<>(List.of("a", "b", "c", "d"));
for (Iterator<String> it = a.iterator(); it.hasNext(); ) {
if (it.next().equals("b")) it.remove();
}
List<String> b = new ArrayList<>(List.of("a", "b", "c", "d"));
b.removeIf(s -> s.equals("b"));
IO.println(a + " " + b);
Map<String, Integer> m = new HashMap<>(Map.of("a", 1, "b", 2, "c", 3));
m.entrySet().removeIf(e -> e.getValue() % 2 == 1);
m.replaceAll((k, v) -> v * 10);
IO.println(m);
}[a, c, d] [a, c, d]
{b=20}what counts as a structural modification?+
only changes to the shape: anything that adds or removes elements (and clear, and a HashMap resize). replacing a value doesn't count. list.set(i, x) during a for-each over an ArrayList is fine, and so is map.put(k, v) for a key that's already there. the one surprise is LinkedHashMap in access order, where a plain get relinks the entry and counts as structural (see LRU caches).
fail-safe iterators
the concurrent collections in java.util.concurrent don't throw CME. they promise something different instead.
CopyOnWriteArrayList copies its whole array on every write. an iterator grabs the array that existed when it was created (a snapshot) and walks that, while writers swap in new arrays:
add("d") built a new 4-element array and pointed the list at it. the iterator still points at the old 3-element snapshot, which never changes, so iterating can't fail and won't see "d".import java.util.concurrent.CopyOnWriteArrayList;
void main() {
List<String> list = new CopyOnWriteArrayList<>(List.of("a", "b", "c"));
for (String s : list) {
if (s.equals("a")) list.add("d");
IO.print(s + " ");
}
IO.println(list);
}a b c [a, b, c, d]
great for lists that are read constantly and changed rarely (listener lists, config), terrible for lists that change a lot (every write is O(n)). ConcurrentHashMap takes a third approach: its iterators are weakly consistent. they never throw, they visit each element at most once, and they may or may not reflect changes made after they started. more in concurrent collections.
- every add/remove bumps
modCount. iterators check it on everynext(). list.removeinside a for-each: CME (or, worse, a silently skipped element).- remove with
it.remove()orremoveIf. update values withset/replaceAll. - fail-fast is a debugging aid, not thread-safety.
- concurrent collections iterate snapshots or weakly consistent views, and never throw CME.