For and For-Each Loop: a tale of how I iterated, iterated, but not iterated

Introduction

Loops are one of the basic structures of programming languages. For example, the Oracle website has a section called " Lesson: Language Basics " that has a separate lesson called " The for Statement " for loops. Let's refresh the memory of the main thing: The loop consists of three expressions (statements): initialization (initialization), condition (termination) and increment (increment): Interestingly, they are all optional, meaning we can, if we want, write:

for (;;){
}

True, in this case we will get an infinite loop, because we do not have a condition for exiting the loop (termination). The initialization expression is only executed once, before the entire loop is executed. It is always worth remembering that the loop has its own scope. This means that initialization , termination , increment , and the loop body all see the same variables. Scope is always easy to define with curly braces. Everything inside the brackets is not visible outside the brackets, but everything outside the brackets is visible inside the brackets. Initializationis just an expression. For example, instead of initializing a variable, you can generally make a method call that will not return anything. Or just skip, leaving a blank space before the first semicolon. The following expression specifies a termination condition . As long as it is true , the loop is executed. And if false , a new iteration will not start. If you look at the picture below, we get an error during compilation and the IDE will swear: our expression in the loop is unreachable. Since we will not have a single iteration in the loop, we will immediately exit, because false: It is worth keeping an eye on the expression in the termination statement : it directly depends on whether there will be infinite loops in your application. Increment is the simplest expression. It is executed after each successful iteration of the loop. And this expression can also be skipped. For example:

int outerVar = 0;
for (;outerVar < 10;) {
	outerVar += 2;
	System.out.println("Value = " + outerVar);
}

As you can see from the example, each iteration of the loop, we will increment by 2, but only as long as the value outerVaris less than 10. Also, since the expression in the increment statement is really just an expression, it can contain anything. Therefore, no one forbids using a decrement instead of an increment, i.e. decrease the value. You should always follow the writing of the increment. +=first performs an increase, and then an assignment, but if in the example above we write the opposite, we will get an infinite loop, because the variable will outerVarnever receive a changed value: in the case it =+will be calculated after the assignment. By the way, ++it's the same with the view increment. For example, we had a loop:

String[] names = {"John","Sara","Jack"};
for (int i = 0; i < names.length; ++i) {
	System.out.println(names[i]);
}

The cycle worked and there were no problems. But here came the refactoring man. He did not understand the increment and simply did this:

String[] names = {"John","Sara","Jack"};
for (int i = 0; i < names.length;) {
	System.out.println(names[++i]);
}

If the increment sign is in front of the value, it means that it will first increase, and then return to the place where it is specified. In this example, we will immediately start getting the element at index 1 from the array, skipping the first one. And then at index 3 we will fall with the error " java.lang.ArrayIndexOutOfBoundsException ". As you may have guessed, this used to work simply because the increment is called after the iteration has been completed. When transferring this expression to an iteration, everything broke. As it turns out, even in a simple loop, you can mess things up) If there is an array, can it be somehow easier to display all the elements?

For each loop

Since Java 1.5, the Java developers have given us a construct for each loopdescribed on the Oracle site in the Guide called " The For-Each Loop " or for version 1.5.0 . In general, it will look like this: In the Java Language Specification (JLS), you can read the description of this construct to make sure that this is not magic at all. This construct is described in the chapter " 14.14.2. The enhanced for statement ". As you can see, for each loop can be used with arrays and with those that implement the java.lang.Iterable interface . That is, if you really want to, you can implement the java.lang.Iterable interface and for each loop can be used with your class. You will immediately say "Yes, an iteration object, but an array is not an object. Sort of." And you will be wrong, because. in Java, arrays are dynamically created objects. This is what the language specification tells us:In the Java programming language, arrays are objects ". In general, arrays are a bit of JVM magic, because how the array is arranged inside is unknown and is located somewhere inside the Java virtual machine. If you are interested, you can read the answers to stackoverflow: " How does array class work in Java? ". It turns out that if we are not using an array, then we should use something that implements Iterable . For example:

List<String> names = Arrays.asList("John", "Sara", "Jack");
for (String name : names) {
	System.out.println("Name = " + name);
}

Here you can just remember that if we use collections ( java.util.Collection ), thanks to this we get exactly Iterable . If an object has a class that implements Iterable, it is required to provide, when calling the iterator method, an Iterator that will iterate over the contents of that object. The code above, for example, would have something like the following bytecode (in IntelliJ Idea you can do "View" -> "Show bytecode" : As you can see, an iterator is indeed being used. If not for for each loop , we would have to write something like:

List<String> names = Arrays.asList("John", "Sara", "Jack");
for (Iterator i = names.iterator(); /* continue if */ i.hasNext(); /* skip increment */) {
	String name = (String) i.next();
	System.out.println("Name = " + name);
}

Iterator

As we saw above, the Iterable interface says that for instances of some object, you can get an iterator, with which you can iterate over the contents. Again, one could say that this is the Single Responsibility Principle from SOLID . The data structure itself shouldn't drive the traversal, but it can provide one who should. The basic implementation of Iterator is that it is usually declared as an inner class that has access to the contents of the outer class and provides the desired element contained in the outer class. Here is an example from a class ArrayListof how an iterator returns an element:

public E next() {
            checkForComodification();
            int i = cursor;
            if (i >= size)
                throw new NoSuchElementException();
            Object[] elementData = ArrayList.this.elementData;
            if (i >= elementData.length)
                throw new ConcurrentModificationException();
            cursor = i + 1;
            return (E) elementData[lastRet = i];
}

As we can see, using ArrayList.thisthe iterator gets access to the external class and its variable elementData, after which it returns an element from there. So getting an iterator is very simple:

List<String> names = Arrays.asList("John", "Sara", "Jack");
Iterator<String> iterator = names.iterator();

Its work boils down to the fact that we can check if there are elements further (the hasNext method ), get the next element (the next method ) and the remove method , which removes the last element received through next . The remove method is optional and is not guaranteed to be implemented. In fact, with the development of Java, interfaces are also being finalized. Therefore, in Java 8, there was also a method forEachRemainingthat allows you to perform some action on the remaining elements not visited by the iterator, some action. What is interesting about an iterator and collections? For example, there is a class AbstractList. This is an abstract class that is the parent of ArrayListandLinkedList. And it is interesting to us because of such a field as modCount . Each change in the contents of the list changes. And what do we have from that? And the fact that the iterator makes sure that during operation there is no change in the collection over which it is iterated. As you understand, the iterator implementation for lists is in the same place as modcount , that is, in the AbstractList. Consider a simple example:

List<String> names = Arrays.asList("John", "Sara", "Jack");
names = new ArrayList(names);
Iterator<String> iterator = names.iterator();
names.add("modcount++");
System.out.println(iterator.next());

Here is the first interesting, albeit off topic. Actually Arrays.asListreturns its own special ArrayList( java.util.Arrays.ArrayList ). It does not implement add methods, so it is non-modifiable. It is written about in JavaDoc: fixed-size . But in fact, it's more than fixed-size . It is also immutable , that is, immutable; remove won't work on it either. And we will also get an error, because having created an iterator, we remembered modcount in it . Then we changed "outside" (i.e. not through the iterator) the state of the collection and executed the iterator method. Therefore, we get an error: java.util.ConcurrentModificationException. To avoid this, the change during iteration must be done through the iterator itself, and not through accessing the collection:

List<String> names = Arrays.asList("John", "Sara", "Jack");
names = new ArrayList(names);
Iterator<String> iterator = names.iterator();
iterator.next();
iterator.remove();
System.out.println(iterator.next());

iterator.remove()As you understand, if you don’t do it before iterator.next(), then because the iterator does not point to any element, then we will get an error. In the example, the iterator will go to the element John , remove it, and then get the element Sara . And here everything would be fine, but that's bad luck, again there are "nuances") java.util.ConcurrentModificationException will be only when hasNext()it returns true . That is, to remove the last element through the collection itself, then the iterator will not fall. For more details, it is better to see the report about Java puzzles with " #ITsubbotnik JAVA section: Java puzzles ". We started such a detailed conversation for the simple reason that exactly all the same nuances apply whenfor each loop, because inside "under the hood" our favorite iterator is used. And all these nuances apply there. The only thing is, we won't have access to the iterator, and we won't be able to safely remove the element. By the way, as you understand, the state is remembered at the moment of creation of the iterator. And safe delete only works where called. That is, this option will not work:

Iterator<String> iterator1 = names.iterator();
Iterator<String> iterator2 = names.iterator();
iterator1.next();
iterator1.remove();
System.out.println(iterator2.next());

Because for iterator2, deletion through iterator1 was "external", that is, it was performed somewhere outside and he knows nothing about it. On the topic of iterators, I would like to note one more thing. Especially for interface implementations, Lista special, extended iterator was made. And they named him ListIterator. It allows you to move not only forward, but also backward, and also allows you to find out the index of the previous element and the next. It also allows you to replace the current element or insert a new one at a position between the iterator's current position and the next one. As you guessed, ListIteratorit is allowed to do so as Listaccess by index is implemented for.

Java 8 and iteration

The release of Java 8 has made life easier for many. Not bypassed and iteration over the contents of objects. To understand how it works, you need to say a few words about this. Java 8 introduced the java.util.function.Consumer class . Here is an example:

Consumer consumer = new Consumer() {
	@Override
	public void accept(Object o) {
		System.out.println(o);
	}
};

Consumer is a functional interface, which means that there is only 1 unimplemented abstract method inside the interface that requires mandatory implementation in those classes that specify implements of this interface. This allows you to use such a magical thing as a lambda. This article is not about that, but we need to understand why we can use it. So, with the help of lambdas, the above Consumer can be rewritten like this: Consumer consumer = (obj) -> System.out.println(obj); This means that Java sees that something called obj will be passed to the input, and then the expression after -> will be executed for this obj. As for iteration, now we can do this:

List<String> names = Arrays.asList("John", "Sara", "Jack");
Consumer consumer = (obj) -> System.out.println(obj);
names.forEach(consumer);

If you jump into the method forEach, you will see that everything is insanely simple. There we all love for-each loop:

default void forEach(Consumer<? super T> action) {
        Objects.requireNonNull(action);
        for (T t : this) {
            action.accept(t);
        }
}

It is also possible to nicely remove an element using an iterator, for example:

List<String> names = Arrays.asList("John", "Sara", "Jack");
names = new ArrayList(names);
Predicate predicate = (obj) -> obj.equals("John");
names.removeIf(predicate);

In this case, the removeIf method takes as input not a Consumer , but a Predicate . It returns boolean . In this case, if the predicate says " true ", then the element will be removed. Interestingly, not everything is obvious here either)) Well, what do you want? It is necessary to give people space to create puzzles at the conference. For example, let's take the following code for removing everything that the iterator can reach after some iteration:

List<String> names = Arrays.asList("John", "Sara", "Jack");
names = new ArrayList(names);
Iterator<String> iterator = names.iterator();
iterator.next(); // Курсор на John
while (iterator.hasNext()) {
    iterator.next(); // Следующий элемент
    iterator.remove(); // Удалor его
}
System.out.println(names);

Okay, everything works here. But we remember that Java 8 is after all. Therefore, let's try to simplify the code:

List<String> names = Arrays.asList("John", "Sara", "Jack");
names = new ArrayList(names);
Iterator<String> iterator = names.iterator();
iterator.next(); // Курсор на John
iterator.forEachRemaining(obj -> iterator.remove());
System.out.println(names);

Has it really become more beautiful? However, there will be a java.lang.IllegalStateException here . And the reason is... a bug in Java. It turns out that it is fixed, but in JDK 9. Here is a link to the task in OpenJDK: Iterator.forEachRemaining vs. Iterator.remove . Naturally, this has already been discussed: Why iterator.forEachRemaining doesnt remove element in the Consumer lambda? Well, another way is directly through the Stream API:

List<String> names = new ArrayList(Arrays.asList("John", "Sara", "Jack"));
Stream<String> stream = names.stream();
stream.forEach(obj -> System.out.println(obj));

conclusions

As we have seen from all the material above, the loop for-each loopis just "syntactic sugar" over the iterator. However, it is now used in many places. In addition, you need to use any tool with caution. For example, harmless forEachRemainingcan cover unpleasant surprises. And this once again proves that unit tests are needed. A good test would be able to identify such a use case in your code. What you can see / read on the topic:

• Conference video: " #ITsubbotnik JAVA section: Java puzzles "
• Video from the conference: " Java 8 Puzzlers. Zurab Bely "
• Good article on Java Rush: " The Iterator Pattern "

#Viacheslav