In the materials on JavaRush, we have already touched on the topic of artificial intelligence (AI), machine learning and other technologies related to AI in one way or another. In particular, we have already talked about how artificial intelligence is related to programming in Java , discussed myths and horror stories about AI , and in a number of articles we mentioned key technologies that are used in this area. Therefore, we decided that now is a good time to delve even deeper into the field of AI, and take a closer look at the technology that actually underlies modern AI and machine learning - neural networks. Namely: how the concept of neural networks originally originated and how this technology was transformed into its current state, how artificial neural networks differ from neurons in the human brain, whether they really know how to learn, and what they are really capable of today.
Artificial neural networks and the human brain. Same?
Initially, computer neural networks received this name because the principle of their design resembles the system of neurons in the human brain. Thus, the average human brain includes more than 100 million tiny cells called neurons. In fact, no one knows exactly how many such cells there could actually be in the human brain - estimates range from 50 to 500 billion. Each neuron consists of a cell body with many so-called dendrites (cell inputs - transmit information to the cell) and one axon ( cell output - transmits information from the cell). Neurons are so small that about a hundred of these cells can be placed in an area of one square millimeter. Inside a computer, the equivalent of a neuron are tiny switching devices called transistors. The latest modern microprocessors (single-chip computers) contain more than 30 billion transistors. Even the first Pentium microprocessor, created about 20 years ago, had about 50 million transistors packed into an integrated circuit with an area of only 25 mm. In fact, experts say, this is where the connection between computer processors and neurons in the human brain ends, because otherwise they are very different from each other and work completely differently. Computer transistors are connected in relatively simple series circuits, each of which can be connected to two or three others, whereas neurons in the brain are tightly connected in complex parallel ways. Thus, scientists have calculated that each neuron can be connected to more than 10 thousand of its neighbors.
What is an artificial neural network?
Thus, the most significant structural difference between computers and neurons in the brain is how they work. The basic idea behind neural networks is to simulate, or, if you will, model, in a simplified form, the operation of many interconnected neurons in the brain. This is what allows a neural network to do things that ordinary microprocessors cannot do, namely, learn, recognize patterns and make decisions on a principle similar to the human brain. However, it is important to emphasize that at the moment, neural networks are a simple software emulation of the work of neurons in the brain. A typical neural network contains from several tens to hundreds, thousands or even millions of artificial neurons, called units, arranged in several layers. Some of these units, called input units, are responsible for receiving various forms of information from the outside world, which the neural network will then try to learn, remember and process in other ways. Other units, called output units, located on the opposite side of the network, are responsible for how the network interacts with this information. Between these two layers are other units that do the computation and make up the bulk of the neural network. Most neural networks are interconnected, meaning each unit is connected to other units in layers on either side. The connections between units are represented by a number called weight. Weight in a neural network can be positive (if one unit activates another) or negative (if one unit suppresses or weakens another). The higher the weight, the greater the influence of one unit on another. A simple neural network designed to solve simple problems may only include three basic layers, as shown in the picture. However, between the input and output layers there can be a number of different layers that perform different functions. Neural networks with more complex structures (than in the illustration above) are called deep neural networks and are usually used to solve complex problems. In theory, the use of deep structure allows a neural network to map any type of input data to any type of output data, thus making it capable of solving any problem. However, for this, the neural network will require serious and lengthy “training,” which consists of processing millions and even billions of examples of information that the network needs to work with.
Neural networks: a little history
Well, we’ve sorted out the basic explanations of what neural networks essentially are, now we can step back a little about the theory and talk about the history of this technology and when it even appeared. This may surprise many, but neural networks appeared in science more than 70 years ago. The concept was first proposed in 1944 by Warren McCullough and Walter Pitts, two researchers at the University of Chicago who moved to the Massachusetts Institute of Technology in 1952, where they founded the first cognitive science research group. Neural networks were actually a major area of research in both neuroscience and computer science until 1969. That is, this technology, which underlies modern artificial intelligence systems, has already been, in modern terms, on hype for more than a whole decade - from about the mid-fifties to the end of the sixties. It is believed that the “killer” of this hype was a book authored by two computer scientists - Marvin Minsky and Seymour Papert - who in their work revealed all the limitations of neural networks. The fact is that early neural networks only had one or two trainable layers. Minsky and Papert showed that such simple networks are mathematically incapable of modeling complex real-world phenomena. As a result, interest in neural networks as a technology was lost for many years - throughout the seventies and the first half of the eighties, objectively little attention was paid to them. Interest was only renewed in 1986, with the publication of a paper that introduced the concept of backpropagation, a practical method for training deep neural networks. Thanks to backpropagation, deep neural networks became more computationally friendly, but still required huge computing resources to operate. And research in the 1990s and 2000s often showed that complex neural networks did not produce the desired results in terms of efficiency. As a result, neural networks fell off the hype wave for a while and only returned in the second decade of the 2000s, largely due to the increased computing power of graphics chips, which allow deep neural networks to work more efficiently than before. The phenomenon of such cyclicity in science as a whole is well explainedTomaso Poggio, a professor at the Massachusetts Institute of Technology: “There is an explanation that ideas in science spread in much the same way as viruses do. For example, there are five or six main strains of the influenza virus, and each one comes back about once every 25 years. People become infected, develop immunity, and are protected from infection for the next 25 years. And then a new generation comes and is again infected with the same strain of the virus. In science, people fall in love with ideas and work on them to the point of exhaustion, so much so that they develop immunity in the form of idea fatigue. And then the idea comes back with another generation of scientists.” Given the fact that neural networks and artificial intelligence technologies are still a fairly new and underdeveloped technology, it is possible that the hype around it will soon begin to wane again, only to return again in another couple of decades.
How do neural networks learn?
Typically, information passes through a neural network in two ways. When the network is just learning or is already running after training, patterns of information enter it through input units, which launch layers of hidden units, which in turn enter output units. This overall structure is called a feedforward network. Not all units work at the same time. Each unit receives incoming data from the units on the left and multiplies it by the weight of the connections it passes through. Each unit adds up all the data it receives in this way and, in the simplest type of network, if the sum exceeds a certain threshold, the unit activates the other units it is connected to (those on the right). One of the key points that allows any neural network to learn is feedback. Just as a child needs to be raised by an adult who will tell him what is good and what is bad, neural networks need feedback and guidance. They use an approach for training called backpropagation . In this method, the network compares the output it produces with the output it should produce, and uses the difference between these two factors to change the weight of the connections between units in the network, from output units, through intermediate units to input units - that is, vice versa. Over time, backpropagation forces the network to learn, reducing the difference between the actual and estimated output to the point where they exactly match.
How do neural networks work and where are they used?
And finally, let’s look at an example of how such a neural network works. Once the neural network has enough examples of data to train on, it reaches a point where the user can feed it completely new data that the network has never processed before and get the desired result from it. For example, let's say you train your neural network to detect a weapon when it appears somewhere within view of a security camera. You train the network by showing it hundreds, thousands, and tens of thousands of images of different types of weapons, such as pistols and rifles. At some point, you upload an image of a new type of weapon that has not been shown to her before, for example, a gun. As a result, the network successfully identifies this object as a weapon based on a number of basic criteria, such as general shape, the presence of a handle and butt, the presence of a trigger, and so on. That is, the neural network determines the weapon in the image based on a number of “learned” patterns. At the same time, of course, this does not mean that the neural network can view new data and independently make a decision, as people do. It receives all information in the form of a binary code, and each of the input units (information into the network) can be in only two positions: on or off. Accordingly, if you, using the same example, train a neural network to find weapons, the information will pass through a set of input units that answer basic “yes/no” questions. For example, such as: does the object have a trigger?; is there a muzzle?; is the item made of metal?; does its shape match that of a pistol? Accordingly, for a gun, the network's response would be: “yes, yes, yes, no” or “1110” in binary code. Of course, in a real system there would be much more such questions, but the essence remains the same: the neural network receives data in the form of a binary code, and with its help learns to distinguish the necessary information from the unnecessary. Now, knowing the basic mechanism of the neural network, it is easy to imagine how this technology is used to solve a variety of problems today. For example, a classic example is the use of neural networks to prevent bank fraud - when the system learns to independently find and instantly block those transfers that for one reason or another seem suspicious. Or, for example, the use of neural networks for the basic autopilot of cars or airplanes: when input units receive information from different control systems, and output units adjust the piloting mode in accordance with a given course.
conclusions
Now you see that a neural network is a relatively uncomplicated, overall, technology that really moves computers and computer systems a step forward, closer to people. On the other hand, it is also obvious that neural networks are still not nearly as complex and efficient as the human brain. So let's be grateful that we are all born with an advanced neural network built right into our brains, and let's use this natural wealth more often, for example, for learning and self-improvement.
Therefore, we decided that now is a good time to delve even deeper into the field of AI, and take a closer look at the technology that actually underlies modern AI and machine learning - neural networks. Namely: how the concept of neural networks originally originated and how this technology was transformed into its current state, how artificial neural networks differ from neurons in the human brain, whether they really know how to learn, and what they are really capable of today. 
A typical neural network contains from several tens to hundreds, thousands or even millions of artificial neurons, called units, arranged in several layers. Some of these units, called input units, are responsible for receiving various forms of information from the outside world, which the neural network will then try to learn, remember and process in other ways. Other units, called output units, located on the opposite side of the network, are responsible for how the network interacts with this information. Between these two layers are other units that do the computation and make up the bulk of the neural network. Most neural networks are interconnected, meaning each unit is connected to other units in layers on either side. The connections between units are represented by a number called weight. Weight in a neural network can be positive (if one unit activates another) or negative (if one unit suppresses or weakens another). The higher the weight, the greater the influence of one unit on another. A simple neural network designed to solve simple problems may only include three basic layers, as shown in the picture. However, between the input and output layers there can be a number of different layers that perform different functions. Neural networks with more complex structures (than in the illustration above) are called deep neural networks and are usually used to solve complex problems. In theory, the use of deep structure allows a neural network to map any type of input data to any type of output data, thus making it capable of solving any problem. However, for this, the neural network will require serious and lengthy “training,” which consists of processing millions and even billions of examples of information that the network needs to work with. 
GO TO FULL VERSION