Exploring the Heart of AI: Neural Networks, Machine Learning, and Deep Learning

This article was written for the Comprendre newsletter: subscribe to get the next article and more!

Understanding how AI works is very easy, you just need to…. Sadly not, the reality is far from easy. After all, we’re attempting to create an artificial being here! Did you think it would be a walk in the park?

To truly grasp the “how” of AI, one must first delve into the foundations upon which scientists have constructed AI. These foundations come in multiple forms, evolving not only with time but also in response to changing needs.

Machine Learning (ML), Deep Learning (DL), Neural Networks (ANN)… These are terms you’ve likely encountered, but you might not be entirely sure how they differ from one another or how they are interconnected. However, they collectively form the backbone of today’s AI technology.

To grasp the fundamentals of AI, it’s essential to understand the underlying of this trio (ML, DL & ANN). Let’s start to understand their principles one by one, starting with the oldest of the bunch: Artificial Neural Network.

Artificial Neural Network

Imagined in the 1950s, Artificial neural networks (shortened to Neural Networks) is a type of model built using principles of neuronal organization observed and imagined on biological neural network constituting animal brains (like ours).

ANNs consist of interconnected nodes, similar to the neurons in our brains. These nodes process information in three layers: input, hidden, and output. Each neuron receives inputs, applies a transformation, and passes the result to the next layer until you reach the output layer, where you get the final result.

Let’s break it down with an example. Imagine an AI trying to determine if a small black-and-white image contains a hand-drawn cross. In an ANN, the input layer consists of nodes representing the image. For a 16x16 pixel image, you’d have 256 input nodes, one for each pixel. Each node has a value between 0 (black) and 1 (white), representing its color.

The output layer’s goal is to provide a true or false statement: True (the image has a cross) or False (the image doesn’t have a cross). It contains two nodes, one for “True” and one for “False,” each with a value between 0 and 1, indicating the probability of being correct. For instance, if the network thinks there’s a 70% chance of a cross (implying a 30% chance of no cross), the output layer would show “True” with a value of 0.7 and “False” with a value of 0.3.

A key principle of Neural Networks is that they provide probabilistic answers. They won’t definitively say “It has a cross” but rather “There’s a 70% probability of a cross.”

Now, what about the mysterious “hidden” layers? These are layers between the input and output layers and can vary in number and complexity. In our example, a single hidden layer with 10 nodes could suffice. Each node connects to every input node and the two output nodes. Their role is to transform inputs into the final output.

How do they perform this transformation? They use a function that takes into account the value of each input node, a “weight” for each value (assigning different importance to each pixel), and a constant called the “bias.” With these inputs, each hidden layer node calculates an output, which is then fed into the output layer to provide the final probability of a cross in the image.

But how are these weights and biases determined? The network is trained. Initially, the entire hidden layer has random variables. Since they’re random, the likelihood of achieving the desired result is almost nil. Therefore, the neural network needs fine-tuning. To do this, it’s given a set of images with known answers and then adjusts its variables (weights and biases) through trial and error. The ANN analyzes an image, checks if it produces the correct answer, and makes small adjustments if necessary. This process is repeated with the provided training set to improve the network’s accuracy.

A crucial concept to remember is that the size of the training set matters. The larger the training set, the more finely tuned the network becomes, and the better it performs.

In short: Artificial Neural Networks are nothing short of revolutionary! Think of them as your brain’s digital twin, brilliantly designed to mimic the neural organization we see in biology. Composed of interconnected nodes that are neatly sorted into input, hidden, and output layers, these networks are engineered to crunch data, recognize patterns, and make mind-blowing decisions! Each node is a tiny data processor, getting inputs, jazzing them up with weights and biases, and then catapulting the output to the next layer. And here’s the kicker: ANNs are learning machines! Feed them a buffet of data, and they’ll gobble it up, fine-tuning their parameters to become increasingly accurate. From face recognition and natural language processing to conquering board games and beyond, ANNs are the superstars of artificial intelligence.

Want to know more? Check this amazing video from 3Blue1Brown beautifully explaining Neural Networks!

Machine Learning

Venturing forth from the complex world of Artificial Neural Networks, we find ourselves amidst the vast landscape of Machine Learning (ML). If ANNs are the digital equivalent of our brains, ML can be thought of as the process of teaching these digital brains.

Machine Learning, at its core, is the idea that a machine can learn and evolve from experience. Rather than being explicitly programmed to perform specific tasks, a machine learning algorithm uses statistical techniques to parse data, recognize patterns, and make informed decisions. The “learning” in machine learning means the model’s improvement over time as it gets more data.

The primary distinction between traditional algorithms and ML algorithms lies in their adaptability. With traditional programming, developers provide strict rules that dictate every action the machine can perform. In contrast, with ML, the algorithm determines the best action on its own by analyzing data. The role of the developer is simply to set the parameters for learning.

Let’s think of an analogy. Suppose you’re trying to teach someone how to recognize the taste of an apple. In traditional programming, you might describe the apple’s taste in great detail, using a set list of descriptors. In machine learning, you’d let the person taste various fruits, including apples, and allow them to differentiate and recognize the taste based on their experiences.

There are various types of machine learning, including supervised learning, unsupervised learning, and reinforcement learning, each with its nuances and applications.

• Supervised Learning: The most common method. Here, the algorithm is fed labeled data and is essentially “taught” by example. Once the learning is done, the model can apply its learned knowledge to new, unlabelled data. This is akin to a student learning a topic with a teacher’s guidance and then taking a test on unfamiliar questions.
• Unsupervised Learning: This method deals with unlabeled data. The algorithm is left to its own devices to find patterns and relationships in the data. It’s like handing someone a jigsaw puzzle without the picture on the box and asking them to figure it out.
• Reinforcement Learning: Here, the algorithm learns by doing. It’s a trial-and-error method where the machine gets feedback on its actions and adjusts accordingly. Imagine teaching a dog a trick. The dog gets a treat (reward) when it does the trick correctly and gets nothing (or a mild scolding) when it doesn’t. Over time, the dog refines its behavior to maximize its rewards.

Machine Learning is the big umbrella under which ANNs and many other algorithms and techniques exist. While the concept might sound simple — feed data and let the machine learn — the mechanics and math behind it are quite intricate. However, the rewards are enormous. From predicting stock market trends and diagnosing medical conditions to personalizing our online shopping experiences, machine learning is reshaping the way we live, work, and play.

Deep Learning

Now, enter Deep Learning (DL). Often confused with ML, Deep Learning is a subset of it. If Machine Learning is the vast ocean of computational intelligence, then Deep Learning is the deepest trench within it.

Deep Learning deals with algorithms known as neural networks, which, as we discussed earlier, are inspired by the structure and function of the brain. However, when we talk about “deep” in Deep Learning, we refer to the depth of these networks. Deep neural networks (DNNs) are those with a large number of layers, often many more than the simple three-layered structure of basic ANNs.

The depth of these networks allows for incredible complexity. While a basic ANN might excel in recognizing simple patterns in data, a DNN can identify intricate patterns across vast data sets. For instance, while an ANN might recognize the face of a cat, a DNN can differentiate between breeds, ages, or even individual cats.

A key advantage of deep learning is its ability to handle unstructured data, like images or speech. This prowess is the reason why DL drives most of the cutting-edge AI applications today, from real-time language translation and autonomous vehicles to artistic style transfer and deepfake generation.

In the end, understanding the intricate dance between Machine Learning, Deep Learning, and Neural Networks is like understanding the layers of our own cognitive processes. As we delve deeper, we begin to appreciate the nuance, the elegance, and the immense potential that these technologies bring to the table. The world of AI is vast, intricate, and endlessly fascinating, offering a glimpse into a future where machines can think, learn, and evolve almost like us.

As vast as the field of AI is in its theoretical underpinnings, its true marvel lies in its practical applications. How do these concepts, which may seem abstract, play out in real-world scenarios? How do they drive the technologies that are increasingly becoming central to our digital existence?

In our upcoming article next week, we will embark on an exploration of the myriad usages of AI. We’ll delve into Large Language Models (LLM) which power some of the most sophisticated linguistic tools and platforms today. We’ll also dive into concepts like Stable Diffusion and many other applications that are making waves in the tech world. Each of these has unique attributes, functionalities, and possibilities that redefine what machines can achieve.

As we transition from understanding the ‘how’ to the ‘what’, it’s a journey from the core of AI’s foundations to its outer applications — from neural ‘neurons’ to digital dialogues and beyond.

Join us next week as we navigate this vast expanse, highlighting the transformative potential of AI in real-world applications and envisioning a future enriched by these technological wonders.