When it comes to Artificial Intelligence (AI), Deep Learning (DL) is one of the hottest topics. It’s an exciting area that’s garnered much interest from AI researchers due to its promising results.

Deep Learning is everywhere now because of the progress of Big Data, which gives us a tremendous amount of storage and data processing power (thanks to the advancement of computing resources like Graphical Processing Units (GPUs) and Tensor Processing Units (TPUs)). It’s used in various fields such as music composition, art creation, object recognition, natural language processing, medical research, etc.

The main goal of this post is to introduce, explain, and get more people interested in Deep Learning: what is its basic foundation? What sets it apart from traditional Machine Learning? We’ll also review some of the most popular AI architectures and briefly discuss some industry best practices for integrating AI into your business.

 

Deep Learning/Machine Learning vs. Traditional Programming

 

First, let’s address a common question: what separates a rule engine (traditional if-else programming) from the well-known Deep Learning (DL) and Machine Learning (ML) techniques?

Classical, or “traditional,” programming combines input data with a predefined set of rules (such as business rules) to produce the desired output. In DL/ML, on the other hand, inputs and outputs are provided, and the rules are determined during the training phase of the models.

 

What Is Deep Learning?

 

First off, Deep Learning is a Machine Learning (ML) paradigm. It works by using neural networks (NN). Their job is to extract features and patterns from the raw data automatically.

 

What Is an Artificial Neural Network?

 

A neural network is a set of neurons inspired by how the human brain works and is used to extract and learn information. These neurons can be stacked on top of one another to make layers. This gives them a certain depth, which is where the term “Deep Learning” comes from.

See an example of a neural network with two hidden layers below.

Note: there must be at least three hidden layers for a neural network to be considered “deep.”

 

What Is an Artificial Neuron?

 

An artificial neuron, also called a perceptron, is the smallest unit in a neural network. A biological neuron and an artificial neural network are compared below.

 

The perceptron consists of:

1. Input layer (xi = input values)
2. Weights (wi)
3. Sum (xi * wi)
4. Activation function
5. Output

 

When Was Deep Learning Created?

 

Are you curious about the origins of Deep Learning, neural networks, and perceptrons now that you understand them better?

As Deep Learning requires massive amounts of data and computing (GPU/TPU), it has only recently been practical, but the original understanding of how neurons work dates back to 1943, and the earliest theorization of Deep Learning dates back to the 1980s.

Graphics card type used for training neural networks

 

Deep Learning vs. Machine Learning

 

Machine Learning is a smart option when working with a small, well-prepared dataset or when you need a basic model quickly.

There are several distinctions between Deep Learning and Machine Learning. In contrast to most Machine Learning methods, Deep Learning relies on neural networks. On top of that, feature extraction is done automatically, but in ML, feature vectors (data descriptors) representing the raw data must be created by hand.

 

We find that traditional Machine Learning algorithms have difficulty extracting relevant data for learning when presented with a complicated dataset. This phenomenon is known as the “curse of dimensionality.” The problem is solved by employing dimension reduction techniques like Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Independent Component Analysis (ICA), etc.

Here is a look at how Deep Learning and Machine Learning perform in terms of accuracy relative to the amount of data. This example isn’t always accurate. In reality, you need to try out many approaches and algorithms before settling on a final solution while keeping in mind that the quality of your data will significantly impact the accuracy of your results.

 

Please see the following articles written by our Chief Technology Officer (CTO), Yann Bilissor, for an introduction to Artificial Intelligence and Machine Learning:

• Artificial Intelligence/Machine Learning: From Perception to Action
• AI/Machine Learning: Creating Intelligent Agents

 

What Are the Neural Network Types?

 

Convolutional neural networks (CNNs)

 

Convolutional neural networks (CNNs) are typically used for computer vision and image analysis. They have proven themselves effective in the analysis and modeling of spatial data, such as 2D and 3D images and videos, outperforming more traditional methods (feature descriptors, like scale-invariant feature transform (SIFT), features from accelerated segment test (FAST), speeded up robust features (SURF), etc.) that need human involvement to provide a more accurate representation of the data.

The two main components of a CNN network are the convolutional and the classification layers. The convolution layer uses a series of filters to extract each image’s features. The classification layer then uses these extracted features to accurately classify the original image.

CNNs can have different architectures, such as:

• AlexNet
• VGGNet
• ResNet

 

Recurrent neural networks (RNNs)

 

Recurrent neural networks (RNNs) are mainly used to model sequential data like text, audio, time series, etc. Text analysis tasks often use them (natural language processing (NLP)).

Classical neural networks were designed to work with data points that can be treated as discrete entities. But if the data is sequential, with each point depending on the one before it, traditional NNs will fail to extract the relevant information properly. For this reason, RNNs were developed to account for the interdependencies between these data points. They have a “memorization” feature that helps them store the states or data of prior inputs and use that knowledge to produce the next output in the sequence.

RNNs can have different architectures, such as:

• Bidirectional recurrent neural networks (BRNNs)
• Gated Recurrent Units (GRUs)
• Long Short-Term Memory (LSTM)

 

 

Generative Adversarial Networks (GANs)

 

Generative adversarial networks are unsupervised algorithms. The way these neural networks are used is different. They are not used for traditional data analysis like classification, where the goal is to discriminate between the different data types as much as possible. GANs try to generate similar data for a given data class.

In practice, the GAN architecture is made up of two neural networks that compete with each other.
The Generator generates new data, and the Discriminator tries to determine if the data produced is original or came from its “opponent,” the Generator.

GANs can have different architectures, such as:

• CycleGAN
• StyleGAN
• DiscoGAN

 

Transformers

Transformers are currently the “rock stars” of Deep Learning. They have successfully replaced RNNs (LSTM and GRU) in natural language processing. They are now also being used for computer vision and may eventually replace convolutional neural networks (CNNs) altogether.

A transformer is a sequence-to-sequence (seq2seq) neural network created in 2017. It has an Encoder and a Decoder and is based on self-attention. This network type can be trained considerably more quickly than traditional RNNs since it doesn’t require the sequential data to be in a specific order.

Transformers have different architectures, such as:

• Bidirectional Encoder Representations from Transformers (BERTs)
• Generative Pre-trained Transformer 3 (GPT-3)
• Robustly Optimized BERT Pre-Training Approach (RoBERTA)

 

Deep Learning and Machine Learning for Your Business or Organization

 

If used correctly, Deep Learning and Machine Learning can be the modernization accelerators your company needs. They give us greater vision and help us identify fresh opportunities.

Some repetitive and mundane tasks can be completed faster with DL/ML. In addition, support from artificial intelligence tools lowers the margin of error, boosts user confidence, and enriches the customer experience.

Using AI in your business or organization doesn’t mean getting rid of people and giving full control to an application. We are a long way off that! Artificial intelligence, however, can be used as a tool to help us do our jobs better, in particular by allowing us to zero in on what really matters for our companies and our core business.

The most popular cloud providers, like Microsoft Azure Cloud, Amazon Cloud, and Google Cloud, all provide several customized and tailored products that make it simple for businesses to adopt AI solutions. For instance, a business with sufficient AI expertise and resources can afford a Machine Learning platform like Azure ML Service, which facilitates the management, control, and acceleration of the ML project cycle (known as MLOps or the industrialization of ML projects), but more importantly, frees up data scientists to focus on their actual tasks.

Most data scientists have an exclusively academic background (engineers or physicians), and this can lead to problems with software engineering best practices like clean code, clean architecture, test-driven development (TDD), DevOps (continuous integration/continuous delivery), etc. This can hinder the progress of a DL/ML project. Obviously, a data scientist doesn’t have to do everything. However, as data science evolves, it’s always a good idea to broaden your skills and learn new ones. To do this, you need the right tools, but more importantly, you need to build comprehensive teams with varied profiles:

• Data platform team
• Dev team
• Data science team
• ML platform team
• Sales team
• Product team
• Manager

Another scenario is a business that is interested in using AI solutions but lacks the necessary expertise in-house. It can use AI service APIs. These can be added to any application. Integrating these APIs doesn’t require deep knowledge of AI and can be done by anybody with development skills.

Microsoft Azure Cloud provides Azure Cognitive Services, which offer a range of services to address various topics such as:

• Vision
• Speech
• Language
• etc.

Azure Cognitive Services (Source: Microsoft Documentation)

 

If you want to learn more about how to implement AI without a data scientist, you can watch our Digital Advisor Nicolas Robert and Justine Charley, Architect and Data Scientist at Microsoft, at Microsoft Build 2022: “Comment embarquer de l’IA dans vos projets sans avoir une équipe de Data Scientists” [How to Embed AI in Your Projects Without a Team of Data Scientists – in French only]

 

Deep Learning and Machine Learning in Business and Research

 

Deep Learning/Machine Learning in business is not the same as Deep Learning/Machine Learning in research. In short, in business (production), we have different stakeholders, and we prioritize fast inference time and the lowest latency possible for our service. The data is constantly changing (data shifting), and, above all, the result must meet the standards of responsible AI. However, in research, we aim for a model with performance near to the state-of-the-art (SOT): we care more about minimizing training time because our data remains static. In most cases, we use benchmark datasets and focus less on the responsible AI aspect (we are much more interested in pure innovation).

As you can see, the two worlds have very different conditions and objectives. Implementing AI systems in business is undoubtedly more complex: it’s estimated that 80% of projects fail at the proof of concept (POC) stage, but when MLOps practices are used, we can see a clear improvement. There are some practices and habits that can hinder the progress of an ML project, however—for example, forgetting to improve model metrics and trying to get better accuracy at all costs and increase accuracy by 0.4% by using more data, calculations, and time. To make matters worse, businesses care less about the algorithms employed and the modified metrics unless there is an effect on the business indicators, which are more relevant and clearer for managers. Whether we like it or not, ML projects that are not making a profit will be scrapped because they are not adding value.

So, as data scientists, we need to understand the real business challenges, the project’s end goal, and, most importantly, how to connect the algorithm metrics to the company’s business metrics and find a balance.

This doesn’t mean you can’t have fun or employ cutting-edge tech. Instead, it calls for a more measured and mature approach that lays the groundwork slowly but steadily so that you don’t lose sight of your eventual aim (starting with a classical model or doing transfer learning).

 

Responsible AI

 

Responsible AI is an important and complicated topic: we could write whole articles or even books about it. In a nutshell, responsible AI is the design, development, and industrialization of an AI product with good intentions and care to ensure a positive impact on individuals and society without discrimination or injustice. There are many definitions and principles for AI in the literature. In this post, we’ll talk about Microsoft’s responsible AI principles:

• Fairness
• Reliability and safety
• Privacy and security
• Inclusiveness
• Transparency
• Accountability

Responsible AI by Microsoft (Source: Microsoft Documentation)

 

If you want to learn more about managing the life cycle of an Machine Learning project in line with the principles of responsible AI, you can read the article by Nathalie Fouet, our Legal Data expert: AI Project Run: Managing the Life Cycle of an ML Model.

If you want an introduction to AI security, our Smart Business CTO, Yann Bilissor, has written a post on the subject:  AI Security: Securing Data During Processing and Storage

 

Applications of Deep Learning/Machine Learning

 

Artificial intelligence may be applied to any area of life by utilizing Deep Learning/Machine Learning techniques, as demonstrated by the following examples:

• Insurance: automating customer complaint handling. Azure Cognitive Services provides a “sentiment analysis” function, which allows us to determine the level of customer satisfaction.
• Finance: personalize customer service and fight fraud and money laundering. With Azure Cognitive Service’s Anomaly Detector, it’s easy to add the ability to detect anomalies in time-series data.
• Health: improving care, promoting preventive medicine, and making diagnostics faster and more accurate.
• Social networks: hyper-personalization of content.
• Automotive: electric and autonomous vehicles.

Note: Azure Cloud’s Azure Machine Learning service and Azure Databricks let you use development frameworks like TensorFlow, PyTorch, and Scikit-learn when you need full control over DL/ML algorithms.

 

Deep Learning/Machine Learning: A Powerful Technology

 

There is no denying that Deep Learning/Machine Learning is a powerful technology for the transformation of any business. Although it may appear daunting at first, adopting AI-powered solutions has never been easier, thanks to the AI services provided by Cloud providers and the introduction of new data and ML/DL methods. We recommend starting small, and gradually implementing this new building block into your business or organization without making a significant financial commitment to see how it could help you.

Do you need help with your Smart Business, AI, or Machine Learning projects? Take a look at the services we offer:

• Smart Automation
• Bot Advisory
• Scalable & Secured Data Platform
• Data Storming
• Empower Your Data