How to Implement the ReLU Function in Python

1. the ReLU Function
2. Implement the ReLU Function in Python

This tutorial will discuss the Relu function and how to implement it in Python.

the ReLU Function

The Relu function is fundamental to Machine Learning and is essential while using Deep Learning.

The term ReLU is an acronym for Rectified Linear Unit and works as an activation function in cases of artificial neural networks, which are the base of Deep Learning. Python, being one of the programming languages suitable to implement both Machine Learning and Deep Learning algorithms, has a scope of the use of the ReLU function.

In simple mathematical terms, the ReLU function can be defined as,

f(x) = max(0,x)

This function is linear concerning x and can zero out all the negative values.

Implement the ReLU Function in Python

To implement the ReLU function in Python, we can define a new function and use the NumPy library.

The NumPy library makes it possible to deal with matrices and arrays in Python, as the same cannot directly be implemented in this programming language. The maximum() function from the NumPy library can be utilized inside our newly created function definition to make a ReLU function.

The following code implements the ReLU function in Python.

import numpy as np


def relu1(a):
    return np.maximum(0, a)


print(relu1(-3))

The above code provides the following output:

0

In the above code, we deal with a single integer. However, the ReLU function that we have created can easily work on anything ranging from a single integer to NumPy arrays and similar objects.

When this function gets a number as an input, the output will always be a number. The same rule is followed when we pass an array to this function.

The type of object inputted with always be the object type returned as the output.

Interestingly enough, when dealing with arrays, we can make use of the plotly library and even create graphs to depict the action of the ReLU function on arrays and similar objects.

To better explain the working of the ReLU function, we will now take an example of a simple array and achieve the task at hand. Moreover, we will also depict a graph and see the live action of the ReLU function on this array.

The following code uses the ReLU function on an array in Python.

import numpy as np
import plotly.express as px


def relu1(a):
    return np.maximum(0, a)


x1 = np.linspace(start=-5, stop=5, num=26)
print(x1)
x2 = relu1(x1)
print(x2)
px.line(x=x1, y=x2)

The above code provides the following output: