The sigmoid Function in Python
-
Implement the Sigmoid Function in Python Using the
math
Module -
Implement the Sigmoid Function in Python Using the
numpy.exp()
Method -
Implement the Sigmoid Function in Python Using the
SciPy
Library
In this tutorial, we will look into various methods to use the sigmoid function in Python. The sigmoid function is a mathematical logistic function. It is commonly used in statistics, audio signal processing, biochemistry, and the activation function in artificial neurons. The formula for the sigmoid function is F(x) = 1/(1 + e^(-x))
.
Implement the Sigmoid Function in Python Using the math
Module
We can implement our own sigmoid function in Python using the math
module. We need the math.exp()
method from the math
module to implement the sigmoid function.
The below example code demonstrates how to use the sigmoid function in Python.
import math
def sigmoid(x):
sig = 1 / (1 + math.exp(-x))
return sig
The problem with this implementation is that it is not numerically stable and the overflow may occur.
The example code of the numerically stable implementation of the sigmoid function in Python is given below.
import math
def stable_sigmoid(x):
if x >= 0:
z = math.exp(-x)
sig = 1 / (1 + z)
return sig
else:
z = math.exp(x)
sig = z / (1 + z)
return sig
Implement the Sigmoid Function in Python Using the numpy.exp()
Method
We can also implement the sigmoid function using the numpy.exp()
method in Python. Like the implementations of the sigmoid function using the math.exp()
method, we can also implement the sigmoid function using the numpy.exp()
method.
The advantage of the numpy.exp()
method over math.exp()
is that apart from integer or float, it can also handle the input in an array’s shape.
Below is the regular sigmoid function’s implementation using the numpy.exp()
method in Python.
import numpy as np
def sigmoid(x):
z = np.exp(-x)
sig = 1 / (1 + z)
return sig
For the numerically stable implementation of the sigmoid function, we first need to check the value of each value of the input array and then pass the sigmoid’s value. For this, we can use the np.where()
method, as shown in the example code below.
import numpy as np
def stable_sigmoid(x):
sig = np.where(x < 0, np.exp(x) / (1 + np.exp(x)), 1 / (1 + np.exp(-x)))
return sig
Implement the Sigmoid Function in Python Using the SciPy
Library
We can also use the SciPy
version of Python’s sigmoid function by simply importing the sigmoid function called expit
in the SciPy
library.
The example code below demonstrates how to use the sigmoid function using the SciPy
library:
from scipy.special import expit
x = 0.25
sig = expit(x)
The expit()
method is slower than the above implementations. The advantage of the expit()
method is that it can automatically handle the various types of inputs like list, and array, etc.
from scipy.special import expit
sig = expit(np.array([0.25, 0.5, 0.6, 0.7, 0.4]))
print(sig)
Output:
[0.5621765 0.62245933 0.64565631 0.66818777 0.59868766]