The sigmoid Function in Python

Muhammad Waiz Khan Feb 02, 2024
  1. Implement the Sigmoid Function in Python Using the math Module
  2. Implement the Sigmoid Function in Python Using the numpy.exp() Method
  3. Implement the Sigmoid Function in Python Using the SciPy Library
The sigmoid Function in Python

In this tutorial, we will look into various methods to use the sigmoid function in Python. The sigmoid function is a mathematical logistic function. It is commonly used in statistics, audio signal processing, biochemistry, and the activation function in artificial neurons. The formula for the sigmoid function is F(x) = 1/(1 + e^(-x)).

Implement the Sigmoid Function in Python Using the math Module

We can implement our own sigmoid function in Python using the math module. We need the math.exp() method from the math module to implement the sigmoid function.

The below example code demonstrates how to use the sigmoid function in Python.

import math


def sigmoid(x):
    sig = 1 / (1 + math.exp(-x))
    return sig

The problem with this implementation is that it is not numerically stable and the overflow may occur.

The example code of the numerically stable implementation of the sigmoid function in Python is given below.

import math


def stable_sigmoid(x):

    if x >= 0:
        z = math.exp(-x)
        sig = 1 / (1 + z)
        return sig
    else:
        z = math.exp(x)
        sig = z / (1 + z)
        return sig

Implement the Sigmoid Function in Python Using the numpy.exp() Method

We can also implement the sigmoid function using the numpy.exp() method in Python. Like the implementations of the sigmoid function using the math.exp() method, we can also implement the sigmoid function using the numpy.exp() method.

The advantage of the numpy.exp() method over math.exp() is that apart from integer or float, it can also handle the input in an array’s shape.

Below is the regular sigmoid function’s implementation using the numpy.exp() method in Python.

import numpy as np


def sigmoid(x):

    z = np.exp(-x)
    sig = 1 / (1 + z)

    return sig

For the numerically stable implementation of the sigmoid function, we first need to check the value of each value of the input array and then pass the sigmoid’s value. For this, we can use the np.where() method, as shown in the example code below.

import numpy as np


def stable_sigmoid(x):

    sig = np.where(x < 0, np.exp(x) / (1 + np.exp(x)), 1 / (1 + np.exp(-x)))
    return sig

Implement the Sigmoid Function in Python Using the SciPy Library

We can also use the SciPy version of Python’s sigmoid function by simply importing the sigmoid function called expit in the SciPy library.

The example code below demonstrates how to use the sigmoid function using the SciPy library:

from scipy.special import expit

x = 0.25
sig = expit(x)

The expit() method is slower than the above implementations. The advantage of the expit() method is that it can automatically handle the various types of inputs like list, and array, etc.

from scipy.special import expit

sig = expit(np.array([0.25, 0.5, 0.6, 0.7, 0.4]))
print(sig)

Output:

[0.5621765  0.62245933 0.64565631 0.66818777 0.59868766]