Memory Leak in Python

1. Memory Leak in Python
2. Identify Memory Leak in Python
3. Fix Memory Leak in Python

Memory leaks are a common programming problem that can be difficult to debug and fix.

This article will explore the Python memory leak with small and larger example programs. We will see how to find the source of the memory leak and how to fix it.

Memory Leak in Python

We will not discuss the internals of the Python memory management system in this article. But, if you are curious about how the Python memory system works, you can refer to the source code of Python’s standard library and the code of other high-level programming languages.

Now, let’s discuss some libraries that we will be using in our examples to study the memory leak in Python. The first library is the Requests library, which allows us to make HTTP requests to a particular URL.

Requests must be learned before moving forward with these technologies, whether REST APIs or Web Scraping. A response is returned when a request is sent to a URL.

Python applications contain built-in management features for both the request and response.

We can easily install it using the following command.

pip install requests

Once we have installed the requests library, now we will install the GC module. It gives us access to the Python garbage collector.

It has options for activating collectors, disabling collectors, tuning collection frequency, debugging, and more.

In lower-level languages like C and C++, the programmer must manually release any resources no longer in use. To put it another way, develop programming to manage the resource.

However, high-level languages such as Python and Java contain a concept of trash collection, which is automated memory management. The trash collection is in charge of allocating and releasing memory for a program.

The methods of the GC module that we will use are get_object(), the method that gives the tracked object from the trash collection. We will also use the collect() method to provide the non-referenced object free from the list that the collectors control.

Identify Memory Leak in Python

Now, we will discuss how we can identify if there was a memory leak while running a program. Let’s start with an example and use the libraries mentioned above to check if there was a leak or not.

We will first import the libraries mentioned earlier in this example.

# python
import requests
import gc

Then we will define a function, getGoogle(), that will request a response from google.com using the Requests library and return the status code received.

Code:

# python
def getGoogle():
    resultGot = requests.get("https://google.com")
    print("Status Code recieved is ", resultGot.status_code)
    return

Now we will define another function, checkMemoryLeak(), in which we will collect garbage objects before and after calling the function getGoogle().

Code:

# python
def checkMemoryLeak():
    print("Memory Leaked before calling getGoogle()")
    print(len(gc.get_objects()))
    getGoogle()
    print("Memory Leaked before calling getGoogle()")
    print(len(gc.get_objects()))


checkMemoryLeak()

Output:

As you can see in the example, before calling the function, the length of the garbage collector was 17472, and after calling our function, the length increased to 17698. It shows that there is always a memory leakage when performing some tasks.

We will now discuss how we can fix the memory leakage and collect as much data as possible without memory leakage.

Fix Memory Leak in Python

We can fix the memory leak in Python using the function gc.collect(). We will use the same program but modify it so that there is no or minimal memory leakage.

In this example, after calling our function getGoogle(), we will directly call gc.collect(), which will reduce the memory leak and fix the problem.

Code:

# python
import requests
import gc


def getGoogle():
    resultGot = requests.get("https://google.com")
    print("Status Code recieved is ", resultGot.status_code)


def checkMemoryLeak():
    print("Memory Leaked before calling getGoogle()")
    print(len(gc.get_objects()))
    getGoogle()
    gc.collect()
    print("Memory Leaked before calling getGoogle()")
    print(len(gc.get_objects()))


checkMemoryLeak()

Output:

As you can see in the example, the number of memory leaks decreases after using gc.collect(). So in this way, we can prevent memory leaks during the function call.