Background Process in Python

1. Background Process in Python
2. Background Process in Windows in Python
3. Background Process in Linux or Mac in Python

We will introduce how to run Python scripts in the background as a background process. We will also introduce pythonw in Python.

Background Process in Python

In the world of automation and data science, we have advanced to such a level that we can easily automate our daily tasks, and they can be processed at the backend without even updating us (Unless we have programmed it to update us).

This article will teach how to create a script and run it as a background process.

The background processes are run one time, and after that, they are automatically processed when we start a computer, and they stop when we turn off our computer.

We can also stop background processes by going to the task manager or using the Python command to kill the process using its ID.

Background Process in Windows in Python

There are two different ways to run Python scripts as a background process; one method is by using pythonw, which is used to run the Python scripts as a background process in Windows.

We will use pythonw to run our Python script as a background process. Pythonw is used to run the Python scripts without using the terminal window.

The syntax to run any Python script without using the terminal window is below.

pythonw backgroundprocess.py

We can replace backgroundprocess.py with the file name we want to run as a background process in the above command.

Background Process in Linux or Mac in Python

To run Python scripts as a background process on Linux or Mac, we use the & operator at the end of the command, which will make it run as a background process.

The syntax to run any Python script as a background process in Linux or Mac is below.

# python
python backgroundprocess.py &

We can replace backgroundprocess.py with the file name we want to run as a background process in the above command.

Once our script is running as a background process, we can check for the process ID to stop the background process whenever we want. We can use the following command in Windows to kill the process we just started.

kill -9 {{process id}}

If we want to kill the process on Linux or Mac devices, we can run the following command.

kill -9 {{process id}} &

In the above commands, -9 means that we want to kill the process as soon as possible. So using this command will kill the process whose id we will provide inside the curly brackets.

Now, if we created an important script for which we want to check how it is working and get an update about the script that we are running as a background process, we can log or flush the output from that script inside another file using the simple command.

The command to log the output from our background process is shown below.

pythonw backgroundprocess.py > backgroundlog

If we want to log the output from our background process in Linux or Mac devices, we can use the following command below.

# python
python backgroundprocess.py > backgroundlog &

One important thing to note is that the output that our background process generates will be first saved in the buffer memory until the background process is stopped. So we can only get the log or flush of output from our background processes once we have stopped the process.

But what if we cannot stop the background process because it can risk losing some data? There is another way to log the output directly into the file without first saving it in a buffer memory.

We can use the following command.

pythonw -u backgroundprocess.py > backgroundlog

If we want to log output in a file directly, we can use this command on Linux or Mac devices, as shown below.

# python
python -u backgroundprocess.py > backgroundlog &

The above command will now directly add the output in our backgroundlog file. But we still have to keep our terminal open to get the output.

If we close the terminal before our background process is complete, all the processes will be stopped. All the processes that we were running will be hung up.

Python always has solutions for every kind of problem. There is another command known as nohup.

This command makes sure that our background processes are complete without any interference.

Even if we close the parent terminal, it will still ensure that our background processes are working. The syntax to use the nohup command is shown below.

nohup python -u backgroundprocess.py

The above example shows that we forgot to log the output in another file to check whenever needed. We didn’t forget that the nohup command automatically generates a log and stores it into a file to see all the records while our background process is running.

The nohup command will store all the logs into the nohup.out file. If we want to kill this process, we can easily use the kill command that we used above to kill this process.

But if we don’t remember the ID of the process we want to delete, we can use another command to take the name of the file that we are running as a background process. And it will find the process ID and use the kill command to kill the process.

The syntax for this command is shown below.

ps ax | grep backgroundprocess.py