How to Clear Docker Container Logs

David Mbochi Njonge Feb 02, 2024
  1. Create a New Project
  2. Define an Image
  3. Build an Image
  4. Run a Container
  5. Check the Container Logs
  6. Check the Container Logs in the Log File
  7. Clear Docker Container Logs Using the echo Command
  8. Clear Docker Container Logs Using the truncate Command
  9. Clear Docker Container Logs Using Log Rotation
  10. Conclusion
How to Clear Docker Container Logs

Logs are information logged by the application when a particular event or state occurs, and they help us to monitor the application and take the necessary actions.

For example, logs help us know whether the application state is up or down when we deploy an application to production. When an error occurs, we can identify the type of error that occurred using the logs.

With this information, we can make the required changes to fix the errors. In a large application being used by many people, it can be challenging to manage the logs, but when using the docker, we can easily manage the logs as it provides features that help to manipulate the logs.

Docker provides commands such as --details, --since, --tail, --timestamps, and --until that we can leverage to manipulate the logs. By default, docker saves logs in JSON format under the folder /var/lib/docker/containers/, but note that we must build images and run their containers as the root user for the file containing the logs to be created.

If this is done without root user privileges, we will get the error File or directory not found . In this tutorial, we will learn the different approaches we can leverage to clear the logs from the */*-json.log file.

Create a New Project

Open WebStorm IDEA and select File > New > Project. On the opened window, select Node.js and change the project name from untitled to docker-logs-clear or use any name preferred.

Finally, press the button labeled Create to generate the project.

Create a file index.js under the docker-logs-clear folder and copy and paste the following code into the file.

console.log('This is an express application')
console.error('An error occurred')
throw new Error('An exception occurred')

This file creates one STDOUT message and two STDERR messages that will get added to the log file when we run the application. We can change this code to be a web application but note that only the STDOUT and STDERR messages will get added to the log file.

Define an Image

Create a file Dockerfile under the docker-logs-clear folder and copy and paste the following instructions into the file.

FROM node:18-alpine
WORKDIR /com/app
ADD package*.json ./
RUN npm install
ADD . .
CMD node index.js

Here, FROM sets the base image on which to create our custom image using subsequent instructions. In this case, we used alpine to use a lightweight image of Node as the base image.

The docker documentation provides a detailed Dockerfile reference where we can learn more about the other instructions.

Build an Image

Note that this stage must be executed as the root user. To achieve this, open a new terminal window using the keyboard shortcut on your computer and run the below command to change the current user to the root user.

david@david-HP-ProBook-6470b:~/WebstormProjects/docker-logs-clear$ sudo su

Output:

root@david-HP-ProBook-6470b:/home/david/WebstormProjects/docker-logs-clear#

Ensure to change the directory cd to the location of your project, as shown above, to be able to build the image. Once we have access to the root user, execute the following command to build an image with the tag docker-logs:latest.

root@david-HP-ProBook-6470b:/home/david/WebstormProjects/docker-logs-clear# docker build --tag docker-logs:latest .

This command executes our Dockerfile, and we can view the instructions being run on the terminal window as shown below.

Step 1/6 : FROM node:18-alpine
18-alpine: Pulling from library/node
213ec9aee27d: Already exists
f379b689aea3: Pull complete
fe299d5780c0: Pull complete
c34a027bbf26: Pull complete
Digest: sha256:f829c27f4f7059609e650023586726a126db25aded0c401e836cb81ab63475ff
Status: Downloaded newer image for node:18-alpine
 ---> 867dce98a500
Step 2/6 : WORKDIR /com/app
 ---> Running in 3b215b9ad992
Removing intermediate container 3b215b9ad992
 ---> aba9cfa2472b
Step 3/6 : ADD package*.json ./
 ---> 6243ccacf178
Step 4/6 : RUN npm install
 ---> Running in 9b90745b171e

added 57 packages, and audited 58 packages in 9s

7 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
Removing intermediate container 9b90745b171e
 ---> e73c696d9743
Step 5/6 : ADD . .
 ---> e5235f761af0
Step 6/6 : CMD node index.js
 ---> Running in 7a857eea0a06
Removing intermediate container 7a857eea0a06
 ---> 29a367a3be2d
Successfully built 29a367a3be2d
Successfully tagged docker-logs:latest

Run a Container

To run a container using the docker-logs image, execute the following command that runs a container named docker-logs-prod. Note that no ports are exposed since it is not a web application.

root@david-HP-ProBook-6470b:/home/david/WebstormProjects/docker-logs-clear# docker run -d --name docker-logs-prod docker-logs:latest

Check the Container Logs

Running the container executes the code in our application. Execute the below command to view the logs generated by the container as a result of the STDOUT and STDERR messages.

root@david-HP-ProBook-6470b:/home/david/WebstormProjects/docker-logs-clear# docker logs -f docker-logs-prod

Output:

This is an express application
An error occurred
/com/app/index.js:3
throw new Error("An exception occurred")
^

Error: An exception occurred
    at Object.<anonymous> (/com/app/index.js:3:7)
    at Module._compile (node:internal/modules/cjs/loader:1149:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1203:10)
    at Module.load (node:internal/modules/cjs/loader:1027:32)
    at Module._load (node:internal/modules/cjs/loader:868:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:23:47

Node.js v18.10.0

Check the Container Logs in the Log File

To verify that the previous logs were added to our log file, use the following command to check the location of the log file where the container saves the generated logs.

root@david-HP-ProBook-6470b:/home/david/WebstormProjects/docker-logs-clear# docker inspect --format='{{.LogPath}}' docker-logs-prod

Output:

/var/lib/docker/containers/1eb27be7b062346872869eaedfb250c526dc77437123dd18adf3dbb5e8b6b7da/1eb27be7b062346872869eaedfb250c526dc77437123dd18adf3dbb5e8b6b7da-json.log

Now that we have the location where the log file is created execute the following command to print the JSON content of the log file.

root@david-HP-ProBook-6470b:/home/david/WebstormProjects/docker-logs-clear# cat /var/lib/docker/containers/1eb27be7b062346872869eaedfb250c526dc77437123dd18adf3dbb5e8b6b7da/1eb27be7b062346872869eaedfb250c526dc77437123dd18adf3dbb5e8b6b7da-json.log

Output:

{"log":"This is an express application\n","stream":"stdout","time":"2022-10-07T10:47:16.594937015Z"}
{"log":"An error occurred\n","stream":"stderr","time":"2022-10-07T10:47:16.596273395Z"}
{"log":"/com/app/index.js:3\n","stream":"stderr","time":"2022-10-07T10:47:16.617728515Z"}
{"log":"throw new Error(\"An exception occurred\")\n","stream":"stderr","time":"2022-10-07T10:47:16.61780931Z"}
{"log":"^\n","stream":"stderr","time":"2022-10-07T10:47:16.617822419Z"}
{"log":"\n","stream":"stderr","time":"2022-10-07T10:47:16.617832094Z"}
{"log":"Error: An exception occurred\n","stream":"stderr","time":"2022-10-07T10:47:16.617846368Z"}
{"log":"    at Object.\u003canonymous\u003e (/com/app/index.js:3:7)\n","stream":"stderr","time":"2022-10-07T10:47:16.617855581Z"}
{"log":"    at Module._compile (node:internal/modules/cjs/loader:1149:14)\n","stream":"stderr","time":"2022-10-07T10:47:16.617864838Z"}
{"log":"    at Module._extensions..js (node:internal/modules/cjs/loader:1203:10)\n","stream":"stderr","time":"2022-10-07T10:47:16.617882182Z"}
{"log":"    at Module.load (node:internal/modules/cjs/loader:1027:32)\n","stream":"stderr","time":"2022-10-07T10:47:16.617890043Z"}
{"log":"    at Module._load (node:internal/modules/cjs/loader:868:12)\n","stream":"stderr","time":"2022-10-07T10:47:16.617898124Z"}
{"log":"    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)\n","stream":"stderr","time":"2022-10-07T10:47:16.617906808Z"}
{"log":"    at node:internal/main/run_main_module:23:47\n","stream":"stderr","time":"2022-10-07T10:47:16.617914665Z"}
{"log":"\n","stream":"stderr","time":"2022-10-07T10:47:16.61792284Z"}
{"log":"Node.js v18.10.0\n","stream":"stderr","time":"2022-10-07T10:47:16.617930182Z"}

From the JSON content returned from the file, we can see all the logs generated from the container, including the stream and the time when the event happened. In the next section, we will learn how to clear logs from this file.

Clear Docker Container Logs Using the echo Command

To clear JSON content from the log file, execute the following command that uses the echo command to overwrite the file with an empty string.

root@david-HP-ProBook-6470b:/home/david/WebstormProjects/docker-logs-clear# echo "" > $(docker inspect --format='{{.LogPath}}' docker-logs-prod)

To verify that our log file has been cleared, execute the docker logs -f docker-logs-prod command, and note that it will not return any logs from the file.

Clear Docker Container Logs Using the truncate Command

Rerun the previous container to generate the logs again, and execute the following command to clear the logs added to the log file. This uses the truncate command to shrink the file to size 0.

root@david-HP-ProBook-6470b:/home/david/WebstormProjects/docker-logs-clear# truncate -s 0 $(docker inspect --format='{{.LogPath}}' docker-logs-prod)

To verify that our log file has been cleared, execute the docker logs -f docker-logs-prod command and note that it will not return any logs from the file.

Clear Docker Container Logs Using Log Rotation

In the previous two examples, we’ve used the echo and truncate commands to clear the log file’s contents manually. These approaches are not recommended as they might affect the docker logging system and cause unexpected behavior.

The recommended way to clear logs from the containers is to use log rotation. To use log rotation with the default logging driver json-file, we can set the max-size and max-file of the log file in the file named daemon.json located under /etc/docker/ folder, as shown below.

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

This should be done with root user privileges. For the changes to take effect, we must reload docker using the command systemctl reload docker.

If we want to specify the log rotation for individual containers, we can specify the --log-driver json-file, --log-opt max-size=10m, and --log-opt max-file=3 options when running the container.

Conclusion

In this tutorial, we’ve learned different approaches that we can use to clear logs from docker containers by clearing the JSON content added to the log files. The approaches we have covered in this tutorial include using the echo command, the truncate command, and the log-rotation.

David Mbochi Njonge avatar David Mbochi Njonge avatar

David is a back end developer with a major in computer science. He loves to solve problems using technology, learning new things, and making new friends. David is currently a technical writer who enjoys making hard concepts easier for other developers to understand and his work has been published on multiple sites.

LinkedIn GitHub