How to Get JSON From URL in Python

Whenever we want to get any data from a web server using a URL, then that data should have some particular format. In general, whenever we want to fetch any data from the web server, we use either JSON or XML formats.

We prefer JSON over XML because it has become quite popular in the software industry. There are various libraries in Python to process JSON.

Throughout this tutorial, we will use json and requests modules, which are available in Python.

The JSON data which we will be fetching is from the below URL. This URL allows us to fetch all the data related to the users like name, email, address, etc. It is free to use.

https://jsonplaceholder.typicode.com/users

In terms of Restful APIs, URLs are called endpoints. Throughout this tutorial, we will be using these two words interchangeably. In the above URL, /users refers to an endpoint.

If you open the above URL in the browser, you will see all the data related to users in JSON format. It is shown below.

[
  {
    "id": 1,
    "name": "Leanne Graham",
    "username": "Bret",
    "email": "Sincere@april.biz",
    "address": {
      "street": "Kulas Light",
      "suite": "Apt. 556",
      "city": "Gwenborough",
      "zipcode": "92998-3874",
      "geo": {
        "lat": "-37.3159",
        "lng": "81.1496"
      }
    },
    "phone": "1-770-736-8031 x56442",
    "website": "hildegard.org",
    "company": {
      "name": "Romaguera-Crona",
      "catchPhrase": "Multi-layered client-server neural-net",
      "bs": "harness real-time e-markets"
    }
  },
]

Note that we have only shown you the JSON data related to a single user in the above code block. This is just for explanation purposes. But if you visit the URL, you will see the data for all the users.

The JSON refers to JavaScript Object Notation. It is used to send and receive data between a server and a client and vice versa. Here, don’t get confused with the word JavaScript; JSON is independent of any programming language. In JSON, we use two types of brackets to format the data; one is the square brackets [] representing an array or a list, and the other is the curly braces {} which represents an object.

The data inside the JSON will always be in the form of key-value pairs within quotation marks "". So, if you want to access any value, you have to use the key associated with that value. For example, let’s say we want to access the user’s name from the above JSON data; we will use the key name to get the value Leanne Graham, which is the first user’s name. Also, make a note that no comments are allowed in JSON.

Get and Access JSON Data in Python

First, we need to import the requests and json modules to get and access the data.

import requests
import json

Fetch and Convert Data From the URL to a String

The first step we have to perform here is to fetch the JSON data using the requests library.

url = requests.get("https://jsonplaceholder.typicode.com/users")
text = url.text
print(type(text))

Output:

<class 'str'>

The requests library has a method called get() which takes a URL as a parameter and then sends a GET request to the specified URL. The response we get from the server is stored in the variable called url.

This response stored inside the url variable needs to be converted into a string with the help of the .text method as url.text. And then, we store the result in the text variable. If you print the type of the text variable, it will be of type <class 'str'>.

Parsethe JSON Data

Parsing is a process of converting string data into JSON format. For that, we will be using the json module.

data = json.loads(text)
print(type(data))

Output:

<class 'list'>

The json module comes with a method called loads(), the s in loads() stands for string. Since we want to convert string data into JSON we will be using this method. Inside this method, we have to pass the text variable that contains the string data json.loads(text) and store it inside the data variable.

Now the data will have the entire JSON response. If you print the type of the data variable, then it will be of type <class 'list'> because in this case, the JSON response starts with square brackets [] and in Python, lists start with square brackets.

Access the JSON Data

Now that we have parsed the JSON data, we are ready to access the individual values which we want using the data variable. To access the details of the first user, like Name and Address, we can do the following.

import json
import requests

url = requests.get("https://jsonplaceholder.typicode.com/users")
text = url.text

data = json.loads(text)

user = data[0]
print(user["name"])

address = user["address"]
print(address)

Output:

Leanne Graham
{'street': 'Kulas Light', 'suite': 'Apt. 556', 'city': 'Gwenborough', 'zipcode': '92998-3874', 'geo': {'lat': '-37.3159', 'lng': '81.1496'}}

To access the first user from the list of users, we have to use the 0 index from the JSON data, which is stored inside a variable called data, and then store the entire details of that first inside user. Now this users is an object which contains all the details related to that user. To access the name, we just have to pass the name key inside the user variable like user['name'] and then print it. It will print the name of the user inside the console.

And to access the address we have to pass the address key inside the user variable i.e user['address'], and then store it inside a variable called address and then print it. This will give you an object which will contain all the specific details related to the address field like Apartment no., street name, zip code, and so on. Let’s say out of these details you want only the zipcode of the user; then you have to pass the zipcode code key inside the address object.