R Cannot Allocate Vector of Size

1. R Cannot Allocate Vector of Size
2. Solution 1: Garbage Collection Using gc()
3. Solution 2: Increase Memory Limit

This tutorial demonstrates the error r cannot allocate vector of size.

R Cannot Allocate Vector of Size

The cannot allocate vector of size error occurs in R when we are trying to deal with a large amount of data. This error is not a syntax error but a logical one that occurs when dealing with an extremely large amount of data because it uses a large amount of virtual memory.

Most of the time, this error occurs when the data is loaded from an external source like a package or any other file. When the compiler cannot control the object’s size, it will throw the error cannot allocate vector of size.

The error cannot allocate vector of size occurs when we create a vector of the object or load a function. This error can also occur with smaller objects when we have many smaller objects.

The error has two solutions, the gc() and memory.limit(), where the gc() doesn’t work in every situation. We demonstrate both methods in this tutorial.

Solution 1: Garbage Collection Using gc()

Garbage collection is the first solution that comes to mind; there is an error cannot allocate vector of size, and it works in many situations. But the GC will not work when the amount of data is way more.

The method gc() is used for the garbage collection in R. Let’s try an example that throws this error and then try to solve it with gc().

demo <- rnorm(5000000000)

The code above will create a vector from randomly distributed values, and according to our system, it should throw the error cannot allocate vector of size. See the output:

Error: cannot allocate vector of size 37.3 Gb
In addition: Warning message:
In library(package, lib.loc = lib.loc, character.only = TRUE, logical.return = TRUE,  :
  there is no package called 'Matrix'

As we can see, the compiler requires the memory of the error: cannot allocate vector of size 37.3 Gb, which is not possible for our system. Not let’s try to run GC and then this code again.

# Run GC
gc()
# Run the above code
demo <- rnorm(5000000000)

The code above will perform the GC and then try to re-run the code; if after GC, there is a memory of 37.3 GB, then the error will be solved; otherwise, it will still throw the error. See the output:

          used (Mb) gc trigger  (Mb) max used (Mb)
Ncells 1004541 53.7    2078938 111.1  1255027 67.1
Vcells 2577662 19.7    8388608  64.0  2836345 21.7

Error: cannot allocate vector of size 37.3 Gb

As we can see, it still throws the same error because the GC is clearing a small amount of memory, which needs more. So the gc() only work for small situations.

Solution 2: Increase Memory Limit

Some time ago, we could increase the memory limit in R using the memory.limit() with the size attribute, solving the error cannot allocate vector of size even in a large data situation. But this method is no longer supported in R, and as per R documentation, we can increase the memory from the OS option.

Follow the step-by-step process to increase memory which will solve the error cannot allocate vector of size.

Now, this code will throw and create a vector named demo. The size 37.3 GB is too large; it might hang your computer if it cannot support the memory.

But whenever the error cannot allocate vector of size occurs, we can use this method to solve it.