Deferred & Remote Function Execution in R

What would you say if you could automatically wrap your R pipeline – which consists of numerous functions and variables – into a single function? What would you say if you could do it repeatedly, with no extra effort regardless of its complexity? Would you store it to document your progress at that particular moment in time? Would you serialize and send it to a remote R session to offload computations? Well, it’s kind-of possible.

defer brings a new capability to R – packaging, as a single R function, a number of inter-connected functions and variables which exist in the current R session. It is something I found useful during fast-prototyping phases of my own work and wanted to share for quite some time now – and at this point I’m interested in seeing if it turns out useful for someone else besides me.

Here’s a simplistic example where defer is given a top-level function h() that requires two other functions (f() and g()), a variable (C) and a library call (base::mean).

C <- 10

f <- function(x) x*x + C
g <- function(y) (f(y) + 10) * f(y+3)
h <- function(z) mean(c(g(z), g(z+1), g(z+2)))

wrapper <- defer(h)
#> Found functions:
#>   g, f
#> variables:
#>   C
#> library calls:
#>   base::mean

rm(C, f, g, h)

#> [1] 29688.67

Here is another example where that same wrapper() can be run in a remote R session (via opencpu) without installing or deploying any extra code on the remote server. First, we serialize wrapper as a base64-encoded string.


buffer <- rawConnection(raw(0), 'w')
saveRDS(wrapper, buffer)
serialized <- base64enc::base64encode(rawConnectionValue(buffer))

Then, we write that string into a temporary file together with a few lines of code to deserialize and run it in a remote R session.

local_script_path <- tempfile(fileext = ".R")

cat(paste0("base64 <- '", serialized, "'\n",
           "bytes <- rawConnection(base64enc::base64decode(base64), 'r')\n",
           "deserialized <- readRDS(bytes)\n",
    file = local_script_path)

Finally, we upload the file on the opencpu server and run it via base::source.

result <- httr::POST(public_opencpu_url,
                     body = list(file = upload_file(local_script_path)))
content(result, 'text')
#> [1] "$value\n[1] 29688.67\n\n$visible\n[1] TRUE\n\n"

And that seems like a number we’ve seen already, right?

The package is currently available only from GitHub as it’s way not ready for CRAN.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s