Start a Backend

This function can be used to start a backend. Check out the Details section for more information.

Usage

start_backend(cores, cluster_type = "psock", backend_type = "async")

Arguments

cores: A positive integer representing the number of cores to use (i.e., the number of processes to start). This value must be between 2 and parallel::detectCores() - 1.
cluster_type: A character string representing the type of cluster to create. Possible values are "fork" and "psock". Defaults to "psock". See the section Cluster Type for more information.
backend_type: A character string representing the type of backend to create. Possible values are "sync" and "async". Defaults to "async". See the section Backend Type for more information.

Value

A Backend instance that can be used to parallelize computations. The methods available on the Backend instance are defined by the Service interface.

Details

This function is a convenience wrapper around the lower-lever API of parabar aimed at developers. More specifically, this function uses the Specification class to create a specification object, and the BackendFactory class to create a Backend instance based on the specification object.

Cluster Type

The cluster type determines the type of cluster to create. The requested value is validated and passed to the type argument of the parallel::makeCluster() function. The following table lists the possible values and their corresponding description.

Cluster	Description
`"fork"`	For Unix-based systems.
`"psock"`	For Windows-based systems.

Backend Type

The backend type determines the type of backend to create. The requested value is passed to the BackendFactory class, which returns a Backend instance of the desired type. The following table lists the possible backend types and their corresponding description.

Backend	Description	Implementation	Progress
`"sync"`	A synchronous backend.	`SyncBackend`	no
`"async"`	An asynchronous backend.	`AsyncBackend`	yes

In a nutshell, the difference between the two backend types is that for the synchronous backend the cluster is created in the main process, while for the asynchronous backend the cluster is created in a backend R process using callr::r_session. Therefore, the synchronous backend is blocking the main process during task execution, while the asynchronous backend is non-blocking. Check out the implementations listed in the table above for more information. All concrete implementations extend the Backend abstract class and implement the Service interface.

Examples

# Create an asynchronous backend.
backend <- start_backend(cores = 2, cluster_type = "psock", backend_type = "async")

# Check that the backend is active.
backend$active
#> [1] TRUE

# Check if there is anything on the backend.
peek(backend)
#> [[1]]
#> character(0)
#> 
#> [[2]]
#> character(0)
#> 

# Create a dummy variable.
name <- "parabar"

# Export the `name` variable in the current environment to the backend.
export(backend, "name", environment())

# Remove the dummy variable from the current environment.
rm(name)

# Check the backend to see that the variable has been exported.
peek(backend)
#> [[1]]
#> [1] "name"
#> 
#> [[2]]
#> [1] "name"
#> 

# Run an expression on the backend.
# Note that the symbols in the expression are resolved on the backend.
evaluate(backend, {
    # Print the name.
    print(paste0("Hello, ", name, "!"))
})
#> [[1]]
#> [1] "Hello, parabar!"
#> 
#> [[2]]
#> [1] "Hello, parabar!"
#> 

# Clear the backend.
clear(backend)

# Check that there is nothing on the backend.
peek(backend)
#> [[1]]
#> character(0)
#> 
#> [[2]]
#> character(0)
#> 

# Use a basic progress bar (i.e., see `parabar::Bar`).
configure_bar(type = "basic", style = 3)

# Run a task in parallel (i.e., approx. 1.25 seconds).
output <- par_sapply(backend, x = 1:10, fun = function(x) {
    # Sleep a bit.
    Sys.sleep(0.25)

    # Compute and return.
    return(x + 1)
})

# Print the output.
print(output)
#>  [1]  2  3  4  5  6  7  8  9 10 11

# Stop the backend.
stop_backend(backend)

# Check that the backend is not active.
backend$active
#> [1] FALSE