Progress Bar for Parallel Tasks

A simple interface in the form of R6 classes for executing tasks in parallel, tracking their progress, and displaying accurate progress bars.

Details

The package is aimed at two audiences: (1) end-users who want to execute a task in parallel in an interactive R session and track the execution progress, and (2) R package developers who want to use parabar as a solution for parallel processing in their packages.

Users

For the first category of users, parabar provides several main functions of interest:

start_backend(): creates a parallel backend for executing tasks according to the specifications provided.
stop_backend(): stops an active backend and makes the R6::R6 eligible for garbage collection.
par_sapply(): is a drop-in replacement for the built-in base::sapply() function when no backend is provided. However, when a backend is provided, the function will execute a task in parallel on the backend, similar to the built-in function parallel::parSapply().
par_lapply(): is a drop-in replacement for the built-in base::lapply() function when no backend is provided. However, when a backend is provided, the function will execute a task in parallel on the backend, similar to the built-in function parallel::parLapply().
par_apply(): is a drop-in replacement for the built-in base::apply() function when no backend is provided. However, when a backend is provided, the function will execute a task in parallel on the backend, similar to the built-in function parallel::parApply().
clear(): removes all variables available on a backend.
peek(): returns the names of all variables available on a backend.
export(): exports objects from a specified environment to a backend.
evaluate(): evaluates arbitrary and unquoted expression on a backend.

parabar also provides a function configure_bar() for configuring the progress bar, and three functions can be used to get and set the package options:

get_option(): gets the value of a package option.
set_option(): sets the value of a package option.
set_default_options(): sets default values for all package options. This function is automatically called on package load.

Developers

For the second category of users, parabar provides a set of classes (i.e., R6::R6Class()) that can be used to create backends (i.e., synchronous and asynchronous) and interact with them via a simple interface. From a high-level perspective, the package consists of backends and contexts in which these backends are employed for executing the tasks in parallel.

Backends

A backend represents a set of operations, defined by the BackendService interface, that can be deployed on a cluster returned by parallel::makeCluster(). Backends can be synchronous (i.e., SyncBackend) or asynchronous (i.e., AsyncBackend). The former will block the execution of the current R session until the parallel task is completed, while the latter will return immediately and the task will be executed in a background R session.

The BackendService interface defines the following operations: start(), stop(), clear(), peek(), export(), evaluate(), sapply(), lapply(), apply(), and get_output().

Check out the documentation for BackendService for more information on each method.

Contexts

A context represents the specific conditions in which the backend operates. The default, regular Context class simply forwards the call to the corresponding backend method. However, a more complex context can augment the operation before forwarding the call to the backend. One example of a complex context is the ProgressTrackingContext class. This class extends the regular Context class and decorates, for example, the backend sapply() operation to log the progress after each task execution and display a progress bar.

The following are the main classes provided by parabar:

BackendService: interface for backend operations.
Backend: abstract class that serves as a base class for all concrete implementations.
SyncBackend: synchronous backend extending the abstract Backend class.
AsyncBackend: asynchronous backend extending the abstract Backend class.
Specification: backend specification used when starting a backend.
TaskState: determine the state of a task deployed to an asynchronous backend.
BackendFactory: factory for creating backend objects.
Context: default context for executing backend operations.
ProgressTrackingContext: context for decorating the sapply(), lapply(), and apply() operations to track and display the execution progress.
ContextFactory: factory for creating context objects.
UserApiConsumer: opinionated wrapper around the other R6::R6 classes used in by the exported functions for the users.

Progress Bars

parabar also exposes several classes for creating and updating different progress bars, namely:

Bar: abstract class defining the pure virtual methods to be implemented by concrete bar classes.
BasicBar: a simple, but robust, bar created via utils::txtProgressBar() extending the Bar abstract class.
ModernBar: a modern bar created via progress::progress_bar extending the Bar abstract class.
BarFactory: factory for creating bar objects.

Finally, parabar uses several base::options() to configure the behavior of the functionality it provides. For more information on the options used and their see default values, see the Options class.

For more information about the design of parabar, check out the documentation and the UML diagram at parabar.mihaiconstantin.com.

Software Design

Author

Maintainer: Mihai Constantin mihai@mihaiconstantin.com (ORCID)