A simple interface in the form of R6 classes for executing tasks in parallel, tracking their progress, and displaying accurate progress bars.
Details
The package is aimed at two audiences: (1) end-users who want to execute a
task in parallel in an interactive R
session and track the execution
progress, and (2) R
package developers who want to use parabar
as a
solution for parallel processing in their packages.
Users
For the first category of users, parabar
provides several main
functions of interest:
start_backend()
: creates a parallel backend for executing tasks according to the specifications provided.stop_backend()
: stops an active backend and makes theR6::R6
eligible for garbage collection.par_sapply()
: is a drop-in replacement for the built-inbase::sapply()
function when no backend is provided. However, when a backend is provided, the function will execute a task in parallel on the backend, similar to the built-in functionparallel::parSapply()
.par_lapply()
: is a drop-in replacement for the built-inbase::lapply()
function when no backend is provided. However, when a backend is provided, the function will execute a task in parallel on the backend, similar to the built-in functionparallel::parLapply()
.par_apply()
: is a drop-in replacement for the built-inbase::apply()
function when no backend is provided. However, when a backend is provided, the function will execute a task in parallel on the backend, similar to the built-in functionparallel::parApply()
.clear()
: removes all variables available on a backend.peek()
: returns the names of all variables available on a backend.export()
: exports objects from a specified environment to a backend.evaluate()
: evaluates arbitrary and unquoted expression on a backend.
parabar
also provides a function configure_bar()
for
configuring the progress bar, and three functions can be used to get and set
the package options:
get_option()
: gets the value of a package option.set_option()
: sets the value of a package option.set_default_options()
: sets default values for all package options. This function is automatically called on package load.
Developers
For the second category of users, parabar
provides a set of
classes (i.e., R6::R6Class()
) that can be used to create backends (i.e.,
synchronous and asynchronous) and interact with them via a simple interface.
From a high-level perspective, the package consists of backends
and
contexts
in which these backends are employed for executing the tasks
in parallel.
Backends
A backend
represents a set of operations, defined by the
Service
interface, that can be deployed on a cluster returned by
parallel::makeCluster()
. Backends can be synchronous (i.e.,
SyncBackend
) or asynchronous (i.e., AsyncBackend
).
The former will block the execution of the current R
session until the
parallel task is completed, while the latter will return immediately and the
task will be executed in a background R
session.
The Service
interface defines the following operations:
start()
, stop()
,
clear()
, peek()
,
export()
, evaluate()
,
sapply()
, lapply()
,
apply()
, and get_output()
.
Check out the documentation for Service
for more information on
each method.
Contexts
A context
represents the specific conditions in which the backend
operates. The default, regular Context
class simply forwards the
call to the corresponding backend method. However, a more complex context can
augment the operation before forwarding the call to the backend. One example
of a complex context is the ProgressTrackingContext
class. This
class extends the regular Context
class and decorates, for
example, the backend sapply()
operation to log the
progress after each task execution and display a progress bar.
The following are the main classes provided by parabar
:
Service
: interface for backend operations.Backend
: abstract class that serves as a base class for all concrete implementations.SyncBackend
: synchronous backend extending the abstractBackend
class.AsyncBackend
: asynchronous backend extending the abstractBackend
class.Specification
: backend specification used when starting a backend.TaskState
: determine the state of a task deployed to an asynchronous backend.BackendFactory
: factory for creating backend objects.Context
: default context for executing backend operations.ProgressTrackingContext
: context for decorating thesapply()
,lapply()
, andapply()
operations to track and display the execution progress.ContextFactory
: factory for creating context objects.UserApiConsumer
: opinionated wrapper around the otherR6::R6
classes used in by the exported functions for the users.
Progress Bars
parabar
also exposes several classes for creating and updating
different progress bars, namely:
Bar
: abstract class defining the pure virtual methods to be implemented by concrete bar classes.BasicBar
: a simple, but robust, bar created viautils::txtProgressBar()
extending theBar
abstract class.ModernBar
: a modern bar created viaprogress::progress_bar
extending theBar
abstract class.BarFactory
: factory for creating bar objects.
Finally, parabar
uses several base::options()
to configure the
behavior of the functionality it provides. For more information on the
options used and their see default values, see the Options
class.
For more information about the design of parabar
, check out the
documentation and the UML
diagram at
parabar.mihaiconstantin.com.
Author
Maintainer: Mihai Constantin mihai@mihaiconstantin.com (ORCID)