# doFuture: Use Foreach to Parallelize via the Future Framework

## TL;DR

To run [`foreach()`](https://rdrr.io/pkg/foreach/man/foreach.html) in
parallel, install R packages
**[doFuture](https://doFuture.futureverse.org)** and
**[futurize](https://futurize.futureverse.org)**, and call:

``` r

library(futurize)
plan(multisession)

y <- foreach(x = 1:4, y = 1:10) %do% {
  z <- x + y
  slow_sqrt(z)
} |> futurize()
```

That’s it - easy!

## Introduction

The **[foreach](https://cran.r-project.org/package=foreach)** package
implements a map-reduce API with functions
[`foreach()`](https://rdrr.io/pkg/foreach/man/foreach.html) and
[`times()`](https://rdrr.io/pkg/foreach/man/foreach.html) that provide
us with powerful methods for iterating over one or more sets of elements
with options to do it in parallel.

The **[future](https://future.futureverse.org)** package provides a
generic API for using futures in R. A future is a simple yet powerful
mechanism to evaluate an R expression and retrieve its value at some
point in time. Futures can be resolved in many different ways depending
on which strategy is used. You can resolve them sequential, in parallel
on your local computer, on remove computers, in the cloud, on a
high-performance compute (HPC) cluster, or via any [future
backend](https://www.futureverse.org/backends.html) available.

The **[doFuture](https://doFuture.futureverse.org)** package provides a
bridge between **foreach** and the **future** parallelization framework.
Specifically, the **doFuture** package provides three alternatives for
using futures with **foreach**:

1.  `y <- foreach(...) %do% { ... } |> futurize()`

2.  `y <- foreach(...) %dofuture% { ... }`

3.  [`registerDoFuture()`](https://doFuture.futureverse.org/reference/registerDoFuture.md) +
    `y <- foreach(...) %dopar% { ... }`.

### Alternative 1: `futurize()` (recommended)

The *first alternative* (recommended) uses
[`futurize()`](https://futurize.futureverse.org/reference/futurize.html)
of the **[futurize](https://futurize.futureverse.org)** package. An
example is:

``` r

library(futurize)
plan(multisession)

y <- foreach(x = 1:4, y = 1:10) %do% {
  z <- x + y
  slow_sqrt(z)
} |> futurize()
```

This alternative is the recommended and most clean way to let
[`foreach()`](https://rdrr.io/pkg/foreach/man/foreach.html) parallelize
via the future framework, especially if you start out from scratch. All
you need to remember is to pipe it to
[`futurize()`](https://futurize.futureverse.org/reference/futurize.html),
and, yes, it is correct to use `%do%` here. In addition to
`multisession`, parallelization can be done via any compliant [future
backend](https://www.futureverse.org/backends.html). Identification of
globals, random number generation (RNG), and error handling is handled
the same way as elsewhere in the future ecosystem. We recommend to use
[`futurize()`](https://futurize.futureverse.org/reference/futurize.html),
because it is consistent with how we parallelize
[`lapply()`](https://rdrr.io/r/base/lapply.html) and
[`purrr::map()`](https://purrr.tidyverse.org/reference/map.html) using
**futurize**. With
[`futurize()`](https://futurize.futureverse.org/reference/futurize.html),
you do not have to explicitly load **doFuture** - instead **doFuture**
will serve
[`futurize()`](https://futurize.futureverse.org/reference/futurize.html)
under the hood.

See
[`help("futurize", package = "futurize")`](https://futurize.futureverse.org/reference/futurize.html)
for more details and examples on this approach.

### Alternative 2: `%dofuture%`

The *second alternative* (formely recommended), which uses `%dofuture%`,
avoids having to use
[`registerDoFuture()`](https://doFuture.futureverse.org/reference/registerDoFuture.md).
The `%dofuture%` operator provides a more consistent behavior than
`%dopar%`, e.g. there is a unique set of foreach arguments instead of
one per possible adapter. An example is:

``` r

library(doFuture)
plan(multisession)

y <- foreach(x = 1:4, y = 1:10) %dofuture% {
  z <- x + y
  slow_sqrt(z)
}
```

This alternative was the recommended way to let
[`foreach()`](https://rdrr.io/pkg/foreach/man/foreach.html) parallelize
via the future framework, but now we recommend using
[`futurize()`](https://futurize.futureverse.org/reference/futurize.html)
instead, especially if you start out from scratch.

See
[`help("%dofuture%", package = "doFuture")`](https://doFuture.futureverse.org/reference/grapes-dofuture-grapes.md)
for more details and examples on this approach.

### Alternative 3: `registerDoFuture()` + `%dopar%`

The *third alternative* is based on the traditional **foreach** approach
where one registers a foreach adapter to be used by `%dopar%`. A popular
adapter is
[`doParallel::registerDoParallel()`](https://rdrr.io/pkg/doParallel/man/registerDoParallel.html),
which parallelizes on the local machine using the **parallel** package.
This package provides
[`registerDoFuture()`](https://doFuture.futureverse.org/reference/registerDoFuture.md),
which parallelizes using the **future** package, meaning any
future-compliant parallel backend can be used.

An example is:

``` r

library(doFuture)
registerDoFuture()
plan(multisession)

y <- foreach(x = 1:4, y = 1:10) %dopar% {
  z <- x + y
  slow_sqrt(z)
}
```

This alternative is useful if you already have a lot of R code that uses
`%dopar%` and you just want to switch to using the future framework for
parallelization. Using
[`registerDoFuture()`](https://doFuture.futureverse.org/reference/registerDoFuture.md)
is also useful when you wish to use the future framework with packages
and functions that use
[`foreach()`](https://rdrr.io/pkg/foreach/man/foreach.html) and
`%dopar%` internally, but still do not support
[`futurize()`](https://futurize.futureverse.org/reference/futurize.html),
e.g. **[NMF](https://cran.r-project.org/package=NMF)**.

See
[`help("registerDoFuture", package = "doFuture")`](https://doFuture.futureverse.org/reference/registerDoFuture.md)
for more details and examples on this approach.

## Installation

R package doFuture is available on
[CRAN](https://cran.r-project.org/package=doFuture) and can be installed
in R as:

``` r

install.packages("doFuture")
```

### Pre-release version

To install the pre-release version that is available in Git branch
`develop` on GitHub, use:

``` r

remotes::install_github("futureverse/doFuture", ref="develop")
```

This will install the package from source.

# Package index

## All functions

- [`doFuture`](https://doFuture.futureverse.org/reference/doFuture.md)
  [`doFuture-package`](https://doFuture.futureverse.org/reference/doFuture.md)
  : doFuture: Foreach Parallel Adapter using Futures

- [`` `%dofuture%` ``](https://doFuture.futureverse.org/reference/grapes-dofuture-grapes.md)
  : Loop over a Foreach Expression using Futures

- [`registerDoFuture()`](https://doFuture.futureverse.org/reference/registerDoFuture.md)
  :

  Use the Foreach `%dopar%` Adapter with Futures

- [`with(`*`<DoPar>`*`)`](https://doFuture.futureverse.org/reference/with.DoPar.md)
  :

  Evaluate an Expression using a Temporarily Registered Foreach
  `%dopar%` Adapter

- [`withDoRNG()`](https://doFuture.futureverse.org/reference/withDoRNG.md)
  :

  Evaluates a foreach `%dopar%` expression with the doRNG adapter

# Articles

### All vignettes

- [doFuture: An Overview on using Foreach to Parallelize via the Future
  Framework](https://doFuture.futureverse.org/articles/doFuture-1-overview.md):
- [Foreach Iteration using Futures via
  %dopar%](https://doFuture.futureverse.org/articles/doFuture-2-dopar.md):
- [Foreach Iteration using Futures via
  %dofuture%](https://doFuture.futureverse.org/articles/doFuture-3-dofuture.md):