<commit_before># arrow

[![cran](https://www.r-pkg.org/badges/version-last-release/arrow)](https://cran.r-project.org/package=arrow)
[![CI](https://github.com/apache/arrow/workflows/R/badge.svg?event=push)](https://github.com/apache/arrow/actions?query=workflow%3AR+branch%3Amaster+event%3Apush)
[![conda-forge](https://img.shields.io/conda/vn/conda-forge/r-arrow.svg)](https://anaconda.org/conda-forge/r-arrow)

[Apache Arrow](https://arrow.apache.org/) is a cross-language
development platform for in-memory data. It specifies a standardized
language-independent columnar memory format for flat and hierarchical
data, organized for efficient analytic operations on modern hardware. It
also provides computational libraries and zero-copy streaming messaging
and interprocess communication.

The `arrow` package exposes an interface to the Arrow C++ library to
access many of its features in R. This includes support for analyzing
large, multi-file datasets (`open_dataset()`), working with individual
Parquet (`read_parquet()`, `write_parquet()`) and Feather
(`read_feather()`, `write_feather()`) files, as well as lower-level
access to Arrow memory and messages.

## Installation

Install the latest release of `arrow` from CRAN with

```r
install.packages("arrow")
```

Conda users can install `arrow` from conda-forge with

```
conda install -c conda-forge --strict-channel-priority r-arrow
```

Installing a released version of the `arrow` package requires no
additional system dependencies. For macOS and Windows, CRAN hosts binary
packages that contain the Arrow C++ library. On Linux, source package
installation will also build necessary C++ dependencies. For a faster,
more complete installation, set the environment variable `NOT_CRAN=true`.
See `vignette("install", package = "arrow")` for details.

## Installing a development version

Development versions of the package (binary and source) are built daily and hosted at
<https://arrow-r-nightly.s3.amazonaws.com>. To install from there:

``` r
install.packages("arrow", repos = "https://arrow-r-nightly.s3.amazonaws.com")
```

Or

```r
arrow::install_arrow(nightly = TRUE)
```

Conda users can install `arrow` nightlies from our nightlies channel using:

```
conda install -c arrow-nightlies -c conda-forge --strict-channel-priority r-arrow
```

These daily package builds are not official Apache releases and are not
recommended for production use. They may be useful for testing bug fixes
and new features under active development.

## Developing

Windows and macOS users who wish to contribute to the R package and
don’t need to alter the Arrow C++ library may be able to obtain a
recent version of the library without building from source. On macOS,
you may install the C++ library using [Homebrew](https://brew.sh/):

``` shell
# For the released version:
brew install apache-arrow
# Or for a development version, you can try:
brew install apache-arrow --HEAD
```

On Windows, you can download a .zip file with the arrow dependencies from the
[nightly repository](https://arrow-r-nightly.s3.amazonaws.com/libarrow/bin/windows/),
and then set the `RWINLIB_LOCAL` environment variable to point to that
zip file before installing the `arrow` R package. Version numbers in that
repository correspond to dates, and you will likely want the most recent.

If you need to alter both the Arrow C++ library and the R package code,
or if you can’t get a binary version of the latest C++ library
elsewhere, you’ll need to build it from source too.

First, install the C++ library. See the [developer
guide](https://arrow.apache.org/docs/developers/cpp/building.html) for details.
It's recommended to make a `build` directory inside of the `cpp` directory of
the Arrow git repository (it is git-ignored). Assuming you are inside `cpp/build`,
you'll first call `cmake` to configure the build and then `make install`.
For the R package, you'll need to enable several features in the C++ library
using `-D` flags:

```
cmake \
  -DARROW_COMPUTE=ON \
  -DARROW_CSV=ON \
  -DARROW_DATASET=ON \
  -DARROW_FILESYSTEM=ON \
  -DARROW_JEMALLOC=ON \
  -DARROW_JSON=ON \
  -DARROW_PARQUET=ON \
  -DCMAKE_BUILD_TYPE=release \
  -DARROW_INSTALL_NAME_RPATH=OFF \
  ..
```

where `..` is the path to the `cpp/` directory when you're in `cpp/build`.

To enable optional features including S3 support, an alternative memory allocator, and additional compression libraries, add some or all of these flags:

```
  -DARROW_S3=ON \
  -DARROW_MIMALLOC=ON \
  -DARROW_WITH_BROTLI=ON \
  -DARROW_WITH_BZ2=ON \
  -DARROW_WITH_LZ4=ON \
  -DARROW_WITH_SNAPPY=ON \
  -DARROW_WITH_ZLIB=ON \
  -DARROW_WITH_ZSTD=ON \
```

Other flags that may be useful:

* `-DARROW_EXTRA_ERROR_CONTEXT=ON` makes errors coming from the C++ library point to files and line numbers
* `-DBOOST_SOURCE=BUNDLED`, for example, or any other dependency `*_SOURCE`, if you have a system version of a C++ dependency that doesn't work correctly with Arrow. This tells the build to compile its own version of the dependency from source.

Note that after any change to the C++ library, you must reinstall it and
run `make clean` or `git clean -fdx .` to remove any cached object code
in the `r/src/` directory before reinstalling the R package. This is
only necessary if you make changes to the C++ library source; you do not
need to manually purge object files if you are only editing R or C++
code inside `r/`.

Once you’ve built the C++ library, you can install the R package and its
dependencies, along with additional dev dependencies, from the git
checkout:

``` shell
cd ../../r
R -e 'install.packages(c("devtools", "roxygen2", "pkgdown", "covr")); devtools::install_dev_deps()'
R CMD INSTALL .
```

If you need to set any compilation flags while building the C++
extensions, you can use the `ARROW_R_CXXFLAGS` environment variable. For
example, if you are using `perf` to profile the R extensions, you may<commit_msg>ARROW-11965: [R][Docs] Simplify install.packages command in R dev docs<commit_after># arrow

[![cran](https://www.r-pkg.org/badges/version-last-release/arrow)](https://cran.r-project.org/package=arrow)
[![CI](https://github.com/apache/arrow/workflows/R/badge.svg?event=push)](https://github.com/apache/arrow/actions?query=workflow%3AR+branch%3Amaster+event%3Apush)
[![conda-forge](https://img.shields.io/conda/vn/conda-forge/r-arrow.svg)](https://anaconda.org/conda-forge/r-arrow)

[Apache Arrow](https://arrow.apache.org/) is a cross-language
development platform for in-memory data. It specifies a standardized
language-independent columnar memory format for flat and hierarchical
data, organized for efficient analytic operations on modern hardware. It
also provides computational libraries and zero-copy streaming messaging
and interprocess communication.

The `arrow` package exposes an interface to the Arrow C++ library to
access many of its features in R. This includes support for analyzing
large, multi-file datasets (`open_dataset()`), working with individual
Parquet (`read_parquet()`, `write_parquet()`) and Feather
(`read_feather()`, `write_feather()`) files, as well as lower-level
access to Arrow memory and messages.

## Installation

Install the latest release of `arrow` from CRAN with

```r
install.packages("arrow")
```

Conda users can install `arrow` from conda-forge with

```
conda install -c conda-forge --strict-channel-priority r-arrow
```

Installing a released version of the `arrow` package requires no
additional system dependencies. For macOS and Windows, CRAN hosts binary
packages that contain the Arrow C++ library. On Linux, source package
installation will also build necessary C++ dependencies. For a faster,
more complete installation, set the environment variable `NOT_CRAN=true`.
See `vignette("install", package = "arrow")` for details.

## Installing a development version

Development versions of the package (binary and source) are built daily and hosted at
<https://arrow-r-nightly.s3.amazonaws.com>. To install from there:

``` r
install.packages("arrow", repos = "https://arrow-r-nightly.s3.amazonaws.com")
```

Or

```r
arrow::install_arrow(nightly = TRUE)
```

Conda users can install `arrow` nightlies from our nightlies channel using:

```
conda install -c arrow-nightlies -c conda-forge --strict-channel-priority r-arrow
```

These daily package builds are not official Apache releases and are not
recommended for production use. They may be useful for testing bug fixes
and new features under active development.

## Developing

Windows and macOS users who wish to contribute to the R package and
don’t need to alter the Arrow C++ library may be able to obtain a
recent version of the library without building from source. On macOS,
you may install the C++ library using [Homebrew](https://brew.sh/):

``` shell
# For the released version:
brew install apache-arrow
# Or for a development version, you can try:
brew install apache-arrow --HEAD
```

On Windows, you can download a .zip file with the arrow dependencies from the
[nightly repository](https://arrow-r-nightly.s3.amazonaws.com/libarrow/bin/windows/),
and then set the `RWINLIB_LOCAL` environment variable to point to that
zip file before installing the `arrow` R package. Version numbers in that
repository correspond to dates, and you will likely want the most recent.

If you need to alter both the Arrow C++ library and the R package code,
or if you can’t get a binary version of the latest C++ library
elsewhere, you’ll need to build it from source too.

First, install the C++ library. See the [developer
guide](https://arrow.apache.org/docs/developers/cpp/building.html) for details.
It's recommended to make a `build` directory inside of the `cpp` directory of
the Arrow git repository (it is git-ignored). Assuming you are inside `cpp/build`,
you'll first call `cmake` to configure the build and then `make install`.
For the R package, you'll need to enable several features in the C++ library
using `-D` flags:

```
cmake \
  -DARROW_COMPUTE=ON \
  -DARROW_CSV=ON \
  -DARROW_DATASET=ON \
  -DARROW_FILESYSTEM=ON \
  -DARROW_JEMALLOC=ON \
  -DARROW_JSON=ON \
  -DARROW_PARQUET=ON \
  -DCMAKE_BUILD_TYPE=release \
  -DARROW_INSTALL_NAME_RPATH=OFF \
  ..
```

where `..` is the path to the `cpp/` directory when you're in `cpp/build`.

To enable optional features including S3 support, an alternative memory allocator, and additional compression libraries, add some or all of these flags:

```
  -DARROW_S3=ON \
  -DARROW_MIMALLOC=ON \
  -DARROW_WITH_BROTLI=ON \
  -DARROW_WITH_BZ2=ON \
  -DARROW_WITH_LZ4=ON \
  -DARROW_WITH_SNAPPY=ON \
  -DARROW_WITH_ZLIB=ON \
  -DARROW_WITH_ZSTD=ON \
```

Other flags that may be useful:

* `-DARROW_EXTRA_ERROR_CONTEXT=ON` makes errors coming from the C++ library point to files and line numbers
* `-DBOOST_SOURCE=BUNDLED`, for example, or any other dependency `*_SOURCE`, if you have a system version of a C++ dependency that doesn't work correctly with Arrow. This tells the build to compile its own version of the dependency from source.

Note that after any change to the C++ library, you must reinstall it and
run `make clean` or `git clean -fdx .` to remove any cached object code
in the `r/src/` directory before reinstalling the R package. This is
only necessary if you make changes to the C++ library source; you do not
need to manually purge object files if you are only editing R or C++
code inside `r/`.

Once you’ve built the C++ library, you can install the R package and its
dependencies, along with additional dev dependencies, from the git
checkout:

``` shell
cd ../../r

Rscript -e '
options(repos = "https://cloud.r-project.org/")
if (!require("remotes")) install.packages("remotes")
remotes::install_deps(dependencies = TRUE)
'

R CMD INSTALL .
```

If you need to set any compilation flags while building the C++
extensions, you can use the `ARROW_R_CXXFLAGS` environment variable. For
example, if you are using `perf` to profile the R extensions, you may
{'fraction_non_alphanumeric': 0.09841137123745819, 'fraction_numerical': 0.002173913043478261, 'mean_word_length': 4.6075949367088604, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 25, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3890678', 'n_tokens_mistral': 3654, 'n_tokens_neox': 3496, 'n_words': 1465}