<commit_before><!---
  Licensed to the Apache Software Foundation (ASF) under one
  or more contributor license agreements.  See the NOTICE file
  distributed with this work for additional information
  regarding copyright ownership.  The ASF licenses this file
  to you under the Apache License, Version 2.0 (the
  "License"); you may not use this file except in compliance
  with the License.  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing,
  software distributed under the License is distributed on an
  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
  KIND, either express or implied.  See the License for the
  specific language governing permissions and limitations
  under the License.
-->

# arrow 8.0.0.9000

* The `arrow.dev_repo` for nightly builds of the R package and prebuilt
  libarrow binaries is now https://nightlies.apache.org/arrow/r/.
* `lubridate::parse_date_time()` datetime parser:
  * `orders` with year, month, day, hours, minutes, and seconds components are supported.
  * the `orders` argument in the Arrow binding works as follows: `orders` are transformed into `formats` which subsequently get applied in turn. There is no `select_formats` parameter and no inference takes place (like is the case in `lubridate::parse_date_time()`).
* New functions `read_ipc_file()` and `write_ipc_file()` are added.
  These functions are almost the same as `read_feather()` and `write_feather()`,
  but differ in that they only target IPC files (Feather V2 files), not Feather V1 files.
* `read_arrow()` and `write_arrow()`, deprecated since 1.0.0 (July 2020), have been removed.
  Instead of these, use the `read_ipc_file()` and `write_ipc_file()` for IPC files, or,
  `read_ipc_stream()` and `write_ipc_stream()` for IPC streams.
* `write_parquet()` now defaults to writing Parquet format version 2.4 (was 1.0). Previously deprecated arguments `properties` and `arrow_properties` have been removed; if you need to deal with these lower-level properties objects directly, use `ParquetFileWriter`, which `write_parquet()` wraps.
* added `lubridate::qday()` (day of quarter)

# arrow 8.0.0
<commit_msg>ARROW-17188: [R] Update news for 9.0.0 (#13726)<commit_after><!---
  Licensed to the Apache Software Foundation (ASF) under one
  or more contributor license agreements.  See the NOTICE file
  distributed with this work for additional information
  regarding copyright ownership.  The ASF licenses this file
  to you under the Apache License, Version 2.0 (the
  "License"); you may not use this file except in compliance
  with the License.  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing,
  software distributed under the License is distributed on an
  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
  KIND, either express or implied.  See the License for the
  specific language governing permissions and limitations
  under the License.
-->

# arrow 8.0.0.9000

## Arrow dplyr queries

* New dplyr verbs:
  * `dplyr::union` and `dplyr::union_all` (ARROW-15622)
  * `dplyr::glimpse` (ARROW-16776)
  * `show_exec_plan()` can be added to the end of a dplyr pipeline to show the underlying plan, similar to `dplyr::show_query()`. `dplyr::show_query()` and `dplyr::explain()` also work and show the same output, but may change in the future. (ARROW-15016)
* User-defined functions are supported in queries. Use `register_scalar_function()` to create them. (ARROW-16444)
* `map_batches()` returns a `RecordBatchReader` and requires that the function it maps returns something coercible to a `RecordBatch` through the `as_record_batch()` S3 function. It can also run in streaming fashion if passed `.lazy = TRUE`. (ARROW-15271, ARROW-16703)
* Functions can be called with package namespace prefixes (e.g. `stringr::`, `lubridate::`) within queries. For example, `stringr::str_length` will now dispatch to the same kernel as `str_length`. (ARROW-14575)
* Support for new functions:
  * `lubridate::parse_date_time()` datetime parser: (ARROW-14848, ARROW-16407)
    * `orders` with year, month, day, hours, minutes, and seconds components are supported.
    * the `orders` argument in the Arrow binding works as follows: `orders` are transformed into `formats` which subsequently get applied in turn. There is no `select_formats` parameter and no inference takes place (like is the case in `lubridate::parse_date_time()`).
  * `lubridate` date and datetime parsers such as `lubridate::ymd()`, `lubridate::yq()`, and `lubridate::ymd_hms()` (ARROW-16394, ARROW-16516, ARROW-16395)
  * `lubridate::fast_strptime()` (ARROW-16439)
  * `lubridate::floor_date()`, `lubridate::ceiling_date()`, and `lubridate::round_date()` (ARROW-14821)
  * `strptime()` supports the `tz` argument to pass timezones. (ARROW-16415)
  * `lubridate::qday()` (day of quarter)
  * `exp()` and `sqrt()`. (ARROW-16871)
* Bugfixes:
  * Count distinct now gives correct result across multiple row groups. (ARROW-16807)
  * Aggregations over partition columns return correct results. (ARROW-16700)

## Reading and writing

* New functions `read_ipc_file()` and `write_ipc_file()` are added.
  These functions are almost the same as `read_feather()` and `write_feather()`,
  but differ in that they only target IPC files (Feather V2 files), not Feather V1 files.
* `read_arrow()` and `write_arrow()`, deprecated since 1.0.0 (July 2020), have been removed.
  Instead of these, use the `read_ipc_file()` and `write_ipc_file()` for IPC files, or,
  `read_ipc_stream()` and `write_ipc_stream()` for IPC streams. (ARROW-16268)
* `write_parquet()` now defaults to writing Parquet format version 2.4 (was 1.0). Previously deprecated arguments `properties` and `arrow_properties` have been removed; if you need to deal with these lower-level properties objects directly, use `ParquetFileWriter`, which `write_parquet()` wraps. (ARROW-16715)
* UnionDatasets can unify schemas of multiple InMemoryDatasets with varying
  schemas. (ARROW-16085)
* `write_dataset()` preserves all schema metadata again. In 8.0.0, it would drop most metadata, breaking packages such as sfarrow. (ARROW-16511)
* Reading and writing functions (such as `write_csv_arrow()`) will automatically (de-)compress data if the file path contains a compression extension (e.g. `"data.csv.gz"`). This works locally as well as on remote filesystems like S3 and GCS. (ARROW-16144)
* `FileSystemFactoryOptions` can be provided to `open_dataset()`, allowing you to pass options such as which file prefixes to ignore. (ARROW-15280)
* By default, `S3FileSystem` will not create or delete buckets. To enable that, pass the configuration option `allow_bucket_creation` or `allow_bucket_deletion`. (ARROW-15906)
* `GcsFileSystem` and `gs_bucket()` allow connecting to Google Cloud Storage. (ARROW-13404, ARROW-16887)


## Arrays and tables

* Table and RecordBatch `$num_rows()` method returns a double (previously integer), avoiding integer overflow on larger tables. (ARROW-14989, ARROW-16977)

## Packaging

* The `arrow.dev_repo` for nightly builds of the R package and prebuilt
  libarrow binaries is now https://nightlies.apache.org/arrow/r/.
* Brotli and BZ2 are shipped with MacOS binaries. BZ2 is shipped with Windows binaries. (ARROW-16828)

# arrow 8.0.0

{'fraction_non_alphanumeric': 0.09784075573549257, 'fraction_numerical': 0.030364372469635626, 'mean_word_length': 4.8034455755677365, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2159828', 'n_tokens_mistral': 2357, 'n_tokens_neox': 2166, 'n_words': 919}