<jupyter_start><jupyter_text>(*** hide ***)<jupyter_code>#nowarn "211"
open System
let airQuality = __SOURCE_DIRECTORY__ + "/data/airquality.csv"<jupyter_output><empty_output><jupyter_text>(**Interoperating between R and Deedle===================================The [R type provider](http://fslab.org/RProvider/) enablessmooth interoperation between R and F. The type provider automatically discovers installed packages and makes them accessible via the `RProvider` namespace.R type provider for F automatically converts standard data structures betwene Rand F (such as numerical values, arrays, etc.). However, the conversion mechanismis extensible and so it is possible to support conversion between other F types.The Deedle library comes with extension that automatically converts between Deedle`Frame` and R `data.frame` and also between Deedle `Series` and the[zoo package](http://cran.r-project.org/web/packages/zoo/index.html) (Z's ordered observations).This page is a quick overview showing how to pass data between R and Deedle.You can also get this page as an [F script file](https://github.com/fslaborg/Deedle/blob/master/docs/content/rinterop.fsx)from GitHub and run the samples interactively.Getting started---------------To use Deedle and R provider together, all you need to do is to install the [**Deedle.RPlugin** package](https://nuget.org/packages/Deedle.RPlugin), whichinstalles both as dependencies. Alternatively, you can use the [**FsLab**package](http://www.nuget.org/packages/FsLab), which also includes additionaldata access, data science and visualization libraries.In a typical project ("F Tutorial"), the NuGet packages are installed in the `../packages`directory. To use R provider and Deedle, you need to write something like this:*)<jupyter_code>#load "../../packages/RProvider/RProvider.fsx"
#load "../../bin/net45/Deedle.fsx"

open RProvider
open RDotNet
open Deedle<jupyter_output><empty_output><jupyter_text>(**If you're not using NuGet from Visual Studio, then you'll need to manually copy thefile `Deedle.RProvider.Plugin.dll` from the package `Deedle.RPlugin` to the directory where `RProvider.dll` is located (in `RProvider/lib`). Once that'sdone, the R provider will automatically find the plugin.Passing data frames to and from R--------------------------------- From R to DeedleLet's start by looking at passing data frames from R to Deedle. To test this, wecan use some of the sample data sets available in the `datasets` package. The Rmakes all packages available under the `RProvider` namespace, so we can justopen `datasets` and access the `mtcars` data set using `R.mtcars` (when typingthe code, you'll get automatic completion when you type `R` followed by dot):*) (*** define-output:mtcars ***)<jupyter_code>open RProvider.datasets

// Get mtcars as an untyped object
R.mtcars.Value

// Get mtcars as a typed Deedle frame
let mtcars : Frame<string, string> = R.mtcars.GetValue()<jupyter_output><empty_output><jupyter_text>(*** include-value:mtcars ***) (**The first sample uses the `Value` property to convert the data set to a boxed Deedleframe of type `obj`. This is a great way to explore the data, but when you want to do some further processing, you need to specify the type of the data frame that you wantto get. This is done on line 7 where we get `mtcars` as a Deedle frame with both rowsand columns indexed by `string`.To see that this is a standard Deedle data frame, let's group the cars by the number ofgears and calculate the average "miles per galon" value based on the gear. To visualizethe data, we use the [F Charting library](https://github.com/fsharp/FSharp.Charting):*) (*** define-output:mpgch ***)<jupyter_code>#load "../../packages/FSharp.Charting/lib/net45/FSharp.Charting.fsx"
open FSharp.Charting

mtcars
|> Frame.groupRowsByInt "gear"
|> Frame.getCol "mpg"
|> Stats.levelMean fst
|> Series.observations |> Chart.Column<jupyter_output><empty_output><jupyter_text>(*** include-it:mpgch ***) (** From Deedle to RSo far, we looked how to turn R data frame into Deedle `Frame`, so let's lookat the opposite direction. The following snippet first reads Deedle data frame from a CSV file (file name is in the `airQuality` variable). We can then use thedata frame as argument to standard R functions that expect data frame.*)<jupyter_code>let air = Frame.ReadCsv(airQuality, separators=";")<jupyter_output><empty_output><jupyter_text>(*** include-value:air ***) (**Let's first try passing the `air` frame to the R `as.data.frame` function (which will not do anything, aside from importing the data into R). To do something slightly more interesting, we then use the `colMeans` R function to calculate averagesfor each column (to do this, we need to open the `base` package):*)<jupyter_code>open RProvider.``base``

// Pass air data to R and print the R output
R.as_data_frame(air)

// Pass air data to R and get column means
R.colMeans(air)
// [fsi:val it : SymbolicExpression =]
// [fsi:  Ozone  Solar.R  Wind  Temp  Month   Day ]
// [fsi:    NaN      NaN  9.96 77.88   6.99  15.8]<jupyter_output><empty_output><jupyter_text>(** As a final example, let's look at the handling of missing values. Unlike R, Deedle does not distinguish between missing data (`NA`) and not a number (`NaN`). For example, in the following simple frame, the `Floats` column has missing value for keys 2 and 3 while`Names` has missing value for the row 2:*)<jupyter_code>// Create sample data frame with missing values
let df = 
  [ "Floats" =?> series [ 1 => 10.0; 2 => nan; 4 => 15.0]
    "Names"  =?> series [ 1 => "one"; 3 => "three"; 4 => "four" ] ] 
  |> frame<jupyter_output><empty_output><jupyter_text>(**When we pass the data frame to R, missing values in numeric columns are turned into `NaN`and missing data for other columns are turned into `NA`. Here, we use `R.assign` whichstores the data frame in a varaible available in the current R environment:*)<jupyter_code>R.assign("x",  df)
// [fsi:val it : SymbolicExpression = ]
// [fsi:     Floats   Names ]
// [fsi: 1       10     one ] 
// [fsi: 2      NaN    <NA> ]
// [fsi: 4       15    four ]
// [fsi: 3      NaN   three ]<jupyter_output><empty_output><jupyter_text>(**Passing time series to and from R---------------------------------For working with time series data, the Deedle plugin uses [the zoo package](http://cran.r-project.org/web/packages/zoo/index.html) (Z's ordered observations). If you do not have the package installed, you can do thatby using the `install.packages("zoo")` command from R or using `R.install_packages("zoo")` fromF after opening `RProvider.utils`. When running the code from F, you'll need to restart your editor and F interactive after it is installed. From R to DeedleLet's start by looking at getting time series data from R. We can again use the `datasets`package with samples. For example, the `austres` data set gives us access to quarterly time series of the number of australian residents:*)<jupyter_code>R.austres.Value
// [fsi:val it : obj =]
// [fsi:    1971.25 -> 13067.3 ]
// [fsi:    1971.5  -> 13130.5 ]
// [fsi:    1971.75 -> 13198.4 ]
// [fsi:    ...     -> ...     ]
// [fsi:    1992.75 -> 17568.7 ]
// [fsi:    1993    -> 17627.1 ]
// [fsi:    1993.25 -> 17661.5 ]<jupyter_output><empty_output><jupyter_text>(**As with data frames, when we want to do any further processing with the time series, we needto use the generic `GetValue` method and specify a type annotation to that tells the Fcompiler that we expect a series where both keys and values are of type `float`:*)<jupyter_code>// Get series with numbers of australian residents
let austres : Series<float, float> = R.austres.GetValue()

// Get TimeSpan representing (roughly..) two years
let twoYears = TimeSpan.FromDays(2.0 * 365.0)

// Calculate means of sliding windows of 2 year size 
austres 
|> Series.mapKeys (fun y -> 
    DateTime(int y, 1 + int (12.0 * (y - floor y)), 1))
|> Series.windowDistInto twoYears Stats.mean<jupyter_output><empty_output><jupyter_text>(**The current version of the Deedle plugin supports only time series with single column.To access, for example, the EU stock market data, we need to write a short R inlinecode to extract the column we are interested in. The following gets the FTSE time series from `EuStockMarkets`:*)<jupyter_code>let ftseStr = R.parse(text="""EuStockMarkets[,"FTSE"]""")
let ftse : Series<float, float> = R.eval(ftseStr).GetValue()<jupyter_output><empty_output><jupyter_text>(** From Deedle to RThe opposite direction is equally easy. To demonstrate this, we'll generate a simpletime series with 3 days of randomly generated values starting today:*)<jupyter_code>let rnd = Random()
let ts = 
  [ for i in 0.0 .. 100.0 -> 
      DateTime.Today.AddHours(i), rnd.NextDouble() ] 
  |> series<jupyter_output><empty_output><jupyter_text>(**Now that we have a time series, we can pass it to R using the `R.as_zoo` function orusing `R.assign` to store it in an R variable. As previously, the R provider automaticallyshows the output that R prints for the value:*)<jupyter_code>open RProvider.zoo

// Just convert time series to R
R.as_zoo(ts)
// Convert and assing to a variable 'ts'
R.assign("ts", ts)
// [fsi:val it : string =
// [fsi: 2013-11-07 05:00:00 2013-11-07 06:00:00 2013-11-07 07:00:00 ...]
// [fsi: 0.749946652         0.580584353         0.523962789         ...]<jupyter_output><empty_output><jupyter_text>(**Typically, you will not need to assign time series to an R variable, because you can use it directly as an argument to functions that expect time series. For example, thefollowing snippet applies the rolling mean function with a window size 20 to the time series.*)<jupyter_code>// Rolling mean with window size 20
R.rollmean(ts, 20)<jupyter_output><empty_output><jupyter_text>(**This is a simple example - in practice, you can achieve the same thing with `Series.window`function from Deedle - but it demonstrates how easy it is to use R packages with time series (and data frames) from Deedle. As a final example, we create a data frame thatcontains the original time series together with the rolling mean (in a separate column)and then draws a chart showing the results:*) (*** define-output:means ***)<jupyter_code>// Use 'rollmean' to calculate mean and 'GetValue' to 
// turn the result into a Deedle time series
let tf = 
  [ "Input" => ts 
    "Means5" => R.rollmean(ts, 5).GetValue<Series<_, float>>()
    "Means20" => R.rollmean(ts, 20).GetValue<Series<_, float>>() ]
  |> frame

// Chart original input and the two rolling means
Chart.Combine
  [ Chart.Line(Series.observations tf?Input)
    Chart.Line(Series.observations tf?Means5)
    Chart.Line(Series.observations tf?Means20) ]<jupyter_output><empty_output><jupyter_text>(**Depending on your random number generator, the resulting chart looks something like this:*) (*** include-it:means ***)<jupyter_code><jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1169343735658559, 'fraction_numerical': 0.02010096374483708, 'mean_word_length': 4.35693215339233, 'pattern_counts': {'":': 0, '<': 73, '<?xml version=': 0, '>': 103, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '3913', 'license': 'MIT', 'n_tokens_mistral': 3501, 'n_tokens_neox': 3265, 'n_words': 1319, 'path': 'notebooks/other/Deedle/rinterop.ipynb', 'repo_name': 'SpaceAntelope/IfCntk', 'size': '10895'}