The **dataset** package

Christian Dietrich 2013-2016

stettberger@dokucode.de

[https://github.com/stettberger/dataset](https://github.com/stettberger/dataset)

2022/03/25 v0.7

## 1 Introduction

Writing scientific texts is a craft. It is the craft of communicating your results to your colleagues and to the curious world public. Often your conclusions are based upon facts and numbers that you gathered during your research for the specific topic. You might have done many experiments and produced lot of data. The craft of writing is to guide your reader through a narrative that is based upon that data. But there may be many versions of that data. Perhaps you found a problem in your experiment, while already writing, that forces you back into the laboratory. After a while, the moon has done its circle many times, you return from that dark place and your methodology has improved as significantly as your data has. But now you have to rewrite that parts of the data that reference the old data points.

The **dataset** is here to help you with managing your data points. It provides you with macro style keys that represent symbolic names for your data points. You can reference those symbolic names with \dref, use them in calculations to have always up-to-date percentage values, define projections between sets of data points and document them. **dataset** also introduces the notion of assertions (\drefassert) for your results to ensure that your prosa text references fit the underlying data.

## 2 Usage (or 32 mice)

```
1Orfset{/medA/micecount}{32}
2Orfset{/medA/recovered}{20}
3Fromthe\dref{/medA/micecount}miceintheexperiment,
4Orfcalc[prefix=/medA]{d(/micecount)-d(/recovered)}died.
```

### Design Principles

Before we jump into the description of **dataset**, let us look a little bit into the design principles of **dataset**. By understanding the principles, you will be more productive and embedding data into your document will become easier.

First of all, dataref is built on top of pgfkeys and pgfmath from the PGF/TiKZ macro packages. While the former provides a usable user interface to provide options to dataref, the later is used to perform computation on your datapoints. If you are interested into these two excellent TeX packages, please look at texdoc pgfmanual for further information.

There are two aspects of dataref: setting datapoints and referencing datapoints. While setting datapoints is kind of boring, we have a wide variety of options when it comes to referencing. The expansion of datapoints is done in multiples phases (see Figure 1).

The dataref macros are different regarding the phases they include or omit and in their default settings. In the following, we will discuss all options and macros you can use to reference your datapoints. By default, the \drefresult is always set to the result of the pipeline.

### Setting Datapoints

\drefset[\(\langle\mathit{options}\rangle\)]{\(\langle\mathit{name}\rangle\)} (@set) \drefset{/med A/mice race}{Black Six} \drefset{/med A/mice count}{32} \drefset{/med A/dead after 24h}{6} \drefset{/med A/dead after 48h}{1} The \drefset command is used to define the symbolic datapoints. The name of a datapoint may contain virtually all characters, including spaces and slashes. It is good practice to use a hierarchy to structure your data point names. The value might be any string, while the focus of dataref is on numerical datapoints. The \drefset command works outside the pipeline model (Figure 1) for performance reasons. It virtually consists only of the @options and @set stage.

\drefsave[\(\langle\mathit{options}\rangle\)]{\(\langle\mathit{name}\rangle\)} (@set)

Figure 1: The dataref Pipeline


\drefvalueof{\(\(name\))} \drefref{\(name)} (@get)

\drefvalueof{/med A/mice race} Since \dref is not expandable, it cannot be used in all circumstances. Therefore, \drefvalueof bypasses all internal bookkeeping and provides access to the raw datapoint value. \drefref can be used to mark the datapoint as used to let it appear in the datagraphy.

\dref/ignoremissing=\(\langle\)_true-or-false_\(\rangle\) (default **true**, initially **false**) (/dref/defaultvalue=\(\langle\)_value_\(\rangle\) (no default, initially **1.0**) By default, dataref produces an error if the referenced datapoint is undefined. If ignoremissing is given, the defaultvalue is used. This key is useful in combination with \drefsave. Furthermore, it is possible to extract the missing keys from the aux file and to retrieve it from a secondary source (e.g. database, wikidata, etc).

\dref*[ignoremissing,defaultvalue=undefined]{blank}

\drefsethelp{\(\langle\)_pattern_\(\rangle\)}

\drefhelp{\(\langle\)_name_\(\rangle\)}

dataref comes with a simple method for defining documentation for data points. This help can for example be used to communicate what is the concrete semantics of the data point. This is of special interest when writer and data gatherer are not the same person. \drefsethelp takes two arguments: first a regular expression that matches the symbolic data point, second the help text.

\drefsethelp{.*/mice race}{The mice race used for experiments heavily influences the outcome of the results}

\drefsethelp{/med A/mice race}

\drefresult

Is set in the @set phase to the result of the currently executed pipeline.

### Calculations and Math Tools

\drefcalc[\(\langle\)_options_\(\rangle\)]{\(\langle\)_expression_\(\rangle\)} (@calc, @print, @set) \drefcalc*[\(\langle\)_options_\(\rangle\)]{\(\langle\)_expression_\(\rangle\)} (@calc, @set)\dreffomat*[\(\langle options\rangle\)]{(\(\langle number\rangle\))} The \drefcalc is the core function of calculating with data points. It is based on the pgfmath engine, but allows also the usage of symbolic datapoints within mathematical expressions. Datapoints can either are inserted into the calculations with the \(\langle\langle path\rangle\rangle\) or the \ata("\(\langle path\rangle\)") notation. The starred variant of \drefcalc does not print the result, but only sets the result macros.

It is important to note, that \drefcalc always uses the /pgf/fpu environment. The FPU feature of pgfmath is used to handle large numbers, which may occur often when handling experiment data points.

\drefcalc{(4+7)/12 * 100}\drefcalc{d(/med A/mice count) * 100}\drefcalc{data("/med A/mice count") * 100}\drefcalc*{1+3}\drefresult

Since the default printing mechanism of dataref utilizes PGF, all options from /pgf/number format can be directly used in the options. \drefformat does only the printing. For documentation on the available options, please consult the PGF manual.

\drefcalc[precision=4]{1/3}\\\\\ \drefcalc[sci]{123456789}\\\\ \drefcalc[prefix=/med A/,frac]{d(recovered)/d(mice count)}\\\\ \drefformat[fixed zerofill, precision=2]{\drefresult}

### Units and Unit Scaling

dataref allows to give the unit of a datapoint and enforces the correct combination of units when using them in calculations. dataref units can be arbitrary combinations of macros and strings, which allows the combination with the SIUnitTX package.

\dref/unit=\(\langle unit\rangle\)

The unit of a datapoint is loaded in the @get phase, and stored in the @set phase of the dataref pipeline.

\drefset[unit=ms]{/duration}{5555}\drefset[unit=\(\langle\)joule\(\rangle\){/power}{1234}

/dref/unit/format=\(\langle formatting style\rangle\) (no default, choice) /dref/unit/format default=\(\langle formatting style\rangle\) (initially **plain**)

If a datapoint with unit is referenced, the unit is printed after the value. The formatting mechanism can be exchanged, in order to omit the unit, or to use SIUnitX for properly print it. By default, the unit/format default is set in the @init phase. If you are using SIUnitX in your document, it is safe to set the default value accordingly. Currently, the values false, plain, and siunitx are valid formatting styles.


with any other dref macro that includes the @calc phase.

All operations operate on the current result, which is initially the given key or value from the mandatory argument.

\drefel[percentof=123]{40}

The different relation operations try to have a speaking name, such that the TeX code is easily understandable by the writer. This explicit notation aims to prevent common mistakes, like confusing denominator and numerator.

The starred version of \drefel does not print any number, but only saves the result in \drefresult. Instead of only taking datapoint keys, \drefel, as well as the relating operations, take either a key or a bare number as you can see from the example.

/dref/scale by=\(\langle\)_key or value\(\rangle\)_

\[\text{(no default)}\]

\[\text{Scales the current value by the given factor, which must be unit less. The resulting unit is untouched.}\]

/dref/percent

\[\text{(no default)}\]

\[\text{(no default)}\]

\[\text{(no default)}\]

\[\text{(no default)}\]

\[\text{(no default)}\]

\[\text{(dref/divide by=\langle\)_key or value\(\rangle\)_

\[\text{(no default)}\]

\[\text{Divides the current value by the given factor, which must be unit less. The resulting unit is untouched.}\]

\[\text{(no default)}\]

\[\text{(dref/abs}\]

\[\text{Calculate the absolute value.}\]

\[\text{(no default)}\]

\[\text{(dref/factor of=\langle\)_key or value\(\rangle\)_

\[\text{(no default)}\]

\[\text{(dref/percent of=\langle\)_key or value\(\rangle\)_

\[\text{(no default)}\]

\[\text{The \bf factor of operation gives the portion the current value in relation to the given base value. In easy words: a division. This macro ensures, that base and current vale have the same unit or are unit less. The result of this operation is unit less. The **percent of** operation, furthermore, scales the result with 100 to get a percentage.

/dref/increase from=\(\langle\)_key or value\(\rangle\)_

\[\text{(no default)}\]

\[\text{(dref/decrease factor from=\langle\)_key or value\(\rangle\)_

\[\text{(no default)}\]

\[\text{(dref/decrease factor from=\langle\)_key or value\(\rangle\)_

\[\text{(no default)}\]

\[\text{(dref/increase percent from=\langle\)_key or value\(\rangle\)_

\[\text{(no default)}\]

\[\text{(dref/decrease percent from=\langle\)_key or value\(\rangle\)_

\[\text{(no default)}\]

In a situation, where a datapoint is the result of a changed experiment setup, the value normally shows an increased or decrease numerical value. This family of operations calculates this delta, assuming it is an increase or decrease. The factor operations scale the result to the base value, and the percent operations give this scaled value as a percentage.

\drefrel[increase from=500]{525}\\\drefrel[increase factor from=500,fixed]{525}\\\drefrel[increase percent from=500]{525}\,\%

### Helper Utilities

\drefrow[\(options)]{\(comma-separated list)\{\(key\ template)\}}

\drefrow*{\(comma-separated list)\{\(template)\}}

Often different columns in a table have to be obtained from your data points. Often those rows and columns are similar. Generating parts of tables within LaTeXis very tricky, so dataset provides you with \drefrow. This macro iterates over a comma-separated list of values and fills out a macro which is interpreted as a symbolic data point. The entries are seperated with & and printed. In the starred variant the resulting text is not interpreted as symbolic name, but as a macro.

Both, unstarred and starred variant take a template (a macro body) that is expanded once for every item in the given list. The first replacement #1 is list item and the second #2 is the current index starting from 0. The unstarred variant interprets the expanded result as a datapoint key and uses \dref to expand it; the optional parameter is passed through to every invocation of \dref. The starred variant does not wrap the result into \dref, and, therefore, is more flexible.

\drefassert{\(expr)}

Sometimes the underlying data changes while you are writing. But what if your prose text relies on certain characteristics of the data. \drefassert uses a pgf-math expression that evaluates to true or false. When the assertion holds (true) nothing happens, only a terminal message is printed. When it does not hold (false) the compilation is aborted.

\drefassert{data("/control group/mice count") > 30}

Of the more than thirty infected mice...

\dref/noassert=\(\langle\)_true or false_\dref(default **true**)

The **noassert** deescalates all assertion errors to mere warnings. This option can also be given at the \usepackage invocation.

\dref/annotate=\(\langle\)_annotation type_\dref(no default, initially **none**, choice)

\dref/annotate=none


\begin{tabular}{|l|l|l|} \hline Keys without Description & Page & Value \\ \hline
**/foo** & 3 & 32 \\
**/bar** & 3 & 32 \\
**blah** & 4 & **undefined** \\
**/duration** & 5 & 5555 ms \\
**/power** & 5 & 1234 J \\ \hline \end{tabular}

_For these keys, no description was given_