<jupyter_start><jupyter_text>BRIL Work Suite<jupyter_code><div class="mynavbar cell">
<ul class="my-nav">
<li><a class="my-nav" href="http://cms-service-lumi.web.cern.ch/cms-service-lumi/dbrss.xml">News</a></li>
<li><a class="my-nav" href="http://cms-service-lumi.web.cern.ch/cms-service-lumi/releaserss.xml">Releases</a></li>
<li><a class="my-nav" href="http://cms-service-lumi.web.cern.ch/cms-service-lumi/brilwsdocs.html">Docs</a></li>
</ul>
</div>
<br>
<br><jupyter_output><empty_output><jupyter_text>A commandline toolkit for CMS Beam Radiation Instrumentation and Luminosity (BRIL). The BRIL Work Suite works on top of the python virtual environment brilconda which is a custom build of the Anaconda Scientific Python distribution. The toolkit is compatible with python2 and python3.As prerequisite, the python virtual environment should be centrally or privately installed.And there is NO requirement on the PYTHONPATH nor LD_LIBRARY_PATH to envoke the software.The virtual environment brilconda has much longer release cycle than the bril toolkit.  Documentation for release 3.6.x**Table Of Contents:**1.   [Prerequisite for running and installing brilws](prerequisite)2.   [Install brilws](installation)3.   [Install brilconda virtual environment](brilcondainstallation)4.   [Command Structure](commandstructure)5.   [brilcalc](brilcalc)    *   [lumi](brilcalclumi)    *   [beam](brilcalcbeam)         *   [trg](brilcalctrg)    6.   [briltag](briltag)     *   [listiov](briltaglistiov)    *   [insertiov](briltaginsertiov)    *   [listdata](briltaglistdata)    *   [insertdata](briltaginsertdata)    7.   [Appendix A : trigger rules](triggerrules)  Prerequisite for running and installing brilws 1.  work with centrally installed virtual environment:   *python2*       ```    export PATH=$HOME/.local/bin:/cvmfs/cms-bril.cern.ch/brilconda/bin:$PATH (bash)     ```        *python3*        ```    export PATH=$HOME/.local/bin:/cvmfs/cms-bril.cern.ch/brilconda3/bin:$PATH (bash)    ```     2.  work with central installationon bril online cluster in .cms        *python2*        ```    export PATH=$HOME/.local/bin:/nfshome0/lumipro/brilconda/bin:$PATH    (bash)    ```        *python3*    ```    export PATH=$HOME/.local/bin:/nfshome0/lumipro/brilconda3/bin:$PATH    (bash)    ```    3.  privately installation of virtual environment    ```    export PATH=/bin:$PATH       (bash)    ```    Note: the **\$PATH** requirement must be fullfilled for the **shell** from which the brilws and pip commands run. Be aware that external scripts, e.g. cmsenv.sh, alter the **\$PATH** environment, users should make sure the environment is as above **at the moment of** running the brilws and pip commands.  Installation of brilws**Please do not attempt this if the prerequisite condition is not satisfied**  Install brilws  cern```pip install --user brilws```  online (on bril host)```pip install --no-index --find-links=file:///nfshome0/lumipro/installers/linux-64 --user brilws```  private virtual environment brilconda```pip install brilws```  Install a specific version Specify the version of the package with == operator: e.g. brilws==3.6.2  Uninstall brilws **Please do not attempt this if prerequisite condition is not satisfied** ```pip uninstall brilws```  Check most recent version (Please check prerequisite condition is satisfied before anything) ```pip show brilws```  Install virtual environment brilconda (Optional) The virtual environment is centrally installed in the following locations:     cvmfs  :  /cvmfs/cms-bril.cern.ch/brilconda (python2), /cvmfs/cms-bril.cern.ch/brilconda3 (python3)    Note: the cvmfs mount is on-demand. It does not appear unless specifically accessed. bril online cluster : /nfshome0/lumipro/brilconda (python2), /nfshome0/lumipro/brilconda3 (python3)                             If local resource permits, users can choose to install the brilconda environment privately for more control over the working environment. Install brilconda on different platforms: Linux-x86_64 with installer python2```wget https://cern.ch/cmslumisw/installers/linux-64/Brilconda-1.1.7-Linux-x86_64.sh   bash Brilconda-1.1.7-Linux-x86_64.sh -b -p ```     python3   ```wget https://cern.ch/cmslumisw/installers/linux-64/Brilconda-3.2.16-Linux-x86_64.sh   bash Brilconda-3.2.16-Linux-x86_64.sh -b -p ``` MacOS-64 with installer     python2    ```wget https://cern.ch/cmslumisw/installers/macos-64/Brilconda-1.1.7-MacOSX-x86_64.shbash Brilconda-1.1.7-MacOSX-x86_64.sh -b -p ``` python3 ```wget https://cern.ch/cmslumisw/installers/macos-64/Brilconda-3.2.16-MacOSX-x86_64.shbash Brilconda-3.2.16-MacOSX-x86_64.sh -b -p ```Regardless the location of the installation, the virtual environment is activated whenever its bin directory is in the PATH, and disactivated when the bin directory is not present in the PATH.<jupyter_code><div class="warn cell">
<p>
<strong>Note:</strong> There is no requirements on the PYTHONPATH nor LD_LIBRARY_PATH to envoke python software.
</p>
</div><jupyter_output><empty_output><jupyter_text>Off-site access     BRIL tools analyse data in the database server at CERN which is closed to the outside. Therefore the most convienient way is to run the toolkit on hosts (private or public) at CERN. If you must use the software installed outside the CERN intranet, a ssh tunnel to the database server at CERN has to be established first. Since the tunneling procedure requires a valid cern computer account, a complete unregistered person will not be able to access the BRIL data in any case.     The following instruction assumes the easiest setup: you have **two** open sessions on the same off-site host, e.g. cmslpc32.fnal.gov, one for the ssh tunnel and another for execution. It is also assumed that all the software are installed and the $PATH variable set correctly on the execution shell.  ssh tunneling session```    ssh -N -L 10121:itrac5117-v.cern.ch:10121 @lxplus.cern.ch```Provide you cern account password and the tunnel will kept open without getting back the shell prompt. open an execution session on the same host that has the ssh tunnel```    brilcalc lumi -c offsite -r 281636```Run bril tools with connection string **-c offsite**.<jupyter_code><div class="warn cell">
<p><strong>Note:</strong> the -c <strong>offsite</strong> parameter is supported only with release version >=2.2.0. </p>
</div>

<!--p>
**Link**: [knowledge base for oracle access from outside CERN intranet](https://cern.service-now.com/service-portal/article.do?n=KB0004111)
<p--><jupyter_output><empty_output><jupyter_text>web-cache access   In addition to online, offline, on-site and off-site access to bril database server, indirect accessvia web cache is also supported for bril tools (read-only). Specificlly, web cache refers to the frontier/squid infrastructure available for all CMS sites. (It is also possible to have your own private setup.)   Using web cache access allows queries and results to be cached close to you therefore reducing the overall load on the database server at CERN. It is much faster to get already cached results. Another convenience is that accessing data from outside the CERN firewall does not require ssh tunnels in this case.   However, keep in mind that web cached are not real time data. The caches are reset every about 5 hours. In another word, the most recent results are not necessarily correct. For example, querying last run from the cache might show 5 lumi sections, while there were actually 200 lumi sections in the database. You get 5 because the cache close to you was already populated by yourself or other peoples since the last cache reset.  Any update operation on the database server will not be reflected by the cache for a few hours.   Weigh carefully advantage and disadvantage and use your own judgement when choosing between direct and indirect accesses.  use the cache server at CERNOption ** -c web ** directs you to the web cache server at CERN ** cmsfrontier.cern.ch:8000/LumiCalc **. e.g.```brilcalc lumi -c web -r 284077```  use any cache serverOption **```-c ```** allows you to use the web cache most conveniently located and configured.The configuration file can be found at each CMS site, it is normally called as "site-local-config.xml".You can use it as it is or use it as a starting example to build your own configuration:```    brilcalc lumi -r 284077 -c /afs/cern.ch/cms/SITECONF/CERN/JobConfig/site-local-config.xml```To build custom cache config xml, only contents enclosed in the `````` tag are required by the bril tool, all other tags are ignored: e.g.```    more /home/me/mycache.xml                                                                                  brilcalc lumi -r 284077 -c /home/me/mycache.xml```Note: if you are working in the .cms network, you can not use the web cache. Anyway there is no need for that. [Back to Top](top)  Commands**Please do not attempt anything bellow if prerequisite condition is not satisfied** The commandlines in the work suite are made of main and sub commands. Each subcommand have its independent help menus. For example, brilcalc lumi -h, where brilcalc is the main command, lumi is the subcommand.```brilcalc -h``````briltag -h``````brilcalc lumi -h``````brilcalc beam -h``````brilcalc trg -h```  brilcalc  common command optionsThere are four categories of common command options: selection, filter, output and display, database connection   *  Selections: select on input lumi section, run, fill number or begin/end time in terms of UTC time or fill/run number. These selections can be overlapping.<jupyter_code><div class="cmdlines cell">
    <p><strong>-i INPUTSELECTION : </strong> input lumi section selection file or string format 
    {run:[[startls,stopls],[startls,stopls],...]} 
   
   </p>
   <p>
    In the string input, the whole string should be quoted not the run number, e.g. "{123456:[[1,4][6,19]]}". In the file input, the run number field must be quoted. 
   </p>
   <p> 
    By default, the command checks the lumi sections specified by the -i option against those actually returned as result and print out the list of mismatches. This feature can be switched off with <b>--without-checkjson</b>.
    </p>    
    <p><strong>--begin BEGIN :</strong> UTC time stamp MM/DD/YY HH24:MI:SS, fill or run number of the beginning lumi section
    </p>
    <p><strong>--end END : </strong>UTC time stamp MM/DD/YY HH24:MI:SS, fill or run number of the ending lumi section
    </p>
    <p><strong>-f FILLNUM :</strong> fill number
    </p>
    <p><strong>-r RUNNUM :</strong> run number
    </p>
</div><jupyter_output><empty_output><jupyter_text>*  Filters: Filter on the result<jupyter_code><div class="cmdlines cell">
    
    <p><strong>-b BEAMSTATUS : </strong> filter on beam status. Choices ["STABLE BEAMS", "FLAT TOP","ADJUST","SQUEEZE"]
    </p>
    <p><strong>--amodetag AMODETAG : </strong> filter on machine mode. Choices ["PROTPHYS","IONPHYS", "PAPHYS"]
    </p>
    <p><strong>--beamenergy BEAMENERGY : </strong> filter on target single beam energy in GeV. e.g. 1650
    </p>
   </div><jupyter_output><empty_output><jupyter_text>*  Output and Display<jupyter_code><div class="cmdlines cell">
   
   <p><strong>-o OUTPUTFILE : </strong> output csv file. "-" is the special file stdout. This file output has precedence over the screen display. 
   </p>
   
   <p><strong>--output-style OUTPUT STYLE : </strong> output style on screen. Choices ["tab","csv","html"]. This option has no effect if there is already -o option. 
   </p>
    
   <p><strong>-n SCALEFACTOR : </strong> apply the scalefactor 1./n to the output. Contrary to older releases, it does NOT automatically change the output luminosity unit.
   </p>
   
   <p><strong>--cerntime : </strong> switch to display result time field in cern local time. This switch has no effect on the input time. 
   </p>
    
   <p><strong>--tssec : </strong> switch to display result time field in unix timestamp. 
   </p>
   
   </div><jupyter_output><empty_output><jupyter_text>*  Database connection Reasonable default parameters are given, readonly users do not need to specify these options.<jupyter_code><div class="cmdlines cell">
   <p><strong>-c CONNECTION : </strong> database service to connect to. Example choices ["offline","online","onlinew","dev"]. The default is "offline".
   </p>
   <p><strong>-p AUTHFILE : </strong> database authentication file. Normal readonly users do not need to specify this option.
    </p>
</div><jupyter_output><empty_output><jupyter_text>lumi Subcommand to show luminosity measured by BRIL. By default, i.e. without --type or --normtag options, the summary of the best known luminosity from online is shown. Switches and options specific to the subcommand are explained below.   --filedata FILEDATA  Raw data can be fetched from files and combined with other data from database to produce calibrated luminosity results. When using input from files, database connection parameters, implicitly or explicitly given, are still required. This option is an extention rather than a replacement of the general database connection options. Results will have the same granularity as that of the input. If inputs from files are in lumi-section, then the output is per LS, if inputs are in NB4, then output is on NB4. Since the NB4 granularity is 16 times larger than the LS granularity, this option should be used on small data sets, for example, those on VDM scans.  The input parameter can be either a directory or a single file. The input files must be in hdf5 format and contain at least the BRIL "tcds" table. The file input option works only together with --byls or --byls --xing. And the output luminosity unit is fixed hz/ub. Using option -u will not change the output unit.In this case, because the output is instantaneous luminosity and the integration time is unknown a priori by the software, the summary sum table cannot be calculated therefore will not be displayed at the end with this option.  --byls switchShow luminosity and  average pileup by lumi section.   --xing switchShow luminosity by lumi section and per bunch crossing. The bxidx field of the output has the range of [1,3564]. More often not all 3564 possible bunches are interesting. Three filters can be specified with this switch to reduce the total output. The filters can be applied accumulatively. * Bunch filters specific to --xing switch<jupyter_code><div class="cmdlines cell">
   
   <p><strong>--xingId BXID : </strong> select bunch by id. Input can be a single number, a comma separated list in file or string format. e.g. --xingId "1,2,3"
   </p>
   <p><strong>--xingTr BXTHRESHOLD : </strong> select bunch with luminosity above the specified fraction of the max bx lumi. Input is a float number between 0 and 1. e.g. 0.85. 
   </p>    
   <p><strong>--xingMin XINGMIN :</strong> select bunch with luminosity greater than the specified value. Note: any scaling effect from -n or -u options are already taken into account. The specified min value cut is applied to the final number.
   </p>
   </div>
<div class="warn cell">
<p>
<strong>Note:</strong> for better performance, it is strongly recommended to use -o OUTPUTFILE or --output-style csv for the **--xing** option. Because in the case of csv output (to file or to screen), the result is streamed out without waiting for the whole display table instance to be fully collected.
</p>
</div><jupyter_output><empty_output><jupyter_text>--normtag NORMTAG Apply calibration/correction function defined by a tag. Each norm tag is bound to a specific luminometer. By using the tag, you have automatically selected a luminometer.When selecting data from a combination of luminometers, a json format file or string is accepted (specified below). In this case, both -i and --normtag options can select run and lumi sections. When both given, the **intersection** of the two are used. Also, it is required that --normtag selection is a superset of the -i selection. On failing this check, warning messages will be printed but the program continues.     The composite selection is a list of lists as described below:  [    [normtag,{run:[[startls,stopls],[startls,stopls]...]} | run, run, ... ],     [normtag,{run:[[startls,stopls],[startls,stopls]...]} | run, run, ... ],     ...  ]  Run, ls selection matrices follow each normtag in which the chosen lumi sections are valid.    Here "|" means "or". Flat format of the inner list [normtag, run, run,...] is a shortcut to indicate that all lumi sections in the run are selected. Internally each run number string is translated to {run:[[1,999999]]}.Example below:  [ ["bcm1fv1",{"251118":[[1,20]]}],    ["hfocv1",{"251118":[[21,21]]}],    ["bcm1fv1",{"251118":[[22,24]]}],    ["bcm1fv1","251119","251131"],    ["hfocv1","251134"],    ["bcm1fv1","251167"],    ["hfocv1","251168"]  ]  When this type of selection file is given --type argument has no more effects.  The same rules apply for quoting as with -i : when in file, **key** fields and **string** fields must be quoted. While as string input, the entire string should be quoted.     -u UNIT  Show luminosity in the specified unit and scale the output value accordingly. Recognised unit strings are/kb , 1e21/cm2 , hz/kb , 1e21/cm2s/b  , 1e24/cm2 , hz/b  , 1e24/cm2s/mb , 1e27/cm2 , hz/mb , 1e27/cm2s/ub , 1e30/cm2 , hz/ub , 1e30/cm2s/nb , 1e33/cm2 , hz/nb , 1e33/cm2s/pb , 1e36/cm2 , hz/pb , 1e36/cm2s/fb , 1e39/cm2 , hz/fb , 1e39/cm2s/ab , 1e42/cm2 , hz/ab , 1e42/cm2s  --type LUMITYPE Show results from the selected luminometer. Current available choices are hfoc,hfet,bcm1f,bcm1fsi,bcm1futca,pltzero,pltslink,dt,pxl,ramses,radmon. This option can be skipped if --normtag is specified because every norm tag is bound to a specific luminometer. If option --normtag is present, then --type is ignored.Option --type is **required** if **--without-correction** switch is specified.  --without-correction  Show raw data taken by a specific luminometer. --type LUMITYPE must be provided if this switch is specified.  --datatag DATATAG A specific version of lumi and beam data. The initial datatag "online" is guaranteed to exist. Other data tags are the patches to its previous version. If not specified, the most recent datatag, i.e. the best version of data, is used.Trigger related data are summary copies of trigger databases, they are not versioned.  --precision PRECISION Define the luminosity value output format and precision. The format string can be "[0-9]f" or "[0-9]e". The integer number specifies the precision and "f" stands for fixed floating point format, "e" for the scientific notation. Without this option, the default luminosity value output is "9f". Choose the precision and format according to your need. Too high precision output slows down the program and bloat the output file on disk. In addition, the luminosity number output precision can be reduced by using -u UNIT or -n SCALEFACTOR options.   --hltpath HLTPATH**HLTPATH** is hlt path name or pattern. The string pattern uses the file system convention fnmatch. \*, ? ,\[seq], \[!seq] operators are recognised. In order not to confuse with the file system, please double quote the hltpath string if there are special char \*, ?, \[] \! Show nominal luminosity scaled down by HLT and L1 prescale factors also taking into account L1 bit masks. Nominal luminosity means that taken during CMS running. This scaling filter has effect on all lumi switches or without, e.g. --byls, --xing. The output format could be different from the default one: one or two fields are removed and replaced with the long HLT path string.Any of the following conditions cause the rejection of a lumisection in the selected path: hlt prescale=0, relavent l1 bits prescale=0 or relavent l1 bits masked.Each output unit is grouped by hlt path name meaning if the output is one per lumi section, then for each of them, all possible hltpaths are shown before changing to the next unit.    Since release >=3.3.0 Sum delivered, Sum recorded fields in the Summary section show the sum luminosity of all groups above. Their meaning depends on the user's context.<jupyter_code><div class="warn cell">
<p>
<strong>Note :</strong> The number of combinatorials of trigger information is huge. If no sufficiently strict --hltpath filter is specified, it is a performance hit on the database and the execution software. If you must use loose filters, switch to csv display style or file might help slightly. 
</p>
</div><jupyter_output><empty_output><jupyter_text>--ignore-mask switch Switch off the effect of L1 bit masks. This should be used only for debugging purpose. The masks should be considered for the result to be correct.  --without-checkjson switch Switch off cross-checking with the json selection.  --minBiasXsec MINBIASXSEC Specify minimum bias cross section (ub) to use when calculating the average pileup column for the --byls output. This option should be used together with --type or --normtag. The program does not accept this option if neither --type nor --normtag is found. **Note:** the average pileup per lumi section is recalculated only if both --byls --normtag or --byls --type are specified. In other cases, the result shown is what calculated online during data acquisition time and cannot be changed. Reference minbias cross section table:Single Beam E (Gev) | Accelerator mode | minBiasXsec (ub) ---  | --- | --- 6500 | pp | 80000  2510 | pp | 65000 6370 | pbpb | 7800000 3500 | pp | 715004000 | pp | 730005000 | ppb | 20610008160 | ppb | 21350002720 | xexe | 5650000  beamSubcommand to show beam information (timeinfo, single beam energy, beam intensities, ncolliding bunches) per lumi section.  --xing switchShow beam intensities per bunch crossing. The bxidx field of the output has the range of [1,3564]<jupyter_code><div class="warn cell">
<p>
<strong>Note :</strong> for better performance, it is strongly recommended to use -o OUTPUTFILE or --output-style csv for the --xing option.
</p>
</div><jupyter_output><empty_output><jupyter_text>trgSubcommand to show L1, HLT bit/path and prescale information per run/per lumi section. The hlt path shown by this subcommand is constraint to only those potentially relevant to luminosity. Paths containing substring such as "Calibration","Random","DQM","Physics","Activity" are ignored.Without any filter options the summary map of the hltconfig to run number is shown. Switches and filter options specific to this subcommand are explained below.  --pathinfo switchShow HLT path and its L1 seed expression as well as the logical relationship between the l1 bits in the expression. Complicated L1seed logics with no simple scaling relationship between L1 bits are excluded. Filter options --hltconfig , -r, --hltpath reduce the output size.   --prescale switchIf no --hltpath filter is specified, show prescale index change during the given run. If --hltpath filter is given, show the real HLT and L1 prescale values of the selected run on the selected hltpath(s). Note: if the selected path is not effective , e.g. prescale=0 or all its seed bits are masked or having prescale=0, no result will be shown. Only truely effective hltpath are shown. You can use the --ignore-mask option to relax the requirement on l1bit masks. But keeping in mind that for the correctness of the result L1 mask should not be ignored, it is the same as prescale value=0.  trg subcommand specific selection options Since L1 and HLT information are not associated with LHC fill numbers, this subcommand supports only one common time selection option -r RUN.<jupyter_code><div class="cmdlines cell">
   <p><strong>-r RUN :</strong> run number 
   </p>
   <p><strong>--hltconfig HLTCONFIG :</strong> numeric hltconfig id or string hltkey. 
   </p>
   <p><strong>--hltpath HLTPATH :</strong> hlt path name or pattern. The string pattern uses the file system convention fnmatch. \*, ? ,\[seq], \[!seq] operators are recognised. 
   </p>
   <p><strong>--ignore-mask :</strong> ignore the effect of masks on L1 bits on prescale values. This should be used only for debugging purpose. 
   </p>
</div>
<div class="warn cell">
<p>
<strong>Note :</strong> The number of combinatorials of trigger information is huge. If no sufficiently strict filters are specified, it is a performance hit on the database and the execution software. Therefore using strict filters are recommended. If you must use loose filters, switch to csv display style or file might help a bit. 
</p>
</div><jupyter_output><empty_output><jupyter_text>[Back to Top](top)  briltag Subcommand to manage norm and data tags. Tags are used for versioning.Calibration and correction factors can be applied offline and having different values at different time period. Norm tag describes the function and parameters applied to the raw data. Each correction has its validity in time( run number ).Sometimes the raw data need to be patched. Data tag is the version of the raw data. For the moment, there is only one version "online". And the tool is not fully developed for handling data tags.  common command options  Database connection parameters -c, -p are common. However, one should distinguish readonly and readwrite operations. listxxx commands are readonly therefore -c, -p option values work as normal. insertxxx commands involve database write operations, only BRIL operators can perform it using specific -c, -p parameters.<jupyter_code><div class="cmdlines cell">
   <p>
   <strong>-c CONNECTION :</strong> database service to connect to. Example choices ["offline","online","onlinew","dev"]. The default is "offline". 
   </p>
   <p>
   <strong>-p AUTHFILE : </strong> database authentication file. 
   </p>
</div><jupyter_output><empty_output><jupyter_text>listiovShow norm tag info of each luminometer. By default, the summary is shown. With **--name** it shows the detail of the selected tag.  Some predefined functions shown below. function | description | parammeters--- | --- | ---**poly1d** | polynomial | coefs: array of coefficients**inversepoly1d** | 1/polynomial | coefs: array of coefficients**afterglow** | afterglow correction by bx threshold | afterglowthresholds: array of (nbx threshold,factor)**poly1dWafterflow** | poly1d followed by afterglow correction | coefs, afterglowthresholds**inversepoly1dWafterglow** | product of inverse poly1d and afterglow | coefs, afterglowthresholds**poly2dlL** | polynomial2d c[i,j]\*x^i*y^j where x=per bunch lumi , y=totallumi | coefs: array of coefficients **poly2dlLWafterglow** | poly2dlL followed by afterglow correction | coefs, afterglowthresholds**afterglowWpoly2dlL** | afterglow correction followed by poly2dlL | coefs, afterglowthresholds **Note** :The correction function is on the **single bunch instantaneous luminosity (SBIL)**. Its effect on the luminosity integrated over all orbits is calculated and applied automatically by the software based on the assumption that the bunches are equally populated.   insertiovTo insert normtag data into database. Data to insert must be defined in an input file of yaml format and given to the -y option.<jupyter_code><div class="cmdlines cell">

   <p><b>-y YAMLFILE</b> : file defining calibration parameters and corresponding interval of validity. Example input yaml file shown below. Note: yaml format requires correct indentations with no tab char. You can use a <a href="http://codebeautify.org/yaml-to-json-xml-csv">yaml validator</a> to validate the  input data before loading to database.
   <pre>
        name: pltzerov1
        applyto: lumi
        datasource: pltzero
        istypedefault: 0
        comments: from mini vdm
        since:
            - 246437:
                func: poly1d
                payload: {'coefs': '0.,0.'}
                comments: 2015, egev 6500, PROTPHYS, uncalibrated
            - 246908:
                func: poly1d
                payload: {'coefs': '21.7524,0.'}
                comments: 2015, egev 6500, PROTPHYS, magnet off
            - 271001:
                func: poly2dlL
                payload: {'coefs': '[[0,0],[0.00132754,1.175e-12]]'}
                comments: with correlated correction of raw and tot_raw
   </pre>
</div><jupyter_output><empty_output><jupyter_text>listdataList all data tags  insertdataCreat a new data tag. lumi beam data are versioned, trigger data are not. The condition to open a new datatag is if there are old data under the most recent tag for the lumi sections to load (patch). Loading new data to the most recent tag does not require opening a new one. For example, if loading run=1,ls=1 to tag v1 would cause a duplicated id error, then you need to create a data tag v2.<jupyter_code><div class="cmdlines cell">

   <p><strong>--name TAGNAME</strong> : Name of the data tag. 
   </p>
   <p><strong>--comments COMMENTS</strong> : Comments on the tag.
   </p>
</div><jupyter_output><empty_output><jupyter_text>[Back to Top](top)  Appendix A : Trigger Rules Effective luminosity in a given HLT path is : total_luminosity/(HLT_prescale*L1_prescale).Rules on deducing L1 contribution to the total prescale factor : 1. Do not consider any masked L1 bit in the seed expression, unless --ignore-mask switch is specified.2. If the path is single seeded, take the seed bit's prescale value.3. If (A.1 OR A.2 OR A.3...) , ignore the triggers with prescale 0, take the minimum of the remaining prescale values.4. If (A AND B AND C...), take the product of the prescale values.5. If NOT, or a combination of AND/OR/NOT, reject this path.6. For HLT_L1seed logic with more than one logical operators, users are considered taking their own risk and should use own judgement.Note: above rules are effective since release 2.1.x.  Two suggested by not implemented rules:1. (NOT IMPLEMENTED) If (A.1 OR A.2 OR A.3...) AND (B.1 OR B.2 OR B.3...) AND ... , for each group, take the minimum of the non-zero prescal, or zero if they are all zero. Then the product of the prescales of each group. Note: not implemented means such path are rejected by brilcalc.2. (NOT IMPLEMENTED) In some cases, L1 masks and prescale should be ignored. Note: not implemented means users are expected know about such cases and use --ignore-mask by themselves. [Back to Top](top)<jupyter_code>from IPython.core.display import HTML
def css_styling():
    styles = open('./style/custom.css','r').read()
    return HTML(styles)
css_styling()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09088815789473684, 'fraction_numerical': 0.019111842105263156, 'mean_word_length': 4.033278145695364, 'pattern_counts': {'":': 3, '<': 254, '<?xml version=': 0, '>': 256, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 8}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '12664', 'license': 'MIT', 'n_tokens_mistral': 9116, 'n_tokens_neox': 8489, 'n_words': 3921, 'path': 'brilwsdoc.ipynb', 'repo_name': 'xiezhen/brilwsdoc', 'size': '30400'}