<jupyter_start><jupyter_text>MadMiner particle physics tutorial Appendix 3: reweighting existing MC samples<NAME>, <NAME>, <NAME>, and <NAME> 2018-2019 What if you already have a bunch of generated events, either simulated with MadMiner with a different benchmark setup, or with stand-alone MadGraph? MadMiner now lets you add the missing event weights to these files.  0. Preparations<jupyter_code>from __future__ import absolute_import, division, print_function, unicode_literals

import logging
import numpy as np
import matplotlib
from matplotlib import pyplot as plt
%matplotlib inline

from madminer.core import MadMiner<jupyter_output><empty_output><jupyter_text>MadMiner uses the Python `logging` module to provide additional information and debugging output. You can choose how much of this output you want to see by switching the level in the following lines to `logging.DEBUG` or `logging.WARNING`.<jupyter_code># MadMiner output
logging.basicConfig(
    format='%(asctime)-5.5s %(name)-20.20s %(levelname)-7.7s %(message)s',
    datefmt='%H:%M',
    level=logging.INFO
)

# Output of all other modules (e.g. matplotlib)
for key in logging.Logger.manager.loggerDict:
    if "madminer" not in key:
        logging.getLogger(key).setLevel(logging.WARNING)<jupyter_output><empty_output><jupyter_text>1. Original event sample We'll start with the event sample from the main tutorial, and assume we forgot to add one benchmark we *really* care about before starting the event generation. Let's try to add this benchmark (and the corresponding weights) after generating the events!We will assume that you have generated events according to the following settings, please adapt them if necessary:<jupyter_code>mg_process_dir = "mg_processes/signal1"
run_name = "run_01"
sampling_benchmark = "sm"<jupyter_output><empty_output><jupyter_text>2. Load setup and add new benchmark<jupyter_code>miner = MadMiner()
miner.load("data/setup.h5")
miner.add_benchmark({'CWL2':10., 'CPWL2':8.}, 'oh_shit_we_forgot_this')
miner.save('data/setup_with_extra_benchmark.h5')<jupyter_output>07:15 madminer.core        INFO    Found 2 parameters:
07:15 madminer.core        INFO       CWL2 (LHA: dim6 2, maximal power in squared ME: (2,), range: (-20.0, 20.0))
07:15 madminer.core        INFO       CPWL2 (LHA: dim6 5, maximal power in squared ME: (2,), range: (-20.0, 20.0))
07:15 madminer.core        INFO    Found 6 benchmarks:
07:15 madminer.core        INFO       sm: CWL2 = 0.00e+00, CPWL2 = 0.00e+00
07:15 madminer.core        INFO       w: CWL2 = 15.20, CPWL2 = 0.10
07:15 madminer.core        INFO       neg_w: CWL2 = -1.54e+01, CPWL2 = 0.20
07:15 madminer.core        INFO       ww: CWL2 = 0.30, CPWL2 = 15.10
07:15 madminer.core        INFO       neg_ww: CWL2 = 0.40, CPWL2 = -1.53e+01
07:15 madminer.core        INFO       morphing_basis_vector_5: CWL2 = -1.68e+01, CPWL2 = -1.72e+01
07:15 madminer.core        INFO    Found morphing setup with 6 components
07:15 madminer.core        INFO    Did not find systematics setup.
07:15 madminer.core        INFO    Added benchmark oh_s[...]<jupyter_text>This doesn't change the morphing setup (which always uses the first benchmarks, i.e. the ones that were already defined in the MadMiner file).  3. Prepare reweighting Here's the crucial step. `reweight_benchmarks` is a list of all the benchmarks that the sample will be reweighted to -- this should *not* include the benchmarks for which the sample already contains weights.<jupyter_code>miner.reweight_existing_sample(
    mg_process_directory=mg_process_dir,
    run_name=run_name,
    param_card_template_file="Cards/param_card_template.dat",
    sample_benchmark=sampling_benchmark,
    reweight_benchmarks=["oh_shit_we_forgot_this"],
)<jupyter_output>07:15 madminer.core        INFO    Reweighting setup
07:15 madminer.core        INFO      Originally sampled from benchmark: sm
07:15 madminer.core        INFO      Now reweighting to benchmarks:     ['oh_shit_we_forgot_this']
07:15 madminer.core        INFO      Reweight card:                     /madminer/cards/reweight_card_reweight.dat
07:15 madminer.core        INFO      Log file:                          reweight.log
07:15 madminer.core        INFO    Creating param and reweight cards in mg_processes/signal1
07:15 madminer.utils.inter INFO    Starting reweighting of an existing sample in mg_processes/signal1
07:16 madminer.core        INFO    Finished running reweighting! Please check that events were succesfully reweighted in the following folder:

 mg_processes/signal1/Events/run_01
{'fraction_non_alphanumeric': 0.08130081300813008, 'fraction_numerical': 0.045264776972094044, 'mean_word_length': 3.9104638619201726, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 23, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '267195', 'license': 'MIT', 'n_tokens_mistral': 1556, 'n_tokens_neox': 1426, 'n_words': 466, 'path': 'examples/tutorial_particle_physics/A3_reweighting_existing_samples.ipynb', 'repo_name': 'cjs220/madminer', 'size': '4576'}