<jupyter_start><jupyter_text>High level interface Prerequisites:- Understanding the gammapy data workflow, in particular what are DL3 events and intrument response functions (IRF). ContextThis notebook is an introduction to gammapy analysis using the high level interface. Gammapy analysis consists in two main steps. The first one is data reduction: user selected observations  are reduced to a geometry defined by the user. It can be 1D (spectrum from a given extraction region) or 3D (with a sky projection and an energy axis). The resulting reduced data and instrument response functions (IRF) are called datasets in Gammapy.The second step consists in setting a physical model on the datasets and fitting it to obtain relevant physical informations.**Objective: Create a 3D dataset of the Crab using the H.E.S.S. DL3 data release 1 and perform a simple model fitting of the Crab nebula.** Proposed approach:This notebook uses the high level `Analysis` class to orchestrate data reduction. In its current state, `Analysis` supports the standard analysis cases of joint or stacked 3D and 1D analyses. It is instantiated with an `AnalysisConfig` object that gives access to analysis parameters either directly or via a YAML config file. To see what is happening under-the-hood and to get an idea of the internal API, a second notebook performs the same analysis without using the `Analysis` class. In summary, we have to:- Create an `~gammapy.analysis.AnalysisConfig` object and edit it to define the analysis configuration:    - Define what observations to use    - Define the geometry of the dataset (data and IRFs)    - Define the model we want to fit on the dataset.- Instantiate a `~gammapy.analysis.Analysis` from this configuration and run the different analysis steps    - Observation selection    - Data reduction    - Model fitting    - Estimating flux pointsFinally we will compare the results against a reference model.  Setup<jupyter_code>%matplotlib inline
import matplotlib.pyplot as plt
from pathlib import Path
from astropy import units as u
from gammapy.analysis import Analysis, AnalysisConfig
from gammapy.modeling.models import create_crab_spectral_model<jupyter_output><empty_output><jupyter_text>Analysis configurationFor configuration of the analysis we use the [YAML](https://en.wikipedia.org/wiki/YAML) data format. YAML is a machine readable serialisation format, that is also friendly for humans to read. In this tutorial we will write the configuration file just using Python strings, but of course the file can be created and modified with any text editor of your choice.Here is what the configuration for our analysis looks like:<jupyter_code>config = AnalysisConfig()
# the AnalysisConfig gives access to the various parameters used from logging to reduced dataset geometries
print(config)<jupyter_output><empty_output><jupyter_text>Setting the data to use We want to use Crab runs from the H.E.S.S. DL3-DR1. We define here the datastore and a cone search of observations pointing with 5 degrees of the Crab nebula. Parameters can be set directly or as a python dict.PS: do not forget to setup your environment variable _$GAMMAPY\_DATA_ to your local directory containing the H.E.S.S. DL3-DR1 (see [here](../../getting-started/quickstart.rstdownload-tutorials)).<jupyter_code># We define the datastore containing the data
config.observations.datastore = "$GAMMAPY_DATA/hess-dl3-dr1"

# We define the cone search parameters
config.observations.obs_cone.frame = "icrs"
config.observations.obs_cone.lon = "83.633 deg"
config.observations.obs_cone.lat = "22.014 deg"
config.observations.obs_cone.radius = "5 deg"

# Equivalently we could have set parameters with a python dict
# config.observations.obs_cone = {"frame": "icrs", "lon": "83.633 deg", "lat": "22.014 deg", "radius": "5 deg"}<jupyter_output><empty_output><jupyter_text>Setting the reduced datasets geometry<jupyter_code># We want to perform a 3D analysis
config.datasets.type = "3d"
# We want to stack the data into a single reduced dataset
config.datasets.stack = True

# We fix the WCS geometry of the datasets
config.datasets.geom.wcs.skydir = {
    "lon": "83.633 deg",
    "lat": "22.014 deg",
    "frame": "icrs",
}
config.datasets.geom.wcs.width = {"width": "2 deg", "height": "2 deg"}
config.datasets.geom.wcs.binsize = "0.02 deg"

# We now fix the energy axis for the counts map
config.datasets.geom.axes.energy.min = "1 TeV"
config.datasets.geom.axes.energy.max = "10 TeV"
config.datasets.geom.axes.energy.nbins = 4

# We now fix the energy axis for the IRF maps (exposure, etc)
config.datasets.geom.axes.energy_true.min = "0.5 TeV"
config.datasets.geom.axes.energy_true.max = "20 TeV"
config.datasets.geom.axes.energy.nbins = 10<jupyter_output><empty_output><jupyter_text>Setting the background normalization maker<jupyter_code>config.datasets.background.method = "fov_background"
config.datasets.background.parameters = {"method": "scale"}<jupyter_output><empty_output><jupyter_text>Setting the exclusion mask In order to properly adjust the background normalisation on regions without gamma-ray signal, one needs to define an exclusion mask for the background normalisation.For this tutorial, we use the following one ``$GAMMAPY_DATA/joint-crab/exclusion/exclusion_mask_crab.fits.gz``<jupyter_code>config.datasets.background.exclusion = (
    "$GAMMAPY_DATA/joint-crab/exclusion/exclusion_mask_crab.fits.gz"
)<jupyter_output><empty_output><jupyter_text>Setting modeling and fitting parameters`Analysis` can perform a few modeling and fitting tasks besides data reduction. Parameters have then to be passed to the configuration object.Here we define the energy range on which to perform the fit. We also set the energy edges used for flux point computation as well as the correlation radius to compute excess and significance maps.<jupyter_code>config.fit.fit_range.min = 1 * u.TeV
config.fit.fit_range.max = 10 * u.TeV
config.flux_points.energy = {"min": "1 TeV", "max": "10 TeV", "nbins": 3}
config.excess_map.correlation_radius = 0.1 * u.deg<jupyter_output><empty_output><jupyter_text>We're all set. But before we go on let's see how to save or import `AnalysisConfig` objects though YAML files.  Using YAML configuration filesOne can export/import the `AnalysisConfig` to/from a YAML file.<jupyter_code>config.write("config.yaml", overwrite=True)
config = AnalysisConfig.read("config.yaml")
print(config)<jupyter_output><empty_output><jupyter_text>Running the analysisWe first create an `~gammapy.analysis.Analysis` object from our configuration.<jupyter_code>analysis = Analysis(config)<jupyter_output><empty_output><jupyter_text>Observation selectionWe can directly select and load the observations from disk using `~gammapy.analysis.Analysis.get_observations()`:<jupyter_code>analysis.get_observations()<jupyter_output><empty_output><jupyter_text>The observations are now available on the `Analysis` object. The selection corresponds to the following ids:<jupyter_code>analysis.observations.ids<jupyter_output><empty_output><jupyter_text>To see how to explore observations, please refer to the following notebook: [CTA with Gammapy](../data/cta.ipynb) or  [HESS with Gammapy](../data/hess.ipynb)   Data reductionNow we proceed to the data reduction. In the config file we have chosen a WCS map geometry, energy axis and decided to stack the maps. We can run the reduction using `.get_datasets()`:<jupyter_code>%%time
analysis.get_datasets()<jupyter_output><empty_output><jupyter_text>As we have chosen to stack the data, there is finally one dataset contained which we can print:<jupyter_code>print(analysis.datasets["stacked"])<jupyter_output><empty_output><jupyter_text>As you can see the dataset comes with a predefined background model out of the data reduction, but no source model has been set yet.The counts, exposure and background model maps are directly available on the dataset and can be printed and plotted:<jupyter_code>counts = analysis.datasets["stacked"].counts
counts.smooth("0.05 deg").plot_interactive()<jupyter_output><empty_output><jupyter_text>We can also compute the map of the sqrt_ts (significance) of the excess counts above the background. The correlation radius to sum counts is defined in the config file.<jupyter_code>analysis.get_excess_map()
analysis.excess_map["sqrt_ts"].plot(add_cbar=True);<jupyter_output><empty_output><jupyter_text>Save dataset to diskIt is common to run the preparation step independent of the likelihood fit, because often the preparation of maps, PSF and energy dispersion is slow if you have a lot of data. We first create a folder:<jupyter_code>path = Path("analysis_1")
path.mkdir(exist_ok=True)<jupyter_output><empty_output><jupyter_text>And then write the maps and IRFs to disk by calling the dedicated `write()` method:<jupyter_code>filename = path / "crab-stacked-dataset.fits.gz"
analysis.datasets[0].write(filename, overwrite=True)<jupyter_output><empty_output><jupyter_text>Model fittingNow we define a model to be fitted to the dataset. Here we use its YAML definition to load it:<jupyter_code>model_config = """
components:
- name: crab
  type: SkyModel
  spatial:
    type: PointSpatialModel
    frame: icrs
    parameters:
    - name: lon_0
      value: 83.63
      unit: deg
    - name: lat_0 
      value: 22.014    
      unit: deg
  spectral:
    type: PowerLawSpectralModel
    parameters:
    - name: amplitude      
      value: 1.0e-12
      unit: cm-2 s-1 TeV-1
    - name: index
      value: 2.0
      unit: ''
    - name: reference
      value: 1.0
      unit: TeV
      frozen: true
"""<jupyter_output><empty_output><jupyter_text>Now we set the model on the analysis object:<jupyter_code>analysis.set_models(model_config)<jupyter_output><empty_output><jupyter_text>Finally we run the fit:<jupyter_code>analysis.run_fit()
print(analysis.fit_result)<jupyter_output><empty_output><jupyter_text>This is how we can write the model back to file again:<jupyter_code>filename = path / "model-best-fit.yaml"
analysis.models.write(filename, overwrite=True)
!cat analysis_1/model-best-fit.yaml<jupyter_output><empty_output><jupyter_text>Flux points<jupyter_code>analysis.config.flux_points.source = "crab"
analysis.get_flux_points()
ax_sed, ax_residuals = analysis.flux_points.plot_fit()<jupyter_output><empty_output><jupyter_text>The flux points can be exported to a fits table following the format defined [here](https://gamma-astro-data-formats.readthedocs.io/en/latest/spectra/flux_points/index.html)<jupyter_code>filename = path / "flux-points.fits"
analysis.flux_points.write(filename, overwrite=True)<jupyter_output><empty_output><jupyter_text>To check the fit is correct, we compute the map of the sqrt_ts of the excess counts above the current model.<jupyter_code>analysis.get_excess_map()
analysis.excess_map["sqrt_ts"].plot(add_cbar=True);<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0840312931431201, 'fraction_numerical': 0.009295904279797516, 'mean_word_length': 5.510485320551228, 'pattern_counts': {'":': 13, '<': 97, '<?xml version=': 0, '>': 97, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '24', 'dir': 'jupyter-structured-clean-dedup', 'id': '84793', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 3330, 'n_tokens_neox': 3189, 'n_words': 1134, 'path': 'docs/tutorials/starting/analysis_1.ipynb', 'repo_name': 'AtreyeeS/gammapy', 'size': '10865'}