<jupyter_start><jupyter_text>DREAM-toolsInstall or upgrade dream-tools:<jupyter_code>!pip install dream-tools --upgrade<jupyter_output>Requirement already up-to-date: dream-tools in c:\users\b031441\appdata\local\continuum\miniconda3\lib\site-packages (0.3.5)
Requirement already satisfied, skipping upgrade: easygui in c:\users\b031441\appdata\local\continuum\miniconda3\lib\site-packages (from dream-tools) (0.98.1)
Requirement already satisfied, skipping upgrade: pandas in c:\users\b031441\appdata\local\continuum\miniconda3\lib\site-packages (from dream-tools) (1.1.0)
Requirement already satisfied, skipping upgrade: pytz>=2017.2 in c:\users\b031441\appdata\local\continuum\miniconda3\lib\site-packages (from pandas->dream-tools) (2020.1)
Requirement already satisfied, skipping upgrade: python-dateutil>=2.7.3 in c:\users\b031441\appdata\local\continuum\miniconda3\lib\site-packages (from pandas->dream-tools) (2.8.1)
Requirement already satisfied, skipping upgrade: numpy>=1.15.4 in c:\users\b031441\appdata\local\continuum\miniconda3\lib\site-packages (from pandas->dream-tools) (1.19.1)
Requirement already satisfied, skippin[...]<jupyter_text>Our convention is to import the *dreamtools* package under the alias *dt*.<jupyter_code>import dreamtools as dt<jupyter_output><empty_output><jupyter_text>Gams PandasDREAM-tools provides a wrapper class *GamsPandasDatabase* that enables convertion between Pandas data structures and a Gams Database.We can either open an existing GDX file as a GamsPandasDatabase or create new database from scratch.  Creating and populating a new GamsPandasDatabaseTo create an empty database we use the GamsPandasDatabase class.The GamsPandasDatabase has 3 main methods for creating new Gams symbols: *create_parameter*, *create_variable*, and *create_set*. Each of these methods adds a Gams symbol to the underlying Gams Database and returns a Pandas object (or a scalar) representing the symbol.To improve readability, we can assign some shorter names (In SnakeCase to indicate that these methods essentially work as classes):<jupyter_code>db = dt.GamsPandasDatabase()
Par, Var, Set = db.create_parameter, db.create_variable, db.create_set<jupyter_output><empty_output><jupyter_text>Define sets<jupyter_code>t = Set("t", range(2010, 2020), "Årstal")
s = Set("s", ["tjenester", "fremstilling"], "Brancher")

st = Set("st", [s, t], "Branche x år dummy")
sub = Set("sub", ["tjenester"], "Subset af brancher", domains=["s"])

one2one = Set("one2one", [(2010, 2015), (2011, 2016)], "1 til 1 mapping", domains=["t", "t"])
one2many = Set("one2many",
               [("tot", "tjenester"), ("tot", "fremstilling")],
               "1 til mange mapping", domains=["*", "s"],
               )<jupyter_output><empty_output><jupyter_text>Define new parameters and variables<jupyter_code>gq = Par("gq", None, "Produktivitets-vækst", 0.01)
fq = Par("fp", t, "Vækstkorrektionsfaktor", (1 + 0.01) ** (t - 2010))
d = Par("d", st, "Dummy")
y = Var("y", [s, t], "Produktion")
p = Var("p", [s, t], "Pris")<jupyter_output><empty_output><jupyter_text>Python representation of Gams symbolsA set is represented by a Pandas Index or a Pandas MultIndex if it has more than one dimension.Scalars, such as *gq*, are simply Floats or Ints, while all other variable or parameters are accessed as Pandas Series:<jupyter_code>print(type(s))
print(type(one2many))

print(type(gq))
print(type(p))<jupyter_output><class 'pandas.core.indexes.base.Index'>
<class 'pandas.core.indexes.multi.MultiIndex'>
<class 'float'>
<class 'pandas.core.series.Series'><jupyter_text>As they are regular Pandas Series, all the usual methods can be used (and googled). E.g. use reset_index to get a DataFrame from a multiindex Series:<jupyter_code>p.reset_index().head()<jupyter_output><empty_output><jupyter_text>Assigning to existing symbolsTo change the values of the underlying Gams symbols, start by assigning values to their Pandas Series representations:<jupyter_code>y["tjenester"], y["fremstilling"] = 7, 3
p[:] = 100<jupyter_output><empty_output><jupyter_text>DREAM-tools defaults to using Pandas Series with multi-indexes rather than DataFrames.That is because using multi-indexes greatly simplifies operations when our variables have multiple dimensions with consistent names. For example, Pandas automatically aligns the *y* and *fq* variables on their shared *t* axis without us having to do any complicated merging operations:<jupyter_code>y *= fq
p * y<jupyter_output><empty_output><jupyter_text>Checking if a symbol name already exists in the database is as simple as<jupyter_code>print("y" in db)
print("free_lunch" in db)<jupyter_output>True
False<jupyter_text>To figure out which type of symbol it is, we can either check which dictionary it belongs to or access the underlying Gams Symbol through the *.symbols* dictionary.<jupyter_code>print("y" in db.sets)
print("y" in db.variables)
print("y" in db.parameters)

print(type(db.symbols["y"]))<jupyter_output>False
True
False
<class 'gams.database.GamsVariable'><jupyter_text>Usually we only want to access the Pandas representation of a symbol. We can access those using either **\[  \]** or **.** notation.<jupyter_code>print(type(db["y"]))
print(db.y is db["y"])<jupyter_output><class 'pandas.core.series.Series'>
True<jupyter_text>Use **\[ \]** notation to assign a new value to an existing scalar<jupyter_code>db["gq"] = 0.02<jupyter_output><empty_output><jupyter_text>Exporting the database to a gdx file<jupyter_code>db.export("test.gdx")<jupyter_output><empty_output><jupyter_text>Working with an existing gdx fileIf we already have a gdx file, such as the one we just exported, we can read that instead of creating a GamsPandasDatabase from scratch using the Gdx class:<jupyter_code>gdx = dt.Gdx("test.gdx")
isinstance(gdx, dt.GamsPandasDatabase)

print(gdx.gq)<jupyter_output>0.02<jupyter_text>Time series analysis If the *Plotly* package is installed, DREAM-tools sets the plotting backend of Pandas to Plotly. That means Plotly is used by default when plotting or printing a Pandas objects instead of Matplotlib. Neither plotting packages handles multi-indexes well. To plot our *y* variable from befroe we could write:<jupyter_code>y.unstack("s").plot()<jupyter_output><empty_output><jupyter_text>We could also use the DREAM-tools plot function:<jupyter_code>from dreamtools import plot
plot(y)<jupyter_output><empty_output><jupyter_text>For more advanced examples, we open a gdx file as a GamsPandasDatabase and set it as the "reference database".The reference database should be our baseline scenario for any analysis.<jupyter_code>dt.REFERENCE_DATABASE = dt.Gdx("Q:/MAKRO/Data/PreviousSolutions/previous_dynamic_calibration")<jupyter_output><empty_output><jupyter_text>We read another gdx file and assign it to a short name for easy access.<jupyter_code>s = dt.Gdx("Q:/MAKRO/Data/PreviousSolutions/previous_dynamic_calibration2")<jupyter_output><empty_output><jupyter_text>Let's try plotting a few national income variables:<jupyter_code>plot([s.vX, s.vC, s.vG, s.vI])<jupyter_output><empty_output><jupyter_text>Note that only the aggregate elements are plotted despite each variable being multidimensional.That is because *dreamtools* is setup with a dictionary defining which elements should be shown for a number of sets if we haven't specified anything.<jupyter_code>dt.DEFAULT_SET_AGGREGATIONS<jupyter_output><empty_output><jupyter_text>If we want other components we can simply be more specific (or supply a different *default_set_aggregations* dictionary):<jupyter_code>plot(s.vC[s.c])<jupyter_output><empty_output><jupyter_text>To compare variables with our baseline (the reference database) instead of writing```s.vX / baseline.vX - 1, s.vX / baseline.vX - 1 ...```we can use the operator "q" (as in Gekko):<jupyter_code>plot([s.vX, s.vC, s.vG, s.vI], "q", 2015, 2025)<jupyter_output><empty_output><jupyter_text>The *prt* function works identically, except instead of returning a plotly figure for plotting, it returns a string for printing.We can set a global time period of interest using the *time* function.3 operators are currently implemented:- "s": show baseline side-by-side (shock, baseline)- "q": relative multiplier (shock / baseline - 1)- "m": absolute multiplyer (shock - baseline)<jupyter_code>from dreamtools import prt
dt.time(2017, 2025)
prt(s.vX, "s")<jupyter_output><empty_output><jupyter_text>Here is an example with a few more useful arguments:<jupyter_code>plot([s.qY, s.qK],
        operator="m",
        start_year=2016,
        end_year=2080,
        reference_database=None,
        default_set_aggregations={"s_": ["tje"], "s": ["tje"]},
        layout={"title": "Example graph", "yaxis": {"title": "Difference from baseline"}},  # See plotly documentation for other layout options
)<jupyter_output><empty_output><jupyter_text>Age profiles2 functions have been added specifically to plot age profiles more easily:<jupyter_code>dt.age_figure_2d(s.vC_a, years=[2017, 2090])
dt.age_figure_3d(s.vC_a, end_age=99, end_year=2099)<jupyter_output><empty_output><jupyter_text>Exporting plotsThere are many ways to export the plots, but one easy way to store and share them is to export an html file:<jupyter_code>figures = [
    plot(s.vC),
    plot(s.vG)
]
dt.figures_to_html(figures, "example.html")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11274247133211875, 'fraction_numerical': 0.018969027971278533, 'mean_word_length': 6.31921568627451, 'pattern_counts': {'":': 8, '<': 109, '<?xml version=': 0, '>': 115, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '27', 'dir': 'jupyter-structured-clean-dedup', 'id': '536161', 'license': 'MIT', 'n_tokens_mistral': 3195, 'n_tokens_neox': 3036, 'n_words': 904, 'path': 'dreamtools/examples.ipynb', 'repo_name': 'MartinBonde/dream-tools', 'size': '9331'}