<jupyter_start><jupyter_text>Example 2 - Using the IEA plugin Futura is written to accept plugins (using the entry point `futura_plugin`). These can be written and installed separately to supplement and extend the features of Futura.The first of these is the IEA plugin which automatically downloads the latest monthly OECD electricity generation statistics from the IEA and updates the high voltage grid mixes of the relevant countries\* accordingly. To run this example you'll need to install the IEA plugin. At the moment the best way to do this is using:`pip install https://github.com/pjamesjoyce/futura_iea/archive/master.zip`(It will get it's own PyPi and Conda release in due course)In this example, we'll load ecoinvent 3.6 into a FuturaLoader, then use the IEA plugin to alter the grid mixes.\*Note: The USA and Canada are excluded currently due to complications arising from sub-grids in ecoinvent  Setting upFirst we import all of the relevant things we need from Futura - note the plugin loads automatically when we import futura<jupyter_code>import futura
from futura.loader import FuturaLoader
from futura.utils import create_filter_from_description
from futura import w
from futura.markets import FuturaMarket<jupyter_output><empty_output><jupyter_text>We create a FuturaLoader, give it a simple recipe and run it. Note: as long as the `ecoinvent_version` and `ecoinvent_system_model` are set, you don't need to have an existing brightway project/database set up - you'll be prompted for your login details and it will create it for you<jupyter_code>f = FuturaLoader()
f.recipe = {
    'metadata': {'base_project': 'Futura_SETUP',
                 'base_database': 'ecoinvent_cut-off36',
                 'output_database': 'ecoinvent_cut-off36_updated',
                 'ecoinvent_version': 3.6,
                 'ecoinvent_system_model': 'cut-off',
                 'description': 'blank version of ecoinvent 3.6'},
    'actions': [{'action': 'load',
                 'tasks': [{'function': 'extract_bw2_database',
                            'kwargs': {'project_name': 'Futura_SETUP',
                                       'database_name': 'ecoinvent_cut-off36'}}]
                }]
}
f.run()<jupyter_output>load started
extract_bw2_database(project_name=Futura_SETUP, database_name=ecoinvent_cut-off36)
Getting activity data<jupyter_text>Seeing where we start fromYou can choose to update all of the locations at once, or specify a subset to use. Here we'll just do the UK and Germany.<jupyter_code>grid_locations = ['DE', 'GB']<jupyter_output><empty_output><jupyter_text>We'll write a filter to get our high voltage electricity markets, convert it into a wurst filter, and run it against the database in the FuturaLoader<jupyter_code>elec_filter_base = [
            {'filter': 'equals', 'args': ['unit', 'kilowatt hour']},
            {'filter': 'startswith', 'args': ['name', 'market for electricity, high voltage']},
            {'filter': 'doesnt_contain_any', 'args': ['name', ['Swiss Federal Railways', 'label-certified']]},
            {'filter': 'either', 'args':
                [{'filter': 'equals', 'args': ['location', x]} for x in grid_locations]
             }
        ]
elec_filter = create_filter_from_description(elec_filter_base)
elec_list = list(w.get_many(f.database.db, *elec_filter))

assert [x['location'] for x in elec_list] == grid_locations<jupyter_output><empty_output><jupyter_text>Now let's see what the current status of the grid mixes are using a FuturaMarketFirst the UK:<jupyter_code>gb = elec_list[1]
gb_fm = FuturaMarket(gb, f.database)
gb_fm.plot<jupyter_output><empty_output><jupyter_text>That's a lot more coal than you'd expect!Next up, Germany:<jupyter_code>de = elec_list[0]
de_fm = FuturaMarket(de, f.database)
de_fm.plot<jupyter_output><empty_output><jupyter_text>Loading the IEA ImporterOK - let's load up the IEA Importer from the plugin. We give it our FuturaLoader so it can make the edits to the grids directly.We'll get some warnings (from `country_converter`) about country codes not being found - this is to do with making sure the regions map correctly onto their ecoinvent counterparts and is nothing to worry about.<jupyter_code>iea_converter = futura.iea.IEA_Importer(f)<jupyter_output>root        : WARNING  OECDTOT not found in regex
root        : WARNING  OECDAM not found in regex
root        : WARNING  OECDAO not found in regex
root        : WARNING  OECDEUR not found in regex
root        : WARNING  IEATOT not found in regex
root        : WARNING  OECDTOT not found in regex
root        : WARNING  OECDTOT not found in regex
root        : WARNING  OECDAM not found in regex
root        : WARNING  OECDAM not found in regex
root        : WARNING  OECDAO not found in regex
root        : WARNING  OECDAO not found in regex
root        : WARNING  OECDEUR not found in regex
root        : WARNING  OECDEUR not found in regex
root        : WARNING  IEATOT not found in regex
root        : WARNING  IEATOT not found in regex
root        : WARNING  AUSTRALI not found in regex
root        : WARNING  NETHLAND not found in regex
root        : WARNING  SWITLAND not found in regex
root        : WARNING  UK not found in ISO2<jupyter_text>Now all we need to do is run the `update_grid` function. We can either pass it a list of countries, or we can run it with no arguments to update all of the available grids.<jupyter_code>iea_converter.update_grid(grid_locations)<jupyter_output>Updated grid mix for Germany
Updated grid mix for United Kingdom<jupyter_text>Let's see what we've doneOk - let's rerun our filter to grab the updated versions of the electricity markets from our FuturaLoader and see what they look like now<jupyter_code>elec_list2 = list(w.get_many(f.database.db, *elec_filter))<jupyter_output><empty_output><jupyter_text>First up, let's take a look at the IEA data for the UK - this is aggregated at a higher level than the ecoinvent data<jupyter_code>iea_converter.aggregated_data.loc['GB'].plot(kind='pie', y='Latest Year Total', legend=None)<jupyter_output><empty_output><jupyter_text>When the IEA Importer does it's thing, it groups the available technologies in the existing market according to the classifications above. Then it reassigns the technologies to the new sections based on their percentage contribution to that technology group - this keeps the proportions of things like imports, and generation technologies in the same ratio, but with a different overall contribution Here's what the UK looks like now:<jupyter_code>gb2 = elec_list2[1]
gb_fm2 = FuturaMarket(gb2, f.database)
gb_fm2.plot<jupyter_output><empty_output><jupyter_text>That looks a lot better - Coal down to around 4%, mainly gas, some nuclear, a high proportion of wind - far more accurate.Let's do the same for Germany, first here's what the IEA data says:<jupyter_code>iea_converter.aggregated_data.loc['DE'].plot(kind='pie', y='Latest Year Total', legend=None)<jupyter_output><empty_output><jupyter_text>And here is the new market mix:<jupyter_code>de2 = elec_list[0]
de_fm2 = FuturaMarket(de2, f.database)
de_fm2.plot<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0766100743650905, 'fraction_numerical': 0.004630279219868107, 'mean_word_length': 4.237325495958854, 'pattern_counts': {'":': 0, '<': 50, '<?xml version=': 0, '>': 50, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '485870', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 2177, 'n_tokens_neox': 2097, 'n_words': 803, 'path': 'examples/Example 2 - IEA plugin.ipynb', 'repo_name': 'pjamesjoyce/futura', 'size': '7127'}