Model pipeline¶
Data processing in oemof-flexmex is divided into 4 main steps:
preprocessing
inferring
optimization
postprocessing
The workflow is managed using the workflow management tool snakemake. Each of the 4 steps is represented by a snakemake rule in the Snakefile, which runs the script of the same name.
The data each step is provided with is held in different forms:
raw data
preprocessed data
optimization results
postprocessed results
Raw data¶
The raw data holds the energy system model definition for all scenarios. It consists of a parameter database (parameters are called scalars) and a bunch of timeseries (sequences or profiles).
The data is expected to be CSV-formatted and is read from data/In
.
The format of timeseries and scalars is described below.
Note
Raw data for FlexMex is not part of the oemof-flexmex github repository but can be provided by the FlexMex project partners.
Scalars¶
The scalars database defines parameters for all scenarios and regions.
It is held in a CSV file called Scalars.csv
.
The following table shows the first lines of an example Scalars.csv
and its format:
Scenario |
Region |
Year |
Parameter |
Unit |
Value |
---|---|---|---|---|---|
FlexMex1 |
ALL |
ALL |
Energy_SlackCost_Electricity |
Eur/GWh |
50000000 |
FlexMex1 |
ALL |
ALL |
Energy_SlackCost_Electricity |
Eur/GWh |
5000000 |
ALL |
ALL |
ALL |
Energy_SlackCost_Heat |
Eur/GWh |
1000000 |
ALL |
ALL |
ALL |
Energy_SlackCost_H2 |
Eur/GWh |
1000000 |
FlexMex1 |
AT |
2050 |
DemandResponse_Capacity_Electricity_Cooling |
MW (el) |
248 |
FlexMex1 |
BE |
2050 |
DemandResponse_Capacity_Electricity_Cooling |
MW (el) |
584 |
FlexMex1 |
CH |
2050 |
DemandResponse_Capacity_Electricity_Cooling |
MW (el) |
183 |
FlexMex1 |
CZ |
2050 |
DemandResponse_Capacity_Electricity_Cooling |
MW (el) |
275 |
FlexMex1 |
DE |
2050 |
DemandResponse_Capacity_Electricity_Cooling |
MW (el) |
2950 |
FlexMex1 |
DK |
2050 |
DemandResponse_Capacity_Electricity_Cooling |
MW (el) |
391 |
FlexMex1 |
FR |
2050 |
DemandResponse_Capacity_Electricity_Cooling |
MW (el) |
3361 |
FlexMex1 |
IT |
2050 |
DemandResponse_Capacity_Electricity_Cooling |
MW (el) |
2304 |
FlexMex1 |
LU |
2050 |
DemandResponse_Capacity_Electricity_Cooling |
MW (el) |
58 |
FlexMex1 |
NL |
2050 |
DemandResponse_Capacity_Electricity_Cooling |
MW (el) |
947 |
FlexMex1 |
PL |
2050 |
DemandResponse_Capacity_Electricity_Cooling |
MW (el) |
1497 |
FlexMex1 |
AT |
2050 |
DemandResponse_Capacity_Electricity_HVAC |
MW (el) |
964 |
FlexMex1 |
BE |
2050 |
DemandResponse_Capacity_Electricity_HVAC |
MW (el) |
2144 |
FlexMex1 |
CH |
2050 |
DemandResponse_Capacity_Electricity_HVAC |
MW (el) |
1027 |
FlexMex1 |
CZ |
2050 |
DemandResponse_Capacity_Electricity_HVAC |
MW (el) |
1117 |
- Scenario: string
Special identifier to address scenario or group of scenarios (‘experiment’)
Note
The keyword
ALL
can be used as a universal quantifier to avoid repetition.- Region: string
Region identifier
Note
The keyword
ALL
can be used as a universal quantifier to avoid repetition.- Year: integer
Year
Warning
Years support is not fully implemented!
- Parameter: string
The parameter name used in the FlexMex project. This is mapped via preprocessing to the components parameters.
- Unit: string
Unit of measurement of the given value
Warning
Unit support is incomplete! Especially, there is no check for unit equivalence nor any automatic unit conversion!
- Value: float
Value of the parameter
Timeseries¶
Timeseries in oemof-flexmex assign a value to every hour of the year (1…8760). They are held in CSV files with one time index-value pair per line and one timeseries per file.
Warning
The time index is ignored at the moment. It will be overwritten by a pandas
datetimeindex
.
The paths to the timeseries are defined in flexmex_config/mapping-input-timeseries.yml
per component.
If a component has no timeseries defined here, an info line is added to the log output.
The found filenames are interpreted according to the following pattern:
{experiment name}_{region code}_{year}.csv
Note
Experiment name
and year
are ignored at the moment.
The following table shows the first lines of an exemplary time series csv file for heat demand in Austria,
which is stored as data/In/Energy/FinalEnergy/Heat/FlexMex1_AT_2050.csv
.
timeindex |
load |
---|---|
1 |
0.000213222 |
2 |
0.000214263 |
3 |
0.0002161 |
4 |
0.000221314 |
5 |
0.000228666 |
And here is the corresponding entry in mapping-input-timeseries.yml
:
heat-demand:
profiles:
heat-demand:
input-path: Energy/FinalEnergy/Heat
Preprocessing¶
Preprocessing brings the raw data into the oemof.tabular format.
In this step, scalars belonging to a component are mapped to the components model parameters and saved within an input CSV file.
Timeseries are attached in a similar way.
The so formed input data is held in a datapackage
format comprising a JSON schema file (meta data) and the CSV files containing the actual data.
Inferring¶
Optimization¶
Optimization is performed by oemof-solph. Specifically, with the help of oemof.tabular, an EnergySystem
is created from the data package
created in preprocessing.
Postprocessing¶
Postprocessing translates the results into an exchange-friendly format defined by the FlexMex project partners. For that, a result template defines the output parameters for each scenario. The oemof-flexmex-internal parameters are recalculated and mapped to the FlexMex parameter names.
The results template is provided by the FlexMex project partners. It consists of an output directory structure and a scaffold Scalars.csv output file (with no values). It should be placed in the path:
flexmex_config/output_template/
The mapping is read from the two CSV files:
flexmex_config/mapping-output-scalars.csv
flexmex_config/mapping-output-timeseries.yml