Statistics Functions API

`statistics_functions`

Statistics functions for PyPSA validation processing.

Each function in this module corresponds to one IAMC variable and extracts the relevant value from a given PyPSA Network. The functions are looked up by name via the mapping defined in configs/mapping.default.yaml.

For statistics-functions, the fixed input is n = pypsa.Network (one network / investment year) and aggregate_per_year: bool = True to switch between yearly aggregation and full snapshot time series.

If a variable-specific function needs additional settings, optional function parameters can be added. Currently, these include config and energy totals. The functions signature is identified automatically.

Each function returns a :class:pandas.Series with MultiIndex, holding at least the indexes location and unit (for aggregate_per_year = True) or a :class:pandas.DataFrame with MultiIndex and snapshots as columns (for aggregate_per_year = False).

Region Level: Regions in the returned Series correspond to the network's bus regions (e.g., "AT1", "AT2", "AT3" for pypsa-at). The postprocessing layer in Network_Processor handles aggregation to country level or keeps regions based on the aggregation_level configuration by the name of the entries of location. All regions starting with the given country-entry are grouped together.

`UNITS_MAPPING = {'MWh_el': 'MWh', 'MWh_LHV': 'MWh', 'MWh_th': 'MWh', 'land transport': 'MWh', 't_co2': 't', '': '', 'MWh': 'MWh'}` `module-attribute`

`kwargs = {'groupby': ['location', 'unit'], 'nice_names': False}` `module-attribute`

`kwargs_filtering = {'groupby': ['name', 'bus', 'carrier', 'location', 'unit'], 'nice_names': False}` `module-attribute`

`kwargs_imports = {'groupby': ['bus0', 'bus1', 'unit'], 'nice_names': False}` `module-attribute`

`Final_Energy_by_Carrier__Coal(n: pypsa.Network, aggregate_per_year: bool = True) -> pd.Series | pd.DataFrame | None`

Extracts Final Energy [by Carrier]|Coal from the PyPSA Network.

Parameters:

Name	Type	Description	Default
`n`	`Network`	PyPSA network to process.	required
`aggregate_per_year`	`bool`	If `True` (default), aggregate over all snapshots and return a :class:`pandas.Series`. If `False`, return a :class:`pandas.DataFrame` with snapshots as columns.	`True`

Returns:

Type	Description
`Series \| DataFrame \| None`	Pandas Series (`aggregate_per_year=True`) or DataFrame (`aggregate_per_year=False`) with MultiIndex of `location` and `unit`. Returns data at regional level as provided by the PyPSA network. Country-level aggregation is handled by Network_Processor._aggregate_to_country() if configured.

Notes

This function includes only the Final Energy consumption of coal. This excludes coal for electricity or CHP, but only focuses on the carrier coal for industry as Final Energy of Coal. Return None, if this carrier is not present in the network.

`Final_Energy_by_Carrier__District_Heat(n: pypsa.Network, aggregate_per_year: bool = True) -> pd.Series | pd.DataFrame`

Extract Final Energy of District Heat from PyPSA Network.

Returns the total final energy demand, district heating facilities have to provide district heating.

Parameters:

Name	Type	Description	Default
`n`	`Network`	PyPSA network to process.	required
`aggregate_per_year`	`bool`	If `True` (default), aggregate over all snapshots and return a :class:`pandas.Series`. If `False`, return a :class:`pandas.DataFrame` with snapshots as columns.	`True`

Returns:

Type	Description
`Series \| DataFrame`	Pandas Series (`aggregate_per_year=True`) or DataFrame (`aggregate_per_year=False`) with MultiIndex including `location` and `unit`. Returns data at regional level as provided by the PyPSA network. Country-level aggregation is handled by Network_Processor._aggregate_to_country() if configured.

Notes

The energy demand of district heating is calculated based on the links from other carriers buses to the urban central heat buses. This includes waste heat from eg. DAC. Taking the withdrawal from the source bus of the link gives the energy in elem of the source carrier.

`Final_Energy_by_Carrier__Electricity(n: pypsa.Network, aggregate_per_year: bool = True) -> pd.Series | pd.DataFrame`

Extract electricity-carrier final energy from a PyPSA Network.

Returns the total final energy demand supplied by electricity across agriculture, residential and commercial, transportation, and industry and DAC.

Parameters:

Name	Type	Description	Default
`n`	`Network`	PyPSA network to process.	required
`aggregate_per_year`	`bool`	If `True` (default), aggregate over all snapshots and return a :class:`pandas.Series`. If `False`, return a :class:`pandas.DataFrame` with snapshots as columns.	`True`

Returns:

Type	Description
`Series \| DataFrame`	Pandas Series (`aggregate_per_year=True`) or DataFrame (`aggregate_per_year=False`) with MultiIndex of `location` and `unit`. Returns data at regional level as provided by the PyPSA network. Country-level aggregation is handled by Network_Processor._aggregate_to_country() if configured.

Notes

Combines electricity withdrawals from the following contributions: agriculture electricity loads, residential/commercial low-voltage loads (excluding dedicated industry/agriculture/charger/distribution categories, BEV charging) industry electricity loads, and DAC electricity demand change in home battery is of magnitude 1e-9 compared to electricity demand. Concerning heat: rural air heat pump, rural ground heat pump, rural resisive heater and urban decentral heatings are included. Central heatings using electricity as fuel are NOT included.

`Final_Energy_by_Carrier__Natural_Gas(n: pypsa.Network, aggregate_per_year: bool = True) -> pd.Series | pd.DataFrame`

Extract natural-gas final energy from a PyPSA Network.

Returns the total final energy demand supplied by natural gas across residential and commercial buildings and industry, while subtracting the estimated non-fossil share of gas consumption.

Parameters:

Name	Type	Description	Default
`n`	`Network`	PyPSA network to process.	required
`aggregate_per_year`	`bool`	If `True` (default), aggregate over all snapshots and return a :class:`pandas.Series`. If `False`, return a :class:`pandas.DataFrame` with snapshots as columns.	`True`

Returns:

Type	Description
`Series \| DataFrame`	Pandas Series (`aggregate_per_year=True`) or DataFrame (`aggregate_per_year=False`) with MultiIndex including `location` and `unit`. Returns data at regional level as provided by the PyPSA network. Country-level aggregation is handled by Network_Processor._aggregate_to_country() if configured.

Notes

The function combines residential and commercial gas withdrawals with industry gas demand, then scales the total by the fossil fraction of gas. The fossil fraction is estimated from the ratio of non-fossil gas production to total gas usage, excluding pipeline flows and limiting the non-fossil share to at most 1. Industry gas demand is reduced by a fixed non-energy-use share for EU-27 based on Eurostat energy balance data.

`Final_Energy_by_Carrier__Oil(n: pypsa.Network, aggregate_per_year: bool = True) -> pd.Series | pd.DataFrame`

Extract fossil final-energy oil demand from a PyPSA Network.

Returns the final energy from oil carriers after removing an estimated renewable-oil share.

Parameters:

Name	Type	Description	Default
`n`	`Network`	PyPSA network to process.	required
`aggregate_per_year`	`bool`	If `True` (default), aggregate over all snapshots and return a :class:`pandas.Series`. If `False`, return a :class:`pandas.DataFrame` with snapshots as columns.	`True`

Returns:

Type	Description
`Series \| DataFrame`	Fossil oil final energy with MultiIndex including `location` and `unit`. Returns data at regional level as provided by the PyPSA network. Country-level aggregation is handled by Network_Processor._aggregate_to_country() if configured.

Notes

Total oil final energy is built from: - agriculture machinery oil (Load), - residential/commercial oil boiler demand (rural + urban decentral), - land transport oil (Load).

naphtha for industry is intentionally excluded because it is treated as non-energy use and therefore not part of Final Energy variables.

Regionalization from the copperplate topology is performed by deriving a region code from demand- or production-bus names and applying :func:create_location_index_from_copperplate before regrouping to kwargs["groupby"].

The renewable-oil fraction is computed per region as:

renewable oil production in region / total oil demand in region.

Renewable production is based on supply from selected renewable-oil carriers, while total oil demand is based on withdrawals from oil-using carriers. If the fraction exceeds 1 (i.e., renewable production is larger than regional oil demand), it is clipped to 1, so the fossil share becomes zero in that region. Cross-regional export/import effects of renewable oil are not represented in this statistic.

UNITS_MAPPING is applied inside this function to enable multiplication with demand-side units of the renewable-oil fraction.

`Final_Energy_by_Sector__Agriculture(n: pypsa.Network, aggregate_per_year: bool = True) -> pd.Series | pd.DataFrame`

Extract agriculture-sector final energy from a PyPSA Network.

Returns the total energy consumed by the transportation sector (excluding transmission / distribution losses) across the pypsa-network.

Parameters:

Name	Type	Description	Default
`n`	`Network`	PyPSA network to process.	required
`aggregate_per_year`	`bool`	If `True` (default), aggregate over all snapshots and return a :class:`pandas.Series`. If `False`, return a :class:`pandas.DataFrame` with snapshots as columns.	`True`

Returns:

Type	Description
`Series \| DataFrame`	Pandas Series (`aggregate_per_year=True`) or DataFrame (`aggregate_per_year=False`) with MultiIndex of `location` and `unit`. Returns data at regional level as provided by the PyPSA network. Country-level aggregation is handled by Network_Processor._aggregate_to_country() if configured.

Notes

Includes carriers ['agriculture electricity','agriculture heat','agriculture machinery electric', 'agriculture machinery oil'] executed on Load-Components. Agriculture machinery oil is also carrier of Links and Buses, as Demand is assumed fixed. Time series of Agriculture demand are assumed to be constant in PyPSA-EUR. Theoretically, machinery oil is a single bus with load and links for energy flow. This bus could be sourced by a bus representing oil usage with carbon capture in agriculture. Therefore the "efficiency loss" of this link must be added.

`Final_Energy_by_Sector__Industry(n: pypsa.Network, aggregate_per_year: bool = True) -> pd.Series | pd.DataFrame`

Extract Industry-sector final energy from a PyPSA Network.

Returns the total energy consumed by the Industry sector (excluding transmission / distribution losses)

Parameters:

Name	Type	Description	Default
`n`	`Network`	PyPSA network to process.	required
`aggregate_per_year`	`bool`	If `True` (default), aggregate over all snapshots and return a :class:`pandas.Series`. If `False`, return a :class:`pandas.DataFrame` with snapshots as columns.	`True`

Returns:

Type	Description
`Series \| DataFrame`	Pandas Series (`aggregate_per_year=True`) or DataFrame (`aggregate_per_year=False`) with MultiIndex of `location` and `unit`. Returns data at regional level as provided by the PyPSA network. Country-level aggregation is handled by Network_Processor._aggregate_to_country() if configured.

Notes

Includes all carriers directly connected to loads in the industry sector. Same Carrier names are also attached to some links, so components-grouping is needed! efficiency losses directly supplying industry loads for gas, biomass and coal carbon capture are included extra.

`Final_Energy_by_Sector__Residential_and_Commercial(n: pypsa.Network, aggregate_per_year: bool = True) -> pd.Series | pd.DataFrame`

Extract residential and commercial-sector final energy from a PyPSA Network.

Returns the total energy consumed by the sector residentials and commercials across the pypsa-network.

Parameters:

Name	Type	Description	Default
`n`	`Network`	PyPSA network to process.	required
`aggregate_per_year`	`bool`	If `True` (default), aggregate over all snapshots and return a :class:`pandas.Series`. If `False`, return a :class:`pandas.DataFrame` with snapshots as columns.	`True`

Returns:

Type	Description
`Series \| DataFrame`	Pandas Series (`aggregate_per_year=True`) or DataFrame (`aggregate_per_year=False`) with MultiIndex of `location` and `unit`. Returns data at regional level as provided by the PyPSA network. Country-level aggregation is handled by Network_Processor._aggregate_to_country() if configured.

Notes

excludes EV-charging, as covered by Sector Transportation. Concerning Carrier heat, the Function distinguishes between central heating systems (Final Energy is Heat) and decentral heating systems (Final Energy is Electricity, Gas, Liquids or Solids).

`Final_Energy_by_Sector__Transportation(n: pypsa.Network, aggregate_per_year: bool = True, energy_totals: Path | None = None) -> pd.Series | pd.DataFrame`

Extract transportation-sector final energy from a PyPSA Network.

Returns the total energy consumed by the Transportation sector (excluding transmission / distribution losses) across the pypsa-network.

Parameters:

Name	Type	Description	Default
`n`	`Network`	PyPSA network to process.	required
`aggregate_per_year`	`bool`	If `True` (default), aggregate over all snapshots and return a :class:`pandas.Series`. If `False`, return a :class:`pandas.DataFrame` with snapshots as columns.	`True`
`energy_totals`	`Path \| None`	Path to an energy totals file used to determine domestic shares for aviation and navigation liquid fuels. If `None`, default domestic shares from :func:`get_energy_totals_domestic_share` are used.	`None`

Returns:

Type	Description
`Series \| DataFrame`	Pandas Series (`aggregate_per_year=True`) or DataFrame (`aggregate_per_year=False`) with MultiIndex including `location` and `unit`. Returns data at regional level as provided by the PyPSA network. Country-level aggregation is handled by Network_Processor._aggregate_to_country() if configured.

Notes

Sums transportation final energy from electricity, hydrogen, and liquid fuels: - Electricity demand from BEV charging loads. - Additional EV charging losses computed from BEV charger link flows and adjusted for V2G participation. - Hydrogen demand from fuel-cell land transport. - Liquid fuels for aviation and navigation scaled by domestic fractions, plus land-transport oil.

Raises:

Type	Description
`ValueError`	If BEV charging flow sign conventions are violated.
`TypeError`	If intermediate statistics return mixed result types.

`Net_Imports__Electricity(n: pypsa.Network, aggregate_per_year: bool = True) -> pd.Series | pd.DataFrame`

Extract net imports of electricity from a PyPSA Network.

Evaluation of imports is performed on regional level, national level is covered as a result of aggregation.

Parameters:

Name	Type	Description	Default
`n`	`Network`	PyPSA network to process.	required
`aggregate_per_year`	`bool`	If `True` (default), aggregate over all snapshots and return a :class:`pandas.Series`. If `False`, return a :class:`pandas.DataFrame` with snapshots as columns.	`True`

Returns:

Type	Description
`Series \| DataFrame`	Pandas Series (`aggregate_per_year=True`) or DataFrame (`aggregate_per_year=False`) with MultiIndex of `location` and `unit`. Returns data at regional level as provided by the PyPSA network. Country-level aggregation is handled by Network_Processor._aggregate_to_country() if configured.

Notes

The evaluation pathway in this function strongly depends on the input of the parameter aggregate_per_year. Non-aggregated data (with aggregate_per_year is False) are processed as :class:pandas.DataFrame in long-format, aggregated data (with aggregate_per_year is True) are processed as they come.

`Net_Imports__Oil(n: pypsa.Network, aggregate_per_year: bool = True) -> pd.Series | pd.DataFrame`

Extract net oil imports from a PyPSA Network.

Calculates the net imports of oil by computing the difference between imports to and exports from each location based on Link components.

Parameters:

Name	Type	Description	Default
`n`	`Network`	PyPSA network to process.	required
`aggregate_per_year`	`bool`	If `True` (default), aggregate over all snapshots and return a :class:`pandas.Series`. If `False`, return a :class:`pandas.DataFrame` with snapshots as columns.	`True`

Returns:

Type	Description
`Series \| DataFrame`	Pandas Series (`aggregate_per_year=True`) or DataFrame (`aggregate_per_year=False`) with net oil imports indexed by location and unit.

Notes

All oil is distributed from cupperplate at "EU oil" bus. Oil link know both direction and multiple carriers are covered by these links, as these links are not just representing transmission, but also technologies with efficiencies.

Locations are derived from bus names and assigned to the respective bus columns.

The evaluation pathway in this function strongly depends on the input of the parameter aggregate_per_year. Non-aggregated data (with aggregate_per_year is False) are processed as :class:pandas.DataFrame in long-format, aggregated data (with aggregate_per_year is True) are processed as they come.

`create_location_index_from_copperplate(raw_input: pd.Series | pd.DataFrame, usage_location_list: list, column_name: str = 'location')`

Replace the values of level column_name of an indexed object. Default column_name to be replaced is "location" for backward compatibility.

This helper rebuilds the index of raw_input from its index frame and overwrites the column_name column with values from usage_location_list. It is mainly used when location information from a copperplate-carrier result must be mapped back to explicit regional labels.

Parameters:

Name	Type	Description	Default
`raw_input`	`Series or DataFrame`	Input object with a (Multi)Index that includes a `location` level. The function preserves data values and index level order/names.	required
`usage_location_list`	`list`	New location values to assign row-by-row. Must have the same length as `raw_input`.	required
`column_name`	`str`	Name of the index level to replace, by default "location".	`'location'`

Returns:

Type	Description
`Series or DataFrame`	A copy of `raw_input` with the same data and a rebuilt index where the `column_name` level has been replaced.

Raises:

Type	Description
`ValueError`	If `usage_location_list` length does not match the number of rows, or if the index cannot be reconstructed with the existing index names.
`ValueError`	If `column_name` is not found in the index levels of `raw_input`.

`get_energy_totals_domestic_share(energy_totals: Path, kind: str) -> pd.Series`

Return the domestic share of energy totals for a given kind.

Parameters:

Name	Type	Description	Default
`energy_totals`	`Path`	Path to the energy totals csv file.	required
`kind`	`str`	The kind of energy totals to calculate the factor for.	required

Returns:

Type	Description
`Series`	The share of national aviation or navigation per country.

`remap_unit_index(series: pd.Series | pd.DataFrame, unit_mapping: dict = UNITS_MAPPING) -> pd.Series | pd.DataFrame`

Remaps unit index to standard value for statistics with need for direct comparison of input-output flows.

Parameters:

Name	Type	Description	Default
`series`	`Series \| DataFrame`	A series with a MultiIndex with a `unit` level.	required
`unit_mapping`	`dict`	Dictionary with old unit values as keys and new unit values as values, by default UNITS_MAPPING	`UNITS_MAPPING`

Returns:

Type	Description
`Series \| DataFrame`	A series with a MultiIndex with a `unit` level remapped.

Statistics Functions API