Data

Data#

Parser, scenario, and conversion utilities for order-book datasets.

Zonal Scenario#

API path: apem.order_book_based_model.euphemia.data.parsing.zonal_scenario

class ZonalScenario(
name,
periods,
step_orders,
block_orders,
complex_orders,
complex_step_orders,
scalable_complex_orders,
scalable_step_orders,
piecewise_linear_orders,
zones=None,
atc=None,
fb_constraints=None,
fb_ptdf=None,
)[source]#

Bases: object

Container for normalized order-book and network inputs used by Euphemia.

The scenario stores all parsed bid tables, period definitions, zone labels, and optional network representations in the canonical structure expected by the master problem and pricing subproblem.

Create a normalized scenario object for a parsed order-book dataset.

Parameters:

  • name (str) – Human-readable scenario name.

  • periods (list) – Ordered market periods modeled by the scenario.

  • step_orders (DataFrame) – Standard step-order bids.

  • block_orders (DataFrame) – Block-order bids.

  • complex_orders (DataFrame) – Parent complex-order definitions.

  • complex_step_orders (DataFrame) – Step rows associated with complex orders.

  • scalable_complex_orders (DataFrame) – Parent scalable-complex-order definitions.

  • scalable_step_orders (DataFrame) – Step rows associated with scalable-complex orders.

  • piecewise_linear_orders (DataFrame) – Piecewise-linear bid segments.

  • zones (list[str] | None) – Optional explicit zone list. Defaults to ["Z1"].

  • atc (DataFrame | None) – Optional ATC network table.

  • fb_constraints (DataFrame | None) – Optional FBMC constraint table.

  • fb_ptdf (DataFrame | None) – Optional FBMC PTDF table.

overview()[source]#

Return a multiline text dump of the scenario contents.

Returns:

Human-readable summary containing all stored tables and metadata.

Return type:

str

Parse Order Book#

API path: apem.order_book_based_model.euphemia.data.parsing.parse_order_book

class ParseOrderBook(path, title)[source]#

Bases: ParseData

Parse a Euphemia-style CSV dataset folder into a normalized scenario.

Store dataset location and display title.

Parameters:

  • path (Path) – Root directory of the order-book dataset.

  • title (str) – Human-readable dataset name used in outputs.

parse_data(day=None)[source]#

Read dataset CSVs and assemble a ZonalScenario.

The parser loads the standard order tables, period definitions, and optional network files from self.path. Zone labels are normalized, missing optional network inputs are replaced with empty canonical tables, and the final result is returned as a model-ready scenario.

Parameters:

day (object | None) – Optional compatibility argument accepted to match the base parser interface. It is currently ignored.

Returns:

Parsed order-book scenario ready for Euphemia.

Return type:

ZonalScenario

ensure_zone_column(df)[source]#

Normalize zone naming in an order table.

Parameters:

df (DataFrame) – Input table that may use a zone alias such as z or bidding_zone.

Returns:

Copy of df with a canonical string zone column. Missing zone information defaults to "Z1".

Return type:

DataFrame

parse_atc(path)[source]#

Load optional ATC network data from atc.csv.

Parameters:

path (Path) – Dataset directory that may contain atc.csv.

Returns:

DataFrame with canonical ATC columns from_zone, to_zone, t, cap and optional ramp_up / ramp_down columns. An empty DataFrame is returned when no ATC file exists.

Raises:

ValueError – If atc.csv is present but missing required columns.

Return type:

DataFrame

parse_fb_constraints(path)[source]#

Load optional FBMC RAM constraints from fb_constraints.csv.

Parameters:

path (Path) – Dataset directory that may contain fb_constraints.csv.

Returns:

DataFrame with canonical columns cnec_id, t, ram and optional lb. An empty DataFrame is returned when no FBMC constraint file exists.

Raises:

ValueError – If the file exists but required columns are missing.

Return type:

DataFrame

parse_fb_ptdf(path)[source]#

Load optional FBMC PTDF rows from fb_ptdf.csv.

Parameters:

path (Path) – Dataset directory that may contain fb_ptdf.csv.

Returns:

DataFrame with canonical columns cnec_id, t, zone, and ptdf. An empty DataFrame is returned when no PTDF file exists.

Raises:

ValueError – If the file exists but required columns are missing.

Return type:

DataFrame

transform_step_orders(orders, periods, sell, order_id=None, scalable=None)[source]#

Convert cumulative step-order quantities into marginal bid segments.

Parameters:

  • orders (DataFrame) – Step-order table for either complex or scalable-complex orders.

  • periods (List[int]) – Ordered list of modeled periods.

  • sell (bool) – When True, transform sell-side cumulative quantities; otherwise transform buy-side cumulative quantities.

  • order_id (int | None) – Optional parent order identifier used to filter rows to a single complex or scalable-complex order.

  • scalable (bool | None) – Whether order_id refers to a scalable-complex order.

Returns:

DataFrame of transformed marginal step-order rows.

Return type:

DataFrame

Data Conversion#

Convert unit-based scenario inputs into order-book-based Euphemia datasets.

API path: apem.order_book_based_model.euphemia.data.conversion.data_conversion

class DataConversion(scenario)[source]#

Bases: object

Convert a unit-based market scenario into order-book Euphemia inputs.

The source scenario uses the unit-based APEM data model with buyer-side demand tables and seller-side supply tables. This helper maps those tables into the order-book structures expected by the Euphemia workflow.

Parameters:

scenario (Scenario)

add_linked_block_orders(
time_commitment,
s,
count,
block_bids,
reduce_orders,
)[source]#

Add linked child blocks for one commitment pattern of one supply unit.

Parameters:

  • time_commitment (list[int]) – Binary commitment profile over the modeled periods.

  • s (int) – Unit identifier from the source unit-based scenario.

  • count (int) – Running index used to build stable order identifiers.

  • block_bids (list[dict]) – Output list to which linked block rows are appended.

  • reduce_orders (bool) – Whether to merge same-price linked segments across active periods.

Return type:

None

block_signature(row)[source]#

Generate a stable signature for one block order.

The signature uses the block type, quantity profile, price, and MAR. Source unit or demand identifiers are intentionally ignored so that economically identical converted block orders can be merged later.

Parameters:

row (Series)

Return type:

str

compress_blocks(df)[source]#

Losslessly merge identical converted block-order chains.

A linked chain consists of one exclusive parent block and zero or more linked child blocks. Two chains are mergeable when they have the same block signatures and the same tree structure.

After merging, identical quantities are summed, the parent keeps MAR = 1, original ids are concatenated for traceability, and child code_prm values are rewired to the merged parent id.

Parameters:

df (DataFrame)

Return type:

DataFrame

compute_buyers_elastic_bids()[source]#

Convert price-elastic demand segments into order-book step orders.

Return type:

DataFrame

compute_buyers_inelastic_bids()[source]#

Convert price-inelastic demand into order-book block orders.

One block order is created per demand entity. For each period t, the block quantity equals the inelastic demand in that period. The resulting order is assigned MAR = 1 and a very large bid price so that it behaves as must-serve demand in the order-book model.

Return type:

DataFrame

generate_contiguous_patterns(min_uptime)[source]#

Generate contiguous commitment patterns for a unit with a given minimum uptime.

The returned patterns assume at most one start and one stop event across the modeled horizon.

Parameters:

min_uptime (int)

Return type:

List[List[int]]

generate_patterns(min_uptime)[source]#

Generate all commitment patterns for a unit with a given minimum uptime.

Parameters:

min_uptime (int)

Return type:

List[List[int]]

generate_positive_no_load_cost_bids(reduce_linked_blocks)[source]#

Convert commitment-coupled unit offers into order-book block-order families.

This routine targets supply units whose behavior cannot be represented by a simple scalable complex order, for example because they have a non-trivial minimum-uptime requirement or a positive no-load cost. Each such unit is translated into one exclusive parent block per commitment pattern together with linked child blocks that represent additional dispatch segments.

Parameters:

reduce_linked_blocks (bool) – When True, merge same-price linked child segments across active periods into one linked block per segment.

Returns:

Block-order dataframe for the converted commitment-coupled offers.

Return type:

DataFrame

generate_write_patterns(use_contiguous_patterns)[source]#

Generate commitment-pattern files used for block-order conversion.

Parameters:

use_contiguous_patterns (bool) – Whether to restrict the generated patterns to contiguous on/off trajectories.

Return type:

None

generate_zero_no_load_cost_bids()[source]#

Convert simple unit-based supply offers into scalable complex Euphemia orders.

This routine targets supply units with zero no-load cost and no meaningful minimum-uptime restriction. Each qualifying unit is mapped to one scalable complex order plus its associated step orders. The MAP_t parameters are taken from the unit’s minimum production profile, while the step orders are derived from the period-wise supply curve segments.

Return type:

Tuple[DataFrame, DataFrame]

set_block_bids()[source]#

Placeholder for a future block-order assembly helper.

Return type:

None

set_step_orders()[source]#

Placeholder for a future step-order assembly helper.

Return type:

None

run_unit_based_to_order_based_conversion(
unit_based_data,
generate_uptime_patterns=True,
reduce_linked_blocks=True,
use_contiguous_patterns=True,
compress_identical_blocks=True,
)[source]#

Convert a unit-based dataset into order-book Euphemia CSV inputs.

Parameters:

  • unit_based_data (ParseData) – Parser class for the source unit-based dataset.

  • generate_uptime_patterns (bool) – Whether to generate and persist commitment patterns for units with minimum-uptime constraints.

  • reduce_linked_blocks (bool) – Whether to merge linked child blocks across multiple active periods when they share the same price segment.

  • use_contiguous_patterns (bool) – Whether to restrict commitment patterns to contiguous on/off trajectories.

  • compress_identical_blocks (bool) – Whether to merge identical converted block-order chains before writing the output CSV files.

Return type:

None

save_df(df, unit_based_data, name)[source]#

Persist one converted dataframe into the mapped order-book dataset folder.

Parameters:

  • df (DataFrame)

  • unit_based_data (ParseData)

  • name (str)

Return type:

None