Supported Datasets

Supported Datasets#

APEM includes built-in datasets for the unit-based and order-book-based workflows. Select them through config.json.

For configuration syntax, see Configuration. For adding your own data, see Using Your Own Datasets.

Unit-Based Datasets#

Use these values under unit_based_model.dataset.

Dataset key

Description

Notes

IEEE_RTS

IEEE Reliability Test System-style unit-based scenario, based on the RTS-96 test system described by Grigg et al. (1999).

Generation-cost and demand-valuation bid/offer curves follow the case studies of Garcia-Bertrand et al. (2006) and Zoltowska (2016). Also available as a converted order-book dataset. Supports the nodal unit-based workflows.

PJM

Single-node PJM day-ahead-market dataset included in the repository.

Repository data is for 28 February 2023. Uses many seller offer blocks and generated buyer valuations. Source data is based on PJM Data Miner 2. Supports the nodal unit-based workflows.

PyPSAEurSmall

Small PyPSA-Eur-derived unit-based scenario for Germany.

Data is for 1 March 2013. Supports the nodal and zonal unit-based workflows.

PyPSAEurLarge

Large PyPSA-Eur-derived unit-based scenario for Germany.

Data is for 1 March 2013. Supports the nodal and zonal unit-based workflows.

ARPA

ARPA-E Grid Optimization Competition-derived unit-based network scenario.

Single-period scenario. Supports the nodal unit-based workflows. Used by APEM PF Relaxations and also available as a converted order-book dataset.

Note

The zonal unit-based workflows (Zonal_NTC_aggregated, Zonal_NTC_multiedge, and Zonal_FBMC) are currently supported for PyPSAEurSmall and PyPSAEurLarge.

Unit-Based Dataset Sizes#

The counts below are computed from the parsed Scenario objects. Branches are transmission-network graph edges. Seller and buyer counts are unique seller and buyer IDs.

Dataset key

Nodes

Branches

Sellers

Buyers

IEEE_RTS

25

34

32

17

PJM

1

0

457

3

PyPSAEurSmall

40

67

246

40

PyPSAEurLarge

328

431

1078

252

ARPA

617

841

94

404

Order-Book-Based Datasets#

Use these values under order_book_based_model.dataset.

Dataset key

Scope

Order content

Notes

GENERATED_SMALL

24 periods; 1 zone.

Step, block, complex, scalable-complex, and piecewise-linear orders.

Synthetic Euphemia-style instance for small order-book experiments.

GENERATED_LARGE

24 periods; 1 zone.

Same order families as GENERATED_SMALL, with more block, complex, and scalable-complex orders.

Synthetic larger instance for stress-testing order-book formulations.

OMIE

24 periods; 1 zone.

Step orders and complex orders.

Based on OMIE day-ahead-market cab and det file formats; see the dataset README under apem/order_book_based_model/euphemia/data/datasets/omie/.

GME

24 periods; 1 zone.

Step orders and block orders.

Bundled Euphemia-style CSV instance derived from GME-style order-book data.

TEST_3NODE

1 period; 3 zones.

Step orders and one block order.

Small validation case with ATC and FBMC inputs.

TEST_3NODE_LOWCAP

1 period; 3 zones.

Step orders and one block order.

Low-capacity variant of TEST_3NODE for network-constraint validation.

IEEE_RTS

24 periods; 1 zone.

Step, block, and scalable-complex orders.

Converted from the unit-based IEEE_RTS scenario.

ARPA

1 period; 1 zone.

Step, block, and scalable-complex orders.

Converted from the unit-based ARPA scenario.

Order-book datasets are stored as Euphemia-style CSV folders under:

apem/order_book_based_model/euphemia/data/datasets/

Dataset Conversion#

APEM can convert selected unit-based scenarios into order-book CSV inputs. This is used for the order-book versions of IEEE_RTS and ARPA.

For implementation details, see apem/order_book_based_model/euphemia/data/conversion/data_conversion.py.

References#