Skip to content

Outputs

This page provides a detailed description of the output files generated by the data generation process. All files are saved under {settings.data_dir}/{network.name}/raw.

Output Files

Log Files

tqdm.log

Progress bar log showing the execution progress of the data generation process.

error.log

Error messages captured during generation.

args.log

YAML dump of the configuration used for this run.

Load Scenario Files

scenarios_{generator}.parquet

Load scenarios (per-element time series) produced by the selected load generator.

scenarios_{generator}.html

Plot of the generated load scenarios.

scenarios_{generator}.log

Generator-specific notes (e.g., bounds for the global scaling factor when using agg_load_profile).

n_scenarios.txt

Metadata file containing the total number of scenarios (used for efficient partition management).

Network Data Files

Note: All network data files are saved as partitioned parquet directories. Each file includes a scenario_partition column used for partitioning, which groups scenarios into partitions (default: 200 scenarios per partition).

bus_data.parquet

Bus-level features for each processed scenario. Columns:

  • scenario: Global scenario index (unique identifier)
  • load_scenario_idx: Index of the load scenario
  • bus: Index of the bus
  • Pd: Active power demand at the bus (MW)
  • Qd: Reactive power demand at the bus (MVAr)
  • Pg: Active power generation at the bus (MW)
  • Qg: Reactive power generation at the bus (MVAr)
  • Vm: Voltage magnitude at the bus (p.u.)
  • Va: Voltage angle at the bus (degrees)
  • PQ: Indicates if the bus is a PQ bus (1 if true, 0 if false)
  • PV: Indicates if the bus is a PV bus (1 if true, 0 if false)
  • REF: Indicates if the bus is a reference bus (1 if true, 0 if false)
  • vn_kv: Nominal voltage level at the bus (kV)
  • min_vm_pu: Minimum voltage magnitude limit (p.u.)
  • max_vm_pu: Maximum voltage magnitude limit (p.u.)
  • GS: Shunt conductance at the bus (p.u. at V = 1.0 p.u.)
  • BS: Shunt susceptance at the bus (p.u. at V = 1.0 p.u.)

If settings.include_dc_res=True, also includes DC power flow columns (DC_BUS_COLUMNS): - Va_dc: DC voltage angle (degrees) - Pg_dc: DC active power generation at the bus (MW)

gen_data.parquet

Generator features per scenario. Columns:

  • scenario: Global scenario index (unique identifier)
  • load_scenario_idx: Index of the load scenario
  • idx: Generator row index (0-based)
  • bus: Bus index where the generator is connected
  • p_mw: Active power output (MW)
  • q_mvar: Reactive power output (MVAr)
  • min_p_mw: Minimum active power limit (MW)
  • max_p_mw: Maximum active power limit (MW)
  • min_q_mvar: Minimum reactive power limit (MVAr)
  • max_q_mvar: Maximum reactive power limit (MVAr)
  • cp0_eur: Constant cost coefficient (EUR)
  • cp1_eur_per_mw: Linear cost coefficient (EUR/MW)
  • cp2_eur_per_mw2: Quadratic cost coefficient (EUR/MW²)
  • in_service: 1 if generator is in service, else 0
  • is_slack_gen: 1 if generator is at the reference bus, else 0

If settings.include_dc_res=True, also includes DC generator column (DC_GEN_COLUMNS): - p_mw_dc: Active power from DC solution (MW)

branch_data.parquet

Branch features per scenario. Columns:

  • scenario: Global scenario index (unique identifier)
  • load_scenario_idx: Index of the load scenario
  • idx: Branch row index (0-based)
  • from_bus: Index of the source bus
  • to_bus: Index of the destination bus
  • pf: Active power flow from source to destination (MW)
  • qf: Reactive power flow from source to destination (MVAr)
  • pt: Active power flow from destination to source (MW)
  • qt: Reactive power flow from destination to source (MVAr)
  • r: Series resistance (p.u.)
  • x: Series reactance (p.u.)
  • b: Total line charging susceptance (p.u.)
  • Yff_r: Real part of the self-admittance at the source bus (p.u.)
  • Yff_i: Imaginary part of the self-admittance at the source bus (p.u.)
  • Yft_r: Real part of the mutual admittance from source to destination bus (p.u.)
  • Yft_i: Imaginary part of the mutual admittance from source to destination bus (p.u.)
  • Ytf_r: Real part of the mutual admittance from destination to source bus (p.u.)
  • Ytf_i: Imaginary part of the mutual admittance from destination to source bus (p.u.)
  • Ytt_r: Real part of the self-admittance at the destination bus (p.u.)
  • Ytt_i: Imaginary part of the self-admittance at the destination bus (p.u.)
  • tap: Transformer tap ratio
  • shift: Phase shift (degrees)
  • ang_min: Minimum angle limit (degrees)
  • ang_max: Maximum angle limit (degrees)
  • rate_a: Rate limit of the branch (MVA)
  • br_status: Branch status (1 if in service, 0 if out of service)

If settings.include_dc_res=True, also includes DC branch columns (DC_BRANCH_COLUMNS): - pf_dc: DC active power flow from source to destination (MW) - pt_dc: DC active power flow from destination to source (MW)

y_bus_data.parquet

Nonzero Y-bus entries per scenario. Columns:

  • scenario: Global scenario index (unique identifier)
  • load_scenario_idx: Index of the load scenario
  • index1: Row index in the Y-bus matrix
  • index2: Column index in the Y-bus matrix
  • G: Conductance value (p.u.)
  • B: Susceptance value (p.u.)

Runtime Data Files

runtime_data.parquet

Runtime data for each scenario. Columns:

  • scenario: Global scenario index (unique identifier)
  • load_scenario_idx: Index of the load scenario
  • ac: AC solver execution time (seconds)

If settings.include_dc_res=True, also includes DC runtime column (DC_RUNTIME_COLUMNS): - dc: DC solver execution time (seconds)

Statistics Files

stats.parquet

Aggregated statistics collected during generation (if settings.no_stats=False), including metrics such as:

  • Number of generators
  • Number of lines and transformers
  • Number of overloads
  • Maximum loading values
  • Other network performance metrics

stats_plot.png

Visualization of the aggregated statistics (if settings.no_stats=False).

Feature Visualization

Feature distribution plots can be generated using the CLI command:

gridfm-datakit plots path/to/data/directory

This creates a feature_plots/ directory with violin plots per feature named distribution_{feature_name}.png.