Our forecasting models

Predico uses ensemble techniques to combine forecasts from multiple forecasters, assigning weights based on their historical performance.

Forecasting Components Breakdown

Predico collaborative forecasting sessions consider two types of ensemble techniques.

1. Forecasts pre-processing: This module analyzes the submitted time-series (i.e., by Forecasters) and removes abnormal submissions (relative to the majority in the group).
2. Weighted Average: This method combines forecasts from multiple forecasters, assigning weights based on their historical performance.
3. Regression models: These models use historical data to predict the future performance of forecasters.

For each challenge, the platform executes both ensemble techniques and stores them internally to be used by a market maker later.

Forecasts pre-processing

Throughout our studies in collaborative forecasting, particularly when running in operational scenarios, it became obvious that:

1. Forecasters do not always have similar forecasts (even if it is for the same target resource)
2. There will unexpected issues regarding the submitted timeseries (e.g., submittion of Q90 instead of Q50)

Below you can find an illustration comparing the submitted forecasts, Predico's ensemble forecasts and the measurement data. Some abnormal situations are also pointed out.

To deal with these issues, we have equiped our platform with outlier detection mechanisms to detect abnormal submissions and remove them before applying our forecast techniques. Currently, we are using a Dynamic Time Warping based approach to identify and remove abnormal submissions.

Our system is also prepared to deal with scenarios when a Forecaster does not submit forecasts (for a single session), which happens quite frequently.

Weighed Average

The weighted average ensemble technique - currently applied to both wind and solar forecasting challenges - combines forecasts from multiple forecasters, assigning weights based on their historical performance. The weights are calculated using the forecast skill scores of each forecaster. The higher the skill score, the higher the weight assigned to the forecaster's forecast.

In contrast to regression models, this approach allows the system to react quickly to changes in forecasters' performance, as the weights are updated daily based on the latest skill scores.

While different strategies can be used to calculate the weights, the Predico platform currently uses a weighted-average approach:

• For Q50 ensemble forecasts, the weights are calculated based on the Root Mean Squared Error (RMSE) of the forecasted values compared to the actual measurements.
• For Q10 and Q90 ensemble forecasts, the weights are calculated based on the Pinball Loss of the quantile forecasts.
• The weights are calculated based on the scores of the last 5 days transformed via an exponential function (to accentuate differences between forecasters).

An outlier detection step is also executed over the day-ahead forecasts to ensure that abnormal submissions are not part of the final ensemble.

Regression Models

Regression models are currently used to produce both wind power and wind power variability forecasts. The latter is used for detecting sudden changes in wind power output (i.e., wind ramps). Besides forecasting it also provides insights regarding the contribution of each variables (i.e., Forecaster data or variables engineered from forecasters data) in the forecast accuracy.

The following components are involved in the forecasting/assessment process:

• Forecasting: This is divided into two parallel processes:
  • Wind Power: Forecasts are generated through a series of steps including feature engineering, hyperparameter optimization, model training, and the final forecast generation.
  • Wind Power Variability: A similar process is followed here, focusing on capturing fluctuations in wind power output, which includes feature engineering, hyperparameter optimization, model training, and forecast generation.
• Wind Ramp Detection: Identifies sudden changes or "ramps" in wind power, which are crucial for managing grid stability and operational decisions.
• Forecast Skill Evaluation: This stage evaluates the performance of forecasters using metrics like Root Mean Squared Error (RMSE). These scores help rank forecasters based on their forecast skill, per challenge participation.
• Data Value Assessment: Using methods such as Permutation Importance and Shapley Values, this component assesses the value of data inputs, helping identify the most influential variables in forecasting accuracy.

We are working on expanding the regression models to include solar power forecasting in the future.