This is just tracking in a more public way PV MetNet results for site level forecasting. Overall, results so far are probably not worth changing from the random forest model.
The best performing PV MetNet model only took NWP data, and has a MAE of ~11% on average across timesteps up to 30 hours into the future, when only looking at times when generation was above 1% (To cut out night) and on the same set of IDs at the random forest model.
Weights and Bias link is here (WIll need to be signed in to OCF account to see it): https://wandb.ai/openclimatefix/PvMetNet
Detailed Description
PVMetNet was trained with a few different sets of modalities, and not all for the same amount of steps, as especially satellite data is very slow to load. For a similar reason, only the best performing model was benchmarked against the validation set. Overall, since Sun position and topo did not seem to make a huge difference, the one benchmarked was just the NWP-only model. The model was trained to predict generation for each 5 minute interval, including night time. So the training MAEs are artificially low versus models that only predict during day time.
| Inputs |
Context Size |
Pixel Size |
Train MAE |
| NWP |
512km |
64 |
0.03583 |
| NWP+Sun |
512km |
32 |
0.03516 |
| NWP+Sun |
512km |
64 |
0.03937 |
| NWP+Topo+Sat+Sun |
128km |
64 |
0.03461 |
| NWP+Topo+Sat+Sun |
128km |
32 |
0.05579 |
| Sat+Sun+Topo |
128km |
16 |
0.04593 |
Training loss plots, helpful-flower, blooming-meadow, and unique-grass are the ones with satellite, trained for a lot less time because of the time to create batches was so high:
Validation results:
This is just tracking in a more public way PV MetNet results for site level forecasting. Overall, results so far are probably not worth changing from the random forest model.
The best performing PV MetNet model only took NWP data, and has a MAE of ~11% on average across timesteps up to 30 hours into the future, when only looking at times when generation was above 1% (To cut out night) and on the same set of IDs at the random forest model.
Weights and Bias link is here (WIll need to be signed in to OCF account to see it): https://wandb.ai/openclimatefix/PvMetNet
Detailed Description
PVMetNet was trained with a few different sets of modalities, and not all for the same amount of steps, as especially satellite data is very slow to load. For a similar reason, only the best performing model was benchmarked against the validation set. Overall, since Sun position and topo did not seem to make a huge difference, the one benchmarked was just the NWP-only model. The model was trained to predict generation for each 5 minute interval, including night time. So the training MAEs are artificially low versus models that only predict during day time.
Training loss plots,
helpful-flower,blooming-meadow, andunique-grassare the ones with satellite, trained for a lot less time because of the time to create batches was so high:Validation results: