Radiation

PVSC44: Himawari 8 enabled real-time distributed PV simulations for distribution networks

Just over a year ago, I announced an exciting new project focused on delivering real-time distributed PV simulations to distribution network service providers (DNSPs). This is an Australian Renewable Energy Agency (ARENA) funded research project, led by the Australian National University. As the Chief Investigator on this research project, I have spent the last year regularly meeting with and networking with DNSPs across the country. As a result, we now have 12 of the 15 DNSPs on board, which is a tremendous outcome.

Each of theses DNSPs is providing us with the metadata for the installed PV system within their networks, which we are using to produce simulations of their power output.  This work extends the efforts of my PhD, where I built the Regional PV Simulation System (RPSS) through collaboration with locality utility ActewAGL, but adds new capability through real-time PV monitoring and derived irradiance estimates from the Himawari 8 satellite.

This week, I am at the PVSC44 conference, co-chairing the session on solar forecasting for grid integration of PV, as well as presenting a paper which outlines our modelling methodology for this new tool. I wanted to take the chance to put up an accompanying blog post, to help make it easily accessible to my fellow researchers. Of course if you want the real details, you should check out the paper :-).  It is important to note that the following methods are the result of a close industry collaboration with industry partner Solcast, and represented the state-of-the-art of its satellite nowcasting technologies from early 2017

[Download the Paper]

Here's how it all works..

Flow diagram of our methodology

Flow diagram of our methodology

We take images from the Himawari-8 satellite, in particular the visible and infrared channels, and apply the general principles of the Heliosat-2 approach for turning these into cloud opacity estimates. What's very interesting about this process, is that the spatial and temporal scales of this data are quite advanced, as compared to other geostationary weather satellites (e.g. GOES 13/15 over the US). At 1-2km^2 resolution with 10 minute update cycles, the radiation modelling enabled by this satellite is quite an exciting advancement for our field.

We use the Himawari 8 data in combination with the GFS NWP model to decompose clouds into lower, middle and upper troposphere layers, and then aggregate these layers to estimate total cloudiness. Total cloud opacity (represented as an index between 0 and 1) is then derived using differences between the lowest visible return value of albedo. With this value in hand, we apply a linear reduction to the Esra clear sky radiation model to produce an estimate of global horizontal irradiance (Gh) for each pixel. Estimates of Gh are afterwards decomposed into direct normal irradiance (Bn) and diffuse horizontal irradiance (Dh) through the Engerer2 separation model.

Satellite imagery decomposition

Satellite imagery decomposition

Radiation Validation on "Estimated Actuals"

estimated actuals

For this conference paper, we have validated the Gh values produced by this system. These "Estimated Actuals" are produced using the satellite scan for each time stamp (no forecasting involved). It is necessary to compare these outputs to the observed radiation conditions at the surface, which we accomplish using data from the Australian Bureau of Meteorology.  Using 7 sites and 2 months of data from the BoM, we were able to determine these estimated actuals to have a Mean Bias Error (MBE) of -7 W m-2 and Root Mean Squared Error (RMSE) of 55 W m-2.  This accomplishes the industry standard of a 'good' radiation model, which is defined in the literature as having an relative MBE less than 5% (1.8% reported) and relative RMSE less than 15% (10.7% reported).

PV Power Estimated Actuals + Validation

The next step in the methodology is to compute PV power output calculations, based on the installed characteristics of the PV systems considered.  This requires information such as the azimuth & tilt of the system, as well as the total installed capacity. We apply the Reindl transposition model to estimate the available plane of array irradiance, followed by a quadratic PV power model to compute the estimated power output. By combining the Estimated Actuals from the satellite system with this model, we also can compute Estimated Actuals for PV system power output.

Screen Shot 2017-06-27 at 14.47.55 pm.png

In order to accomplish this step & validate it, we collected data from 78 PV systems across the Canberra, Australia region for a period of 6 months.  Using these time-series data, we then applied our QCPV methodology to both quality control the reported power outputs, but also to determine the actual azimuths & tilts for the POA radiation modelling step.  For this validation, we report an MBE of 0.04 W/Wp and RMSE of 0.15 W/Wp

Overall scatter in the validation data shows well behaved results, but with accuracy losses nearly double that of the radiation validation. We note that this validation reports uneven bias across the distribution of PV system measurement values, suggesting further refinement of the quadratic PV model coefficients could be required.

Scaling it up for DNSPs

Given the relatively good performance of the modelling system (much improvement needed still for operational purposes) we apply this system to a network wide simulation.  By utilising the metadata for 15,500+ embedded solar PV systems in the ActewAGL distribution network, we produce Estimated Actuals for the PV power systems in this network via this methodology. Additionally, we implement a simple cross-validation using 6 PV systems in a real-time scenario and provide the demo in video format:

In the provided image, we see the Estimated Actuals for each individual PV system in the ActewAGL network, as provided for the 23rd of June 2016. This day was selected based upon its complicated, multi-layered cloud conditions to best demonstrate the capability of this system in a challenging circumstance. A heat map colour profile shows the relative power output from each PV system based on its installed capacity.  This is accompanied by the data from the 6 PV cross-validation sites in the circles which are colour filled according to their observed power output.

At bottom right of this image, the Estimated Actuals for these systems are displayed in grey, with the red line showing the observed power output values. Overall, we observe a slightly positive bias in the Estimated Actuals of 0.038 W/Wp and a relatively lower RMSE value of 0.12 W/Wp.

A Great Start: Where Next?

Overall, our team is quite happy with this first set of results, but we've many improvements to make to this system moving forward. As a part of this validation, we've already found & corrected issues with positive cloud opacity bias during overcast conditions, and we have implemented a dynamic kernel which corrects the cloud shadow displacement at low sun angles. Up next for us, is improving the radiation modelling routines. For one, Engerer2 was not designed for this purpose and our linear reduction on the Esra model needs to be updated.

The good news is, we've filled up our team with postdocs, PhDs, honours students and software developers, and are working hard to make these updates.  Soon, we'll even highlight the forecasting capability we've added to the system in an upcoming issue of Progress in Solar Energy.  In the meantime, we're deploying this system for our participating DNSP partners right now, and will iterate forward based on their feedback. So check back soon and connect with us on Research Gate to stay up to date & to collaborate.

It's all a part of teaming up for that solar-powered future! And in that spirit be sure to check out our data sharing campaign, where we are making the output of this system available for your research purposes. Read more at this link!

Lastly, the ANU team would like thank & acknowledge our industry partner Solcast's contributions to this project. 


The Engerer2 Diffuse Fraction Model (Global Radiation Separation Model)


August 2016 Update: You can learn how to compute the Engerer2 model in my Rpackage here!

This blog post explores the success of the Engerer 2 model as laid out in: Engerer, Nicholas A.  Minute resolution estimates of the diffuse fraction of global irradiance for southeastern Australia. Solar Energy. 116, June 2015, pp. 215-237.

[download the paper here]


One of the key outcomes from my PhD thesis was the validation of two different types of radiation models: clear sky and separation, against one minute resolution data. For the clear sky validation, I found suitable performance from several models for use in Australia, but the available separation models, however, did not have acceptable performance

The main issue with the available separation models (models that take a global radiation measurement from a pyranometer and separate it into its direct and diffuse components), is that they are regression based, with the original data being hourly averages of radiation.  At minute-resolution timescales, the relationship with global and diffuse/direct radiation is very different.  For one, there are very rapid fluctuations in the incoming radiation budget across these timescales.  Another big difference is the influence of cloud enhancement which is where radiation arriving at the surface exceeds the clear sky value because of non-linear interactions with some types of cloud decks. 

The Engerer2 model, with the diffuse fraction (Kd), plotted against the clearness index (Kt)

Making big changes, in the name of science

Thus, when I formulated my model, I knew that I had to make some significant advancements upon the existing methods/literature.  The principal improvements made with this model are four-fold:

  1. Inclusion of a physical model (REST2 clear sky model), making the model 'quasi-physical', much like the DISC model written by Eugene Maxwell (Maxwell 1987)
  2. The model is the only one of its class (as of the time of publication) that has been fit to minute resolution data (most other models have been designed for hourly data)
  3. There are two new variables, which have not previously been utilised in a separation model.  These are delta_Ktc (deviation of observed clearness index from clear sky value of clearness index)
  4. and K_de (the portion of the diffuse fraction that is attributable to cloud enhancement events)
The Engerer2 model formulation.  Please  read the paper  for more information.

The Engerer2 model formulation.  Please read the paper for more information.

independent assessment of the model: it works very, very well

The result is an impressive performance of the model against the current suite representing the state-of-the-art.  In a recent study, Gueymard and Ruiz-Arias 2015, radiation data from 54 sites around the globe were used to validate 140 separation models.  In this study, the Engerer2 model was the best!  Here it is, as described in the text:

“It is found that two models stand out over the arid, temperate and tropical climate zones: ENGERER2 and PEREZ2. These two models share two important features: (i) They include a variability predictor; and (iii) They leverage clear-sky irradiance estimates. The reason why ENGERER2 performs consistently better than PEREZ2 or other models is most likely because it was actually derived from 1-min data (compared to hourly data for PEREZ2 or most other models tested here). Based on the ensemble of statistical results obtained here, it is concluded that ENGERER2 has the best generalization skill, and can thus be considered a ‘‘quasi-universal” 1-min separation model, wherever and whenever low-albedo conditions prevail.”
Figure 2 in Gueymard & Ruiz-Arias 2015. Displaying the stations at which radiation data was used for model evaluation.

Figure 2 in Gueymard & Ruiz-Arias 2015. Displaying the stations at which radiation data was used for model evaluation.

And well... was that ever quite the compliment (especially coming from a scientist whom I've looked up to for so long)!  I am very pleased with this result, because now, my Engerer2 model is the ‘'quasi-universal' 1-min separation model" and has been accepted to be of global standard.  That makes my inner nerd quite happy, I'll admit. "Chuffed" as my Aussie friends would say :-).  And now I've have been given a reason to write a long overdue blog post about this research work.  As well as deliver a little surprise...

NOw, the Engerer2 model is in demand

As a result of this excellent outcome, I have several researchers in the community who would like to use my model, and I am quite happy to oblige.  So with this post, I am also announcing that a beta version of my Rpackage "anusolar" is now available, on request.  You can read more about this software at nickengerer.org/rpackage and where you can find out how to use the Engerer 2 separation model!  This package will allow you to do more than that, including PV simulation, KPV calculation and creating output from clear-sky radiation models.  So go check it out!

#MODSIM2015 - Extracting Solar Radiation Measurements from PV System Power Output

 A bit of background info on my 21st International Congress on Modelling and Simulation (MODSIM2015) paper

[download the paper]

#MOSIM2015 is underway! And lucky me, I’m here at our wonderful sunny Gold Coast location awaiting my opportunity to present an innovative modeling approach which estimates solar radiation using PV system power output.

Without further delay, let’s dig right in...

The Premise

A pyranometer!

A pyranometer!

Here’s the premise:  photovoltaic solar panels are not altogether different from pyranometers.  In most cases, pyranometers – the global ‘go-to’ solar radiation sensor – use a silicon wafer to measure the global horizontal irradiance at a given location.  Interestingly, this silicon wafer is based on the same technology as most PV modules.  The result is that the power output from a PV system has a first order relationship with the global solar radiation arriving in the plane-of-array.  As a scientist, this creates a sense of curiousity within me!  I wonder, can we use the power output from a PV system to then work our way back to a standard radiation measurement?

The Hypothesis

Let’s put that pondering within the framework of the scientific method, forming a hypothesis:

“Solar panels are not altogether different from pyranometers, thus one should be able to use their power output as an input to a ‘separation’ model to estimate the diffuse and beam components of solar radiation”

What is a separation model?

Over the past several decades, there have been many (as in hundreds) of models developed for the purpose of separating the diffuse and direct (beam) components from a global horizontal irradiance measurement.  In my recent paper (link), I describe this more thoroughly, so dig into that if you’re interested in learning more (or if you are a bit lost on the above bold terms)

The Engerer 2 model

The Engerer 2 model

What I’ve done with my MODSIM2015 paper, is use my own separation model format – the “Engerer 2” model - and modified it to accept PV power output measurements as the primary input, in place of pyranometer measurements.  The overall goal here, is to estimate the diffuse component of radiation using only the measured PV power output through this model and then test how well it performed.

In this paper, we use a scientific approach to test this.  Two different model formats were tested.  One with a single input, and another with four inputs.  We compare a standard pyranometer based model (Kt), fit to the same data, to the PV power output based model (KPV).  This allows us to directly compare the two approaches in a controlled, scientific fashion. 

Now, in order to do this, we need data.  So, I turned to my usual sources of data, PVOutput.org and the Bureau of Meteorology solar radiation monitoring sites.  Fortunately, I was able to identify 18 PV sites in Adelaide and Melbourne which were of very good quality and located within 10km of a solar radiation monitoring site. 

Next, I processed these data through quality control algorithms (read more in my PhD thesis) and then averaged them to hourly values.  With this QC complete, we’re able to start fitting and testing the models. 

There are a lot of details here, which I’ll glaze over in this blog post, but you can review in our paper.  Such as what QC methods I used, how I chose the PV sites, etc, etc.  I’ll leave it to you to investigate this further!  I’d rather spend my time here talking about what we found, because it is pretty cool.

The Results

First, in fitting and testing the single parameter model, the general approach undertaken was validatedThe idea works.  The relationship used in my diffuse fraction model (Engerer 2) relies on a generalized logistic function based on the relationship between the Diffuse Fraction (Kd) and the Clearness Index (Kt).  However, in the PV power output based model, we use KPV in place of Kt,  KPV is the clear-sky index for photovoltaics, and allows for the removal of all the individual nuances from the PV power output time series (use the link above to learn more)

This is a great start, but the results for a single input model aren’t very good.  This is because the model is too simple for the complex relationship between radiation and cloud cover.  But never fear, that’s where the Engerer 2 model comes in.

For those of you who are not familiar with this field, it's worth mentioning that the Engerer 2 model was recently declared the "quasi-universal" separation model in a global study using 54 sites to test 140 different models in a study by Chris Gueymard, a leader in the solar radiation modelling field.  So we're taking a modern approach to this problem.

Herein, we use the Engerer 2 model format to further test our hypothesis.  Except for the solar PV power output based model, we change another one of the input variables so that it is based on PV power output instead of a pyranometer measurement. This is the deviation from clear-sky variable (read more in the paper).

Kt versus KPV based modelling within the Engerer 2 model framework. 

Kt versus KPV based modelling within the Engerer 2 model framework. 

The results were very encouraging! The figure above demonstrates that by using only the PV power output, we are able to use our generalised logistic function based model to estimate the diffuse fraction of solar radiation with only a 5% increase in rRMSE and 0.7% increase in rMBE over a pyranometer based model.  Moreover, the observed relative errors are within the ‘good’ modeling requirements established by Gueymard and Meyers 2008.

Great, now what?

Moving forward with this result, there is now more work to do! For example, with a pyranometer measurement, once we have the diffuse component, we get the direct component through the closure equation. With PV system power output, this isn't possible.   So we'll need to test our ability to do extract the beam component further.  Possibly, this might require the joint development of another model to extract it

Despite the further work required, the results herein remain exciting. The approach used in our paper show that extracting solar radiation measurements from PV system power output can be done. This is a world first for such a result, and it is exciting to know that further work in this space is realistic. 

I am very hopeful that this result will encourage others in the field to further explore this conceptual work.  If we can extract solar radiation measurements from PV power output, we suddenly can create a very rich dataset of radiation measurements globally, greatly augmenting our ability to anlayse the spatio-temporal evolution of incoming solar radiation.  Such a dataset can assist in the validation of satellite measurements or global climate/meteorological modeling tools. 

So, with this very awesome scientific finding, I’ll leave you with the above thoughts, and hope that you’ll go on to read the paper, cite the results and further develop the prospect of having millions of additional solar radiation sensors globally.

[download the paper]

[download the presentation]

View the presentation below:


What's Happening at #MODSIM2015

Find Stuff!


Solar 2014: Estimating Hourly Energy Generation of Distributed Photovoltaic Arrays

 

If you were tasked with estimating the energy generation from an entire city of PV systems - how would you do it?

A simulation probably jumps into your mind right away.  Scale up a model of PV system performance and that must get you close right?  Well that's a step in the right direction, and you could do that very accurately if you knew the amount of radiation arriving at the surface of all those PV arrays.

But that's a bit trickier than it sounds!  First, where are we going to get an estimate of the available solar radiation at a given location in the city?

The most common answer I get to this is: a pyranometer.  And that's a great start - you'd get a measurement of global horizontal irradiance (GHI) at a point location, which is very helpful.  But you're left with two major problems:

1. How representative is that pyranometer of the rest of the city's radiation resource?  Those clouds are tricky!

2. How do you estimate the amount of radiation arriving on all of those various tilted surfaces around the city?

So, OK, now we need multiple pyranometer sites around the city and at each site we need to tilt and orient them in various directions in order to get a representative sample.

Well, the bad news is that pyranometers cost a few thousand dollars each, need regular cleaning/calibration/maintenance and it's actually pretty difficult to find appropriate sites for them.  If you'd like to find out just how difficult it is to install scientific equipment on buildings - be my guest! (Hint: paperwork, approvals, PITAs galore!).

But I think I've got a better idea...

But I think I've got a better idea - what if we used the photovoltaic arrays that are already installed in a given region as our primary input to our city-scale modelling project?

They're pointed in many different directions, there are many of them already reporting data publicly in real-time, someone else has paid for the equipment AND they're representative of the systems we are trying to estimate in the first place. Sounds like a pretty sweet deal to me!

But, they are subject to shading, soiling and wiring inefficiencies, not to mention that they are not really the most scientific form of equipment.  Still - they are inherently a type of radiation sensor.  And we can probably deal with a lot of those things with some fancy machine-learning algorithms.

So, here is where I introduce our paper:

Estimating Hourly Energy Generation of Distributed Photovoltaic Arrays: a Comparison of Two Methods

J. Tan, N. A. Engerer and F. P. Mills

[download it]

 In it, we compare a two methods for estimating the energy generation of distributed PV arrays.  

The first uses pyranometers, radiation models and PV system modelling for the estimation.

Method 1: Based off of my Masters Thesis (see the 'Publications' page')

Method 1: Based off of my Masters Thesis (see the 'Publications' page')


The second uses a monitored PV system and my KPV methodology to make the estimation.  

Method 2: Based off of my KPV Methodology, Read the Solar Energy journal publication at the 'Publications' page

Method 2: Based off of my KPV Methodology, Read the Solar Energy journal publication at the 'Publications' page

 

I'll let you read the paper to get the details, as that's not the point of blogging (all the boring stuff is for the papers - Ok, I really do actually think that stuff is fun too, #supernerd).  But I will let you know that we've found a few interesting things:

1.  The pyranometer methods does tend to do a bit better (RMSE  15-20% versus 15-25%)

2. BUT when we start to leverage the prolific availability of the PV systems (there are many more of them out there!), we find the KPV method actually does best! (for distances less than 5km) 

3. We actually detected a calibration error in one of the pyranometers using the PV systems - so much for pyranometers being the pinnacle of scientific monitoring!

Overall, I find this result very encouraging.  If we can use PV systems as our primary input to our city-wide modelling idea, then we are one step closer to making the estimate we need.  And we can do it on the cheap - which is really good for solar! 

Now it's time to scale it up, test it on different time scales and handle all those pesky quality control issues.  But don't worry, you can count on me to bring you the results soon! 

Until then, enjoy my new webpage!

-Nick-

 

 

 

/*