Himawari

PVSC44: Himawari 8 enabled real-time distributed PV simulations for distribution networks

Just over a year ago, I announced an exciting new project focused on delivering real-time distributed PV simulations to distribution network service providers (DNSPs). This is an Australian Renewable Energy Agency (ARENA) funded research project, led by the Australian National University. As the Chief Investigator on this research project, I have spent the last year regularly meeting with and networking with DNSPs across the country. As a result, we now have 12 of the 15 DNSPs on board, which is a tremendous outcome.

Each of theses DNSPs is providing us with the metadata for the installed PV system within their networks, which we are using to produce simulations of their power output.  This work extends the efforts of my PhD, where I built the Regional PV Simulation System (RPSS) through collaboration with locality utility ActewAGL, but adds new capability through real-time PV monitoring and derived irradiance estimates from the Himawari 8 satellite.

This week, I am at the PVSC44 conference, co-chairing the session on solar forecasting for grid integration of PV, as well as presenting a paper which outlines our modelling methodology for this new tool. I wanted to take the chance to put up an accompanying blog post, to help make it easily accessible to my fellow researchers. Of course if you want the real details, you should check out the paper :-).  It is important to note that the following methods are the result of a close industry collaboration with industry partner Solcast, and represented the state-of-the-art of its satellite nowcasting technologies from early 2017

[Download the Paper]

Here's how it all works..

Flow diagram of our methodology

Flow diagram of our methodology

We take images from the Himawari-8 satellite, in particular the visible and infrared channels, and apply the general principles of the Heliosat-2 approach for turning these into cloud opacity estimates. What's very interesting about this process, is that the spatial and temporal scales of this data are quite advanced, as compared to other geostationary weather satellites (e.g. GOES 13/15 over the US). At 1-2km^2 resolution with 10 minute update cycles, the radiation modelling enabled by this satellite is quite an exciting advancement for our field.

We use the Himawari 8 data in combination with the GFS NWP model to decompose clouds into lower, middle and upper troposphere layers, and then aggregate these layers to estimate total cloudiness. Total cloud opacity (represented as an index between 0 and 1) is then derived using differences between the lowest visible return value of albedo. With this value in hand, we apply a linear reduction to the Esra clear sky radiation model to produce an estimate of global horizontal irradiance (Gh) for each pixel. Estimates of Gh are afterwards decomposed into direct normal irradiance (Bn) and diffuse horizontal irradiance (Dh) through the Engerer2 separation model.

Satellite imagery decomposition

Satellite imagery decomposition

Radiation Validation on "Estimated Actuals"

estimated actuals

For this conference paper, we have validated the Gh values produced by this system. These "Estimated Actuals" are produced using the satellite scan for each time stamp (no forecasting involved). It is necessary to compare these outputs to the observed radiation conditions at the surface, which we accomplish using data from the Australian Bureau of Meteorology.  Using 7 sites and 2 months of data from the BoM, we were able to determine these estimated actuals to have a Mean Bias Error (MBE) of -7 W m-2 and Root Mean Squared Error (RMSE) of 55 W m-2.  This accomplishes the industry standard of a 'good' radiation model, which is defined in the literature as having an relative MBE less than 5% (1.8% reported) and relative RMSE less than 15% (10.7% reported).

PV Power Estimated Actuals + Validation

The next step in the methodology is to compute PV power output calculations, based on the installed characteristics of the PV systems considered.  This requires information such as the azimuth & tilt of the system, as well as the total installed capacity. We apply the Reindl transposition model to estimate the available plane of array irradiance, followed by a quadratic PV power model to compute the estimated power output. By combining the Estimated Actuals from the satellite system with this model, we also can compute Estimated Actuals for PV system power output.

Screen Shot 2017-06-27 at 14.47.55 pm.png

In order to accomplish this step & validate it, we collected data from 78 PV systems across the Canberra, Australia region for a period of 6 months.  Using these time-series data, we then applied our QCPV methodology to both quality control the reported power outputs, but also to determine the actual azimuths & tilts for the POA radiation modelling step.  For this validation, we report an MBE of 0.04 W/Wp and RMSE of 0.15 W/Wp

Overall scatter in the validation data shows well behaved results, but with accuracy losses nearly double that of the radiation validation. We note that this validation reports uneven bias across the distribution of PV system measurement values, suggesting further refinement of the quadratic PV model coefficients could be required.

Scaling it up for DNSPs

Given the relatively good performance of the modelling system (much improvement needed still for operational purposes) we apply this system to a network wide simulation.  By utilising the metadata for 15,500+ embedded solar PV systems in the ActewAGL distribution network, we produce Estimated Actuals for the PV power systems in this network via this methodology. Additionally, we implement a simple cross-validation using 6 PV systems in a real-time scenario and provide the demo in video format:

In the provided image, we see the Estimated Actuals for each individual PV system in the ActewAGL network, as provided for the 23rd of June 2016. This day was selected based upon its complicated, multi-layered cloud conditions to best demonstrate the capability of this system in a challenging circumstance. A heat map colour profile shows the relative power output from each PV system based on its installed capacity.  This is accompanied by the data from the 6 PV cross-validation sites in the circles which are colour filled according to their observed power output.

At bottom right of this image, the Estimated Actuals for these systems are displayed in grey, with the red line showing the observed power output values. Overall, we observe a slightly positive bias in the Estimated Actuals of 0.038 W/Wp and a relatively lower RMSE value of 0.12 W/Wp.

A Great Start: Where Next?

Overall, our team is quite happy with this first set of results, but we've many improvements to make to this system moving forward. As a part of this validation, we've already found & corrected issues with positive cloud opacity bias during overcast conditions, and we have implemented a dynamic kernel which corrects the cloud shadow displacement at low sun angles. Up next for us, is improving the radiation modelling routines. For one, Engerer2 was not designed for this purpose and our linear reduction on the Esra model needs to be updated.

The good news is, we've filled up our team with postdocs, PhDs, honours students and software developers, and are working hard to make these updates.  Soon, we'll even highlight the forecasting capability we've added to the system in an upcoming issue of Progress in Solar Energy.  In the meantime, we're deploying this system for our participating DNSP partners right now, and will iterate forward based on their feedback. So check back soon and connect with us on Research Gate to stay up to date & to collaborate.

It's all a part of teaming up for that solar-powered future! And in that spirit be sure to check out our data sharing campaign, where we are making the output of this system available for your research purposes. Read more at this link!

Lastly, the ANU team would like thank & acknowledge our industry partner Solcast's contributions to this project. 


A Data Sharing Compaign to Enable Researchers

A Call to University Researchers to "Get Applied"

It's no big secret that Australia's electricity sector is undergoing a rapid transformation, one that continues to surprise even the most aggressive growth targets for solar and storage, and has our regulators and market operators in a very demanding position. High electricity wholesale and spot prices, mean the costs of electricity are going up. Meanwhile, the cost of renewables and storage are falling quickly.  All across Australia, innovative companies are popping up which are providing solutions in the space of demand management, energy storage and energy trading and driving this transformation from the ground up.

in this electricity sector transformation, what role can universities play?

In this mix, I wonder, what role can Universities play? That's what my Academic mindset wants to know: how can we as researchers contribute to this challenging transformation?  I've had the great privilege of traveling much of Australia lately to visit with many segments of the electricity industry, and in each of those instances, I am filled with ideas on where Universities could contribute to the challenges that this sector is facing. 

But in wondering how best to make these relationships, these BIG ideas happen, I find myself exceptionally time-constrained. I simply can't work with everyone directly! Yet I feel a strong drive to help my fellow University researchers reach the real-world - how best to reconcile these?

Solar Data Enables

In contemplating this, I came to an interesting conclusion: high-resolution solar data is actually an enabling resource.  Let me explain what I mean - want to decide how to charge/discharge a battery? You need to know when the sun will or will not shine. Want to build a demand management algorithm for a commercial building? You need to understand how the rooftop solar array's power output will fluctuate during peak demand periods.  Need to orchestrate the dispatch of hundreds of MW of storage technologies? When will the next solar ramp event arrive and drive 100+MW shortfall of supply over the energy market zone you're trading in?

In the future grid, solar data is everything.  In fact, in my recent talk to the Energy Networks Australia "Grid Edge" Event [watch it here], I proposed that solar intermittency is the next frontier of opportunity for future grid technologies.

If we, as Universities, want to contribute to the future grid, this incredible transformation, then the research we complete will inherently require access to good solar data. And by 'data', I more specifically mean solar radiation or solar power generation, at high spatial and temporal resolutions.

Data Access Can Be Hard

However, solar data access is not easy. Most solar data in Australia is based on Typical Meteorological Years (TMY) climatologies, comes from only a handful of solar radiation sites scattered about the continent, or comes bundled on a harddrive at hourly resolutions and full of bias error. Other options include paying significant cash to commercial companies, which are happy to make you pay thousands of dollars for small datasets - no thanks! Shouldn't there be a better solution?

Building a Solar Forecasting System is Not Easy!

Example forecasts for small-scale solar in South Australia from 22 March 2017. Probability bounds are provided in the shaded regions of each forecast.

Example forecasts for small-scale solar in South Australia from 22 March 2017. Probability bounds are provided in the shaded regions of each forecast.

In addition to historical solar radiation data, researchers equally need to be able to access solar forecasting information. If you want to build any of the future grid enabling technologies which seize the opportunity of intermittency, they must be built in a fashion which captures the uncertainty of solar forecasting. 

At ANU/Solcast, we are pretty good at predicting the near-term availability of solar radiation, but we're not perfect either.  Given the stochastic nature of cloud formation & dissipation, their fast-moving, fast-changing characteristics and their complex optical properties, we are never going to get a cloud/solar radiation forecast 100% correct, 100% of the time.  But what we can do, is provide researchers with access to a state-of-the-art solar forecasting technology with world-class accuracy, that includes probabilistic forecast information. This way, they can build tech with these uncertainties built in - a key part of orchestrating the future grid!

Solar Data + Solar Forecasts = Easy to Share

Our Himawari 8 derived solar irradiance datasets at right, compared with the Bureau's old satellite data at left.

Our Himawari 8 derived solar irradiance datasets at right, compared with the Bureau's old satellite data at left.

In my personal quest to enable other researchers, I've realised, that in my position, with my team at the ANU, and our work with our start-up company Solcast, we can fix this problem.  We can openly and freely share solar radiation data, PV system power output datasets and solar forecasts through our joint ARENA project.   As project Chief Investigator, I have decided to inject some ARENA project funds into a data sharing campaign as part of our knowledge sharing efforts, to support the computational requirements of servicing this data to external parties. 

To be clear, that means that YOU, the researcher, get to extract the data for free for R&D purposes

And with the #SolcastAPI it is actually that easy. Within minutes of registering, you can access historical "Estimated Actuals" derived from the Himawari 8 satellite (right now, reaching back 7 days, in the near future - much deeper!) and solar radiation forecasts for GHI, DNI and DHI (powered by my Engerer2 model).  You can also use the API to simulate solar PV system power output, either via the Estimated Actuals or through the forecasts (0-7 days our, 30 minute increments).

To get started, visit the data sharing page I placed on my webpage, to provide more information about this service.  There you'll find some links for getting started. In the near future, I'll add some demos which show how to access this data, and will provide some research ideas for using high-resolution solar data for applied university research.

It's all part of teaming-up for the solar powered future! I look forward to seeing what you come up with!


Find More Stuff:

/*