ramp events

#SWC2015 - Launching the Regional PV Simulation System in Daegu, Korea at the Solar World Congress

Our conference paper is entitled: Real-time Simulations of 15,000+ Distributed PV Arrays at Sub-Grid Level using the Regional PV Simulation System (RPSS)

[download it here]

It's an absolute pleasure to be writing to you from Daegu, Korea, where a few hundred solar energy scientists are gathered for the International Solar Energy Society's 2015 Solar World Congress.  I've been inspired by the work that I've seen here, and I'm quite proud to be a part of this community.  

At this conference, it is my great pleasure to officially launch my latest research outcome, which progresses my PhD thesis out of the laboratory, and into the real-world.  It's been a fun challenge to make this leap from science to technology, and I'm honestly quite enthused by the outcome.

From Science to Tech, How We Did It

RPSS stands for Regional PV Simulation System, which was developed by Nicholas Engerer during his PhD thesis, and is now undergoing further development to achieve operational deployment in Australia.

Over the past several months, I have been working with ANU Engineering student James Hansard to get the Regional PV Simulation System up and running, live, in the Canberra, Australia region.  This has been made possible through several mechanisms, the foremost of which was excellent ingenuity and determination from James (you can download his thesis here). 


There are 100+ sites reporting their power output into the RPSS as of November 2015

There are 100+ sites reporting their power output into the RPSS as of November 2015

Using an allocation on the National Computational Infrastructure, the RPSS is now running in near real-time via a series of three virtual machines.  The first machine is gathering PV power output data in near real-time (about 10-15 minute delay) from 100+ sites via PVOutput.org, a webpage where users publicly report their PV system's power output.  The second machine runs the RPSS algorithms and produce the output files/graphics.  The latter of these three VMs, is a web server, which hosts the latest simulations at http://rpss.info.  At this webpage, you can see the RPSS output for the day, mapped to ActewAGL's distribution network.  

RPSS version 2: 15,000+ PV Systems Simulated by Transformer Node

In the latest version of the RPSS, I am now simulating 15,000+ PV systems, grouped by transformer.  This is a significant advancement from the 12,000+ PV systems mapped by suburb in my PhD thesis, which progresses the RPSS towards industry relevant outcomes.  If we can quantify the contributions of PV to the grid at a high resolution, and forecast any significant changes ahead of time, then the intermittency of distributed PV can be mitigated.  Getting the RPSS running in near real-time is the first step in this process.

Quantifying PV Power Output by Distribution Node (Transformer)

When we break down our simulations by transformer, we start to observe very interesting behaviours from the collectives of distributed PV installed behind them that have not previously been observed in Australia.  Let me show you what I mean!  First, let's watch a simulation (RPSS output) from 5 March 2014:

Now, if you don't think that video is pretty cool, then you must be much less nerdy than I am.  I think its fantastic how, using only input PV data from a 100+ sites, we can observe cloud-shadow like features in the simulation! (That's the power of KPV!)

Now, let's look in detail, at the simulated power output grouped by transformer node (below)..

The three different coloured lines above, each correspond to the simulated power output at a given transformer (node).  These are shown on the right side of the simulation video above.  What is very interesting, is the observed differences in ramp event timing that occur between the three nodes during the strong negative collective ramp event that occurs around 11AM.  From this, we can see that the timing of these events are very important to predict precisely, in both space and time, as the strong ramping can occur at different times for each of them.  In the strong positive ramp that follows, there is a similar mismatch in the timing of the ramp peaks.  For a scientist, this is fascinating.  For a distribution company, its a clear example of the challenge of PV intermittency.  

And all of it motivates further development and deployment of this tool

If you found this post interesting, you can dig into the science a bit deeper by downloading our paper below, or by viewing the slides from the presentation.  There is also a tag cloud below, to help you navigate through other content in my webpage.

[You can download our ISES SWC 2015 paper here]

[Or you can download a PDF of the talk]

Or view it in the slideshow below:

SWC2015 Action:

Tag Cloud (Find More Stuff)

PhD Thesis: What I did, What I found, Why it Matters

Several years ago, I decided I wanted to write a Masters Thesis that didn't just sit on a shelf, collecting dust.  Call me idealistic, but I wanted to adopt a philosophy of "science off the shelf" 

[April 2016 Update: My PhD Thesis is now finalised! Download it here: Part 1  |  Part 2 ]

[download my submitted thesis - part 1]

[download my submitted thesis - part 2]

Back at the University of Oklahoma School of Meteorology, there was a running joke amongst graduate students that they could stick a $20 bill in their thesis, check back in 5-10 years, and find it still nestled between the pages, safe and sound.  While it was humorous, and mostly in jest, it reflected the mentality of postgraduate education/research work - what we were doing was mostly just to tick boxes & get our degree so that we could one day do something that mattered. In reality, our 'research' would sit on a library shelf, acquiring dust.

Yearly kWh production for a simulated 2kW Sanyo array in Oklahoma

Yearly kWh production for a simulated 2kW Sanyo array in Oklahoma

But I wasn't content to resign myself to this fate. I felt compelled to produce something that mattered, that would have real-world relevance.  So I brought on an industry partner, a local solar installer in Oklahoma, and put together a solar map for Oklahoma, not based on solar radiation - but on kWh hours of electricity generation, which immediately translates to payoff times and dollars. [download my M.S. Thesis]

It was a tremendously successful experience, and as I moved forward into my PhD project, I was very determined to not only maintain that approach, but to expand it.  Now, looking back, I can see that approach has paid off in great ways, which I'd like to share with you.  Because what I've discovered, what I've developed and what I am now positioned to do with the technology I created, is all highly relevant, commercially viable and ready to hit the ground running.

What I did and why

Australian PV installations by year, as provided by the Australian PV Institute

There is ~4GW of solar energy installed in Australia, which is dominated by ~1.4 million small-scale photovoltaic (PV) arrays.  These arrays are relatively small (average size 1.5kW in 2011, growing to 4.5kW by the end of 2014), and the vast majority of these installations are un-monitored.  This means that their minute-by-minute performance is not recorded.  In fact, the only information collected for most of these systems is the total quarterly production as reported on electricity bills.  Long story short, this lack of information limits the number of PV systems that can be installed in a given region (like here).

Measured PV power output, divided by that PV systems clear sky power output

Measured PV power output, divided by that PV systems clear sky power output

So what did I do about it? I developed a method for estimating the power output from many thousands of PV systems using a small selection of monitored PV systems (which report their minute-by-minute or hour-by-hour generation).  This method is called  “the clear-sky index for photovoltaics”, KPV. [read more]

Along the way, I had to do significant work in the field of solar radiation modelling, validating clear sky radiation models and developing a new type of solar radiation model fit to Australian radiation data (a "separation model").  I also had to show that my new method (using solar panels as a sensor network) was able to do the job just as well as professional grade radiation equipment.  

Pyranometer based methods versus KPV based methods.  The takeaway? Get the sites close enough (within 5km) and my new method is just as good as using a more sparse network of professional grade solar radiation sensors.  

It was also necessary to develop new quality control routines for the PV system power output data.  This type of data is messy, often provided by non-experts and hasn't been used in the way I've proposed before.  This was accomplished through the development of a new quality control routine called QCPV (now going through the review process in a major journal).

~200 solar PV systems installed in Canberra, which are reporting their data in real-time

Once I sorted out the quality control work, it became possible to work with data from many hundreds of solar PV systems, as well as scale my KPV method up to many thousands of systems.  So I forged ahead, using Canberra as a proof-of-concept, creating a city-wide distributed PV simulation of its 12,000+ embedded PV generators (based on December 2012 installation data).  

I then paired this simulation system with weather events that cause broad-scale, rapid changes in the power output of all of the PV systems at the same time [check it out].  It is these types of events which are the most likely to cause future grid stability problems.  The basic idea is that, when you have a wide-spread solar network, the negative effects of partly cloudy days are "smoothed out" by those systems being positioned over broad region, but during certain weather events, it is not possible to smooth out these impacts, because the cloud features are too widespread, sudden and thick.

"when you have a wide-spread solar network, the effects of partly cloudy days are "smoothed out" by those systems being positioned over large region, but during certain weather events, it is not possible to smooth out these impacts, because the cloud features are too widespread, sudden and thick"

What I did find?

Let me keep this as focused and brief as possible...

KPV estimates (color) versus measurements (black) under a positive ramp event

Firstly, I demonstrated that my newly proposed KPV method was much better than existing methods, showing that it was well-behaved under all cloud cover conditions, and performed well under positive and negative ramp events. [download publication][read blog post]

Next I found that for clear sky radiation models, operating in Australia, global clear sky simulations, are best computed by the Solis, Esra and REST2 approaches, while the Iqbal, Esra and REST2 methods are the most proficient clear sky beam models. [download publication][read blog post]

The Engerer 2 separation model at work (blue model estimates, grey observations)

 After that, I found that only the Perez separation model performed satisfactorily for high resolution (one minute) solar radiation data.  In response to this, I developed three new separation models, which gave slight improvements over the Perez model and greatly exceeded the performance of all other existing model techniques. [download publication]

Once that was handled, I compared radiation sensor based methods to my PV data based approaches, with a student project.  This study found that the approaches were equally as good for separation distances of 5km or less. Given that PV sensors are "cheap" (someone else pays for them) - this was a great finding. [download publication][read blog post]

Post QC KPV estimates, very tight correlation, great results

Then I dug into the development of the QCPV algorithm (quality control), demonstrating that the method I created can result in a 43% reduction in Mean Absolute Percent Error (MAPE) over the raw data. [pre-print coming soon]

Second to last, with another student project, we categorised the weather events that cause those large scale, collective changes in PV power output discussed earlier.  Positive collective ramp events (sudden clearing) were caused by Australian northwest cloud bands and radiation fog dissipation. Negative collective ramp (sudden cloud cover arrival) events were caused most frequently by the passage of cold fronts and thunderstorms.[download manuscript][read blog post]

Finally, I put it all together, with the city-wide PV simulation system, using it to simulate the changes in total power and energy output from these collective ramp events.  I was able, for the first time, to quantify (aka determine a representative number) the amount of power that (dis)appears on the electrical grid during these events.  For example, a thunderstorm event on 19 February 2014 removed 20.78 MW of power generation from the local grid over an 85 minute period, which equates to approximately 14.54 MWh of energy generation forgone over that period.  That's probably enough to change prices on the energy market - not very much, but as the solar installation numbers continue to grow, that influence will grow significantly.  

Here's the thunderstorm ramp event from 19 February 2014, along with some satellite imagery.

What is my overall conclusion?

The overall conclusion, is that the developed regional simulation system for distributed solar PV, made possible by an upscaling of my KPV methodology, represents a significant, unique and promising tool for scientific, engineering and operational purposes.  

In the simplest of terms: I built a very handy tool, with cheap inputs that can be run anywhere that solar PV systems are reporting their power output data.

Where next?

I have a full-time lecturer position ("professor" in the American use of the word) at The Australian National University, where I work in the Fenner School of Environment and Society (employed since July 2013).  I am using the freedom and security this position provides me with, to apply for funding to scale this simulation system up, Australia-wide.  I'll join it with the new Himawari 8/9 satellite data, and pair up with the energy market/utilities in Australia, in order to help large amounts of distributed solar to be added to the grid.  

You could say the future is sunny and bright (#punny).  And with HUGE amounts of solar being installed globally, the solar century is before us.  There are plenty of opportunities for this science to stay off of that dust shelf.  So I'd say, overall, this whole PhD thing has been a smashing success! Even if it was a bit of a wild ride.  I hope to have more exciting news soon - for now, I'll get back to working getting this simulator to run real-time in Canberra...

The Impact of Weather Events on Solar Energy Generation in Canberra

In a wonderful union of my two favourite scientific subjects, I've been working with ANU student Sonya Wellby to identify the weather events that results in large scale, collective PV ramp events in the Canberra region.

And our findings are fun and exciting!  We'll be presenting them today (13 November 2014) to policy members, ActewAGL (the local distributor), fellow scientists and the general public.  More on the venue, timing, etc can be found here

This blog post is meant to play a supporting role to that presentation, so that those of you who are interested in looking at this subject a bit more in-depth, may do so.  We'll be producing a scientific publication, which I will inform you about in early 2015, that will get into the nitty-gritty of what we've done.  I also will post video/slides/audio from today's presentation in another blog post for your reference.

let's get down to business..

There are several key points that help introduce the subject matter:

  1. Solar PV systems have a power output that varies according to the available solar radiation
  2. The supply of solar radiation is often interrupted by clouds
  3. Cloud cover is driven by (and is a part of) meteorological events
  4. With tens of thousand of PV systems installed in relatively dense geographic regions in Australia (e.g. capital cities) - it is possible for a cloud event to cover all of them at once
  5. Weather events are generally repeatable/follow patterns
  6. We decided to identify the weather events that create sudden, broad-scale changes in all the solar PV systems in Canberra

This exercise in identifying and categorising weather events is a first step in learning how much energy generation from PV systems in a region changes during these events, and then predicting the events ahead of time so that the supply-demand of electricity can be appropriately managed. 

A negative ramp event; black line is observed power output from 200+ systems, dotted blue line is clear-sky output.

Tropical, convective clouds move over Canberra from the NE (marked by purpled dot)

significant, collective ramp events

There were 12,000+ solar PV systems installed in Canberra by the end of 2012.  Through cooperation with ActewAGL, our local distributor, we've been provide the postcode and rated capacity of these installations.

We've also been able to collect data from 200+ sites in Canberra which have reported their power output publicly, drawing from a variety of available web servers.  

Using these 200+ sites, Sonya searched through images of their collective power output (meaning all 200+ sites added together) searching for times when the collective power output from these PV systems changed very suddenly - this is termed a ramp event.

what's a ramp event?

I've addressed this topic a few other times on this webpage (see here or here), but I'm a nice guy, so I'll do it again.  A ramp event is a term used to describe a situation where the power output from a  PV system experiences a large change very suddenly.  A positive ramp event implies the power output changes from a low value to a high one, and a negative ramp event is the opposite.

In our case, Sonya was searching for ramp events where all the PV systems experienced a ramp at approximately the same time (we term these 'collective ramp events').  These types of events are important, because they indicate that all the PV systems in the region (12,000+) experience the same change - and that means big changes in the supply-demand balance of the electrical grid.

who cares? 

This type of event doesn't matter unless there is a high penetration of solar PV systems in a given region, meaning that more than 20% of the electricity is being supplied by the collective generation of all the installed systems.  It just so happens that in the ACT, we have a 90% Renewable Energy target (one of the best in the world!).  As a result,  we are installing several large solar farms (20MW plant at Royalla just opened a few months ago) and the uptake of rooftop PV continues to steadily increase.

So if there is anywhere in the world where solar forecasts matter, it's likely that Canberra is it.  

Now I can only say 'likely' because this whole electricity grid thing is really complicated and we simply don't have enough information about the location/arrangement of the PV systems in Canberra to make a final conclusion.  But that's part of what today's presentation is about - working with ActewAGL to make sure the continued integration of heaps of solar in the Territory can continue.

what did we find out?

Over a two year period, Sonya identified 19 positive and 16 negative significant, collective PV ramp events in the ACT.  She then categorised the weather events that lead to these events, which are presented in the following table:

This is a great news, because all of these weather events are things that our weather models and forecasters at the BoM are able to predict with lead-times of several hours.  With that type of information, any time we see that the sun won't shine - we can deliver electrical energy from alternative sources.  On the flip-side, when the clouds are about to clear out - we can stop purchasing electrical energy we won't need.  It's a beautiful thing!

let's take a look at some cool stuff

Enough talk, let's take a look at some of these weather events in action.  The following videos come from my research efforts, using my KPV methodology (link to publication) to simulate all the performance of all of the PV system in the ACT.

negative ramp event - 30 March 2014

In the first video, we have a negative ramp event that results from a thunderstorm event - watch how the clouds move in from the SW to obscure all the PV generators in less than 1 hour!  A 20MW+ change in total power production (I estimate that number increases to 60MW with the addition of 2013/2014 PV installations, including the Royalla solar farm).

Radar at 1:38PM

Radar at 2:38PM

positive ramp event - 30 March 2014

The next video shows a positive ramp event, where fog and low cloud burn off over the course of ~90minutes.  This is a positive feedback cycle, where the more the fog/clouds clear, the more sunlight makes it to the surface and the more rapidly mixing out of the fog/cloud occurs.  It's pretty amazing to watch.  Notice how the fog/low cloud sticks around Lake Burley Griffin/Ginninderra the longest.  Meteorology is awesome!

where to from here?

First, I hope this will encourage cooperation between policy makers, solar farm operators, ActewAGL, the BoM and my research program.  The open sharing of information will allow for world-class research to be undertaken here in the ACT.

Second, we'll use this research and the resulting publication as a launching point for a grant application to ARENA to repeat the same exercise in all high-concentration regions of solar generators (e.g. capital cities).  I've already line-up some students for the initial work.

Third, continue the proliferation of open PV data sharing, particularly from members of the public, via projects such as our machine-learning forecasting efforts, and across research groups. All of my research code will soon be available via an R-package, which I hope will encourage future research like this everywhere its needed.

in closing...

Here's a Brief of Research Findings prepared by Sonya Wellby, who completed the identification of these weather events via a Special Topics Course (ENVS3016) in the Fenner School of Environment and Society at The Australian National University.  She's the most capable Australian student I've ever met - so watch out for her in the future, she's going to make things happen!

That's all for now - stay tuned for the presentation 

Questions, Comments? nicholas.engerer@anu.edu.au

please connect using the social icons at top right and bottom of this page.