PhD Thesis: What I did, What I found, Why it Matters

Several years ago, I decided I wanted to write a Masters Thesis that didn't just sit on a shelf, collecting dust.  Call me idealistic, but I wanted to adopt a philosophy of "science off the shelf" 

[April 2016 Update: My PhD Thesis is now finalised! Download it here: Part 1  |  Part 2 ]

[download my submitted thesis - part 1]

[download my submitted thesis - part 2]

Back at the University of Oklahoma School of Meteorology, there was a running joke amongst graduate students that they could stick a $20 bill in their thesis, check back in 5-10 years, and find it still nestled between the pages, safe and sound.  While it was humorous, and mostly in jest, it reflected the mentality of postgraduate education/research work - what we were doing was mostly just to tick boxes & get our degree so that we could one day do something that mattered. In reality, our 'research' would sit on a library shelf, acquiring dust.

Yearly kWh production for a simulated 2kW Sanyo array in Oklahoma

Yearly kWh production for a simulated 2kW Sanyo array in Oklahoma

But I wasn't content to resign myself to this fate. I felt compelled to produce something that mattered, that would have real-world relevance.  So I brought on an industry partner, a local solar installer in Oklahoma, and put together a solar map for Oklahoma, not based on solar radiation - but on kWh hours of electricity generation, which immediately translates to payoff times and dollars. [download my M.S. Thesis]

It was a tremendously successful experience, and as I moved forward into my PhD project, I was very determined to not only maintain that approach, but to expand it.  Now, looking back, I can see that approach has paid off in great ways, which I'd like to share with you.  Because what I've discovered, what I've developed and what I am now positioned to do with the technology I created, is all highly relevant, commercially viable and ready to hit the ground running.

What I did and why

Australian PV installations by year, as provided by the Australian PV Institute

There is ~4GW of solar energy installed in Australia, which is dominated by ~1.4 million small-scale photovoltaic (PV) arrays.  These arrays are relatively small (average size 1.5kW in 2011, growing to 4.5kW by the end of 2014), and the vast majority of these installations are un-monitored.  This means that their minute-by-minute performance is not recorded.  In fact, the only information collected for most of these systems is the total quarterly production as reported on electricity bills.  Long story short, this lack of information limits the number of PV systems that can be installed in a given region (like here).

Measured PV power output, divided by that PV systems clear sky power output

Measured PV power output, divided by that PV systems clear sky power output

So what did I do about it? I developed a method for estimating the power output from many thousands of PV systems using a small selection of monitored PV systems (which report their minute-by-minute or hour-by-hour generation).  This method is called  “the clear-sky index for photovoltaics”, KPV. [read more]

Along the way, I had to do significant work in the field of solar radiation modelling, validating clear sky radiation models and developing a new type of solar radiation model fit to Australian radiation data (a "separation model").  I also had to show that my new method (using solar panels as a sensor network) was able to do the job just as well as professional grade radiation equipment.  

Pyranometer based methods versus KPV based methods.  The takeaway? Get the sites close enough (within 5km) and my new method is just as good as using a more sparse network of professional grade solar radiation sensors.  

It was also necessary to develop new quality control routines for the PV system power output data.  This type of data is messy, often provided by non-experts and hasn't been used in the way I've proposed before.  This was accomplished through the development of a new quality control routine called QCPV (now going through the review process in a major journal).

~200 solar PV systems installed in Canberra, which are reporting their data in real-time

Once I sorted out the quality control work, it became possible to work with data from many hundreds of solar PV systems, as well as scale my KPV method up to many thousands of systems.  So I forged ahead, using Canberra as a proof-of-concept, creating a city-wide distributed PV simulation of its 12,000+ embedded PV generators (based on December 2012 installation data).  

I then paired this simulation system with weather events that cause broad-scale, rapid changes in the power output of all of the PV systems at the same time [check it out].  It is these types of events which are the most likely to cause future grid stability problems.  The basic idea is that, when you have a wide-spread solar network, the negative effects of partly cloudy days are "smoothed out" by those systems being positioned over broad region, but during certain weather events, it is not possible to smooth out these impacts, because the cloud features are too widespread, sudden and thick.

"when you have a wide-spread solar network, the effects of partly cloudy days are "smoothed out" by those systems being positioned over large region, but during certain weather events, it is not possible to smooth out these impacts, because the cloud features are too widespread, sudden and thick"

What I did find?

Let me keep this as focused and brief as possible...

KPV estimates (color) versus measurements (black) under a positive ramp event

Firstly, I demonstrated that my newly proposed KPV method was much better than existing methods, showing that it was well-behaved under all cloud cover conditions, and performed well under positive and negative ramp events. [download publication][read blog post]

Next I found that for clear sky radiation models, operating in Australia, global clear sky simulations, are best computed by the Solis, Esra and REST2 approaches, while the Iqbal, Esra and REST2 methods are the most proficient clear sky beam models. [download publication][read blog post]

The Engerer 2 separation model at work (blue model estimates, grey observations)

 After that, I found that only the Perez separation model performed satisfactorily for high resolution (one minute) solar radiation data.  In response to this, I developed three new separation models, which gave slight improvements over the Perez model and greatly exceeded the performance of all other existing model techniques. [download publication]

Once that was handled, I compared radiation sensor based methods to my PV data based approaches, with a student project.  This study found that the approaches were equally as good for separation distances of 5km or less. Given that PV sensors are "cheap" (someone else pays for them) - this was a great finding. [download publication][read blog post]

Post QC KPV estimates, very tight correlation, great results

Then I dug into the development of the QCPV algorithm (quality control), demonstrating that the method I created can result in a 43% reduction in Mean Absolute Percent Error (MAPE) over the raw data. [pre-print coming soon]

Second to last, with another student project, we categorised the weather events that cause those large scale, collective changes in PV power output discussed earlier.  Positive collective ramp events (sudden clearing) were caused by Australian northwest cloud bands and radiation fog dissipation. Negative collective ramp (sudden cloud cover arrival) events were caused most frequently by the passage of cold fronts and thunderstorms.[download manuscript][read blog post]

Finally, I put it all together, with the city-wide PV simulation system, using it to simulate the changes in total power and energy output from these collective ramp events.  I was able, for the first time, to quantify (aka determine a representative number) the amount of power that (dis)appears on the electrical grid during these events.  For example, a thunderstorm event on 19 February 2014 removed 20.78 MW of power generation from the local grid over an 85 minute period, which equates to approximately 14.54 MWh of energy generation forgone over that period.  That's probably enough to change prices on the energy market - not very much, but as the solar installation numbers continue to grow, that influence will grow significantly.  

Here's the thunderstorm ramp event from 19 February 2014, along with some satellite imagery.

What is my overall conclusion?

The overall conclusion, is that the developed regional simulation system for distributed solar PV, made possible by an upscaling of my KPV methodology, represents a significant, unique and promising tool for scientific, engineering and operational purposes.  

In the simplest of terms: I built a very handy tool, with cheap inputs that can be run anywhere that solar PV systems are reporting their power output data.

Where next?

I have a full-time lecturer position ("professor" in the American use of the word) at The Australian National University, where I work in the Fenner School of Environment and Society (employed since July 2013).  I am using the freedom and security this position provides me with, to apply for funding to scale this simulation system up, Australia-wide.  I'll join it with the new Himawari 8/9 satellite data, and pair up with the energy market/utilities in Australia, in order to help large amounts of distributed solar to be added to the grid.  

You could say the future is sunny and bright (#punny).  And with HUGE amounts of solar being installed globally, the solar century is before us.  There are plenty of opportunities for this science to stay off of that dust shelf.  So I'd say, overall, this whole PhD thing has been a smashing success! Even if it was a bit of a wild ride.  I hope to have more exciting news soon - for now, I'll get back to working getting this simulator to run real-time in Canberra...

The Impact of Weather Events on Solar Energy Generation: Recorded Presentation

Happy new year! As promised, here is a recorded version of our November 2014 presentation on weather events and solar energy generation in Canberra, Australia.

Please use the below recording from SoundCloud (press play!) and the below gallery of slides (click through them manually) to review our presentation.  There are two videos that will not work within the Gallery, but that's okay!  You can find them here, along with a summary of our research findings.

[download the presentation here]


The Impact of Weather Events on Solar Energy Generation in Canberra

In a wonderful union of my two favourite scientific subjects, I've been working with ANU student Sonya Wellby to identify the weather events that results in large scale, collective PV ramp events in the Canberra region.

And our findings are fun and exciting!  We'll be presenting them today (13 November 2014) to policy members, ActewAGL (the local distributor), fellow scientists and the general public.  More on the venue, timing, etc can be found here

This blog post is meant to play a supporting role to that presentation, so that those of you who are interested in looking at this subject a bit more in-depth, may do so.  We'll be producing a scientific publication, which I will inform you about in early 2015, that will get into the nitty-gritty of what we've done.  I also will post video/slides/audio from today's presentation in another blog post for your reference.

let's get down to business..

There are several key points that help introduce the subject matter:

  1. Solar PV systems have a power output that varies according to the available solar radiation
  2. The supply of solar radiation is often interrupted by clouds
  3. Cloud cover is driven by (and is a part of) meteorological events
  4. With tens of thousand of PV systems installed in relatively dense geographic regions in Australia (e.g. capital cities) - it is possible for a cloud event to cover all of them at once
  5. Weather events are generally repeatable/follow patterns
  6. We decided to identify the weather events that create sudden, broad-scale changes in all the solar PV systems in Canberra

This exercise in identifying and categorising weather events is a first step in learning how much energy generation from PV systems in a region changes during these events, and then predicting the events ahead of time so that the supply-demand of electricity can be appropriately managed. 

A negative ramp event; black line is observed power output from 200+ systems, dotted blue line is clear-sky output.

Tropical, convective clouds move over Canberra from the NE (marked by purpled dot)

significant, collective ramp events

There were 12,000+ solar PV systems installed in Canberra by the end of 2012.  Through cooperation with ActewAGL, our local distributor, we've been provide the postcode and rated capacity of these installations.

We've also been able to collect data from 200+ sites in Canberra which have reported their power output publicly, drawing from a variety of available web servers.  

Using these 200+ sites, Sonya searched through images of their collective power output (meaning all 200+ sites added together) searching for times when the collective power output from these PV systems changed very suddenly - this is termed a ramp event.

what's a ramp event?

I've addressed this topic a few other times on this webpage (see here or here), but I'm a nice guy, so I'll do it again.  A ramp event is a term used to describe a situation where the power output from a  PV system experiences a large change very suddenly.  A positive ramp event implies the power output changes from a low value to a high one, and a negative ramp event is the opposite.

In our case, Sonya was searching for ramp events where all the PV systems experienced a ramp at approximately the same time (we term these 'collective ramp events').  These types of events are important, because they indicate that all the PV systems in the region (12,000+) experience the same change - and that means big changes in the supply-demand balance of the electrical grid.

who cares? 

This type of event doesn't matter unless there is a high penetration of solar PV systems in a given region, meaning that more than 20% of the electricity is being supplied by the collective generation of all the installed systems.  It just so happens that in the ACT, we have a 90% Renewable Energy target (one of the best in the world!).  As a result,  we are installing several large solar farms (20MW plant at Royalla just opened a few months ago) and the uptake of rooftop PV continues to steadily increase.

So if there is anywhere in the world where solar forecasts matter, it's likely that Canberra is it.  

Now I can only say 'likely' because this whole electricity grid thing is really complicated and we simply don't have enough information about the location/arrangement of the PV systems in Canberra to make a final conclusion.  But that's part of what today's presentation is about - working with ActewAGL to make sure the continued integration of heaps of solar in the Territory can continue.

what did we find out?

Over a two year period, Sonya identified 19 positive and 16 negative significant, collective PV ramp events in the ACT.  She then categorised the weather events that lead to these events, which are presented in the following table:

This is a great news, because all of these weather events are things that our weather models and forecasters at the BoM are able to predict with lead-times of several hours.  With that type of information, any time we see that the sun won't shine - we can deliver electrical energy from alternative sources.  On the flip-side, when the clouds are about to clear out - we can stop purchasing electrical energy we won't need.  It's a beautiful thing!

let's take a look at some cool stuff

Enough talk, let's take a look at some of these weather events in action.  The following videos come from my research efforts, using my KPV methodology (link to publication) to simulate all the performance of all of the PV system in the ACT.

negative ramp event - 30 March 2014

In the first video, we have a negative ramp event that results from a thunderstorm event - watch how the clouds move in from the SW to obscure all the PV generators in less than 1 hour!  A 20MW+ change in total power production (I estimate that number increases to 60MW with the addition of 2013/2014 PV installations, including the Royalla solar farm).

Radar at 1:38PM

Radar at 2:38PM

positive ramp event - 30 March 2014

The next video shows a positive ramp event, where fog and low cloud burn off over the course of ~90minutes.  This is a positive feedback cycle, where the more the fog/clouds clear, the more sunlight makes it to the surface and the more rapidly mixing out of the fog/cloud occurs.  It's pretty amazing to watch.  Notice how the fog/low cloud sticks around Lake Burley Griffin/Ginninderra the longest.  Meteorology is awesome!

where to from here?

First, I hope this will encourage cooperation between policy makers, solar farm operators, ActewAGL, the BoM and my research program.  The open sharing of information will allow for world-class research to be undertaken here in the ACT.

Second, we'll use this research and the resulting publication as a launching point for a grant application to ARENA to repeat the same exercise in all high-concentration regions of solar generators (e.g. capital cities).  I've already line-up some students for the initial work.

Third, continue the proliferation of open PV data sharing, particularly from members of the public, via projects such as our machine-learning forecasting efforts, and across research groups. All of my research code will soon be available via an R-package, which I hope will encourage future research like this everywhere its needed.

in closing...

Here's a Brief of Research Findings prepared by Sonya Wellby, who completed the identification of these weather events via a Special Topics Course (ENVS3016) in the Fenner School of Environment and Society at The Australian National University.  She's the most capable Australian student I've ever met - so watch out for her in the future, she's going to make things happen!

That's all for now - stay tuned for the presentation 

Questions, Comments?

please connect using the social icons at top right and bottom of this page.

Solar 2014: Which Clear-Sky Radiation Model is Best for Use in Australia?

Update 2015: This research is now available as a paper in Solar Energy journal!

[download paper]

Citation: Engerer, Nicholas A and Franklin P. Mills. Validating Nine Clear Sky Radiation Models in Australia. Solar Energy. 120, October 2015, pp. 9-24.

Have you ever wondered which clear-sky solar radiation model you should use in your research project or solar energy simulation?  When I was designing my KPV method for estimating PV system power output I needed to figure out which clear-sky model would be the best one to use.  But there was a problem - I couldn't find a single validation study for clear-sky radiation modelling for Australia!

So in the paper, I had to do a quick model validation using one year of radiation data from Wagga Wagga, from which I decided on the Esra model.  But that simple validation left me wondering which model REALLY was the best?  

Thus I embarked on a scientific journey to discover which model was the best for use in Australia using the solar radiation data from 14 sites in Australia:


First, I set a few ground rules.  I wasn't going to use any radiation model that was overly complicated, nor was I going to use atmospheric variables that were difficult to obtain.

This meant using climatological values for input values such as the Linke Turbidity coefficient or ozone content - rather than using direct measurements from a photometer (because who honestly has spectral data?).  I think this is important, because a validation study should focus on models that are widely applicable so that it is widely useful.

In the end, I chose nine models from the options from both beam (direct) and global radiation: 

In the many Australian presentations and publications I've read and attended over the past few years, the most common clear-sky radiation model used is the Ineichen-Perez model.  

However, my research shows that the Ineichen-Perez model is not even in the top-three best choices.  So we really shouldn't be using it in our research as it is introducing unnecessary errors into our collective research knowledge.

What models are the best to use?  For the beam models, the top three choices are the Iqbal-C, Esra and REST2 models.  And for the global models, they are the Solis, Esra and REST2 models.

If we were to chose the best overall model, that would be the Esra model.  Which edges out the REST2 model, due to is large errors at high zenith angles.

You can read more about this in my Solar 2014 Poster Presentation [direct download] and the hopefully an upcoming article in Solar Energy journal (fingers crossed).

Until then, the quick answer is...

The best clear-sky radiation model for use in Australia is the Esra model!