Outreach

A Data Sharing Compaign to Enable Researchers

A Call to University Researchers to "Get Applied"

It's no big secret that Australia's electricity sector is undergoing a rapid transformation, one that continues to surprise even the most aggressive growth targets for solar and storage, and has our regulators and market operators in a very demanding position. High electricity wholesale and spot prices, mean the costs of electricity are going up. Meanwhile, the cost of renewables and storage are falling quickly.  All across Australia, innovative companies are popping up which are providing solutions in the space of demand management, energy storage and energy trading and driving this transformation from the ground up.

in this electricity sector transformation, what role can universities play?

In this mix, I wonder, what role can Universities play? That's what my Academic mindset wants to know: how can we as researchers contribute to this challenging transformation?  I've had the great privilege of traveling much of Australia lately to visit with many segments of the electricity industry, and in each of those instances, I am filled with ideas on where Universities could contribute to the challenges that this sector is facing. 

But in wondering how best to make these relationships, these BIG ideas happen, I find myself exceptionally time-constrained. I simply can't work with everyone directly! Yet I feel a strong drive to help my fellow University researchers reach the real-world - how best to reconcile these?

Solar Data Enables

In contemplating this, I came to an interesting conclusion: high-resolution solar data is actually an enabling resource.  Let me explain what I mean - want to decide how to charge/discharge a battery? You need to know when the sun will or will not shine. Want to build a demand management algorithm for a commercial building? You need to understand how the rooftop solar array's power output will fluctuate during peak demand periods.  Need to orchestrate the dispatch of hundreds of MW of storage technologies? When will the next solar ramp event arrive and drive 100+MW shortfall of supply over the energy market zone you're trading in?

In the future grid, solar data is everything.  In fact, in my recent talk to the Energy Networks Australia "Grid Edge" Event [watch it here], I proposed that solar intermittency is the next frontier of opportunity for future grid technologies.

If we, as Universities, want to contribute to the future grid, this incredible transformation, then the research we complete will inherently require access to good solar data. And by 'data', I more specifically mean solar radiation or solar power generation, at high spatial and temporal resolutions.

Data Access Can Be Hard

However, solar data access is not easy. Most solar data in Australia is based on Typical Meteorological Years (TMY) climatologies, comes from only a handful of solar radiation sites scattered about the continent, or comes bundled on a harddrive at hourly resolutions and full of bias error. Other options include paying significant cash to commercial companies, which are happy to make you pay thousands of dollars for small datasets - no thanks! Shouldn't there be a better solution?

Building a Solar Forecasting System is Not Easy!

 Example forecasts for small-scale solar in South Australia from 22 March 2017. Probability bounds are provided in the shaded regions of each forecast.

Example forecasts for small-scale solar in South Australia from 22 March 2017. Probability bounds are provided in the shaded regions of each forecast.

In addition to historical solar radiation data, researchers equally need to be able to access solar forecasting information. If you want to build any of the future grid enabling technologies which seize the opportunity of intermittency, they must be built in a fashion which captures the uncertainty of solar forecasting. 

At ANU/Solcast, we are pretty good at predicting the near-term availability of solar radiation, but we're not perfect either.  Given the stochastic nature of cloud formation & dissipation, their fast-moving, fast-changing characteristics and their complex optical properties, we are never going to get a cloud/solar radiation forecast 100% correct, 100% of the time.  But what we can do, is provide researchers with access to a state-of-the-art solar forecasting technology with world-class accuracy, that includes probabilistic forecast information. This way, they can build tech with these uncertainties built in - a key part of orchestrating the future grid!

Solar Data + Solar Forecasts = Easy to Share

 Our Himawari 8 derived solar irradiance datasets at right, compared with the Bureau's old satellite data at left.

Our Himawari 8 derived solar irradiance datasets at right, compared with the Bureau's old satellite data at left.

In my personal quest to enable other researchers, I've realised, that in my position, with my team at the ANU, and our work with our start-up company Solcast, we can fix this problem.  We can openly and freely share solar radiation data, PV system power output datasets and solar forecasts through our joint ARENA project.   As project Chief Investigator, I have decided to inject some ARENA project funds into a data sharing campaign as part of our knowledge sharing efforts, to support the computational requirements of servicing this data to external parties. 

To be clear, that means that YOU, the researcher, get to extract the data for free for R&D purposes

And with the #SolcastAPI it is actually that easy. Within minutes of registering, you can access historical "Estimated Actuals" derived from the Himawari 8 satellite (right now, reaching back 7 days, in the near future - much deeper!) and solar radiation forecasts for GHI, DNI and DHI (powered by my Engerer2 model).  You can also use the API to simulate solar PV system power output, either via the Estimated Actuals or through the forecasts (0-7 days our, 30 minute increments).

To get started, visit the data sharing page I placed on my webpage, to provide more information about this service.  There you'll find some links for getting started. In the near future, I'll add some demos which show how to access this data, and will provide some research ideas for using high-resolution solar data for applied university research.

It's all part of teaming-up for the solar powered future! I look forward to seeing what you come up with!


Find More Stuff:

#saveARENA - There Has Never Been a More Critical Time for Renewables Research in Australia

I Have an Urgent & Important Message for You about The Australian Renewable Energy Agency

For those of you who follow my page, or have adventured here from beyond, it should be readily apparent to you that I’m an academic working in solar energy related research, more specifically in the modelling & forecasting of distributed PV power output.

To date, the majority of my work has been funded by ARENA, the Australian Renewable Energy Agency.  A funding body that has been absolutely crucial in supporting Australia as a global leader in renewable energy research over the past several years.

But ARENA is Under Imminent Threat

But right now, ARENA is under threat. The Coalition Government is planning legislation that will strip it of more than $1B of funding, and Labor are saying they won’t oppose it.

Let me explain to you why these cuts to ARENA research funding are a bad idea.

ARENA is currently funding nine university led research projects across Australia with more than $17M in funding, which are focused on partnering industry and researchers together to solve the major challenges we face integrating renewables into our electricity grid. What’s more, they intend to produce commercial outcomes valuable to the Australian economy.  (Isn't this exactly what the Australian government wants universities to accomplish? i.e. the National Innovation Science Agenda)

As an example, look no further than my own project, which received $1M from ARENA and raised $300k of industry cash to build distributed PV modelling and forecasting software for six distribution networks: ActewAGL, Ergon, Essential Energy, Western Power Horizon Power & Power and Water.

 Perhaps this isn't quite enough industry partnership activity for the Coalition government?

Perhaps this isn't quite enough industry partnership activity for the Coalition government?

Don't believe the lies about a lack of value: ARENA has shifted toward driving commercially-focused outcomes

At the ANU, we're not just developing this distributed PV modelling technology to write some fancy publications.  We are delivering our software into the operations of our distribution network partners and creating a technology that will enable higher penetrations of solar to be added to the grid.  Going one step further, this software is being commercialised by an ANU start-up company, which we have already formed and is starting to complete this work now.  In the future, we hope to take this technology beyond Australia, and into other countries around the world.

Without ARENA funded research like mine, fewer solar PV systems will be permitted on our distribution networks.  Without research projects like the other eight industry-researcher collaborations in ARENA's R&D funding round, what other innovative technologies will Australia miss out on

Some Awesome Examples of Current ARENA R&D:

ARENA & Industry Funded: Drones that survey solar farms

ARENA & Industry Funded: Integrating solar thermal into alumina processing

ARENA & Industry Funded: Batteries & Solar PV into Apartment Buildings

If my project, and others like it, aren’t part of the “Innovation” agenda that the government wants, part of its “Ideas Boom”, part of the technology driven future of the Australian economy where it produces new ideas of global relevance, I don’t know what is.

If these projects aren't part of the "Ideas Boom"... I don't know what is!

We simply have a government that is demanding innovation on one hand, but clearly saying “just not for renewables” on the other.

What do we do about it?

Across Australia, our renewables powered people groups are springing into action, and our universities aren't letting this go down without a fight:

Solar Citizens has made it easy for you to tell your local parliamentarian to stop this madness

GetUp has put together a #saveARENA campaign where you can share your story about working on an ARENA funded project to your social media accounts.

I want YOU to join with me to #saveARENA

So I want YOU, to join with me, to tell our Parliament to #saveARENA. The legislation to cut $1B from ARENA is set to be introduced to Parliament early next week as part of an “omnibus bill”, so we must act now.

Let me be clear: There has never been a more critical moment for the future of renewables in Australia, or for our potential to be a leader in renewable energy technologies throughout the Asia-Pacific, and more broadly, across the globe.

Renewably Yours,

Dr. Nick Engerer


Be sure to retweet my twitter video:

And Follow #saveARENA on Twitter:

 

 

PhD Thesis: What I did, What I found, Why it Matters

Several years ago, I decided I wanted to write a Masters Thesis that didn't just sit on a shelf, collecting dust.  Call me idealistic, but I wanted to adopt a philosophy of "science off the shelf" 

[April 2016 Update: My PhD Thesis is now finalised! Download it here: Part 1  |  Part 2 ]
 


[download my submitted thesis - part 1]

[download my submitted thesis - part 2]

Back at the University of Oklahoma School of Meteorology, there was a running joke amongst graduate students that they could stick a $20 bill in their thesis, check back in 5-10 years, and find it still nestled between the pages, safe and sound.  While it was humorous, and mostly in jest, it reflected the mentality of postgraduate education/research work - what we were doing was mostly just to tick boxes & get our degree so that we could one day do something that mattered. In reality, our 'research' would sit on a library shelf, acquiring dust.

 Yearly kWh production for a simulated 2kW Sanyo array in Oklahoma

Yearly kWh production for a simulated 2kW Sanyo array in Oklahoma

But I wasn't content to resign myself to this fate. I felt compelled to produce something that mattered, that would have real-world relevance.  So I brought on an industry partner, a local solar installer in Oklahoma, and put together a solar map for Oklahoma, not based on solar radiation - but on kWh hours of electricity generation, which immediately translates to payoff times and dollars. [download my M.S. Thesis]

It was a tremendously successful experience, and as I moved forward into my PhD project, I was very determined to not only maintain that approach, but to expand it.  Now, looking back, I can see that approach has paid off in great ways, which I'd like to share with you.  Because what I've discovered, what I've developed and what I am now positioned to do with the technology I created, is all highly relevant, commercially viable and ready to hit the ground running.

What I did and why

Australian PV installations by year, as provided by the Australian PV Institute

There is ~4GW of solar energy installed in Australia, which is dominated by ~1.4 million small-scale photovoltaic (PV) arrays.  These arrays are relatively small (average size 1.5kW in 2011, growing to 4.5kW by the end of 2014), and the vast majority of these installations are un-monitored.  This means that their minute-by-minute performance is not recorded.  In fact, the only information collected for most of these systems is the total quarterly production as reported on electricity bills.  Long story short, this lack of information limits the number of PV systems that can be installed in a given region (like here).

 Measured PV power output, divided by that PV systems clear sky power output

Measured PV power output, divided by that PV systems clear sky power output

So what did I do about it? I developed a method for estimating the power output from many thousands of PV systems using a small selection of monitored PV systems (which report their minute-by-minute or hour-by-hour generation).  This method is called  “the clear-sky index for photovoltaics”, KPV. [read more]

Along the way, I had to do significant work in the field of solar radiation modelling, validating clear sky radiation models and developing a new type of solar radiation model fit to Australian radiation data (a "separation model").  I also had to show that my new method (using solar panels as a sensor network) was able to do the job just as well as professional grade radiation equipment.  

Pyranometer based methods versus KPV based methods.  The takeaway? Get the sites close enough (within 5km) and my new method is just as good as using a more sparse network of professional grade solar radiation sensors.  

It was also necessary to develop new quality control routines for the PV system power output data.  This type of data is messy, often provided by non-experts and hasn't been used in the way I've proposed before.  This was accomplished through the development of a new quality control routine called QCPV (now going through the review process in a major journal).

~200 solar PV systems installed in Canberra, which are reporting their data in real-time

Once I sorted out the quality control work, it became possible to work with data from many hundreds of solar PV systems, as well as scale my KPV method up to many thousands of systems.  So I forged ahead, using Canberra as a proof-of-concept, creating a city-wide distributed PV simulation of its 12,000+ embedded PV generators (based on December 2012 installation data).  

I then paired this simulation system with weather events that cause broad-scale, rapid changes in the power output of all of the PV systems at the same time [check it out].  It is these types of events which are the most likely to cause future grid stability problems.  The basic idea is that, when you have a wide-spread solar network, the negative effects of partly cloudy days are "smoothed out" by those systems being positioned over broad region, but during certain weather events, it is not possible to smooth out these impacts, because the cloud features are too widespread, sudden and thick.

"when you have a wide-spread solar network, the effects of partly cloudy days are "smoothed out" by those systems being positioned over large region, but during certain weather events, it is not possible to smooth out these impacts, because the cloud features are too widespread, sudden and thick"

What I did find?

Let me keep this as focused and brief as possible...

KPV estimates (color) versus measurements (black) under a positive ramp event

Firstly, I demonstrated that my newly proposed KPV method was much better than existing methods, showing that it was well-behaved under all cloud cover conditions, and performed well under positive and negative ramp events. [download publication][read blog post]

Next I found that for clear sky radiation models, operating in Australia, global clear sky simulations, are best computed by the Solis, Esra and REST2 approaches, while the Iqbal, Esra and REST2 methods are the most proficient clear sky beam models. [download publication][read blog post]

The Engerer 2 separation model at work (blue model estimates, grey observations)

 After that, I found that only the Perez separation model performed satisfactorily for high resolution (one minute) solar radiation data.  In response to this, I developed three new separation models, which gave slight improvements over the Perez model and greatly exceeded the performance of all other existing model techniques. [download publication]

Once that was handled, I compared radiation sensor based methods to my PV data based approaches, with a student project.  This study found that the approaches were equally as good for separation distances of 5km or less. Given that PV sensors are "cheap" (someone else pays for them) - this was a great finding. [download publication][read blog post]

Post QC KPV estimates, very tight correlation, great results

Then I dug into the development of the QCPV algorithm (quality control), demonstrating that the method I created can result in a 43% reduction in Mean Absolute Percent Error (MAPE) over the raw data. [pre-print coming soon]

Second to last, with another student project, we categorised the weather events that cause those large scale, collective changes in PV power output discussed earlier.  Positive collective ramp events (sudden clearing) were caused by Australian northwest cloud bands and radiation fog dissipation. Negative collective ramp (sudden cloud cover arrival) events were caused most frequently by the passage of cold fronts and thunderstorms.[download manuscript][read blog post]

Finally, I put it all together, with the city-wide PV simulation system, using it to simulate the changes in total power and energy output from these collective ramp events.  I was able, for the first time, to quantify (aka determine a representative number) the amount of power that (dis)appears on the electrical grid during these events.  For example, a thunderstorm event on 19 February 2014 removed 20.78 MW of power generation from the local grid over an 85 minute period, which equates to approximately 14.54 MWh of energy generation forgone over that period.  That's probably enough to change prices on the energy market - not very much, but as the solar installation numbers continue to grow, that influence will grow significantly.  

Here's the thunderstorm ramp event from 19 February 2014, along with some satellite imagery.

What is my overall conclusion?

The overall conclusion, is that the developed regional simulation system for distributed solar PV, made possible by an upscaling of my KPV methodology, represents a significant, unique and promising tool for scientific, engineering and operational purposes.  

In the simplest of terms: I built a very handy tool, with cheap inputs that can be run anywhere that solar PV systems are reporting their power output data.

Where next?

I have a full-time lecturer position ("professor" in the American use of the word) at The Australian National University, where I work in the Fenner School of Environment and Society (employed since July 2013).  I am using the freedom and security this position provides me with, to apply for funding to scale this simulation system up, Australia-wide.  I'll join it with the new Himawari 8/9 satellite data, and pair up with the energy market/utilities in Australia, in order to help large amounts of distributed solar to be added to the grid.  

You could say the future is sunny and bright (#punny).  And with HUGE amounts of solar being installed globally, the solar century is before us.  There are plenty of opportunities for this science to stay off of that dust shelf.  So I'd say, overall, this whole PhD thing has been a smashing success! Even if it was a bit of a wild ride.  I hope to have more exciting news soon - for now, I'll get back to working getting this simulator to run real-time in Canberra...

The Impact of Weather Events on Solar Energy Generation: Recorded Presentation

Happy new year! As promised, here is a recorded version of our November 2014 presentation on weather events and solar energy generation in Canberra, Australia.

Please use the below recording from SoundCloud (press play!) and the below gallery of slides (click through them manually) to review our presentation.  There are two videos that will not work within the Gallery, but that's okay!  You can find them here, along with a summary of our research findings.

[download the presentation here]

 
 

Introducing a Community Powered Solar Energy Forecasting Project for Canberra

In Our Renewables-Powered Future - Rooftop Solar Energy Will Play a Dominant Role.

In January 2014, Australia had nearly 1.2 million solar energy installations, with nearly all of them on rooftops.  Their combined capacity now exceeds 3GW, and is forecast to grow to as much as 23GW  by 2030 - with most of the new installations showing up on Australian homes and businesses.  

Now, if those numbers don't mean much to you - let's just say that's an enormous amount of energy generation and has very significant consequences for the way the electricity market in Australia works.

As I want to keep this post short and straight to the point, I won't go into the specifics of how, why and when high penetrations of distributed PV systems will start causing problems - but we're getting very close.

This is precisely why ARENA has funded our distributed solar energy forecasting project

Starting now, we're deploying up to 300 data loggers to homes with solar PV systems in the ACT region.  Our aim is to cover all of Canberra with a 'sensor' network of rooftop PV systems reporting live data.  By joining this dataset with the deployment of 10-12 sky imagers - we're going to be able to predict how and when the total solar power output of a city changes in real-time.  Once we prove we can do it - we'll replicate the project across all major cities in Australia.

Not convinced?  Let me give you two concrete examples that show why this project is important:

Consider these two scenarios:

  1. Clearing fog results in a rapid increase in PV power production across the whole of Canberra in less than an hour.
  2. Cloud bands suddenly obscure an entire city's PV systems in under 30 minutes

Well lucky you, I have some videos showing that this actually happens:

Positive ramp event from clearing fog

Cloud bands cause negative and positive ramp events 

Currently, these 'ramp events' are manageable.  But what if we double or triple our total installed capacity?  At some point, very soon, it's going to start being a problem.


Here's the fun part:  with our new project, you get to be part of the solution!

We have a team of scientists already working on this problem, and we're ready for more data.  Our ANU-NICTA partnership needs YOU to sign-up for our project as a volunteer to have your home solar energy system report its energy generation.  It's FREE and it allows you to become part of our science project.  

If you're interested, all you have to do is fill in the information at this link:

http://www.nicta.com.au/solar-monitoring-portal/


Want to learn more?

For starters, I've done an informative interview which I've posted to YouTube - most of the important bits are in Part 2:

Interview Part 1 of 2

Interview Part 2 of 2

If you're interested in the more technical information behind our project, I suggest you check out my scientific presentations and stay tuned for future posts (RSS at top-right).   I've purposefully left out a lot of the technical information to keep this post simple.  

You can also read more at the NICTA project page 

I look forward to seeing volunteers fill-up our inbox!  Tell your family and friends all about this project - and help solar succeed in Australia!

/*