A Data Sharing Compaign to Enable Researchers

A Call to University Researchers to "Get Applied"

It's no big secret that Australia's electricity sector is undergoing a rapid transformation, one that continues to surprise even the most aggressive growth targets for solar and storage, and has our regulators and market operators in a very demanding position. High electricity wholesale and spot prices, mean the costs of electricity are going up. Meanwhile, the cost of renewables and storage are falling quickly.  All across Australia, innovative companies are popping up which are providing solutions in the space of demand management, energy storage and energy trading and driving this transformation from the ground up.

in this electricity sector transformation, what role can universities play?

In this mix, I wonder, what role can Universities play? That's what my Academic mindset wants to know: how can we as researchers contribute to this challenging transformation?  I've had the great privilege of traveling much of Australia lately to visit with many segments of the electricity industry, and in each of those instances, I am filled with ideas on where Universities could contribute to the challenges that this sector is facing. 

But in wondering how best to make these relationships, these BIG ideas happen, I find myself exceptionally time-constrained. I simply can't work with everyone directly! Yet I feel a strong drive to help my fellow University researchers reach the real-world - how best to reconcile these?

Solar Data Enables

In contemplating this, I came to an interesting conclusion: high-resolution solar data is actually an enabling resource.  Let me explain what I mean - want to decide how to charge/discharge a battery? You need to know when the sun will or will not shine. Want to build a demand management algorithm for a commercial building? You need to understand how the rooftop solar array's power output will fluctuate during peak demand periods.  Need to orchestrate the dispatch of hundreds of MW of storage technologies? When will the next solar ramp event arrive and drive 100+MW shortfall of supply over the energy market zone you're trading in?

In the future grid, solar data is everything.  In fact, in my recent talk to the Energy Networks Australia "Grid Edge" Event [watch it here], I proposed that solar intermittency is the next frontier of opportunity for future grid technologies.

If we, as Universities, want to contribute to the future grid, this incredible transformation, then the research we complete will inherently require access to good solar data. And by 'data', I more specifically mean solar radiation or solar power generation, at high spatial and temporal resolutions.

Data Access Can Be Hard

However, solar data access is not easy. Most solar data in Australia is based on Typical Meteorological Years (TMY) climatologies, comes from only a handful of solar radiation sites scattered about the continent, or comes bundled on a harddrive at hourly resolutions and full of bias error. Other options include paying significant cash to commercial companies, which are happy to make you pay thousands of dollars for small datasets - no thanks! Shouldn't there be a better solution?

Building a Solar Forecasting System is Not Easy!

Example forecasts for small-scale solar in South Australia from 22 March 2017. Probability bounds are provided in the shaded regions of each forecast.

Example forecasts for small-scale solar in South Australia from 22 March 2017. Probability bounds are provided in the shaded regions of each forecast.

In addition to historical solar radiation data, researchers equally need to be able to access solar forecasting information. If you want to build any of the future grid enabling technologies which seize the opportunity of intermittency, they must be built in a fashion which captures the uncertainty of solar forecasting. 

At ANU/Solcast, we are pretty good at predicting the near-term availability of solar radiation, but we're not perfect either.  Given the stochastic nature of cloud formation & dissipation, their fast-moving, fast-changing characteristics and their complex optical properties, we are never going to get a cloud/solar radiation forecast 100% correct, 100% of the time.  But what we can do, is provide researchers with access to a state-of-the-art solar forecasting technology with world-class accuracy, that includes probabilistic forecast information. This way, they can build tech with these uncertainties built in - a key part of orchestrating the future grid!

Solar Data + Solar Forecasts = Easy to Share

Our Himawari 8 derived solar irradiance datasets at right, compared with the Bureau's old satellite data at left.

Our Himawari 8 derived solar irradiance datasets at right, compared with the Bureau's old satellite data at left.

In my personal quest to enable other researchers, I've realised, that in my position, with my team at the ANU, and our work with our start-up company Solcast, we can fix this problem.  We can openly and freely share solar radiation data, PV system power output datasets and solar forecasts through our joint ARENA project.   As project Chief Investigator, I have decided to inject some ARENA project funds into a data sharing campaign as part of our knowledge sharing efforts, to support the computational requirements of servicing this data to external parties. 

To be clear, that means that YOU, the researcher, get to extract the data for free for R&D purposes

And with the #SolcastAPI it is actually that easy. Within minutes of registering, you can access historical "Estimated Actuals" derived from the Himawari 8 satellite (right now, reaching back 7 days, in the near future - much deeper!) and solar radiation forecasts for GHI, DNI and DHI (powered by my Engerer2 model).  You can also use the API to simulate solar PV system power output, either via the Estimated Actuals or through the forecasts (0-7 days our, 30 minute increments).

To get started, visit the data sharing page I placed on my webpage, to provide more information about this service.  There you'll find some links for getting started. In the near future, I'll add some demos which show how to access this data, and will provide some research ideas for using high-resolution solar data for applied university research.

It's all part of teaming-up for the solar powered future! I look forward to seeing what you come up with!

Find More Stuff:

#saveARENA - There Has Never Been a More Critical Time for Renewables Research in Australia

I Have an Urgent & Important Message for You about The Australian Renewable Energy Agency

For those of you who follow my page, or have adventured here from beyond, it should be readily apparent to you that I’m an academic working in solar energy related research, more specifically in the modelling & forecasting of distributed PV power output.

To date, the majority of my work has been funded by ARENA, the Australian Renewable Energy Agency.  A funding body that has been absolutely crucial in supporting Australia as a global leader in renewable energy research over the past several years.

But ARENA is Under Imminent Threat

But right now, ARENA is under threat. The Coalition Government is planning legislation that will strip it of more than $1B of funding, and Labor are saying they won’t oppose it.

Let me explain to you why these cuts to ARENA research funding are a bad idea.

ARENA is currently funding nine university led research projects across Australia with more than $17M in funding, which are focused on partnering industry and researchers together to solve the major challenges we face integrating renewables into our electricity grid. What’s more, they intend to produce commercial outcomes valuable to the Australian economy.  (Isn't this exactly what the Australian government wants universities to accomplish? i.e. the National Innovation Science Agenda)

As an example, look no further than my own project, which received $1M from ARENA and raised $300k of industry cash to build distributed PV modelling and forecasting software for six distribution networks: ActewAGL, Ergon, Essential Energy, Western Power Horizon Power & Power and Water.

Perhaps this isn't quite enough industry partnership activity for the Coalition government?

Perhaps this isn't quite enough industry partnership activity for the Coalition government?

Don't believe the lies about a lack of value: ARENA has shifted toward driving commercially-focused outcomes

At the ANU, we're not just developing this distributed PV modelling technology to write some fancy publications.  We are delivering our software into the operations of our distribution network partners and creating a technology that will enable higher penetrations of solar to be added to the grid.  Going one step further, this software is being commercialised by an ANU start-up company, which we have already formed and is starting to complete this work now.  In the future, we hope to take this technology beyond Australia, and into other countries around the world.

Without ARENA funded research like mine, fewer solar PV systems will be permitted on our distribution networks.  Without research projects like the other eight industry-researcher collaborations in ARENA's R&D funding round, what other innovative technologies will Australia miss out on

Some Awesome Examples of Current ARENA R&D:

ARENA & Industry Funded: Drones that survey solar farms

ARENA & Industry Funded: Integrating solar thermal into alumina processing

ARENA & Industry Funded: Batteries & Solar PV into Apartment Buildings

If my project, and others like it, aren’t part of the “Innovation” agenda that the government wants, part of its “Ideas Boom”, part of the technology driven future of the Australian economy where it produces new ideas of global relevance, I don’t know what is.

If these projects aren't part of the "Ideas Boom"... I don't know what is!

We simply have a government that is demanding innovation on one hand, but clearly saying “just not for renewables” on the other.

What do we do about it?

Across Australia, our renewables powered people groups are springing into action, and our universities aren't letting this go down without a fight:

Solar Citizens has made it easy for you to tell your local parliamentarian to stop this madness

GetUp has put together a #saveARENA campaign where you can share your story about working on an ARENA funded project to your social media accounts.

I want YOU to join with me to #saveARENA

So I want YOU, to join with me, to tell our Parliament to #saveARENA. The legislation to cut $1B from ARENA is set to be introduced to Parliament early next week as part of an “omnibus bill”, so we must act now.

Let me be clear: There has never been a more critical moment for the future of renewables in Australia, or for our potential to be a leader in renewable energy technologies throughout the Asia-Pacific, and more broadly, across the globe.

Renewably Yours,

Dr. Nick Engerer

Be sure to retweet my twitter video:

And Follow #saveARENA on Twitter:



Announcing My ARENA Project: Real-time Operational PV Simulations for Distribution Network Service Providers

ARENA Project Launch - 13 April 2016

Members of the successful ARENA projects - who is that handsome guy front right? ;-)

Members of the successful ARENA projects - who is that handsome guy front right? ;-)

This has been a truly incredible week.  I have just returned from Perth, Australia where the Australian Renewable Energy Agency (ARENA) announced the outcomes of their Industry-Researcher development funding round.  All up, 9 projects were funded, totaling over $17M in funding, comprising a very impressive array of projects.  Proudly, the ANU is leading 3 projects and pulled in $4.7M from the scheme!  And looking across the remaining 6 projects, I can't help but feel incredibly proud of the innovative work Australian researchers are doing, as well as be excited about the future of renewable energy Down Under!

What's great about this funding round, is that it required industry and researchers to work together, and put up projects which have clear commercialisation potential.  That means projects which are "real-world relevant" were required, which, if you know me, meant that I was pretty darn excited!  This funding round is exactly where my passions and interest lie, and it lines up very well with the government's desired outcomes from universities in Australia, and is well aligned with the big push for innovation across the higher education sector

So, Why was I there?

Why do this? Lemme explain..

Why do this? Lemme explain..

To answer that question, we need some background.

I have spent the last few years working on another ARENA project which has focused on distributed small scale solar forecasting via machine learning and computer vision techniques.  This was an ANU-NICTA collaboration, which will wrap up mid-2016, (just before NICTA is subsumed into Data61/CSIRO - cue the doomsday music!).  One of the outcomes of this project, was my PhD work on the Regional PV Simulations System (RPSS).

RPSS v1 during a positive collective ramp event

RPSS v1 during a positive collective ramp event

Version 1 of the RPSS was developed within my thesis, and produced a modelling environment which simulated 12,000+ small-scale solar PV systems in Canberra (based on December 2012 installation data from ActewAGL).  This used data from PV systems that reported their power output on PVOutput.org, which through the magic of KPV, I was able to upscale into city-wide simulations

In my thesis, I used several critical collective ramping events to demonstrate how quickly the power output from 12,000+ PV systems can change across an entire region.  This was a fun tool, but only worked with historical data and at surburb-level.  Read between the lines here: it was cool, but ultimately not directly useful. This is the type of science that would stay 'on-the-shelf'.

Enter RPSS Version 2

After I submitted my PhD, I continued to work on the RPSS, developing it up to version 2, which I launched at the Solar World Congress in Daegu, Korea in November 2015. This version featured two major upgrades:

  1. It worked with near real-time data from the PVOutput.org live solar API, updating every 5 minutes (check out the beta at http://rpss.info).
  2. It mapped the simulated PV systems to the transformer nodes on ActewAGL's distribution network (now it's actually useful!)
RPSS v2, live data and mapped to ActewAGL's network

RPSS v2, live data and mapped to ActewAGL's network

These significant advancements demonstrated my ability to take the science off the shelf, get it out of the lab, and into relevance.  Working with live data is not easy (which is why the current version is merely a clunky beta!), nor is convincing an electrical utility to give you detailed data about their distribution network, but it teaches you about the many things required to advance a science to a technology.  And if you're like me, you'll fall in love with the challenge.

It was this second version of the RPSS that I took forward to ARENA, requesting funding to deploy it to distribution networks around Australia in order to address the challenges with integrating high-penetrations of solar PV into Australian electricity networks.  Does that sound like fun to you? Because it sure as hell does to me!

The Challenge: High penetration solar PV

High penetration solar in a neighbourhood in Canberra - solar PV on every roof!

High penetration solar in a neighbourhood in Canberra - solar PV on every roof!

With more than 1.5 million solar PV systems installed to date, totaling more than 4.5GW of capacity, the maximum penetration levels of solar PV are being reached in some areas of Australia. (e.g. Ergon 3.5 kW per system limits, Horizon Power PV limits on radial style networks). 

This 'maximum penetration' level refers to the maximum allowable amount of solar PV a utility will allow on a given part of their network.  The key word here is 'allow'.  For the most part, Distribution Network Service Providers (DNSPs) are taking preventative approaches to distributed PV intermittency, imposing maximum penetration levels that are significantly lower than are technically achievable.

Ergon is limiting the size of individual solar PV systems on parts of their distribution network

Ergon is limiting the size of individual solar PV systems on parts of their distribution network

The Key issue: solar PV = ?

There is one key item that is holding DNSPs back from being more liberal with their maximum PV penetration levels: most Australian DNSPs have no active feedback quantifying how much electricity their embedded PV generators are currently producing at any given time.  On that note, they also don't know how much PV variability will occur in the near future, nor what has occurred in the past.  So how could they possibly manage the inevitable solar future where everyone has solar PV on their roof if they have no idea how much power they are generating at any given time?

This is the key knowledge gap my project will address:  quantifying the current and expected distributed PV power production across distribution networks in near real-time.  That means quantifying distributed PV power output with enough lead-time to do something about PV-induced voltage fluctuations.  In other words, using technology to enable proactive, rather than reactive, grid management. 

What We'll do in response to the Challenge

The solar knowledge gap: how much solar & when/where?

The solar knowledge gap: how much solar & when/where?

This project will take the Regional PV Simulation System (RPSS v2) and develop and deploy it as an operational software that provides distribution network service providers (DNSPs) with real-time distributed photovoltaic (PV) simulations that are mapped to their distribution network.  The output of these simulations will be directly aimed at the knowledge gap that exists between distributed PV integration challenges and their solutions (e.g. energy storage technologies and/or remote demand/supply management). 

HOW we'll do it: Key PArtnerships

Distributed solar PV simulations are not totally unique to my research, and are used by many other researchers in Australia and the rest of the world.  They are integrated into the APVI Solar Map, they are used in Clean Power Research's PV FleetView product, as well as in many related projects around the world.  Where this project sets itself apart from all others is in the consortium I pulled together to accomplish the task and the unique data that they will provide.

This project will be led by myself (Chief Investigator), with the ANU being the sole research partner. Everyone else is an industry partner, and I'm excited to tell you about them.  They fall into a few key categories:

1. Distribution Network Service Providers (DNSP)

DNSP partners ARENA

This project includes active participation from 6 DNSPs, who will provide information about the solar PV systems installed on their networks, so that we can deploy the RPSS to their service region. I am hopeful that more DNSPs will join this project in the near future.

Photo taken in lobby of Western Power's Head Office in Perth CBD

Photo taken in lobby of Western Power's Head Office in Perth CBD

Photo taken in Horizon Power's Head Office in Bentley, WA

Photo taken in Horizon Power's Head Office in Bentley, WA

2. TWo Inverter companies, an inverter wholesaler & a solar installer

One of the primary inputs to the RPSS is monitored solar PV data, which we currently get from PVOutput.org in the online beta version.  This isn't quite good enough, as this data is slow and unreliable, mostly because it comes from non-professional sources. 

This project will work with two inverter companies, SMA Australia and Fronius, to develop real-time, rapid update monitored PV data inputs to the RPSS.  The number of internet connected PV inverters is sky-rocketing, with Fronius installing 50/week at present.  This is an excellent and rapidly growing source of data.

Key project supporter & CTO/Owner of Si Clean Energy, Peter Bulanyi

Key project supporter & CTO/Owner of Si Clean Energy, Peter Bulanyi

This project is receiving significant support from inverter wholesaler, Si Clean Energy, with its involvement led by Owner/CTO Peter Bulanyi.  Peter has been the project's number one supporter from day one, and I would like publicly thank him for his adamant, steadfast support!  Si Clean Energy will also grant access to the AllSolus monitoring network of 1000+ irradiance sensors and PV sites across Australia. 


Benn Masters, Director of SolarHub & key project supporter

Benn Masters, Director of SolarHub & key project supporter

We will also be working with solar & battery installer SolarHub, who are a progressive, high-quality company based in Canberra.  SolarHub has been connecting a large number of monitoring systems to PV installations across Canberra, and will grant access to these data over the course of the project.  They have also been a key supporter of the project from its very beginnings, and I would like to also publicly thank Benn Masters, Director at SolarHub for his significant & strategic contributions to the project.

3. ADvisory support

Patrick Dale, Director, Aeris Capital

Patrick Dale, Director, Aeris Capital

Jesse Warburg, Associate Director Aeris Capital

Jesse Warburg, Associate Director Aeris Capital

We have two companies providing in-kind support to the project.  The first is Aeris Capital, whose Director Pat Dale and Associate Director Jesse Warburg are both very excited about the consortium in place and the technology we plan to develop.  Their role in the project is to help drive it toward valuable outcomes for industry and guide us through the commercialisation of the technology. 

Second, we'll have some input from local solar installer and owner of a solar PV monitoring device called the 'esquid', Soly Ltd., whom will provide technical advice into the project.

4. outreach support: Australian Photovoltaic Institute

Finally, we have negotiated a partnership with the Australian Photovoltaic Institute (APVI) to work simulations from the RPSS into the APVI Live Solar Map, which will expand upon the incredible work being done by this progressive and very important voice for solar in Australia.  This is a great place to insert kudos to Dr. Anna Bruce at UNSW for her excellent work on this APVI Live Solar Map, which is an invaluable contribution to solar PV science outreach.  I am very excited to help build upon their Live Solar Map tool, and contribute to improving on this impressive work out of UNSW. 

What we'll do: merging datasets, Deploying RPSS

RPSS modeled ramp events across 3 transformer nodes in ActewAGL's network.

RPSS modeled ramp events across 3 transformer nodes in ActewAGL's network.

Using our unique consortium of partners, we'll be deploying increasingly advanced versions of the RPSS to the distribution networks of each DNSP.  In the first year of the project (2016-2017), we'll deploy the RPSS v2, based on currently available PV monitoring data-streams like PVOutput.org or the AllSolus network.  This version will be about 10 minutes behind 'real-time' and will be based heavily off the existing work that I've done during my PhD.

In 2017-2018, we will advanced the RPSS to include real-time input data from the Himawari 8 satellite as well as rapid-update PV monitoring input from SMA Australia and Fronius, closing the gap between 'near real-time' and 'real-time' operations.  This version will include updates from the latest in PV simulation and solar radiation radiation research.

A selfie from my April 2016 visit with Western Power

A selfie from my April 2016 visit with Western Power

Then finally, in 2018-2019, we'll have all the kinks worked out of the simulation system, having advanced it to true real-time status across all participating DNSP networks.  This version will then be used in the control rooms of these DNSPs to appropriately manage intermittent, distributed PV power production. 

It is highly likely that they will be pairing distributed PV modelling data with technologies like energy storage, to raise the maximum penetration levels of PV across their electricity network.  After all, that is the entire goal!  I look forward to updating this blog throughout the course of the project.

Up Next: Putting together an enthusiastic talented Team!

This project will kick off from July 2016, so I am now looking to put together a team of creative and inspired Originals, who are ready to dedicate their energy to pulling off a fantastic, influential and real-world driven project.  At this time the team I envision looks something like this:

Are you a unique, passionate person with awesome ideas? Then I want YOU participating on this project!

Are you a unique, passionate person with awesome ideas? Then I want YOU participating on this project!

  • A project manager/power systems engineer
  • 1-2 software engineers
  • 2 postdoctoral researchers
  • 6-8 ANU PhD candidates
  • 6-8 International PhD candidate visiting students
  • 8-10 ANU Masters research students
  • 20-30 ANU undergraduate research students

The core team consists of the engineer, programmers & postdoctoral researchers. Each of this persons will be fully funded by the project. Each of these positions will be advertised via the ANU over the next few months.

The secondary team will consist of promising PhD candidates, masters students and undergraduates.  The project has limited funds for scholarships, so I am searching for students who can self-fund themselves.  This would mean an APA scholarship for domestic PhD students, or country of origin funding sources for international PhD students.  Masters and undergraduate students who are enrolled at The ANU can participate in the project through coursework credit. 

I will be putting together a "How to Work with Me" webpage for all ranges of students in the next few days! Stay tuned!

I am very excited to see what our team will look like in the near future and sincerely hope that anyone who wants to participate will get in contact with me.

That's enough for now!

Thanks for reading about this exciting project, and looking into my latest research update.  I am excited to share more information with you, as this project moves forward, and plan to ensure our experiences from this project are shared with research scientists Australia-wide.  Stay tuned!

See ya next time!

See ya next time!

Find More Stuff:

The Engerer2 Diffuse Fraction Model (Global Radiation Separation Model)

August 2016 Update: You can learn how to compute the Engerer2 model in my Rpackage here!

This blog post explores the success of the Engerer 2 model as laid out in: Engerer, Nicholas A.  Minute resolution estimates of the diffuse fraction of global irradiance for southeastern Australia. Solar Energy. 116, June 2015, pp. 215-237.

[download the paper here]

One of the key outcomes from my PhD thesis was the validation of two different types of radiation models: clear sky and separation, against one minute resolution data. For the clear sky validation, I found suitable performance from several models for use in Australia, but the available separation models, however, did not have acceptable performance

The main issue with the available separation models (models that take a global radiation measurement from a pyranometer and separate it into its direct and diffuse components), is that they are regression based, with the original data being hourly averages of radiation.  At minute-resolution timescales, the relationship with global and diffuse/direct radiation is very different.  For one, there are very rapid fluctuations in the incoming radiation budget across these timescales.  Another big difference is the influence of cloud enhancement which is where radiation arriving at the surface exceeds the clear sky value because of non-linear interactions with some types of cloud decks. 

The Engerer2 model, with the diffuse fraction (Kd), plotted against the clearness index (Kt)

Making big changes, in the name of science

Thus, when I formulated my model, I knew that I had to make some significant advancements upon the existing methods/literature.  The principal improvements made with this model are four-fold:

  1. Inclusion of a physical model (REST2 clear sky model), making the model 'quasi-physical', much like the DISC model written by Eugene Maxwell (Maxwell 1987)
  2. The model is the only one of its class (as of the time of publication) that has been fit to minute resolution data (most other models have been designed for hourly data)
  3. There are two new variables, which have not previously been utilised in a separation model.  These are delta_Ktc (deviation of observed clearness index from clear sky value of clearness index)
  4. and K_de (the portion of the diffuse fraction that is attributable to cloud enhancement events)
The Engerer2 model formulation.  Please read the paper for more information.

The Engerer2 model formulation.  Please read the paper for more information.

independent assessment of the model: it works very, very well

The result is an impressive performance of the model against the current suite representing the state-of-the-art.  In a recent study, Gueymard and Ruiz-Arias 2015, radiation data from 54 sites around the globe were used to validate 140 separation models.  In this study, the Engerer2 model was the best!  Here it is, as described in the text:

“It is found that two models stand out over the arid, temperate and tropical climate zones: ENGERER2 and PEREZ2. These two models share two important features: (i) They include a variability predictor; and (iii) They leverage clear-sky irradiance estimates. The reason why ENGERER2 performs consistently better than PEREZ2 or other models is most likely because it was actually derived from 1-min data (compared to hourly data for PEREZ2 or most other models tested here). Based on the ensemble of statistical results obtained here, it is concluded that ENGERER2 has the best generalization skill, and can thus be considered a ‘‘quasi-universal” 1-min separation model, wherever and whenever low-albedo conditions prevail.”
Figure 2 in Gueymard & Ruiz-Arias 2015. Displaying the stations at which radiation data was used for model evaluation.

Figure 2 in Gueymard & Ruiz-Arias 2015. Displaying the stations at which radiation data was used for model evaluation.

And well... was that ever quite the compliment (especially coming from a scientist whom I've looked up to for so long)!  I am very pleased with this result, because now, my Engerer2 model is the ‘'quasi-universal' 1-min separation model" and has been accepted to be of global standard.  That makes my inner nerd quite happy, I'll admit. "Chuffed" as my Aussie friends would say :-).  And now I've have been given a reason to write a long overdue blog post about this research work.  As well as deliver a little surprise...

NOw, the Engerer2 model is in demand

As a result of this excellent outcome, I have several researchers in the community who would like to use my model, and I am quite happy to oblige.  So with this post, I am also announcing that a beta version of my Rpackage "anusolar" is now available, on request.  You can read more about this software at nickengerer.org/rpackage and where you can find out how to use the Engerer 2 separation model!  This package will allow you to do more than that, including PV simulation, KPV calculation and creating output from clear-sky radiation models.  So go check it out!

#MODSIM2015 - Extracting Solar Radiation Measurements from PV System Power Output

 A bit of background info on my 21st International Congress on Modelling and Simulation (MODSIM2015) paper

[download the paper]

#MOSIM2015 is underway! And lucky me, I’m here at our wonderful sunny Gold Coast location awaiting my opportunity to present an innovative modeling approach which estimates solar radiation using PV system power output.

Without further delay, let’s dig right in...

The Premise

A pyranometer!

A pyranometer!

Here’s the premise:  photovoltaic solar panels are not altogether different from pyranometers.  In most cases, pyranometers – the global ‘go-to’ solar radiation sensor – use a silicon wafer to measure the global horizontal irradiance at a given location.  Interestingly, this silicon wafer is based on the same technology as most PV modules.  The result is that the power output from a PV system has a first order relationship with the global solar radiation arriving in the plane-of-array.  As a scientist, this creates a sense of curiousity within me!  I wonder, can we use the power output from a PV system to then work our way back to a standard radiation measurement?

The Hypothesis

Let’s put that pondering within the framework of the scientific method, forming a hypothesis:

“Solar panels are not altogether different from pyranometers, thus one should be able to use their power output as an input to a ‘separation’ model to estimate the diffuse and beam components of solar radiation”

What is a separation model?

Over the past several decades, there have been many (as in hundreds) of models developed for the purpose of separating the diffuse and direct (beam) components from a global horizontal irradiance measurement.  In my recent paper (link), I describe this more thoroughly, so dig into that if you’re interested in learning more (or if you are a bit lost on the above bold terms)

The Engerer 2 model

The Engerer 2 model

What I’ve done with my MODSIM2015 paper, is use my own separation model format – the “Engerer 2” model - and modified it to accept PV power output measurements as the primary input, in place of pyranometer measurements.  The overall goal here, is to estimate the diffuse component of radiation using only the measured PV power output through this model and then test how well it performed.

In this paper, we use a scientific approach to test this.  Two different model formats were tested.  One with a single input, and another with four inputs.  We compare a standard pyranometer based model (Kt), fit to the same data, to the PV power output based model (KPV).  This allows us to directly compare the two approaches in a controlled, scientific fashion. 

Now, in order to do this, we need data.  So, I turned to my usual sources of data, PVOutput.org and the Bureau of Meteorology solar radiation monitoring sites.  Fortunately, I was able to identify 18 PV sites in Adelaide and Melbourne which were of very good quality and located within 10km of a solar radiation monitoring site. 

Next, I processed these data through quality control algorithms (read more in my PhD thesis) and then averaged them to hourly values.  With this QC complete, we’re able to start fitting and testing the models. 

There are a lot of details here, which I’ll glaze over in this blog post, but you can review in our paper.  Such as what QC methods I used, how I chose the PV sites, etc, etc.  I’ll leave it to you to investigate this further!  I’d rather spend my time here talking about what we found, because it is pretty cool.

The Results

First, in fitting and testing the single parameter model, the general approach undertaken was validatedThe idea works.  The relationship used in my diffuse fraction model (Engerer 2) relies on a generalized logistic function based on the relationship between the Diffuse Fraction (Kd) and the Clearness Index (Kt).  However, in the PV power output based model, we use KPV in place of Kt,  KPV is the clear-sky index for photovoltaics, and allows for the removal of all the individual nuances from the PV power output time series (use the link above to learn more)

This is a great start, but the results for a single input model aren’t very good.  This is because the model is too simple for the complex relationship between radiation and cloud cover.  But never fear, that’s where the Engerer 2 model comes in.

For those of you who are not familiar with this field, it's worth mentioning that the Engerer 2 model was recently declared the "quasi-universal" separation model in a global study using 54 sites to test 140 different models in a study by Chris Gueymard, a leader in the solar radiation modelling field.  So we're taking a modern approach to this problem.

Herein, we use the Engerer 2 model format to further test our hypothesis.  Except for the solar PV power output based model, we change another one of the input variables so that it is based on PV power output instead of a pyranometer measurement. This is the deviation from clear-sky variable (read more in the paper).

Kt versus KPV based modelling within the Engerer 2 model framework. 

Kt versus KPV based modelling within the Engerer 2 model framework. 

The results were very encouraging! The figure above demonstrates that by using only the PV power output, we are able to use our generalised logistic function based model to estimate the diffuse fraction of solar radiation with only a 5% increase in rRMSE and 0.7% increase in rMBE over a pyranometer based model.  Moreover, the observed relative errors are within the ‘good’ modeling requirements established by Gueymard and Meyers 2008.

Great, now what?

Moving forward with this result, there is now more work to do! For example, with a pyranometer measurement, once we have the diffuse component, we get the direct component through the closure equation. With PV system power output, this isn't possible.   So we'll need to test our ability to do extract the beam component further.  Possibly, this might require the joint development of another model to extract it

Despite the further work required, the results herein remain exciting. The approach used in our paper show that extracting solar radiation measurements from PV system power output can be done. This is a world first for such a result, and it is exciting to know that further work in this space is realistic. 

I am very hopeful that this result will encourage others in the field to further explore this conceptual work.  If we can extract solar radiation measurements from PV power output, we suddenly can create a very rich dataset of radiation measurements globally, greatly augmenting our ability to anlayse the spatio-temporal evolution of incoming solar radiation.  Such a dataset can assist in the validation of satellite measurements or global climate/meteorological modeling tools. 

So, with this very awesome scientific finding, I’ll leave you with the above thoughts, and hope that you’ll go on to read the paper, cite the results and further develop the prospect of having millions of additional solar radiation sensors globally.

[download the paper]

[download the presentation]

View the presentation below:

What's Happening at #MODSIM2015

Find Stuff!