API

Launching Our Grid-based Solar Forecasting API at #CIDER17

CIDER17

Today, I’m presenting at the #CIDER17 conference in Sydney.

This conference, is focused on DER integration with networks in the Asia-Pacific, and nowhere is this more relevant than here in Australia.  In my presentation, I am launching of our grid-based solar forecasting API, which has now gone live in beta with a total of 8 Australian distribution networks now available (3 more added soon...).

This API deployment is a part of our ARENA funded distributed solar forecasting and modelling project where we are tasked with the integration solar forecasts into LV network operations.  Here, we have two key outcomes. First, to make accessing solar forecasts easy, scalable and to directly integrate them with the existing software in use by the DNSPs.  And second, raise the allowable penetrations of solar PV in these networks through #orchestration and proactive (rather than reactive) LV network management.

Integration via an API Framework

With that first outcome in mind, we have chosen an API framework for the delivery of our solar forecasting service.  This API is straightforward and built to start the integration process. Data from the API can be previewed in an HTML page for ease of use or testing, but can also be downloaded in specific data formats including JSON, XML and CSV via your own scripts.  Over the next year, we’ll be focusing in on building integrations for your load flow & network modelling software of choice – so be sure to provide us your feedback on what development you’d like to see in this space.

Getting to the grid-based API specifics, lets start off with “the how” we create the forecasting aggregations exposed in this API service.  Using the installed DER metadata for small-scale solar PV sites, provided by DNSPs, we have been able to:

  1. Generate PV power output forecasts & estimated actuals by matching the radiation estimates from our satellite nowcasting system to the locations and characteristics of the installed PV sites
  2. We then aggregate the PV power output forecasts to the upstream distribution transformers and zone substations; providing the total PV power output forecast for those networks assets
  3. Finally, we expose this forecast data via an API framework, which currently makes these aggregations available the zone substation level

Once these aggregations are completed, participating DNSPs are able to retrieve forecasts (as well as estimated actuals, more on that soon) via a wide variety of methods and file formats (as discussed above).  DNSPs first must register for the Solcast API services, and then are granted access to these aggregations for their network via their unique API key. We have instructions available on how to retrieve these forecasts & use the API, which is available on request (just send us an email!).

At this time, we have a total of 3 API endpoints unique to each DNSP:

  1. The ability to retrieve the API’s list of zone substation names available for forecast retrieval. This makes it easy to access the next two endpoints…
  2. An endpoint which provides a 0-7 day forecast, at 30 minute intervals (updated behind the scenes every 10 minutes) for the specified zone substation
  3. An endpoint which allows the DNSP to retrieve the estimated actuals for the downstream PV sites at that asset.  The estimated actuals are a look back at the past 7 days, at 30 minute intervals using the valid satellite scan from that time [no forecasting involved] (Note: this will be expanded to reach back further soon!)

#CIDER17 Demo of the API:

In the interest of showing off the capabilities of this API system, I am attaching the following screenshots and images of the API in actions. Several of these images are from forecasts & estimated actuals retrieved just prior to this talk, the remainder are from other events of interest. 

(Follow-up to Solar Analytics talk yesterday - ANU solar PV Simulations from 2014....)

 

Help Us Improve: Our Priority Here is DNSP Testing & Feedback

At this stage, we are asking DNSPs to access, analyse and critically review this API offering.  We want to know how they will use it, what features they’d like to see added and where we can integrate with existing software and operational strategies.

Looking Ahead: LV Network Orchestration

Getting back to outcome #2, raising the allowable penetration levels of solar PV in our LV networks, we turn to what I believe becomes possible with this grid-based solar forecasting API: using this service in the #orchestration of DER services to LV networks.  By mapping solar variability to our LV networks, we can pair it up with solutions in demand management, energy storage and smart inverters.  With 10GW of small-scale solar, 4GW of large scale solar and 500k energy storage systems on the way by 2020 – I believe this our API can directly contribute to the successful integration of the incoming high penetrations of DER. 

Zooming out, orchestration is really the forefront of the renewables powered transition now underway across the Asia Pacific.  In this effort, we must look to turn intermittency into opportunity (on that topic, watch the video embedded here). Our ANU/Solcast team is working on precisely that! And we plan to make the API open to collaboration in that regard (have you seen our data sharing campaign for researchers?), wherever our DNSP partners determine that to be beneficial.

In Closing, Our Message: Let's Collaborate!

Thanks for taking the time to read this blog post and review the above information. If you are an Australian DNSP project partner, all of the API related services are open to use across your entire organisation. Please share it widely and freely – we can handle any amount of requests that you throw at the API, so have fun.

Himawari 8 domain/imagery

Himawari 8 domain/imagery

If you are interested in deploying a similar solution (bottom up DER mapping of solar, deployment of a solar forecasting API service for solar farms and small-scale solar sites) elsewhere across the Asia-Pacific, get in touch with Solcast directly.  We’re looking for opportunities to collaborate across the Himawari 8 domain!

It’s all part of teaming up for the solar powered future!

-Dr. Nick Engerer-

A Data Sharing Compaign to Enable Researchers

A Call to University Researchers to "Get Applied"

It's no big secret that Australia's electricity sector is undergoing a rapid transformation, one that continues to surprise even the most aggressive growth targets for solar and storage, and has our regulators and market operators in a very demanding position. High electricity wholesale and spot prices, mean the costs of electricity are going up. Meanwhile, the cost of renewables and storage are falling quickly.  All across Australia, innovative companies are popping up which are providing solutions in the space of demand management, energy storage and energy trading and driving this transformation from the ground up.

in this electricity sector transformation, what role can universities play?

In this mix, I wonder, what role can Universities play? That's what my Academic mindset wants to know: how can we as researchers contribute to this challenging transformation?  I've had the great privilege of traveling much of Australia lately to visit with many segments of the electricity industry, and in each of those instances, I am filled with ideas on where Universities could contribute to the challenges that this sector is facing. 

But in wondering how best to make these relationships, these BIG ideas happen, I find myself exceptionally time-constrained. I simply can't work with everyone directly! Yet I feel a strong drive to help my fellow University researchers reach the real-world - how best to reconcile these?

Solar Data Enables

In contemplating this, I came to an interesting conclusion: high-resolution solar data is actually an enabling resource.  Let me explain what I mean - want to decide how to charge/discharge a battery? You need to know when the sun will or will not shine. Want to build a demand management algorithm for a commercial building? You need to understand how the rooftop solar array's power output will fluctuate during peak demand periods.  Need to orchestrate the dispatch of hundreds of MW of storage technologies? When will the next solar ramp event arrive and drive 100+MW shortfall of supply over the energy market zone you're trading in?

In the future grid, solar data is everything.  In fact, in my recent talk to the Energy Networks Australia "Grid Edge" Event [watch it here], I proposed that solar intermittency is the next frontier of opportunity for future grid technologies.

If we, as Universities, want to contribute to the future grid, this incredible transformation, then the research we complete will inherently require access to good solar data. And by 'data', I more specifically mean solar radiation or solar power generation, at high spatial and temporal resolutions.

Data Access Can Be Hard

However, solar data access is not easy. Most solar data in Australia is based on Typical Meteorological Years (TMY) climatologies, comes from only a handful of solar radiation sites scattered about the continent, or comes bundled on a harddrive at hourly resolutions and full of bias error. Other options include paying significant cash to commercial companies, which are happy to make you pay thousands of dollars for small datasets - no thanks! Shouldn't there be a better solution?

Building a Solar Forecasting System is Not Easy!

Example forecasts for small-scale solar in South Australia from 22 March 2017. Probability bounds are provided in the shaded regions of each forecast.

Example forecasts for small-scale solar in South Australia from 22 March 2017. Probability bounds are provided in the shaded regions of each forecast.

In addition to historical solar radiation data, researchers equally need to be able to access solar forecasting information. If you want to build any of the future grid enabling technologies which seize the opportunity of intermittency, they must be built in a fashion which captures the uncertainty of solar forecasting. 

At ANU/Solcast, we are pretty good at predicting the near-term availability of solar radiation, but we're not perfect either.  Given the stochastic nature of cloud formation & dissipation, their fast-moving, fast-changing characteristics and their complex optical properties, we are never going to get a cloud/solar radiation forecast 100% correct, 100% of the time.  But what we can do, is provide researchers with access to a state-of-the-art solar forecasting technology with world-class accuracy, that includes probabilistic forecast information. This way, they can build tech with these uncertainties built in - a key part of orchestrating the future grid!

Solar Data + Solar Forecasts = Easy to Share

Our Himawari 8 derived solar irradiance datasets at right, compared with the Bureau's old satellite data at left.

Our Himawari 8 derived solar irradiance datasets at right, compared with the Bureau's old satellite data at left.

In my personal quest to enable other researchers, I've realised, that in my position, with my team at the ANU, and our work with our start-up company Solcast, we can fix this problem.  We can openly and freely share solar radiation data, PV system power output datasets and solar forecasts through our joint ARENA project.   As project Chief Investigator, I have decided to inject some ARENA project funds into a data sharing campaign as part of our knowledge sharing efforts, to support the computational requirements of servicing this data to external parties. 

To be clear, that means that YOU, the researcher, get to extract the data for free for R&D purposes

And with the #SolcastAPI it is actually that easy. Within minutes of registering, you can access historical "Estimated Actuals" derived from the Himawari 8 satellite (right now, reaching back 7 days, in the near future - much deeper!) and solar radiation forecasts for GHI, DNI and DHI (powered by my Engerer2 model).  You can also use the API to simulate solar PV system power output, either via the Estimated Actuals or through the forecasts (0-7 days our, 30 minute increments).

To get started, visit the data sharing page I placed on my webpage, to provide more information about this service.  There you'll find some links for getting started. In the near future, I'll add some demos which show how to access this data, and will provide some research ideas for using high-resolution solar data for applied university research.

It's all part of teaming-up for the solar powered future! I look forward to seeing what you come up with!


Find More Stuff:

/*