PV_Live API v1 User Guide

Julian Briggs & Jamie Taylor

solar@sheffield.ac.uk

April 2018

Contents

Quick Start

Legacy API (v0)

New API (v1)

Real-time National

Real-time Regional

Historical National

Historical Regional

Retrieve as CSV

Introduction

Compression

Throttling

API Endpoints

PV_Live

Endpoint

Request Parameters

Response

Response Fields

Default fields

Extra fields

Example request

Example response

Region Shapes

Endpoint

Request Parameters

Response Fields

Example request

Example response WKT

Example response GeoJSON

GSP List

Endpoint

pvlive/v1/gsp_lists

Request Parameters

Response Fields

Example request

Example response

Querying the API

Web Browser

Example requests

UNIX Command Line

Python

Error Codes

Data Formats

pvlive

Request

Response

Default fields

Extra Fields

Quick Start

Legacy API (v0)

The original API (v0) was decommissioned on 23rd May 2018. See PV_Live_API_v0 UserGuide

New API (v1)

The default response has 3 fields (region_id, datetime_gmt, generation_mw) in JSON format.

We strongly recommend using JSON libraries to extract data to protect against changes in white space.

Real-time National

To retrieve the latest real-time national result:

https://api0.solar.sheffield.ac.uk/pvlive/v1

Real-time Regional

To retrieve the latest real-time result for a given region:

https://api0.solar.sheffield.ac.uk/pvlive/v1?region_id=123

Historical National

To retrieve the historical real-time national result:

https://api0.solar.sheffield.ac.uk/pvlive/v1?start=2017-12-31T12:00:00&end=2017-12-31T23:59:59

Historical Regional

To retrieve historical result for a given region:

https://api0.solar.sheffield.ac.uk/pvlive/v1?region_id=123&start=2017-12-31T12:00:00&end=2017-12-31T23:59:59

Retrieve as CSV

To retrieve results as csv (with single header row), add parameter: data_format=csv, eg

https://api0.solar.sheffield.ac.uk/pvlive/v1?data_format=csv

Introduction

This document describes how to use the public PV Live Web API (v1) and gives example API queries. The API is under active development so both the query and/or the data it provides may be subject to change.

The API provides nationally and regionally-aggregated PV generation outturn estimates from the PV_Live project. The API offers no guarantee with regards to availability and resilience and as such should always be supported by a more resilient, high-availability fall-back process. The API is optimised for transfer of data between Sheffield Solar server and user applications using json by default (with an option for CSV). Some browsers (e.g. Mozilla Firefox) can pretty-print the json for human viewing.

Compression

The API compresses the response if

  1. The uncompressed response is >=500 bytes and 
  2. The request headers indicate that the client can handle compression. 

We strongly recommend using compression, especially for larger responses. To configure your client to request compressed responses, add a request header. Here’s an example using the Python Requests library:

requests.get(url, headers={'Content-Type': 'application/json',

                           'Accept': 'application/json',

                           'Accept-Encoding': 'gzip, deflate',})

Throttling

To provide a reliable service we limit server load and mitigate the impact of Denial of Service (DoS) and Distributed DoS (DDos) attacks by throttling API requests. To support multiple requests (e.g. for several regions) we throttle to a maximum number of requests per half hour.

API Endpoints

PV_Live

Grid-connected PV generation in GB.

The default response contains these fields:

To add extra response fields use the extra_fields parameter, see below

Endpoint

pvlive/v1

Request Parameters

All request parameters are lower case:

See Data Format section below for further details

The request parameters (start, end, region_id) determine the response data. The response is determined by the first matching condition in this table:

Request Parameters

Response Data

Neither start nor end given

Latest (national or single region or all regions)

region_id=’all’

All regions at the start time if given

or end time if given and start not given

or latest if neither start nor end given

Both start and end given

Single region (or national)  between start and end

Start given (end not given)

Single region (or national) between start and 1 day after start

End given (start not given)

Single region (or national) between 1 day before end and end

Response

The response is a JSON Object[2] with 2 elements, data and metadata.

The data is an array of tuples. Tuples correspond to database rows.

The metadata is an array of field-names, corresponding (in order) to the data. E.g.:

{"data":[[0,"2018-04-15T11:00:00Z",3150.0], …]],

 "meta":["region_id","datetime_gmt","generation_mw"]

}

We strongly recommend using JSON libraries to extract the data. This avoids dependencies on the format (especially whitespace) in the response.

Response Fields

Default fields

  • region_id
  • datetime_gmt
  • generation_mw

Unique ID for the region of interest. (int)

Result is for the 30 minute period ending at this (GMT) timestamp

The PV generation estimate for the 30 minute period (Megawatts)

Extra fields

All lower case

  • lcl_mw

Lower Confidence Limit, 90% probability that the actual PV  generation is between LCL and UCL (Megawatts)

  • ucl_mw

Upper Confidence Limit, 90% probability that the actual PV generation is between LCL and UCL (Megawatts).

  • capacity_mwp

Estimate of total effective PV capacity for the region including estimated performance degradation (Megawatt-peak).

  • installedcapacity_mwp

Estimate of total installed PV capacity (Megawatt-peak)

  • version_id

Version_id of the derivation

  • site_count
  • updated_gmt        

Number of sites used to estimate generation

The timestamp that the derivation was run (we re-run derivations to improve the accuracy by including fuller data-sets and methodological updates)

See Data Format section below for further details

Example request

https://api0.solar.sheffield.ac.uk/pvlive/v1?region_id=123&start=2017-12-31T12:00:00&end=2017-12-31T12:30:00&data_format=csv

Example response

region_id,datetime_gmt,generation_mw

123,2017-12-31T12:00:00Z,0.0492583

123,2017-12-31T12:30:00Z,0.0684962

Region Shapes

Region shape definitions using WKT MultiPolygons with WGS84 World Mercator projection.

The region definitions are intersected with the high precision GB coastline shapefile provided by the Office for National Statistics and as such this query returns a significant quantity of data (>70 MB). An alternative API endpoint is available upon request to access the region definitions one region at a time.

Endpoint

pvlive/v1/region_shapes

Request Parameters

Response Fields

Example request

https://api0.solar.sheffield.ac.uk/pvlive/v1/region_shapes

Example response WKT

[{"data":[

[1,"MULTIPOLYGON(((-3.489394694287177 50.53375464013588, …

[2,"MULTIPOLYGON(((-3.180366025378127 56.42803879464555,-3.163815554364628, ...

 51.92751485793918)))"]],"meta":['region_id', 'WKT']

Example response GeoJSON

{"data":[[1,{"coordinates":[[[[-388437.6405447996,6539231.050030668],

[-7132.961860245987,6787029.807344022]]]],"type":"MultiPolygon"}]],"meta":['region_id','GEOJSON']}

GSP List

A list of GSPs provided by National Grid. This list gives the region_id used elsewhere along with the corresponding GSP Plexos and location. (Each agreed substation has its own region, but regions are identified by a numerical ID, rather than the four letter code used to identify substations).

Endpoint

pvlive/v1/gsp_lists

Request Parameters

Response Fields

Example request

https://api0.solar.sheffield.ac.uk/pvlive/v1/gsp_list

Example response

[{"data":[[1,"Abham","ABHA",50.47108,-3.72977],[2,"Abernethy","ABNE",56.33949,-3.29406],[3,"Aberthaw","ABTH",51.38785,-3.40198],[4,"Aldwarke","ALDW",53.44523,-1.31903],[5,"Alness","ALNE",57.70741,-4.2757],

Weybridge","WWEY",51.35286,-0.47801],[327,"Wymondley","WYMO",51.92672,-0.24865]],"meta":['region_id', 'sitename', 'plexos', 'latitude', 'longitude']}]

Querying the API

You can query the PV Live API using:

  1. Web browser: Chrome, Firefox, Internet Explorer etc.
  2. Command line: curl, wget etc.
  3. Python (we recommend the Python Requests and JSON libraries)
  4. Most other programming languages

The response is in JSON by default.

For a CSV response, use the parameter: data_format=csv

Web Browser

Querying the API from a web browser (Firefox, Chrome etc) may be useful for development and debugging.

The default response is in JSON with http header:

Content-Type: application/json

Some browsers (Firefox and Chrome with plugin) recognise json and provide options for display.

Example requests

https://api0.solar.sheffield.ac.uk/pvlive/v1

https://api0.solar.sheffield.ac.uk/pvlive/v1?data_format=csv

UNIX Command Line

$ curl https://api0.solar.sheffield.ac.uk/pvlive/v1

$ wget https://api0.solar.sheffield.ac.uk/pvlive/v1

Python

Below is a basic example of how to download results from the PV_Live API using the Python programming language.

#!/usr/bin/python3

"""

Minimal Python to query the PV_Live API (V1)

@author: Sheffield Solar

@date 23 Apr 2018

"""

import requests

class Querier(object):

    def __init__(self, endpoint):

        self.endpoint = endpoint

    def query(self):

        return requests.get(self.endpoint)

if __name__ == "__main__":

    endpoint = "https://api0.solar.sheffield.ac.uk/pvlive/v1"

    querier = Querier(endpoint)

    response = querier.query()

    print(response.json())

This example is provided for demonstration purposes and should not be used in production. A production script should, at least:

  1. Add error handling (web-server down etc.)
  2. Write results to database or file
  3. Log queries, warnings, errors. Notify warnings, errors etc.

Error Codes

Data Formats

pvlive

Request

Request Parameter

Description

region_id=0

national (GB grid connected) PV generation

region_id=n

PV generation by region (1 to 327): We use Thiessen Polygons to produce GIS geometries defining the nearest neighbour region corresponding to each transmission system grid supply point (GSP), of which there are 327.

region_id=all

all regions at the single time given by end (end must match the timestamp exactly, so omitting end which defaults to now is unlikely to return results)

start=yyyy-mm-ddTHH:MM:SSZ

GMT, ISO8601 format (several other formats accepted) start of period (response gives readings at or after this time). Default: end - 24h (24 hours before end)

end=yyyy-mm-ddTHH:MM:SSZ

GMT, ISO8601 format (several other formats accepted) end of period (response gives readings at or before this time). Default: now (current time)

data_format=csv

csv: single header row. Data rows, one per line

data_format=json

json: (default) data, metadata (field names)

extra_fields=<field0>,<field1>,<field2>,...

Specify extra (case insentive) response fields, as below. Unrecognised field-names are quietly ignored. Examples:

  • extra_fields=lcl_mw,ucl_mw,capacity_mwp
  • extra_fields=XXX   #ignored

Response

Default fields

Field Name

Description

Unit / Format

Data Type

region_id

Unique ID for the region of interest

n/a

int

datetime_gmt

Result is for the 30 minute period ending at this timestamp

ISO8601
yyyy-mm-ddTHH:MM:SSZ (
GMT)

string

generation_mw

The PV generation estimate for the 30 minute period

MegaWatts

float

Extra Fields

Field Name

Description

Unit / Format

Data Type

lcl_mw

Lower Confidence Limit, 90% probability that the actual PV  generation is between LCL and UCL (Megawatts)

MegaWatts

float

ucl_mw

Upper Confidence Limit, 90% probability that the actual PV generation is between LCL and UCL (Megawatts).

MegaWatts

float

capacity_mwp

Estimate of total effective PV capacity for the region including estimated performance degradation (Megawatt-peak).

MegaWatts

float

installedcapacity_mwp

Estimate of total installed PV capacity (Megawatt-peak)

MegaWatts

float

version

Version

n/a

int


Updated: Mon Sep 17 2018 15:07:00 GMT+0100 (BST)


[1] As of 2017-10-16

[2] https://www.w3schools.com/js/js_json_objects.asp