Published using Google Docs
PV_Live_API_v4.1_User_Guide
Updated automatically every 5 minutes

PV_Live API v4.1 User Guide

Julian Briggs & Jamie Taylor

solar@sheffield.ac.uk

August 2024

Contents

Quick Start

API

Real-time National

Historical by DNO Licence Area (a.k.a PES Region)

Historical by GSP Region

Retrieve data as CSV

PES List

GSP List

Introduction

Compression

Throttling

Timeout

API Endpoints

PV_Live Outturn Estimates

Nationally Aggregated

Aggregated by PES region

Aggregated by GSP region

Request Parameters

Extra fields

Response

Response Fields

Default fields

Example request

Example response

Default Behaviours

PES Region List

Request Parameters

Response Fields

Example request

Example response

GSP Region List

Request Parameters

Response Fields

Example request

Example response

PES and GSP Region Shapes

Querying the API

Web Browser

Example requests

UNIX Command Line

Python

Error Codes

Data Formats Additional Information

PV_Live Outturn Estimates

Request

Response

Default fields

Extra Fields

FAQ

Historical Data

Pagination of Records

Record Limit

Quick Start

API

The default response has 3 fields ("gsp_id", "datetime_gmt", "generation_mw") in JSON format.

We strongly recommend using JSON libraries to extract data to protect against changes in white space.

Some fields may be NULL. For example bias_error, uncertainty_mw and site_count during hours of darkness.

Real-time National

To retrieve the latest nationally-aggregated PV outturn data:

https://api.solar.sheffield.ac.uk/pvlive/api/v4/gsp/0

Historical by DNO Licence Area (a.k.a PES Region)

To retrieve historical PV outturn data aggregated by PES region:

https://api.solar.sheffield.ac.uk/pvlive/api/v4/pes/10?start=2022-07-01T00:00:00&end=2022-07-02T00:00:00

Historical by GSP Region

To retrieve historical outrun data aggregated by GSP region:

https://api.solar.sheffield.ac.uk/pvlive/api/v4/gsp/1?start=2022-07-01T00:00:00&end=2022-07-02T00:00:00

Retrieve data as CSV

To retrieve results in CSV format (with single header row), add parameter: data_format=csv

e.g. To retrieve the for a given interval nationally-aggregated PV outturn data:

https://api.solar.sheffield.ac.uk/pvlive/api/v4/gsp/0?start=2019-01-01T00:00:00&end=2019-01-02T00:00:00&data_format=csv

PES List

The retrieve a list of PES regions:

https://api.solar.sheffield.ac.uk/pvlive/api/v4/pes_list

GSP List

The retrieve a list of GSP regions:

https://api.solar.sheffield.ac.uk/pvlive/api/v4/gsp_list

Introduction

This document describes how to use the public PV_Live Web API (v4) and gives example API queries. The API is under active development and so both the query and/or the data it provides may be subject to change.

The API provides PV outturn estimates at three levels of geographical resolution: nationally-aggregated, regionally-aggregated by DNO Licence Area (a.k.a "PES" region) and regionally-aggregated by Grid Supply Point (GSP).

DNO Licence Areas, also known as Public Electricity Supplier (PES) regions or GSP Group Regions, represent the approximate geographical areas served by 6 Distribution Network Operators who own and operate the GB electricity distribution network. GIS definitions for the DNO Licence Areas can be found on National Grid ESO’s Data Portal[1]. In the PV_Live API, we use the acronym PES for convenience.

Grid Supply Points (GSPs) represent the interface between the transmission network and the distribution network. The geographical boundaries associated with GSPs can be found on the ESo Data Portal[2].

The public API offers no guarantee with regards to availability and resilience and as such should always be supported by a more resilient, high-availability fall-back process. The API is optimised for transfer of data between Sheffield Solar servers and user applications using JSON by default (with an option for CSV). Some browsers (e.g. Mozilla Firefox) can pretty-print the json for human viewing.

Compression

The API compresses the response if:

  1. The uncompressed response is >=500 bytes and 
  2. The request headers indicate that the client can handle compression. 

We strongly recommend using compression, especially for larger responses. To configure your client to request compressed responses, add a request header. Here’s an example using the Python Requests library:

requests.get(url, headers={'Content-Type': 'application/json',

                           'Accept': 'application/json',

                           'Accept-Encoding': 'gzip, deflate',})

Throttling

To provide a reliable service we limit server load and mitigate the impact of Denial of Service (DoS) and Distributed DoS (DDos) attacks by throttling API requests. To support multiple requests (e.g. for several regions) we throttle to a maximum number of requests per half hour.

Timeout

To provide a reliable service we limit server load and mitigate the impact of Denial of Service (DoS) and Distributed DoS (DDos) attacks by enforcing a timeout on API requests (30s at 2024-12-06).

API Endpoints

PV_Live Outturn Estimates

Nationally Aggregated

Grid-connected PV generation in GB. This endpoint takes the form:

.../pvlive/api/v4/gsp/0?param1=<value1>&<param2>=<value2>

Aggregated by PES region

Grid-connected PV generation in GB aggregated by PES region. This endpoint takes the form:

.../pvlive/api/v4/pes/<pes_id>?<param1>=<value1>&<param2>=<value2>

Aggregated by GSP region

Grid-connected PV generation in GB aggregated by GSP region. This endpoint takes the form:

.../pvlive/api/v4/gsp/<gsp_id>?<param1>=<value1>&<param2>=<value2>

Request Parameters

All request parameters are lower case:

Uptime_gmt_from uptime_gmt_to

The web request options:

provide an additional filter for the response data returned from an API query.

Example

The following query returns national outturn between start and end

https://api.pvlive.uk/pvlive/api/v4/gsp/0?start=2024-12-15&end=2024-12-17

The following query returns national outturn between start and end which has been updated between updated_gmt_from and updated_gmt_to:

https://api.pvlive.uk/pvlive/api/v4/gsp/0?start=2024-12-15&end=2024-12-17&updated_gmt_from=2024-12-16T12&updated_gmt_to=2024-12-16T16

The start and end logic remains unchanged:

  1. Start and end given: limit start to 366 days before end
  2. Start alone given set end to start + 1 day
  3. End alone given: set start to end - 1 day
  4. Neither start nor end given: return the latest single row.
  5. Limit end to at or before now

Extra fields

All lower case.

bias_error

Estimate of the Mean Normalised Bias Error (MnBE) on the modelled yield, calculated using K-fold cross validation (%)

capacity_mwp

Estimate of total effective PV capacity for the region including estimated performance degradation (Megawatt-peak).

installedcapacity_mwp

Estimate of total installed PV capacity (Megawatt-peak)

lcl_mw

Lower Confidence Limit, 90% probability that the actual PV  generation is between LCL and UCL (Megawatts)

stats_error

Estimate of the Mean Absolute Percent Error (MAPE) on the modelled yield, calculated using K-fold cross validation (%)

ucl_mw

Upper Confidence Limit, 90% probability that the actual PV generation is between LCL and UCL (Megawatts).

uncertainty_MW

Estimate of the Root Mean Square Error (RMSE) on the modelled yield multiplied by the effective capacity, calculated using K-fold cross validation (Megawatts)

site_count

Number of sites used in derivation

updated_gmt

Timestamp of the latest update to the outturn estimate in ISO8601 format (GMT/UTC). Useful to check for retrospective revisions. (NOTE: updated_gmt as an extra field adds this column to the response. The request parameters: updated_gmt_from and updated_gmt_to (see above) provide an additional filter on the records returned.

Response

The default response is a JSON Object[3] with two keys: data and metadata.

The value of the data key is an array of tuples, with each tuple giving the outturn estimate for a particular interval.

The value for the metadata key is an array of field-names, corresponding (in order) to the data e.g.

{

    "data":[[1,"2019-01-01T00:00:00Z",1,0.0,332.42415,343.97426], ...]],

    "meta":["pes_id","datetime_gmt","generation_mw"]

}

We strongly recommend using JSON libraries to extract the data. This avoids dependencies on the format (especially whitespace) in the response.

Response Fields

Default fields

The response will include the following fields, plus any extra fields specified with the extra_fields parameter:

pes_id/gsp_id

Unique integer ID for the PES/GSP region of interest (int). Note we add pes_id/gsp_id 0 as a proxy for nationally aggregated.

datetime_gmt

Outturn is for the period ending at this (GMT) timestamp

generation_mw

The average PV power estimate for the period (MegaWatts)

Example request

https://api.solar.sheffield.ac.uk/pvlive/api/v4/pes/10?start=2019-01-01T12:00:00&end=2019-01-01T13:00:00&data_format=csv

Example response

pes_id,datetime_gmt,generation_mw

10,2019-01-01T12:00:00Z,219.57995299999993

10,2019-01-01T12:30:00Z,239.79673999999997

10,2019-01-01T13:00:00Z,209.40142

Default Behaviours

The request start and end parameters determine the response data. The response is determined by the first matching condition in this table:

Request Parameters

Response Data

Neither start nor end given

Latest (national or single PES/GSP region)

Both start and end given

Outturns between start and end

Start given (end not given)

Outturns  between start and (start + 24h)

End given (start not given)

Outturns  between (end - 24h) and end

PES Region List

A list of PES regions. This list gives the pes_id used elsewhere along with the more commonly used string identifiers (e.g. “_A”) and the descriptive names for the regions (e.g. “WPD (East Midlands)”).

This endpoint takes the form:

.../pvlive/api/v4/pes_list?<param1>=<value1>&<param2>=<value2>

Request Parameters

Response Fields

pes_id

Unique integer ID for the PES region of interest (int). Note we add pes_id 0 as a proxy for nationally aggregated.

pes_name

String identifiers for the regions (e.g. “_A”).

pes_longname

Descriptive long name for each region.

Example request

https://api.solar.sheffield.ac.uk/pvlive/api/v4/pes_list

Example response

{

    "data":[[0,"_0","NATIONAL"],[10,"_A","UKPN (East)"],[11,"_B","WPD (East Midlands)"],...,[23,"_M","NPG (Yorkshire Electric)"]],

    "Meta":["pes_id","pes_name","pes_longname"]

}

GSP Region List

A list of GSP regions. This list gives the gsp_id used elsewhere along with the more commonly used string identifiers (e.g. “ABHA1”) and the pes_id denoted the DNO Licence Area which the GSP serves.

This endpoint takes the form:

.../pvlive/api/v4/gsp_list?<param1>=<value1>&<param2>=<value2>

Request Parameters

Response Fields

gsp_id

Unique integer ID for the GSP region of interest (int). Note we add gsp_id 0 as a proxy for nationally aggregated.

gsp_name

String identifiers for the regions (e.g. “ABHA1”). Note that some GSPs have been grouped together because it was not possible to discern which geographical regions they each serve. In these cases, the gsp_name field will be a pipe-separated list of names (e.g. “RAVE|WISH”). The maximum length is 34 characters.

pes_id

Unique integer ID for the GSP Group Region to which the GSP belongs (e.g. 10).

Example request

https://api.solar.sheffield.ac.uk/pvlive/api/v4/gsp_list

Example response

{

    "data":[[0,"NATIONAL",0],[1,"ABHA1",22],[2,"ABNE_P",17],…,[317,"WYMOM_1",10]],

    "meta":["gsp_id","gsp_name","pes_id"]}

PES and GSP Region Shapes

Previous versions of this API have provided an endpoint for accessing the GIS definitions of the boundaries associated with PES and GSP regions. These endpoints are no longer supported. Instead, users should refer to the GIS boundary definitions hosted on National Grid ESO’s Data Portal:

PV_Live outturn data will maintain compatibility with these definitions. The ESO Data Portal provides its own API interface for accessing the GIS definitions programmatically.

Note that as of 19th July 2022, the PV_Live regionally-aggregated outturns by GSP will use the 20220314 GSP boundary definitions

Querying the API

You can query the PV_Live API using:

  1. Web browser: Chrome, Firefox, Internet Explorer etc.
  2. Command line: curl, wget etc.
  3. Python (we recommend the Python Requests and JSON libraries)
  4. Most other programming languages

Web Browser

Querying the API from a web browser (Firefox, Chrome etc) may be useful for development and debugging.

The default response is in JSON with HTTP header:

Content-Type: application/json

Some browsers (Firefox and Chrome with plugin) recognise json and provide options for display.

Example requests

https://api.solar.sheffield.ac.uk/pvlive/api/v4

https://api.solar.sheffield.ac.uk/pvlive/api/v4?data_format=csv

UNIX Command Line

$ curl https://api.solar.sheffield.ac.uk/pvlive/api/v4

$ wget https://api.solar.sheffield.ac.uk/pvlive/api/v4

Python

Sheffield Solar is maintaining an open source Python library for interfacing with the PV_Live API, hosted on GitHub: https://github.com/SheffieldSolar/PV_Live-API

Error Codes

Data Formats Additional Information

PV_Live Outturn Estimates

Request

Request Parameter

Additional Information

start=yyyy-mm-ddTHH:MM:SSZ

ISO8601 format in GMT timezone.

Several other formats may be accepted but with undefined behaviour.

Response gives readings whose end-of-interval timestamps fall at or after this time.

Default: end - 24h (24 hours before end)

end=yyyy-mm-ddTHH:MM:SSZ

ISO8601 format in GMT timezone.

Several other formats may be accepted but with undefined behaviour.

Response gives readings whose end-of-interval timestamps fall at or before this time.

Default: now (current time)

data_format=csv

Data will be returned in Comma-Separated-Variable format with a single header row and one row per interval.

data_format=json

Data will be returned in minified JSON format.

extra_fields=<field0>,<field1>,<field2>,...

Extra fields are case sensitive in the request.

Unrecognised field names will result in an error message from the server.

Response

Default fields

Field Name

Description

Unit / Format

Data Type

datetime_gmt

Result is for the interval ending at this timestamp

ISO8601
yyyy-mm-ddTHH:MM:SSZ (
GMT)

string

generation_mw

The average PV power estimate for interval

MegaWatts

float

Extra Fields

Field Name

Description

Unit / Format

Data Type

lcl_mw

Lower Confidence Limit, 90% probability that the actual PV  generation is between LCL and UCL.

MegaWatts

float

ucl_mw

Upper Confidence Limit, 90% probability that the actual PV generation is between LCL and UCL.

MegaWatts

float

capacity_mwp

Estimate of total effective PV capacity for the region including estimated performance degradation (MegaWatt-peak).

MegaWatts

float

installedcapacity_mwp

Estimate of total installed PV capacity (MegaWatt-peak)

MegaWatts

float


FAQ

Historical Data

2025-01-07

Can historical data be fetched using the live API?

If yes, up to how many years back can we retrieve the historical data from it?

For instance, will it contain the data of the last seven years?

Yes. The API serves historical data.

From 2010-01-01 to present.

However to reduce the risk of (Distributed) Denial of Service (DDOS) attack responses have a timeout (currently, 2025-01-05, 30s).

Pagination of Records

2025-01-07

Does the Sheffield Solar API support pagination of records?

No.

Record Limit

2025-01-07

Is there any upper limit on the number of records that can be pulled by this API?

If yes, how can we retrieve the records beyond that limit please?

No. There is no upper limit on the number of records.

However to reduce the risk of (Distributed) Denial of Service (DDOS) attack responses have a timeout (currently, 2025-01-05, 30s).

We provide a dedicated bulk download API endpoint for partner organisations (e.g. NESO). Users in partner organisations may email us for details of this.


Updated: Wed Jan 08 2025 14:51:44 GMT+0000 (Greenwich Mean Time)


[1] https://data.nationalgrideso.com/system/gis-boundaries-for-gb-dno-license-areas

[2] https://data.nationalgrideso.com/system/gis-boundaries-for-gb-grid-supply-points

[3] https://www.w3schools.com/js/js_json_objects.asp