- Why are records PROVISIONAL?
- About whole and rounded temperatures
- What records are presented?
- How the records are presented?
- What are the codes in the Comments column?
- Records colour-coded blue
- Quality control issues
- Getting quality controlled data
Why are records PROVISIONAL?
The system producing the records section of the Daily Weather Summary is run automatically in real time with updates through the day as the readings come in. Most of the information is collected from computerised weather and rainfall stations, so from when the computer in a station sitting, perhaps, on a lonely hilltop "reads" its instruments to when you see its data here, it's possible you are the first human to apply thinking and common sense to the figures. Mistakes happen, both with data from automatic stations and human observers, and because those mistakes often provide extreme readings, they usually show up as records.
The records are shown as PROVISIONAL records, as they still have to pass a large number of tests to check their validity. Obvious errors are corrected or deleted by AWN within a few days when full data is available, and the letter "C" for "Checked" will appear in the comments column with additional notes if needed. However, detailed quality management is a job for the BoM, and with milllions of pieces of data being received each month, that takes time. The records in the Daily Weather Summary have only been given a quick, common-sense check; to get the final official information you will need to visit the Bureau's site some months or sometimes years after the event and information on how to do this is given below.
About whole and rounded temperatures
Only whole (rounded) temperature figures are available until AWN receives detailed data from the Bureau of Meteorology early each afternoon. Minimum and grass minimum temperatures will show a decimal as ".0" until then. Maximum temperatures are rounded and only for the period 09:00 to 15:00 when they first become available, with the maximum for the official period of 09:00 to 09:00 available after 09:00 the next day and are rounded until mid-afternoon. The initial maximum temperatures for the period 09:00 to 15:00 may miss a late-occurring high temperature.
When the decimal figure arrives, stations previously shown as breaking a record may no longer do so. For example, if the record maximum is 41.8 and a rounded maximum of 42 is received it seems to be a record-breaker. But if, when the full figure comes in, it proves to be 41.7, then it no longer breaks the record. Similarly, some stations may fail to break a record with their rounded figure but do when the full figure comes in. Records should therefore only be taken as potential records until the full figure to one decimal place is received.
What records are presented
AWN shows in each day's Daily Weather Report the provisional monthly and all-time records set by any Australian rainfall and weather station, depending on what it measures:
- Record highest and lowest air maximum and minimum temperature
- Record lowest grass minimum temperature
- Record highest daily rainfall and wind gust
"All-time" records are records across all months for as long as the station has existed.
Highest and lowest air maximum and minimum temperatures, along with real-time temperatures, are measured in a Stevenson screen 1.2m above the ground in a well-exposed area. The screen shelters thermometers from rain and snow, as well as shielding them from radiation from the sun or other objects by day. It is painted white and has louvres which allow air to pass through, so that the thermometers are measuring the temperature of the air and can be compared to measurements at any other station, in Australia or internationally. Thermometers in the screen are 1.5m above the ground. The BoM has a good explainer here and short video here.
Lowest "grass minimum" temperatures, also known as the ground or terrestrial minimum, is recorded by an ordinary minimum thermometer left overnight with its bulb at grass-tip height, fully exposed to the sky. Although it is only 1.2m lower than the thermometer in the screen, it can be 5° or more colder on still winter nights. It is mostly used to determine the temperature experienced by crops and the presence of frost, and is only found at relatively few stations.
Both the air and grass minimum temperature are for the 24 hours TO 0900 local time while the maximum temperature is for the 24 hours FROM 0900. All are measured to tenths of a degree Celcius.
Highest Daily rainfall is for the 24 hours to 0900 local time, and is measured to a tenth of a millimetre. All stations measure rainfall.
Highest wind gusts are measured in kilometres per hour to the nearest whole figure and are given for the calendar day, midnight to midnight. They are measured by an anemometer on a mast 10m above ground level. Almost all Bureau automatic weather stations (AWS) measure wind gusts, while only a few of the manually-read stations at major airports do so.
How the records are presented
The columns in the Records report show:
- State and rainfall district, site number and station name
- Today:
- The reading that produced the record on this day.
- Previous monthly record
:
- The record for this month that was broken or equalled.
- The most recent date on which this occurred.
- The number of years for which the station has records for this month, including the current month. Records are only produced for stations with at least 10 previous years of observations for the element concerned.
- Previous all time record (only given if an all time record for the station has been equalled or set). "All-time" means for all months that the station has been operating.
- The all time record that was broken or equalled.
- The most recent date on which this occurred.
- The number of years for which the station has records for all or part of a year, including the current year. This may differ from the monthly years column.
- The comments column automatically notes if the record has been equalled as distinct from broken. It also notes if the previous record was set or equalled in the same month. I also manually add notes here if I believe the record is suspect.
What are the codes in the Comments column
In addition to plain text notes, the Comments column carries a few quality control codes. Here's what they mean:
- The number "1" = the rainfall or temperature observation is for one day. A figure greater than 1 should not occur as it leaves the date the record occurred open to doubt
- C = Checked by AWN for obvious errors but should not be regarded as quality controlled
- N = From BoM climate database and not quality controlled by BoM
- P = From BoM real-time synoptic data and not quality controlled
- S or s = Marked suspect by BoM (S) or AWN (s). Notes will indicate why and wrong record coloured RED if checked by AWN.
- W or w = Marked wrong by BoM (W) or AWN (w). Notes will incidate why and wrong record coloured RED if checked by AWN.
- Y = From BoM climate database, quality controlled and acceptable.
Records colour-coded blue
Records may be coloured BLUE as part of AWN's checking process with a note placed in the comments column. This means that the AWN and BoM climate databases disagree on this figure, and can happen for many reasons such as:
- The data was received, altered or quality controlled by the BoM longer ago than the 180-day window during which AWN receives data from the BoM.
- BoM has added additional earlier data for the station, sometimes extending it back many years or decades.
- AWN and BoM have differed in their quality control of that piece of data.
Some quality control (QC) issues
Unfortunately, programs designed to pick out extremes are also very good at finding observer and AWS errors. Typical errors include:
- Omitting a decimal point, so 54.2mm of rain becomes 542mm (human observers)
- Misreading a thermometer by 5 or 10 degrees, so a maximum of 35.6 becomes 41.6 or 45.6 (human observers)
- Entering a reading on the wrong day, especially rainfall observations (human observers)
- Entering a rainfall observation as a 24-hour period when in fact it has been for a longer period (human observers)
- Power spikes can give wild minimum temperatures (e.g. -41.7) or maximum temperatures (e.g. 58.9), or give a reading of 0.0 (AWSs)
- Rainfall in AWSs is measured by rain entering the gauge via a funnel into a tipping bucket system. As AWSs operate unattended, a blocked funnel or tipping bucket may go unnoticed for a while so that rainfall is lost. More important for record rainfall errors is that the tipping bucket mechanism may be hindered so that it begins to tip rapidly, counting huge amounts of non-existent rain (AWSs)
- A technician working on an AWS may reset all values to the time his work was finished, so data recorded will only reflect temperatures, rainfall, wind etc. since the time of reset and not the full 24 hours (AWSs).
Quality control of the data goes through 3 stages:
- Do-it-yourself: Records produced in real time have no quality control whatsoever. Use the notes above, and compare any suspect figures with nearby locations. The recent weather pages are good for this, as they present all observations in a district together making it easy to pick out obvious errors at one station.
- Done by AWN: I regularly scan for obvious errors. They will be corrected or deleted within a few days when full data is available, and the letter "C" for "Checked" will appear in the comments column with additional notes if needed.
- Done by BoM: The Bureau of Meteorology's Quality Control System is an ongoing process. Automatic Weather Stations run an internal routine each day at 9am, and their figures are usually deemed to be quality controlled, though in my experience rare errors will remain. In the manual reports, some errors are picked up within days, however it may take months or sometimes years for the QC to reach all stations. Even then, with millions of pieces of weather data in the BoM database, some errors will remain. The most recent data on the BoM Climate Data Online pages offer the most recent quality controlled version available. I receive data each day for the last 120 days for climate stations and 180 days for rainfall stations which picks up most of the BoM QC corrections and also reports received after real time by BoM. This is routinely updated in the AWN Daily Climatic Data pages and archives. The Daily Weather Summaries, however, rely only on my extraction system and alertness to keep them clean until they are recreated 6 months after real time to catch BoM quality control.
Getting quality controlled data
To get the Bureau's quality controlled (QC) data, firstly you need to know which station you want, secondly you must get to the data and thirdly check if it has yet been quality controlled. Be warned in advance that it is not possible online to check whether grass/ground minima or wind gusts have been quality controlled.
The quickest way to find the station you want is to get its 6-digit station ID number. This is given next to the station's name in the AWN records section of each Daily Weather Summary, or for all stations in the AWN Daily Climatic Data. As an example to work through the following, use the station ID for Sydney Airport: 066037.
Rainfall
- To find rainfall data and records, start by going here.
- Enter the 6-digit station number in section 3, "Get the data" and click the "Get Data" button.
- You should now be looking at the current year's daily rainfall readings for the station. Below all the readings is a section called "Summary statistics for all years". The last line gives the highest daily rainfall for that month for the whole time the station has been operating. Hover your mouse over each figure to get the date of the record.
- If a figure is in italics, it has not yet been quality controlled.
- Finally, if the figure you are expecting is not there, check the date the page was created, given in the last line on the page above the footer links. BoM may not yet have caught up with it. If the page creation date is after the date of the reading you're questioning, the reading may have been invalidated by quality control. Check for the reading in the daily readings above.
Temperatures and wind gusts
- To find temperature and wind gust records, start by going here.
- Enter the 6-digit station number in section 3, "Get the data" and click the "Get Data" button.
- In the long box just above all the figures, select "All available".
- You should now be looking at a full set of monthly and annual averages, extremes and other data for the station.
- If the figure you are expecting is not there, check the date the page was created, given in the last line on the page above the footer links. BoM may not yet have caught up with it. If the page creation date is after the date of the reading you're questioning, the reading may have been invalidated by quality control.
- Unfortunately, this page doesn't distinguish between QC data and non-QC data by using italics. To see if a maximum or minimum temperature for a day has had QC performed on it, go to this page.
- From the drop-down list, select Temperature, then select Maximum or Minimum temperature, enter the station number and click "Get Data".
- You should now be looking at the current year's daily maximum or minimum temperature readings for the station. Below all the readings is a section called "Summary statistics for all years". The last two lines give the highest and lowest maximum or minimum temperature for that month for the whole time the station has been operating. Hover your mouse over each figure to get the date of the record.
- If you've got this far, you'd think that the figures on the bottom rows would follow the italics rule, where data in italics is not yet quality controlled. Sadly, they don't, and you'll have to find the figure in the columns of daily data to see whether it is italicised or not. Use the year selector top left to move to another year.
- Finally if you're trying to find whether grass/ground minima or wind gusts have yet been quality controlled, you're out of luck. That daily data is not available online.
Last updated: 30/04/2022 |