ABARE farm surveys methodology

Farm surveys conducted by ABARE have been a prime source of physical and financial information for the Australian farm sector for the past forty years. This information has been collected through close cooperation, in operational and financial terms, between ABARE and key research and development funding organisations. It has been used to undertake economic research into industry and government policy areas.

Surveys undertaken for 2001-02 included the Australian agricultural and grazing industries survey, which covers much of the broadacre sector of agriculture, and the Australian dairy industry survey. These surveys provide an integrated set of physical, financial and socio-economic information for industries that include around 75 per cent of Australian farm businesses, and account for around 98 per cent of Australia’s agricultural production. Some of the data presented in this package is from these surveys. Supplementary surveys, such as the resource management survey, can be attached to the main or telephone surveys.

Between June and November, ABARE survey officers visit sample farms. These officers interview farmers to obtain physical and financial details of the farm business for the latest financial year ended 30 June. Further information is subsequently obtained from accountants, selling agents and marketing organisations on the signed authority of responding farmers.

Information is collected on production, share farming, livestock, cropping, irrigation, fertiliser, land tenure, labor, costs, returns, debts and capital inventory. Considerable effort is made to reconcile the information obtained from the various sources to produce an accurate description of the physical and financial characteristics of each sample farm in the survey. Respondents to the surveys are also contacted by telephone in October each year to obtain estimates of production and expected receipts and costs for the current financial year.

ABARE surveys are designed and samples selected on the basis of a framework drawn from the Business Register maintained by the Australian Bureau of Statistics (ABS). This framework includes agricultural establishments in each statistical local area classified by size and major industry. The industry definitions are based on the Australian and New Zealand Standard Industrial Classification (ANZSIC), which is in line with an international standard that is applied comprehensively across Australian industry, permitting comparisons between industries, both within Australia and internationally. Farms assigned to a particular ANZSIC class have a high proportion of their total output characterised by that class. Further information on ANZSIC and on the farming activities included in each of these industries is provided in ABS, Australian and New Zealand Standard Industrial Classification, 1993 (ABS cat. no. 1292.0).

The estimates in this data package cover establishments with an estimated value of agricultural operations of $22 500 or more. A definition of the estimated value of agricultural operations is given in ABS, Australian Standard Industrial Classification, 1983 (ABS cat. no. 1201.0).

The reliability of the estimates of population characteristics presented in this data package depends on the design of the sample and the accuracy of the measurement of characteristics for the individual sample farms.

The data collected from each sample farm are weighted to calculate population estimates. Estimates derived from these farms are likely to be different from those that would have been obtained if information had been collected from a census of all farms. Any such differences are called ‘sampling errors’.

The size of the sampling error is most influenced by the survey design and the estimation procedures, as well as the sample size and the variability of farms in the population. The larger the sample size, the lower the sampling error is likely to be. So regional estimates are likely to have greater sampling errors than state estimates, and state estimates greater sampling errors than national estimates.

To give a guide to the reliability of the survey estimates, sampling errors have been calculated for the 2001-02 estimates. These estimated errors, expressed as percentages of the survey estimates and termed ‘relative standard errors’, are given next to each estimate either in parentheses or in italics.

When comparing estimates between different industries, it is important to recognise that the differences are subject to sampling error. As a rough rule of thumb, a conservative estimate (an overestimate) of the standard error of the difference can be constructed by adding the squares of the estimated standard errors of the component estimates and then taking the square root of the result. An example is given below.

Suppose the estimates of total cash receipts were $100 000 in the beef industry and $125 000 in the sheep industry — a difference of $25 000 — and the relative standard error is given as 6 per cent for each estimate. The standard error of the difference can be estimated as

[(0.06 x $100 000)² + (0.06 x $125 000)²]^1/2

= $9605

so the relative standard error of the difference is:

($9605 /$25 000) x 100 = 38%.

Similar calculations can be made when comparing estimates of change from year to year. However, it should be noted that there are changes in the industry populations from one year to the next. If these population changes are substantial, differences in estimates, such as farm incomes, might be due more to the changes in population than changes in incomes of farmers between years. There may also be differences in data quality between the two estimates being compared: final estimates are more reliable than preliminary estimates.

ABARE’s survey system is designed to produce data of a quality suitable for research and analysis at the unit level. This involves a set of quality controls, with procedures being tailored to the specific requirements of individual surveys. The key to the success of the system is employing specialist highly experienced survey officers and statisticians to guide the design and operation of the data collection and estimation process.

With voluntary surveys, the first critical control point is maximising the response rate of the selected survey sample. Having staff with appropriate people skills is essential. Nevertheless, low response rates can be unavoidable in some surveys. Problems of data quality arising from this source are reduced by the use of procedures to guide the selection of replacement farms, and the use of statistical modeling in the estimation process.

Data quality is also enhanced by checks against available external data sources and by internal consistency checks. The first of these checks takes place at the time of collection. With expert survey staff and training in the specific survey topic, much of the checking for internal consistency of data is done most effectively and efficiently as part of the interview. After the collection of the survey information, the data are passed through a series of automated and manual edits to check against any data collected at the unit level from other sources and as a final check for internal consistency. Extreme observations are also identified and, if necessary, checked by a second contact with the survey respondent.

Further discussion on ABARE survey methodology can be found in Australian Farm Surveys Report 2003 (ABARE 2003).

ABARE 2003, Australian Farm Surveys Report 2003, Canberra.