ABARE farm surveys methodology

Farm surveys conducted by ABARE have been a prime source of physical and financial information for the Australian farm sector for the past forty years. This information has been collected through close cooperation, in operational and financial terms, between ABARE and key research and development funding organisations. It has been used to undertake economic research into industry and government policy areas.

Surveys undertaken for 1998-99 included the Australian agricultural and grazing industries survey, which covers much of the broadacre sector of agriculture, and the Australian dairy industry survey. These form the basis for much of the data presented in this data package. ABARE’s annual surveys of Australian broadacre industries provide a unique database that integrates detailed financial and physical information for just over 70 per cent of Australian farm business units.

Between June and November, ABARE survey officers visit sample farms. These officers interview farmers to obtain physical and financial details of the farm business for the latest financial year ended 30 June. Further information is subsequently obtained from accountants, selling agents and marketing organisations on the signed authority of responding farmers.

Information is collected on production, sharefarming, livestock, cropping, irrigation, fertiliser, land tenure, labor, costs, returns, debts and capital inventory. Considerable effort is made to reconcile the information obtained from the various sources to produce an accurate description of the physical and financial characteristics of each sample farm in the survey. Respondents to the surveys are also contacted by telephone in October each year to obtain estimates of production and expected receipts and costs for the current financial year. ABARE used the responses received in October 1998 to calculate estimates for 1998-99.

ABARE surveys are designed and samples selected on the basis of a framework drawn from the Business Register maintained by the Australian Bureau of Statistics (ABS). This framework includes agricultural establishments in each statistical local area classified by size and major industry.

The estimates in this data package cover establishments with an estimated value of agricultural operations of $22 500 or more. A definition of the estimated value of agricultural operations is given in ABS, Australian Standard Industrial Classification, 1983 (ABS cat. no. 1201.0).

The broadacre industries in ABARE’s Australian agricultural and grazing industries survey are:

Wheat and other crops industry (ANZSIC class 0121): farms engaged mainly in growing cereal grains, coarse grains, oilseeds and/or pulses.

Mixed livestock–crops industry (ANZSIC class 0122): farms engaged mainly in running sheep or beef cattle and growing cereal grains, coarse grains, oilseeds and/or pulses.

Sheep industry (ANZSIC class 0124): farms engaged mainly in running sheep.

Beef industry (ANZSIC class 0125): farms engaged mainly in running beef cattle.

Sheep–beef industry (ANZSIC class 0123): farms engaged mainly in running both sheep and beef cattle.

The Australian dairy industry survey includes:

Australian dairy industry (ANZSIC class 0130): farms engaged mainly in dairying.

Industry definitions are based on the Australian and New Zealand Standard Industrial Classification (ANZSIC). This classification conforms to an international standard that is applied comprehensively across Australian industry, permitting comparisons between industries, both within Australia and internationally.

Farms assigned to a particular ANZSIC class have a high proportion of their total output characterised by that class. Further information on ANZSIC and on the farming activities included in each of these industries is provided in ABS, Australian and New Zealand Standard Industrial Classification, 1993 (ABS cat. no. 1292.0).

The reliability of the estimates of population characteristics presented in this data package depends on the design of the sample and the accuracy of the measurement of characteristics for the individual sample farms.

Only a small number of farms out of the total number of farms in a particular industry are surveyed. Estimates derived from these farms are likely to be different from those that would have been obtained if information had been collected from a census of all farms. How closely the survey results represent the population is influenced by the number of farms in the sample, the variability of farms in the population and most importantly the design of the survey and the estimation procedures used.

In the design for the broadacre survey the population is stratified according to farm size, industry and region to ensure a broad representation of farms across Australia. The data collected from each sample farm are weighted to calculate population estimates. To increase the efficiency of the estimation process in generating measures of farm financial performance, the sample weights are based on variables that are linked to farm income and profits. Broadly, sample weights are calculated so that sample estimates of numbers of farms, areas of crops and numbers of livestock in various geographic regions and industries correspond as closely as possible to Australian Bureau of Statistics data and/or to reliable data obtained from other sources (Bardsley and Chambers 1984).

Despite the use of efficient sample design and estimation techniques, the estimates presented in this data package are likely to be different from those that would have been obtained if information had been collected from all farms. To give a guide to the reliability of the survey estimates, measures of sampling variation have been calculated. These measures, expressed as percentages of the survey estimates and termed ‘relative standard errors’, are given next to each estimate in parentheses.

These relative standard errors can be used to calculate ‘confidence intervals’ for the survey estimate. The standard error can be obtained by multiplying the relative standard error by the survey estimate and dividing by 100. For example, if average total cash receipts are estimated to be $100 000 with a relative standard error of 6 per cent, the standard error for this estimate is $6000. There is roughly a two in three chance that the ‘census value’ (the value that would have been obtained if all farms in the target population had been surveyed) is within one standard error of the survey estimate. There is roughly a nineteen in twenty chance that the census value is within two standard errors of the survey estimates. Thus, in this example, there is approximately a two in three chance that the census value is between $94 000 and $106 000, and approximately a nineteen in twenty chance that the census value is between $88 000 and $112 000.

When comparing estimates between different groups of farms, it is important to recognise that the differences are also subject to sampling variation. As a rough rule of thumb, a conservative estimate (an overestimate) of the standard error of the difference can be constructed by adding the squares of the estimated standard errors of the component estimates and then taking the square root of the result. An example is given below.

Suppose estimates of total cash receipts for farms in two different regions were $100 000 and $125 000 — a difference of $25 000 — and the relative standard error is given as 6 per cent for each estimate. The standard error of the difference can be estimated as

[(0.06 x $100 000)² + (0.06 x $125 000)²]^1/2

= $9605

so the relative standard error of the difference is:

($9605 /$25 000) x 100 = 38 per cent.

The values obtained in a survey or a census can be affected by errors of measurement and reporting. To minimise these sources of error, ABARE obtains data from a range of sources and crosschecks the data for consistency. The data sources include farmer estimates obtained in face to face interviews, farm business records, various sales outlets, and callbacks when inconsistencies are identified. Consistency checks are made progressively as information becomes available with the result that final estimates for a year are generally based on more accurate data than either the ‘preliminary’ or ‘provisional’ estimates.

Nonresponse to a survey can lower the quality of the survey’s results. Some people who participated in ABARE's main survey of broadacre and dairy farms choose not to respond to the resource management supplementary survey. While there was minimal nonresponse from dairy farms, around 6 per cent of farms that responded to ABARE’s survey of broadacre farms did not respond to the resource management survey.

Biases in the estimates can occur if the reason a person does not respond to the survey is related to the questions asked. For instance, if farms with higher debt levels were more likely not to respond to the survey, then the estimated debt levels for the respondents would be lower than for the actual population. This nonresponse means that the weights generated on the basis of variables collected from the main questionnaire may not be appropriate to obtain population estimates for variables in the supplementary survey.

When the reason for nonresponse can be linked to the known characteristics in the farm population the survey weights can be adjusted to correct for bias, and ABARE selected this approach. The surveys were first manually checked to investigate why people may not have responded. Inferences from this investigation were then incorporated into a model investigating nonresponse through logistic regressions. Using the modeled response mechanism, weights were first adjusted to account for nonresponse. These weights were then recalibrated to ensure the weighted sample conformed to the known population for selected characteristics. For further information on nonresponse in the resource management supplementary survey see ABARE’s report prepared for the Natural Heritage Trust (Alexander et al 2000). Further discussion on ABARE survey methodology can be found in Australian Farm Surveys Report 2000 (ABARE 2000).

ABARE 2000, Australian Farm Surveys Report 2000, Canberra.

Alexander, F., Brittle, S., Ha, A., Gleeson, T. and Riley, C. 2000, Landcare and Farm Forestry: Providing a Basis for Better Resource Management on Australian Farms, ABARE Report to the Natural Heritage Trust, Canberra, November.

Bardsley, P. and Chambers, R. L. 1984, ‘Multipurpose estimation from unbalanced samples’, Journal of the Royal Statistical Society, Series C (Applied Statistics), vol. 33, pp.290-9.