How to measure the performance of a Public Service Broadcaster – the ABC experience

I'd like to share briefly some of the Australian Broadcasting Corporation's experience in measuring audience appreciation which is one of our key measures for the performance of the corporation. Just to refresh memories in terms of the scope and extent of ABC, I' ll run through what the ABC offers to its domestic Australian audience. We' ll focus on why we do annual appreciation surveys, how we do it and the information we are able to derive from the surveys.

It provides us with the key measures of the ABC's value, in terms of the value the community places in the ABC. It allows us to measure our performance against our charter obligations. It also provides key quality measures for our radio, TV and ABC online. I'll also provide you with a snippet of the sort of data we gathered which we use strategically as research tools.

Key Measures

ABC Television is available to 90 percent of the population, similarly with ABC radio it is available through most of the five networks throughout the country. We have an internet station, as most broadcasters do nowadays to stay hip and cool. We have an extensive website, ABC Online and more information about this survey is available on our website.

ABC TV is at any given week watched by over 13 million Australians, ABC Radio is listened to by over six and a half million Australians and the net reach of both ABC TV and Radio has approximately over 75 % of the population using ABC services. We have an extensive website which is visited by over a million Australians in any given month. In terms of the measures of our appreciation and usage, some indicative data from this year in terms of share of audience, ABC TV commands 18% of the free to air market, ABC radio 21% of the market.

We do a host of other research, most of that data are derived from our viewers ratings data for television, radio and online. We also do a myriad of other research exploring attitudes, relationships to television programmes, radio programmes and online content.

Why do we do a community appreciation survey? This is a chance for the public to feed back to the corporation its performance on some key measures. One of the key measures is does the public or the community value the ABC. An equally important question is whether the ABC is doing a good job delivering its charter obligations.

Key objectives we're going to look at include the overall value of the ABC to the community, the performance against the charter, the quality of programming on television, radio and online, whether the ABC provides programming people like to watch or listen to, the usage and attitudes to the ABC website, comparison with commercial media. It also goes into the amount of sports coverage, even-handedness when reporting news, the amount of coverage given to various news and current affairs issues, coverage of country, news and information.

Survey Methodology

Just briefly on the methodology, our last survey was in June 2003, we have another survey coming up in June 2004. The research is undertaken by an external market research agency. The agency that we've used for the four chunks of the survey is an organization called Newspoll which conducts a lot of opinion polling in Australia. The field work is conducted by telephone, using a sample size of 1900 people aged over 14 years. The data is post-weighted by the figures of the Australian Bureau of Statistics.

One of the key questions is the value of the ABC, and the audience is asked how they would “rate the ABC and its services to the Australian community”. About 90% rate the ABC as good and classified it as valuable. Another question is the performance of our charter obligations and are measured against 9 items namely:

• Being innovative
  
• Being balanced and even-handed when reporting news and current affairs
  
• Encouraging and promoting Australian performing arts such as music and drama
  
• Broadcasting drama of an educational nature
  
• Being distinctively Australian and contributing to Australia's national identity
  
• Being efficient and well-managed
  
• Broadcasting programmes that are different from the sort of programmes you would get on commercial television and commercial radio stations
  
• Broadcasting programmes that would be different from the sort of programmes of wide appeal and broadcasting programmes that appeal to people with special interest
  
• Broadcasting programmes that would reflect the cultural diversity of the Australian community
  

One of the key measures for television is the quality of programming for ABC TV compared to the commercial media. Commercial television is characterised as television with advertising or pay-TV. 80% regard the quality of ABC TV as good. We also ask the number of programmes or shows ABC TV provides that they personally like to watch. 70% of the respondents said the ABC TV is doing a good job in this regard, and 50% for commercial television.

Similar questions are asked for radio, the quality of programming on ABC Radio and on commercial radio. Interestingly, for those who have visited Australia, there are a myriad of radio stations and listening opportunities as well as five networks of the ABC. Much closer results are reported in terms of quality of programming on ABC Radio. Six out of ten people say ABC provides good quality programming, with a small amount saying it's poor. For commercial radio, five and a half say it's good, and a larger proportion say the quality of programming is not so good. And in radio, similar to television, the amount of radio programmes people like to listen to, five and a half say it provides programmes they like to listen to and a similar amount for the commercial radio, but again with a higher amount of dissatisfaction for the commercial radio programming.

Finally, for the online service, we ask whether they regard the quality of the information on the site as good or poor. And overwhelmingly, the quality of information on ABC online site is regarded by those who use it as very good or at least good.

Community Perception

A look at the summary of our most recent data, the key message that came from the community, largely painted a positive picture of the ABC, and you have seen that from some of the snap shots of data we have looked at. Little changed in terms of response, most feel that ABC performs a valuable role, provides quality content, and is doing a good job at fulfilling its charter obligations. There are small improvements concerning perceptions about being efficient and well managed, distinctively Australian, balanced and even-handed, broadcasting educational programmes. Not so positive was the relationship between ABC and the youth which is still not a strong one. Though there have been some improvement among 18 to 24's on the appeal of ABC TV and usage of the website, the youth market continues to be a hard one.

More detailed analysis could be viewed on ABC Online at: www.abc.net.au

■Lisa Walsh
Manager, Audience Research,Australian BroadcastingCorporation (ABC)
(* Extracted from : The Quality Surveying of Television Programmes and Public Broadcasting Service Seminar, organized by RTHK on 19th March, 2004 www.rthk.org.hk/special/tvai/2003/discussion.htm)