The role of JNLR Committee is to ensure the radio industry in Ireland and its stakeholders have access to the most robust and reliable listenership data available. As you know, the JNLR, Ireland’s national listenership survey, is the largest media survey in the country. It is balanced and stratified in each local radio franchise area, leading to a combined national sample of 16,500 respondents annually.
Over the past 24 months the JNLR Committee has been actively listening to industry voices in relation to debate around the methodology currently used to gather the JNLR research. As a result, the Committee has undertaken a review of the approach currently adopted in the Irish market, comparing and contrasting it with alternative methodologies used in other markets, with a particular focus on the approaches being used and under consideration for the UK market.
Currently there are three broad philosophies for radio research measurement worldwide.
- Day-After Aided Recall - what was listened to the previous day
- Diary self-completion - where a respondent records their radio listening in a diary over seven days
- Passive measurement - where a device, carried by the respondent, records the radio stations that the device/respondent could hear.
The current JNLR uses Day-After Aided Recall, interviewing people in their own homes and asking about radio listening on the previous day. This approach has been adopted since 1989 and has provided the industry with a remarkable level of data reliability. But is it still the optimal solution?
In the UK, a Diary Self-Completion approach is used, and the JNLR Committee decided to trial this approach in Ireland to find out if the data delivered was of a higher quality than the Day-After Aided Recall approach.
For the Diary Self-Completion trial, a sample of 650 respondents were recruited across the Dublin and South East (Carlow, Kilkenny, Wexford, Waterford and Tipperary SR) regions to match the profile of sample required for the JNLR. Similar to the standard JNLR survey, a range of quota controls were applied in terms of age, gender, social and working class. Each recruit was given the choice of completing a pen-and-paper or an online diary. Participants were asked to complete the diary every day for seven days, although due to the self-completion nature of the task it is not possible to say exactly how much time did elapse between listening and completing the diary.
The diary trial findings revealed no significant data differences between the pen-and-paper self -completion diary and the current JNLR. For the online self-completion diary, listenership levels were unusually low, suggesting poor levels of compliance possibly driven by the inconvenience of having to go online and log in each time a respondent needed to record a listening event. Overwhelmingly, even among younger respondents, the pen-and-paper self-completion diary was preferred over the online diary as the more convenient option.
The JNLR Committee concluded that a diary approach (pen and paper or online) would not provide better data, may impact data continuity and would require a move to a smaller sample size to reflect the increased difficulty in recruiting and monitoring diary respondents.
The third option open to the radio industry is to passively measure listenership by maintaining a panel of respondents who are required to carry a listening device at all times which would record the radio stations listened to. Over the past decade, the JNLR Committee has been closely following developments in passive measurement, with particular attention being paid to developments in the UK. On a number of occasions, international experts have been invited to present on latest developments in passive measurement to the Committee.
Passive measurement remains cost prohibitive for a market as rural and geographically distributed as Ireland. While the cost of the devices used has fallen (with the use of smartphone, to ‘hear’ now a possibility), the costs of recruiting and maintaining a panel, and the costs associated with maintaining an audio-matching or encoding infrastructure, have not decreased. While the JNLR Committee concluded that cost is a key barrier, it is not the only barrier, as a move to passive measurement would also impact data continuity and there is no guarantee that passive would produce more accurate data: the technology measures the radio we can ‘hear’ and not the radio we ‘listen’ to. Additionally, passive technology relies on achieving a high level of compliance from panellists.
Having undertaken a full review, at considerable cost, and having explored international best practice in radio audience measurement, the Committee concluded the current Day-After Aided Recall is best practice for the Irish market at this current time.
While the Committee is satisfied that Day-After Aided Recall is the best approach for radio in Ireland currently, the Committee is open to considering new ways of collecting the data beyond the current in-home approach, providing the integrity of the JNLR sample is maintained. New approaches to data collection will form part of the JNLR Research Tender 2020 – 2022 for which expressions of interest are currently being sought.
The JNLR Committee would like to thank all the members of the Committee for their time in conducting this review and we are delighted that all members of the Committee, including members who represent advertising agencies (IAPI), advertisers (AAI) and the regulator (BAI), fully endorse its conclusions. It has always been important for radio in Ireland that the industry and all its stakeholders speak with one voice and work in partnership to promote the wonderful medium of radio.
The JNLR Committee are aware that you may have questions arising from this, so they very much welcome the opportunity to discuss this in more detail with you and your team. Please contact your IAPI representative on the JNLR Committee - Helen O'Rourke - to discuss this further.
