AD for communion of visually impaired and normally sighted persons

    Audio Description (AD) project serves to provide service for selected RTHK television programmes on its digital terrestrial television (DTT) channel, which would render them more accessible to the visually impaired people by providing an audio description of the visual image on the screen during the gaps between dialogues. 

　　To achieve communion of visually impaired and normally sighted people, technologies are applied to allow the synchronization of the AD sound track (streamed through internet to the mobile phone of the visually impaired people) with the standard sound track (transmitted via DTT off-air to the TV set).  In this way, the visually impaired people cannot only enjoy RTHK’s TV programmes with AD on their own, but they can also share the “viewing” experience with their family members sitting together, who listen to the standard sound track from TV sets simultaneously.

Service Descriptions
　　AD renders TV services more accessible to the visually impaired people by providing an explanation as to what is happening on the screen during the gaps between dialogues, thereby making them easier to follow the programme.

　　Audio descriptions of changes of location, actions, facial expressions, gestures and so on give context and set the scene. They are fitted between dialogues and/or commentary to maintain the flow of the programme.

　　Since 2009, Hong Kong Society for the Blind has been promoting AD through organising screenings at cinemas with AD facilities.  DVDs of some well-known movies were also released with AD sound track.

　　However, there are only few TV programmes in Hong Kong supporting AD and so far no TV programme that can enable the visually impaired people to listen to the AD sound track while others can listen to the standard sound track simultaneously.

　　RTHK, as Hong Kong’s public service broadcaster, bears the mission of providing to Hong Kong people professional and high-quality radio, television and new media services.  Extending such services to the visually impaired community through the adoption of state-of-the-art technology allows RTHK to further its audience reach in Hong Kong and facilitate the building of a more inclusive and caring society.

　　AD is especially useful in certain drama programmes which may occasionally contain scenes with actions but not dialogue.  Providing AD for RTHK’s TV programmes would benefit all visually impaired/difficult people in Hong Kong, who would be able to enjoy such programmes from the comfort of their own homes.  This initiative goes one step further as it will permit them to enjoy the programmes accompanied by their family members, who can listen to the standard sound track.

Technologies to be Adopted
　　The AD is recorded on a separate sound track and streamed in real time during the TV broadcast via the public internet through a content delivery network.  The visually impaired people can then access the AD through a customized mobile application on their mobile phones.  Synchronization between the AD sound track and the standard sound track in real time can be achieved using the following technologies:

(i) Achieved a consistent time delay for all households between the mobile phone reception of the AD sound track through internet streaming and TV signal transmit out from the TV broadcast centre. 

• This is an important step for later synchronization with the received sound signal from off-air DTT signal, whose arrival time to TV set differs from household to household owing to the variations in the off-air transmission delay according to the geographical locations and the decoding time delay according to the model of TV set/Set-top box.

(ii) Achieved synchronization between the streamed AD sound track and the off-air standard sound track

• This involves the embedding of pilot signal within the standard audio sound track which is inaudible to the audience.  The mobile application can “listen” to the pilot signal contained in the TV output sound, allowing it to align the AD sound track playing through the mobile application with the standard audio sound track transmitted via DTT.  This enables the visually impaired people to listen to the AD sound track on their mobile phones, while others listen to the standard sound track through the TV set simultaneously.

• Fig.1  synchronization between the IP streamed AD sound track and the DTT off-air standard sound track

Technical challenges encountered

• Analysis of the timing difference between the two audio signals decoded from TV set and mobile device respectively must be extremely accurate.
• Under a noisy environment or when the sound level of TV is too low, the target pilot signal output from TV set is difficult to be recognized.
• Diverse use of mobile devices with different operating systems, headsets and system clock settings adds to the complexity of consistent synchronization.

Implementation Plan
　　The project involves a combination of both direct purchase and customized development.  As the application of the above technology in AD for TV programme is entirely new to the industry, the market research and subsequent tendering exercise involved sophisticated processes, including comprehensive discussions with potential suppliers, demonstration and proof of concept sessions, evaluation of technical proposals as well as consideration of total cost of ownership among different solutions from various suppliers.

　　During the development phase, special focus has been placed on the technical trials and user trials for the prototype product, in order to ensure the developed solution fully fulfill the needs of our target user groups.  In fact there has been a number of modifications and fine tunings on the solution based on users’ feedbacks. 

　　The project has entered into the final user trial stage and this service is expected to be launched in early 2021.