Public spaces are full of moving images. Whether we find them in shop windows, in train stations, at trade fairs, in waiting rooms or in the medical practice – they all have one thing in common: Since the videos are not relevant for all viewers at the same time and to not bother anyone, these films are usually silent or played very quietly. This limits the possibilities of the medium to the extreme – and the likelihood of the message getting across.
Effortless Audio Access via QR Code
Museums are the only place where headsets are occasionally made available, but their hygiene is only half-way guaranteed by frequent disinfection. In contrast, Nubart’s patent-pending audio streaming solution, allows the audio to be easily played in any language on the user’s own smartphone – in perfect lip-sync. Unlike other existing systems, Nubart Sync does not require any additional hardware or special applications. Users simply scan a QR code and access the audio source in their browser via WLAN or mobile data. Museum Sydøstdanmark has been using the system at several locations since 2021. For their exhibition media, they have chosen Brightsign players with an internet connection, which can be integrated with Nubart Sync in just a few clicks.
With voice, sound effects and music, it is in the nature of video to have more audio than visual information channels. The most direct of these is speech, as information can be conveyed directly and clearly with synchronous spoken contributions, for example in an interview, or with an off-camera voiceover, which is ideal for image and product videos or reportage. The visuals can complement and reinforce what is being said. In addition, sound effects and music create a spatial atmosphere that links the locations and scenes together and also supports the video’s message on an emotional level. ‘The rule is: the more senses that are addressed, the more intense the effect of the video on the recipient,’ says Dr Rosa Sala, CEO of Nubart. ‘The disadvantages of playing videos without sound in public are correspondingly great’.
However, in order to exploit the full potential of video as a medium, billboards or information displays are often played at a low volume. In public spaces, however, the sound is quickly lost and people do not hear it. At the same time, videos are limited to a single language, with or without subtitles. In addition, multiple videos in direct proximity clash with one another and are mutually distracting. For this reason, museums and other cultural venues often provide headsets that allow visitors to hear the audio at a comfortable volume while blocking out unwanted background noise.
However, maintaining this hardware requires complex logistics. In addition, the high number of users means that the hardware is prone to failure and, even with frequent disinfection, it is difficult to maintain the required hygiene standards. ‘Over the years, more and more of our customers have been asking us if it would be possible to synchronise the audio with the video in our PWA audio guide,’ says Dr Sala. ‘Until now, this has only been possible with expensive hardware and software such as i-beacons or RFID trigger systems, which are not really financially feasible for most museums and cultural institutions.’
Passers-by stream lip-synched audio to their smartphone browsers
With its audio guide PWAs, and for around eight years, Nubart has been already providing a cost-effective alternative to the high-maintenance devices and expensive apps previously used by museums for multimedia exhibitions and multilingual guided tours. In 2020, as a complement to those audio guides, CTO Simon Effing developed the concept behind Nubart Sync: As previously, the operator hosts the video file locally and plays it back in the browser, in a Brightsign player, or in a dedicated app. Local playback offers the advantage that the video player solely requires an Internet connection with a minimal bandwidth.
Passers-by or museum visitors use their smartphones to scan a QR code, optionally placed in the video or near the screen, and access the audio through their browsers. If multiple audio tracks are available in different languages, the desired language can be selected at this point. Accessibility solutions can be also built in, such as an audio description, for instance, or a version in simplified language. In the background, Nubart Sync aligns the audio and video time stamps with each other in the Cloud and thus enables lip-synced playback of the audio track on a variety of terminals simultaneously.
At the heart of the system is the synchronisation of the audio and video tracks without any additional hardware or software and without any delay – and this was also the biggest challenge during development. Nubart solves the problem by having the terminals not only count back the time stamps in respect of the connection’s latency, but also take device-specific offsets into consideration during audio playback.
‘The big advantage of Nubart Sync is that the system communicates the time stamps exclusively via the server,’ Dr Sala explains. ‘The audio file can be streamed or alternatively pre-loaded to the browser, meaning that only a minimal Internet bandwidth is required for latency-free playback.’ Conversely, this means that the operator is not obliged to provide users with a stable, full-coverage WIFI connection: a conventional mobile data connection is sufficient; even streaming with 4G is easily possible.
Museum Sydøstdanmark integrates Nubart Sync in Nubart Audio Guide
Museums and cultural institutions can integrate Nubart Sync in the same provider’s multimedia guide PWA, which also works entirely via the smartphone browser, without any additional hardware or apps.
For example, Museum Sydøstdanmark has been successfully using Nubart Sync at various locations since 2021, including Køge Museum, Danmarks Borgcenter and temporary exhibitions at Holmegaards Vaerk. ‘No matter what device our visitors are using, there is absolutely no latency as long as the internet connection is stable,’ says Carl-Henrik Hansen, CTO of Museum Sydøstdanmark.
The Danish museums use Brightsign players for their exhibition media. For audio synchronisation, all that is required is the creation of a video instance in the Nubart platform and the storage of an HTML file provided by the system together with the video track in Brightsign. If necessary, the offset for individual video players can also be adjusted in the backend. However, this is a stable value that does not need to be changed afterwards. ‘Most of our visitors are entirely unaware that it’s not self-evident to be hearing perfectly synchronised audio tracks accompanying an external video on their own phone “just like that”. As far as they’re concerned, it’s completely invisible technology,’ summarises Hansen.
