Applications of Second Screen Synchronization
This chapter examines recent literature related to second screen synchronization mostly on challenges and ways to overcome those challenges. According to Spilo et al. (2015), 84% of smartphones users and 86% of tablet owners in the United States in the second quarter of 2012 indicated that they utilized second screen while watching television at least once in a month. However, the study also established that 41% and 39% of tablet and smartphones owners respectively multitask at least once in 24 hours.
Howson et al. (2011) note that most of digital network strategists have realized an opportunity in the people’s attachment to a second screen while watching TV, whereby currently, some of the largest cable programmers and broadcast networks are investigating ways to offer more information on the second screen. However, Aditya (2010), postulates that the technology behind second screen possesses numerous technical challenges, which must be addressed.
Some of the challenges involved in second screen synchronization include embedding and detecting of marks in real time. Since second screen synchronization involves sending of some of the content over the internet, Non Crogh & Ebay (2016) insist that appropriate marking is required to keep off the ever-popular cyber-attacks.
According to Ton et al. (2011), technology such as visual encrypting and water marking are essential in ensuring that second screen synchronization works as expected. However, Sandoval et al. (2013) note that much of the previous studies on the topic has not covered information related to digital content marking for second screen synchronization. Therefore, the technology still is at its infancy and faces many challenges in its current form.
Overview of Second Screen Synchronization
According to Amir & Shahrokh (2012), second screen is utilizing a computing device such as a tablet, laptop, smartphone to offer an improved experience for another device’s content; for instance a TV. Specifically, the technique refers to the utilization of a device to offer interactive aspects when content is being broadcasted; for instance, a program on TV and postings on social media networks such as twitter and Facebook related to the content on television. Using a second screen enhances social TV and generation of conversation on the online platforms around the content being broadcasted.
Perumanam (2012) indicates that second screen television relies on strong synchronization of every device in participation for better management of interactive visualization of the additional content. According to Sandoval et al. (2012), several techniques of device synchronization exists which include time code synch, closed captions, direct link, manual synch, visual, and audio synch. However, Chen & Zhou (2011) indicates that most of these approaches possess numerous challenges in terms of synchronization.
Nature of Video Based Encoding
According to Howson et al. (2011), video encoding technology is necessary to enhance second screen synchronization. Video encoding also known as transcoding is a technique of converting digital video files to a format different from the original ‘source’. Video encoding is critical because different browsers and devices support varying video formats; therefore, if the source content is not compatible with the destination device format, conversion is required to enhance viewability. Different video formats, with particular variables including containers such as (.FLOV, .OGG, .MOV, WebM and .WMV), Codecs (ProRes, VP6, and H264), and bitrates, currently exists. In the context of second screen synchronization, video encoding is necessary because the TV and the second screen utilize different types of formats, with specific requirements.
However, Howson et al. (2011), note that the existing video encoding technologies are not as robust as the audio ones. Therefore, Spilo et al. (2015) argue that various challenges exists in the video encryption technology whereby incidences such as cyber-attacks are a real threat to second screen synchronization, whereby attackers can hijack the process and introduce malicious information to the second screen. Extracting the watermarks from the content in real time is also an immense issue to the second screen synchronization because users require information in real-time.
Despite the challenges existing in the video encoding, several studies have been conducted regarding the issues. For instance, some strategies have been proposed to address the issues of real time mark extracting. In addition, mechanisms to improve the robustness of encryption to enhance security have been developed and others proposed. Therefore, as Manoj & Arnold (2013) argue, the state of second screen synchronization is expected to become better in future, whereby, the experience of users is expected to be highly improved through robust security, visualization, and improved quality of information. According to Howson et al. (2011), increased applications, with elegant technologic capabilities regarding second screen synchronization, presence is also expected in the market.
Challenges of Second Screen Synchronization
According to Sandoval et al. (2013), second screen involves sharing of content over two displays that are spatially disconnected; therefore, introducing a particular category of MDE (multi display environment) with a bigger screen; for instance, a television or a computer, and the second smaller screen especially a smartphone or a tablet. Since devices possess different intrinsic characteristics, and second screen include the requirement to switch attention between the involved screens, multi display environments include unique issues to the interactive interfaces design.
Chen & Zhou (2011) argue that immense studies have been conducted regarding human computer interaction and the cooperative work supported by computers to improve the understanding of the design characteristics. However, Sandoval et al. (2012) note that there is some previous research concerning the manner in which data visualization functions in MDEs. One of the previous work on data visualization in multi display environments is WeSpace, which involved users sharing visual content on their laptops from a larger screen. Perumanam (2012) utilized a similar approach while researching on multi display environments in the automotive sector. Additionally Zhou & Zeng (2012) developed an idea for managing smart view for smart rooms, whereby the technique puts together different system views taking into account the position of the users, semantics, and direction of views to be displayed. Sandoval et al. (2013) also suggested a middleware framework for the implementation of the visualization application in multi display environments with changing user directions. Although some of the previous studies have tried to tackle issues related to second screen visualization, more research is still required to ensure that the technology improves user experience. In addition, most of the issues regarding second screen synchronization still exist even when the demand is constantly high.
As indicated earlier, some of the existing approaches to solve synchronization issues in the second screen technologies possess various problems that make them undesirable. For instance manual synchronization, which involves embedding a visible or audible trigger into the content that is broadcasted, whereby the user is required to press a button or select a specific position on the timeline to synch. According to Howson et al. (2011), although the approach is easy; it cannot uphold synchronization after pausing streaming.
Use of time codes or directly linking devices via the use of web server or a WIFI, are alternatives to the manual synch. However, Sandoval et al. (2012) explains that the approaches require special hardware; hence, limiting wide applicability in varying scenarios. Closed caption is another alternative; however, much of the content does not offer them. Visual sync triggers is another approach to synchronization whereby natural features as well as QR codes are used, but the technique is computationally expensive and is highly dependent on the situation of lighting. With all these types of approaches possessing numerous challenges, Spilo et al. (2015) argue that alternative methods for synchronization are required to address the issues identified above. Additionally, none of the above techniques above adequately addresses the issues of video-based marking technology, whereby a marker, which not supposed to intrude on the content, is embedded on the content. Removing the marker from the content in a changing range and resolution of the captured content is another source of challenge regarding synchronization.
Strategies to Overcome Second Screen TV Synchronization Challenges
According to Chen & Zhou (2011) there exist several ways of addressing numerous issues related to video encoding which include video watermarking by use of techniques such as visual cryptography and data hiding techniques or a combination of both.
For instance, Sandoval et al. (2012) proposed an algorithm of watermarking for video sequences, which was based on the idea of visual cryptography and was conducted on a spatial domain. In this technique, owners’ shares mark including a visible logo, are the embedded watermarks, which appear as pseudo-spread spectrum sequences. The algorithm for the detection of the watermark is based on the knowledge that stacking which contains the logo’s corresponding shares, achieves an improved correlation with that logo, in comparison to stacking of frames with the logo’s shares that are not relevant. The approach, according to Howson et al. (2011) is a perfect solution to geometric distortions and video processing strategies that are non-pressing. The idea is also a countermeasure to critical challenges to video watermarking such as collusion attack.
According to Sandoval et al. (2013), the method can also be used in marking video content for second synchronization since it provides watermarks that are not visible to the user and are easily decrypted. Chen & Zhou (2011) argue that the approach enhances second screen synchronization due to applicability in a wide range of content and lighting. Perumanam (2012) also developed a watermarking technology to monitor content broadcast monitoring.
Acknowledging the challenges of broadcast monitoring, the scholar presented several technical options. The uniqueness of the technique lies in the manner it exploits shift invariance to acquire an improved payload and measure of reliability for every detection. The technology has been applied on various platforms indicating real time detection and embedding feasibility. In addition, VIVA tests revealed high performance of the technology in terms of robustness and visibility. Therefore, the method is also a perfect technology for second screen synchronization due to the ability of the solution to improve visibility and robustness, which are major source of concern in 2nd screen synchronization. In addition, using the technique, embedding, and detection of watermarks can be done in real time, which is a primary demand of second screen synchronization. However, Howson et al. (2011) note that the technology requires some future improvement regarding rotation and scaling strengths.
Perumanam (2012) also, to improve the robustness of the content for the second screen, proposed a sophisticated approach to ensure that the watermarks are uncorrupted in case of attacks. The technique proposed by the scholar addressed retrieval of data without any loss while upholding high data security.
The primary goal of the scholar was to come up with a video watermarking technique that was invisible and adequately robust to carryout increased payload by applying a secure watermark distributed throughout the entire video file. The strategy also utilizes a layered strategy that enhances correct extract of the watermark once it has been embedded within a video; hence, improving the system security.
The proposed solution was supposed to be capable of performing reversible data hiding. According to the scholar, the solution to the existing video encrypting issues is a hybrid model involving wavelet, histogram, and cryptography approaches. The hybrid solution was expected to improve the robustness of the copyright image, watermark data, and the payload as well. The proposed watermarking mechanism was also expected to enhance original information extraction after several distortion attacks.
From the literature examined above, it is apparent second screen synchronization is an important aspect of the modern television watching. Therefore, TV networks are required to ensure that they offer the necessary applications to support a second screen on most or all of their programs. However, currently, there exists a challenge in video encoding whereby issues such as those regarding security exist. In addition, embedding and detecting watermarks in real time is also a problem of second screen encryption.
In that regard, several methods have been tried and tested and others proposed to offer a solution to the issues of second screen encryption. Techniques such as video cryptography and the histogram approach have been proposed to monitor, and improve the security of content sent on the online platforms as well as enhance embedding and extracting of the watermarks in real time. Some of the existing studies have also revealed methods of improving visualization of the content on the second screen, which is an immense problem in the second screen synchronization.
Howson, C., Gautier, E., Gilberton, P., Laurent, A. and Legallais, Y., 2011, September. Second screen TV synchronization. In Consumer Electronics-Berlin (ICCE-Berlin), 2011 IEEE International Conference on (pp. 361-365). IEEE.
Sandoval, F. and Kakatsakis, G., Cable Television Laboratories, Inc., 2013. Synchronization of 2nd screen applications. U.S. Patent 8,516,528.
Montagud, M., Boronat, F., Stokking, H. and van Brandenburg, R., 2012. Inter-destination multimedia synchronization: schemes, use cases and standardization. Multimedia systems, 18(6), pp.459-482.
Spilo, M. and Daub, J., Netgear, Inc., 2015. Method and system for synchronization of digital media playback. U.S. Patent 8,973,063.
Perumanam, R., Cahnbley, J. and Mandrekar, I., Thomson Licensing, 2012. Method and apparatus for multimedia stream synchronization. U.S. Patent Application 13/732,023.
Aditya, V, Rajarathnam, N, Amitabha, 2010. Watermarking video content using visual cryptography and scene averaged image. IEEE International Conference. Multimedia and Expo
Amir, H, Shahrokh, G, 2012. A Novel Video Watermarking Method Using Visual Crygraphy. Sharif University of Technology
Manoj, K, Arnold, H, 2013. Robust Digital Video Watermarking using Reversible Data Hiding and Visual Cryptography. Network Security and Digital Forensics Group
Ton, K, Geert, D, Jaap, H, Maurice M, 2011. A Video Watermarking System for Broadcast Monitoring. Philips Research
Chen, X & Zhou, P 2011, “Video watermarking algorithm based on moving object detection”, in , Journal of Computer Applications, vol. 31, no. 1, pp. 258-259.
Zhou, Y & Zeng, F 2012, “Video watermarking algorithm based on compressive sensing”, in , Journal of Computer Applications, vol. 31, no. 6, pp. 1508-1511.