The proliferation of surveillance cameras has greatly improved the physical security of many security-critical properties including buildings, stores, and homes. However, recent surveillance camera looping attacks demonstrate new security threats - adversaries can replay a seemingly benign video feed of a place of interest while trespassing or stealing valuables without getting caught. Unfortunately, such attacks are extremely difficult to detect in real-time due to cost and implementation constraints. In this paper, we propose SurFi to detect these attacks in real-time by utilizing commonly available Wi-Fi signals. In particular, we leverage that channel state information (CSI) from Wi-Fi signals also perceives human activities in the place of interest in addition to surveillance cameras. SurFi processes and correlates the live video feeds and the Wi-Fi CSI signals to detect any mismatches that would identify the presence of the surveillance camera looping attacks. SurFi does not require the deployment of additional infrastructure because Wi-Fi transceivers are easily found in the urban indoor environment. We design and implement the SurFi system and evaluate its effectiveness in detecting surveillance camera looping attacks. Our evaluation demonstrates that SurFi effectively identifies attacks with up to an attack detection accuracy of 98.8 and 0.1 false positive rate
Recent body of work studies how to utilize heterogeneous sensing modalities for added benefits. Researchers correlate video camera image with other sensing modalities such as RSSI value or IMU data to provide additional verification in different applications @cite_8 @cite_5 @cite_0 . We are inspired by these techniques but addresses an inherently more difficult challenge of performing cross-modality correlation without the need of deploying specific sensors on users or objects, but rather utilizing the sensed information from existing infrastructure.
@cite_0 @cite_5 @cite_8
2785936395 2813832714 2060598636
It is known that the inconsistent distribution and representation of different modalities, such as image and text, cause the heterogeneity gap that makes it challenging to correlate such heterogeneous data. Generative adversarial networks (GANs) have shown its strong ability of modeling data distribution and learning discriminative representation, existing GANs-based works mainly focus on generative problem to generate new data. We have different goal, aim to correlate heterogeneous data, by utilizing the power of GANs to model cross-modal joint distribution. Thus, we propose Cross-modal GANs to learn discriminative common representation for bridging heterogeneity gap. The main contributions are: (1) Cross-modal GANs architecture is proposed to model joint distribution over data of different modalities. The inter-modality and intra-modality correlation can be explored simultaneously in generative and discriminative models. Both of them beat each other to promote cross-modal correlation learning. (2) Cross-modal convolutional autoencoders with weight-sharing constraint are proposed to form generative model. They can not only exploit cross-modal correlation for learning common representation, but also preserve reconstruction information for capturing semantic consistency within each modality. (3) Cross-modal adversarial mechanism is proposed, which utilizes two kinds of discriminative models to simultaneously conduct intra-modality and inter-modality discrimination. They can mutually boost to make common representation more discriminative by adversarial training process. To the best of our knowledge, our proposed CM-GANs approach is the first to utilize GANs to perform cross-modal common Sketch-based image retrieval (SBIR) has become a prominent research topic in recent years due to the proliferation of touch screens. The problem is however very challenging for that photos and sketches are inherently modeled in different modalities. Photos are accurate (colored and textured) depictions of the real-world, whereas sketches are highly abstract (black and white) renderings often drawn from human memory. This naturally motivates us to study the effectiveness of various cross-modal retrieval methods in SBIR. However, to the best of our knowledge, all established cross-modal algorithms are designed to traverse the more conventional cross-modal gap of image and text, making their general applicableness to SBIR unclear. In this paper, we design a series of experiments to clearly illustrate circumstances under which cross-modal methods can be best utilized to solve the SBIR problem. More specifically, we choose six state-of-the-art cross-modal subspace learning approaches that were shown to work well on image-text and conduct extensive experiments on a recently released SBIR dataset. Finally, we present detailed comparative analysis of the experimental results and offer insights to benefit future research. Cross modal (e.g., text-to-image or image-to-text) retrieval has received great attention with the flushed multi-modal social media data. It is of considerable challenge to stride across the heterogeneous gap between modalities. Existing methods project different modalities into a common space by minimizing the distance within the heterogeneous pairs (intra-pair) of the new latent space. However, the relationship among these multi-modal pairs (inter-pair) are neglected, which are beneficial to eliminate the heterogeneity. In this paper, we propose a novel algorithm based on canonical correlation analysis by considering the high-order relationship among pairs (HCCA) for cross-modal retrieval. Supervised with additional semantic labels and unsupervised without semantic labels are simultaneously considered by treating the intra- and inter-pair correlation discriminatively. Moreover, kernel tricks are also performed on HCCA to learn a non-linear projection, termed HKCCA. Extensive experiments conducted on three public datasets demonstrate the superiority of the proposed methods compared with the state-of-the-art approaches in cross modal retrieval. HighlightsHigh-order relationship is used to discover the structure of cross-modal data.A kernel version is developed to deal with the nonlinear cross-modal correlation.Both supervised and unsupervised scenarios are considered for cross-modal learning.High-order, semantic and nonlinear is included for cross-modal correlation.