18355899
A microscopic view on community detection in complex networks
The community structure is a basic characteristic of complex networks. A strong community structure has high modularity. It has been proven an NP-Complete problem to identify the community structure with the highest modularity. Many approximate algorithms have been proposed to alleviate the problem. However, they suffer from inefficiency or low quality. In this paper, we propose a two-step method. The first step of our method analyze the vertex similarity of the network, which is a microscopic view. If a pair of vertices are similar enough, they will be put into the same community. The second step of our method focuses on the increment of modularity of the similarity-based communities generated by the first step. If the number of edges between two communities is greater than the expected number based on random choice, the two communities will be merged. The second step is implemented by the CNM algorithm or its improvement CNM+HE'. The similarity-based community remedies the defect on microscope introduced by CNM or CNM+HE'. Our method runs efficiently and finds meaningful communities effectively. We tested the method on more than twenty datasets. The modularity of community structure found by the method is higher than the state-of-the-art algorithm.
2882341
Effective Criterion Functions for Efficient Agglomerative Clustering on Very Large Networks
As the agglomerative clustering algorithm is widely used in data mining, image processing, bioinformatics and pattern recognition. it has attracted great interests from both academical and industrial communities. However, existing studies neglect the decisive factor of the efficiency of the agglomerative clustering algorithm for large complex networks and usually use criterion functions which lead to inefficiency. In this paper, we propose three effective criterion functions for improving performance of agglomerative clustering algorithm. We note that clustering efficiency is determined by two factors: a) the number of neighbors of two merged clusters in each merge step; b) the number of neighbors shared by the two clusters. Based on these observations, we propose a framework for designing criterion functions in order to efficiently find clusters in very large networks. We devise three criterion functions that can effectively control the number of neighbors of clusters, and they can efficiently produce high-quality clusters. We have implemented our method and compared with existing studies on real networks, and our method outperforms state-of-the-art approaches significantly on large networks.
12791479
Group-by Query Process in Middleware of Large Scale Data Intensive Systems
Large scale data intensive systems are available in many fields in recent years, and it’s a severe challenge for group-by query of large volume of data in a cluster based on shared-nothing architecture. This paper proposes a design of a parallel query engine (PQE) and its asynchronous improvement (APQE) for group-by queries. PQE and APQE support for pipelined query processing and develop maximum degree of pipeline parallelism. APQE further eliminates the synchronous overhead of multi nodes parallelism, and returns part of final result as early as possible if no data dependency exists. Experimental results demonstrate that, compared to previous 2-step query engine, PQE and APQE can make a significant performance improvement for group-by query of large data sets in a shared-nothing cluster system, as well as obviously better scalability.
18897982
An Efficient SAN-Level Caching Method Based on Chunk-Aging
SAN-level caching can manage caching within a global view so that global hot data can be identified and cached. However, two problems may be encountered in the existing SAN-level caching methods: first, too many migrations from disks to cache resulting from cache thrashing; second, too few cache hits resulting from insufficient caching. This paper proposes a SAN-level caching method employing chunk-aging to control the migrations of chunks of data from disks to cache, which takes the temporal distribution of chunk accesses into account and solves the two problems above. Then by employing two LRU lists to manage the cache replacement, this method separates and manages two types of hot data chunks, one of which have burst accesses and the other have long-term frequent accesses, thus preventing them from interfering with each other in cache. The simulation results demonstrate that our method can achieve a high hit ratio while maintaining a relatively low migration rate.
40027798
Multiple Ranker Method in Document Retrieval
In this paper, we propose a multiple-ranker approach to make learning to rank methods more effective for document retrieval application. In traditional learning to rank methods, a ranker is learned from a set of queries together with their corresponding document rankings labeled by experts, and it is then used to predict the document rankings for new queries. But in practice, user queries vary in large diversity, which makes the single ranker learned from a close set of data not representative. The single ranker cannot be guaranteed with the best ranking result for every single query, and this becomes the bottleneck of traditional learning to rank approaches. To address this problem, we propose a multi-ranker approach. We train multiple diverse rankers which can cover diverse categories of queries, instead of an isolated one, and take an ensemble of these rankers for final prediction. We verify the proposed multipleranker approach over real-world datasets. The experimental results indicate that the proposed approach can outperform existing ‘learning to rank’ methods significantly.
55331569
'No representation without information flow': measuring efficacy and efficiency of representationan: an information theoretic approach
Representation is a key concept for semiotics and for information systems. Stamper's framework may be seen outlining what is required for the efficacy of signs in standing for (i.e., representing) something else in an organization, among many others. We explore how the efficacy and the efficiency of representation may be measured, which seems overlooked in available literature of information systems and organizational semiotics. And we approach this problem from the perspective of what we call the 'information carrying' relation between the representation and the represented. We model the represented as an information source and the representation an information carrier, and the 'representing' relationship between them as that of 'information carrying'. That is, information is carried and therefore flows. These then are further modeled mathematically as random variables and random events, and a special relationship between random events. This approach enables us to reveal a necessary condition for the efficacy and the efficiency of representation, and to measure it. To this end we extend Dretske's semantic theory of information. The conviction that we put forward here is 'No representation without information flow', based upon which the efficacy and efficiency of a representation system may be measurable.
18765129
Security Analysis of Enterprise Network Based on Stochastic Game Nets Model
In this paper, we propose a novel modeling method, Stochastic Game Nets (SGN), and use it to model and analyze the security issues in enterprise networks. Firstly, the definition and modeling algorithm of Stochastic Game Nets are given. And then we apply the Stochastic Game Nets method to describe the attack and defense course in the enterprise networks successfully, and find a Nash equilibrium. Finally we analyze the confidentiality and integrity of the enterprise network quantificationally based on the model. The method can also be applied to other areas with respect to a game.
7758752
A comprehensive approach to teaching CS238 (visual basic programming)