2882341
Effective Criterion Functions for Efficient Agglomerative Clustering on Very Large Networks
As the agglomerative clustering algorithm is widely used in data mining, image processing, bioinformatics and pattern recognition. it has attracted great interests from both academical and industrial communities. However, existing studies neglect the decisive factor of the efficiency of the agglomerative clustering algorithm for large complex networks and usually use criterion functions which lead to inefficiency. In this paper, we propose three effective criterion functions for improving performance of agglomerative clustering algorithm. We note that clustering efficiency is determined by two factors: a) the number of neighbors of two merged clusters in each merge step; b) the number of neighbors shared by the two clusters. Based on these observations, we propose a framework for designing criterion functions in order to efficiently find clusters in very large networks. We devise three criterion functions that can effectively control the number of neighbors of clusters, and they can efficiently produce high-quality clusters. We have implemented our method and compared with existing studies on real networks, and our method outperforms state-of-the-art approaches significantly on large networks.
12791479
Group-by Query Process in Middleware of Large Scale Data Intensive Systems
Large scale data intensive systems are available in many fields in recent years, and it’s a severe challenge for group-by query of large volume of data in a cluster based on shared-nothing architecture. This paper proposes a design of a parallel query engine (PQE) and its asynchronous improvement (APQE) for group-by queries. PQE and APQE support for pipelined query processing and develop maximum degree of pipeline parallelism. APQE further eliminates the synchronous overhead of multi nodes parallelism, and returns part of final result as early as possible if no data dependency exists. Experimental results demonstrate that, compared to previous 2-step query engine, PQE and APQE can make a significant performance improvement for group-by query of large data sets in a shared-nothing cluster system, as well as obviously better scalability.
18765129
Security Analysis of Enterprise Network Based on Stochastic Game Nets Model
In this paper, we propose a novel modeling method, Stochastic Game Nets (SGN), and use it to model and analyze the security issues in enterprise networks. Firstly, the definition and modeling algorithm of Stochastic Game Nets are given. And then we apply the Stochastic Game Nets method to describe the attack and defense course in the enterprise networks successfully, and find a Nash equilibrium. Finally we analyze the confidentiality and integrity of the enterprise network quantificationally based on the model. The method can also be applied to other areas with respect to a game.
3235615
Application of optical proximity correction technology
As process technology scales down to very deep sub-micron (VDSM) in semiconductor manufacturing technology, intrinsic size becomes close to or even shorter than the wavelength used for optical lithography. Thus, some distortions and deformations are introduced by optical proximity effects (OPE) mainly caused by the diffraction and interference of exposure light when layout patterns on a mask are transcribed to a wafer, which influence on the yield and performance of IC circuit. In order to compensate for the deformations, optical proximity correction (OPC) is the most commonly used methodology. Presently, the OPC method is to use a unitary toleration on the whole chip layer, which makes the run time of OPC algorithm longer, causes the size of GDSII files to follow exponential growth, and results in the cost of making mask grow immensely. Firstly, this paper proposes a self-adaptation OPC method with preprocessing function of patterns classification. According to the need of the correction precision, the OPC system divides patterns corrected into two groups with different toleration: critical patterns and general patterns, which enhance the efficiency of the OPC approach. Secondly, a model-based OPC method is presented based on pattern subsection and classification, which keeps the precision of the correction as well as enhances the efficiency. We also propose a rule-based OPC method with general, concise and complete correction rules, and achieve automatic-built rules-based and its looking-up. Thirdly, we also implement an OPC system, called MR-OPC; the MR-OPC system integrates both rule-based OPC and model-based OPC into a whole, so it can solve the confliction between the efficiency and precision. Experimental results show that the MR-OPC system we suggested has advantages of the efficiency and expansibility.
1772294
Scalable real-time system design using preemption thresholds
The maturity of schedulability analysis techniques for fixed-priority preemptive scheduling has enabled the consideration of timing issues at design time using a specification of the tasking architecture and estimates of execution times for tasks. While successful, this approach has limitations since the preemptive multi-tasking model does not scale well for a large number of tasks, and the fixed-priority scheduling theory does not work well with many object-oriented design methods. In this paper, we present an approach that scales well even when the design consists of a large number of concurrent jobs. The approach avoids any unnecessary preemptability in the system, thereby resulting in reduced run-time overheads from preemptions and associated context switches. It also allows significant memory savings by grouping jobs into non-preemptive groups and then sharing the stack space between them. Our approach is based on our earlier work on scheduling using preemption thresholds that allows parametric control over preemptability in a priority-based system. We show that our approach provides significant advantages over one using a fixed-priority preemptive scheduling architecture. The benefits include higher schedulability for small numbers of tasks and lower run-time overheads, and hence better scalability. We develop algorithms that allow design-time consideration of schedulability and automatic synthesis of an implementation model to minimize run-time overheads.
6225988
A global optimal algorithm for class-dependent discretization of continuous data
This paper presents a new method to convert continuous variables into discrete variables for inductive machine learning. The method can be applied to pattern classification problems in machine learning and data mining. The discretization process is formulated as an optimization problem. We first use the normalized mutual information that measures the interdependence between the class labels and the variable to be discretized as the objective function, and then use fractional programming (iterative dynamic programming) to find its optimum. Unlike the majority of class-dependent discretization methods in the literature which only find the local optimum of the objective functions, the proposed method, OCDD, or Optimal Class-Dependent Discretization, finds the global optimum. The experimental results demonstrate that this algorithm is very effective in classification when coupled with popular learning systems such as C4.5 decision trees and Naive-Bayes classifier. It can be used to discretize continuous variables for many existing inductive learning systems.
269891
Respecting Users' Individual Privacy Constraints in Web Personalization
Web personalization has demonstrated to be advantageous for both online customers and vendors. However, its benefits may be severely counter acted by privacy constraints. Personalized systems need to take users' privacy concerns into account, as well as privacy laws and industry self-regulation that may be in effect. In this paper, we first discuss how these constraints may affect web-based personalized systems. We then explain in what way current approaches to this problem fall short of their aims, specifically regarding the need to tailor privacy to the constraints of each individual user. We present a dynamic privacy-enhancing user modeling framework as a superior alternative, which is based on a software product line architecture. Our system dynamically selects personalization methods during runtime that respect users' current privacy concerns as well as the privacy laws and regulations that apply to them.