extfieldsofstudy:
Computer Science
Mathematics

text:
On the Consistency of Quick Shift

Quick Shift is a popular mode-seeking and clustering algorithm. We present finite sample statistical consistency guarantees for Quick Shift on mode and cluster recovery under mild distributional assumptions. We then apply our results to construct a consistent modal regression algorithm.

Introduction
Quick Shift [16] is a clustering and mode-seeking procedure that has received much attention in computer vision and related areas. It is simple and proceeds as follows: it moves each sample to its closest sample with a higher empirical density if one exists in a τ radius ball, where the empirical density is taken to be the Kernel Density Estimator (KDE). The output of the procedure can thus be seen as a graph whose vertices are the sample points and a directed edge from each sample to its next point if one exists. Furthermore, it can be seen that Quick Shift partitions the samples into trees which can be taken as the final clusters, and the root of each such tree is an estimate of a local maxima.
Quick Shift was designed as an alternative to the better known mean-shift procedure [4,5]. Mean-shift performs a gradient ascent of the KDE starting at each sample until -convergence. The samples that correspond to the same points of convergence are in the same cluster and the points of convergence are taken to be the estimates of the modes. Both procedures aim at clustering the data points by incrementally hill-climbing to a mode in the underlying density. Some key differences are that Quick Shift restricts the steps to sample points and has the extra τ parameter. In this paper, we show that Quick Shift can surprisingly attain strong statistical guarantees without the second-order density assumptions required to analyze mean-shift.
We prove that Quick Shift recovers the modes of an arbitrary multimodal density at a minimax optimal rate under mild nonparametric assumptions. This provides an alternative to known procedures with similar statistical guarantees; however such procedures only recover the modes but fail to inform us how to assign the sample points to a mode which is critical for clustering. Quick Shift on the other hand recovers both the modes and the clustering assignments with statistical consistency guarantees. Moreover, Quick Shift's ability to do all of this has been extensively validated in practice.
A unique feature of Quick Shift is that it has a segmentation parameter τ which allows practioners to merge clusters corresponding to certain less salient modes of the distribution. In other words, if a local mode is not the maximizer of its τ -radius neighborhood, then its corresponding cluster will become merged to that of another mode. Current consistent mode-seeking procedures [6,12] fail to allow one to control such segmentation. We give guarantees on how Quick Shift does this given an arbitrary setting of τ .
We show that Quick Shift can also be used to recover the cluster tree. In cluster tree estimation, the known procedures with the strongest statistical consistency guarantees include Robust Single Linkage (RSL) [2] and its variants e.g. [13,7]. We show that Quick Shift attains similar guarantees.
Thus, Quick Shift, a simple and already popular procedure, can simultaneously recover the modes with segmentation tuning, provide clustering assignments to the appropriate mode, and can estimate the cluster tree of an unknown density f with the strong consistency guarantees. No other procedure has been shown to have these properties.
Then we use Quick Shift to solve the modal regression problem [3], which involves estimating the modes of the conditional density f (y|X) rather than the mean as in classical regression. Traditional approaches use a modified version of mean-shift. We provide an alternative using Quick Shift which has precise statistical consistency guarantees under much more mild assumptions. The procedure returns one tree, whose head is the sample with highest empirical density. Right: τ set to a lower value. The edges with length greater than τ are no longer present when compared to the left. We are left with three clusters.

Setup
Let X [n] = {x 1 , ..., x n } be n i.i.d. samples drawn from distribution F with density f over the uniform measure on R d . Assumption 1 (Hölder Density). f is Hölder continuous on compact support x ∈ X and some 0 < α ≤ 1 and C α > 0.
The next assumption says that the level sets are continuous w.r.t. the level in the following sense where we denote the -interior of A as A := {x ∈ A, inf y∈∂A d(x, y) ≥ } (∂A is the boundary of A): Assumption 2 (Uniform Continuity of Level Sets). For each > 0, there exists δ > 0 such that for 0 < λ ≤ λ ≤ ||f || ∞ with |λ − λ | < δ, then L f (λ) ⊆ L f (λ ). Remark 1. Procedures that try to incrementally move points to nearby areas of higher density will have difficulties in regions where there is little or no change in density. The above assumption is a simple and mild formulation which ensures there are no such flat regions. Remark 2. Note that our assumptions are quite mild when compared to analyses of similar procedures like mean-shift, which require at least second-order smoothness assumptions. Interestingly, we only require Hölder continuity.

KDE Bounds
We next give uniform bounds on KDE required to analyze Quick Shift. Definition 3. Define kernel function K : R d → R ≥0 where R ≥0 denotes the non-negative real numbers such that R d K(u)du = 1. We make the following mild regularity assumptions on K. Assumption 3. (Spherically symmetric, non-increasing, and exponential decays) There exists nonincreasing function k : R ≥0 → R ≥0 such that K(u) = k(|u|) for u ∈ R d and there exists ρ, C ρ , t 0 > 0 such that for t > t 0 , k(t) ≤ C ρ · exp(−t ρ ). Remark 3. These assumptions allow the popular kernels such as Gaussian, Exponential, Silverman, uniform, triangular, tricube, Cosine, and Epanechnikov. Definition 4 (Kernel Density Estimator). Given a kernel K and bandwidth h > 0 the KDE is defined by Here we provide the uniform KDE bound which will be used for our analysis, established in [11]. Lemma 1.
[ ∞ bound for α-Hölder continuous functions. Theorem 2 of [11]] There exists positive constant C depending on f and K such that the following holds with probability at least

Mode Estimation
In this section, we give guarantees about the local modes returned by Quick Shift. We make the additional assumption that the modes are local maxima points with negative-definite Hessian. This assumption leads to the following.
which contains x 0 and does not intersect with other modes.
The next assumption ensures that the level sets don't become arbitrarily thin as long as we are sufficiently away from the modes. Assumption 5.
[Level Set Regularity] For each σ, r > 0, there exists η > 0 such that the following holds for all connected components We next give the result about mode recovery for Quick Shift. It says that as long as τ is small enough, then as the number of samples grows, the roots of the trees returned by Quick Shift will bijectively correspond to the true modes of f and the estimation errors will match lower bounds established by Tsybakov [15] up to logarithmic factors. We defer the proof to Theorem 2 which is a generalization of the following result. Theorem 1 (Mode Estimation guarantees for Quick Shift). Let τ < r M /2 and Assumptions 1, 2, 3, 4, and 5 hold. Choose h such that (log n) 2/ρ · h → 0 and log n/(nh d ) → 0 as n → ∞. Let M be the heads of the trees in G (returned by Algorithm 1). There exists constant C depending on f and K such that for n sufficiently large, with probability at least 1 − 1/n, and |M| = | M|. In particular, taking h ≈ n −1/(4+d) optimizes the above rate to d(M, M) = O(n −1/(4+d) ). This matches the minimax optimal rate for mode estimation up to logarithmic factors.
We now give a stronger notion of mode that fits better for analyzing the role of τ . In the last result, it was assumed that the practitioner wished to recover exactly the modes of the density f by taking τ sufficiently small. Now, we analyze the case where τ is intentionally set to a particular value so that Quick Shift produces segmentations that group modes together that are in close proximity to higher density regions.
In other words, an (r, δ) + -mode is a mode that is also a maximizer in a larger ball of radius r by at least δ when outside of the region where there is quadratic decay and smoothness (B(x 0 , r M )). An (r, δ) − -mode is a mode that is not the maximizer in its radius r ball by a margin of at least δ.
The next result shows that Algorithm recovers the (τ + , δ) + -modes of f and excludes the (τ − , δ) −modes of f . The proof is in the appendix. Theorem 2. (Generalization of Theorem 1) Let δ, > 0 and suppose Assumptions 1, 2, 3, 4, and 5 hold. Let h ≡ h(n) be chosen such that h → 0 and log n/(nh d ) → 0 as n → ∞. Then there exists C > 0 depending on f and K such that the following holds for n sufficiently large with probability Moreover, | M| ≤ |M| − |M − τ − ,δ |. In particular, taking → 0 and δ → 0 gives us an exact characterization of the asymptotic behavior of Quick Shift in terms of mode recovery.

Assignment of Points to Modes
In this section, we give guarantees on how the points are assigned to their respective modes. We first give the following definition which formalizes how two points are separated by a wide and deep valley. Definition 6. x 1 , x 2 ∈ X are (r s , δ)-separated if there exists a set S such that every path from x 1 and x 2 intersects with S and sup x∈S+B(0,rs) Lemma 3. Suppose Assumptions 1, 2, 3, 4, and 5 hold. Let τ < r s /2 and choose h such that (log n) 2/ρ · h → 0 and log n/(nh d ) → 0 as n → ∞. Let G be the output of Algorithm 1. The following holds with probability at least 1 − 1/n for n sufficiently large depending on f , K, δ, and τ uniformly in all x 1 , x 2 ∈ X . If x 1 and x 2 are (r s , δ)-separated, then there cannot exist a directed path from x 1 to x 2 in G.
Proof. Suppose that x 1 and x 2 are (r s , δ)-separated (with respect to set S) and there exists a directed path from x 1 to x 2 in G. Given our choice of τ , there exists some point x ∈ G such that x ∈ S+B(0, r s ) and x is on the path from x 1 to x 2 . We have f (x) < f (x 1 )−δ. Choose n sufficiently large such that by Lemma 1, , which means a directed path in G starting from x 1 cannot contain x, a contradiction. The result follows. The results in this section will show that in such cases, these pairs of points will not be assigned to the same cluster.
This leads to the following consequence about how samples are assigned to their respective modes. Theorem 3. Assume the same conditions as Lemma 3. The following holds with probability at least 1 − 1/n for n sufficiently large depending on f , K, δ, and τ uniformly in x ∈ X and x 0 ∈ M. For each x ∈ X and x 0 ∈ M, if x and x 0 are (r s , δ)-separated, then x will not be assigned to the tree corresponding to x 0 from Theorem 1. Remark 4. In particular, taking δ → 0 and r s → 0 gives us guarantees for all points which have a unique mode in which it can be assigned to.
We now give a more general version of (r s , δ)-separation, in which the condition holds if every path between the two points dips down at some point. The same results as the above extend for this definition in a straightforward manner. Definition 7. x 1 , x 2 ∈ X are (r s , δ)-weakly-separated if there exists a set S, with x 1 , x 2 ∈ S + B(0, r s ), such that every path P from x 1 and x 2 satisifes the following. (1) P ∩ S = ∅ and (2) sup x∈P∩S+B(0,rs) where x 1 , x 2 are defined as follows. Let P 1 be the path obtained by starting at x 1 and following P until it intersects S, and P 2 be the path obtained by following P starting from the last time it intersects S until the end. Then x 1 and x 2 are points which respectively attain the highest values of f on P 1 and P 2 .

Cluster Tree Recovery
The connected components of the level sets as the density level varies forms a hierarchical structure known as the cluster tree. Definition 8 (Cluster Tree). The cluster tree of f is given by Definition 9. Let G(λ) be the subgraph of G with vertices x ∈ X [n] such that f h (x) > λ and edges between pairs of vertices which have corresponding edges in G. LetG(λ) be the sets of vertices corresponding to the connected components of G(λ).

Definition 10.
Suppose that A is a collection of sets of points in R d . Then define Link(A, δ) to be the result of repeatedly removing pairs A 1 , A 2 ∈ A from A (A 1 = A 2 ) that satisfy inf a1∈A1 inf a2∈A2 ||a 1 − a 2 || < δ and adding A 1 ∪ A 2 to A until no such pairs exist.
Parameter settings for Algorithm 2: Suppose that τ ≡ τ (n) is chosen as a function of n such such that τ → 0 as n → ∞, τ (n) ≥ (log 2 n/n) 1/d and h ≡ h(n) is chosen such that h → 0 and log n/(nh d ) → 0 as n → ∞.
The following is the main result of this section, the proof is in the appendix.  Shift's output to samples have empirical density above a certain threshold and connecting nearby clusters, then this approximates the connected components of the true density level set. Moreover, we give guarantees that such points will be assigned to clusters which correspond to modes within its connected component.

Modal Regression
Suppose that we have joint density f (X, y) on R d × R w.r.t. to the Lebesgue measure. In modal regression, we are interested in estimating the modes of the conditional f (y|X = x) given samples from the joint distribution. Initialize directed graph G with vertices Y and no edges. Theorem 5 (Consistency of Quick Shift Modal Regression). Suppose that τ ≡ τ (n) is chosen as a function of n such such that τ → 0 as n → ∞, τ (n) ≥ (log 2 n/n) 1/d and h ≡ h(n) is chosen such that h → 0 and log n/(nh d+1 ) → 0 as n → ∞. Let M x be the modes of the conditional density f (y|X = x) and M x be the output of Algorithm 3. Then with probability at least 1 − 1/n uniformly in x such that f (y|X = x) and K satisfies Assumptions 1, 2, 3, 4, and 5,

Related Works
Mode Estimation. Perhaps the most popular procedure to estimate the modes is mean-shift; however, it has proven quite difficult to analyze. Arias-Castro et al. [1] made much progress by utilizing dynamical systems theory to show that mean-shift's updates converge to the correct gradient ascent steps. The recent work of Dasgupta and Kpotufe [6] was the first to give a procedure which recovers the modes of a density with minimax optimal statistical guarantees in a multimodal density. They do this by using a top-down traversal of the density levels of a proximity graph, borrowing from work in cluster tree estimation. The procedure was shown to recover exactly the modes of the density at minimax optimal rates.
In this work, we showed that Quick Shift attains the same guarantees while being a simpler approach than known procedures that attain these guarantees [6,12]. Moreover unlike these procedures, Quick Shift also assigns the remaining samples to their appropriate modes. Furthermore, Quick Shift also has a segmentation tuning parameter τ which allows us to merge the clusters of modes that are not maximal in its τ -radius neighborhood into the clusters of other modes. This is useful as in practice, one may not wish to pick up every single local maxima, especially when there are local maxima that can be grouped together by proximity. We formalized the segmentation of such modes and identify which modes get returned and which ones become merged into other modes' clusters by Quick Shift.
Cluster Tree Estimation. Work on cluster tree estimation has a long history. Some early work on density based clustering by Hartigan [9] modeled the clusters of a density as the regions {x : f (x) ≥ λ} for some λ. This is called the density level-set of f at level λ. The cluster tree of f is the hierarchy formed by the infinite collection of these clusters over all λ. Chaudhuri and Dasgupta [2] introduced Robust Single Linkage (RSL) which was the first cluster tree estimation procedure with precise statistical guarantees. Shortly after, Kpotufe and Luxburg [13] provided an estimator that ensured false clusters were removed using used an extra pruning step. Interestingly, Quick Shift does not require such a pruning step, since the points near cluster boundaries naturally get assigned to regions with higher density and thus no spurious clusters are formed near these boundaries. Sriperumbudur and Steinwart [14], Jiang [10], Wang et al. [17] showed that the popular DBSCAN algorithm [8] also estimates these level sets. Eldridge et al. [7] introduced the merge distortion metric for cluster tree estimates, which provided a stronger notion of consistency. We use their framework to analyze Quick Shift and show that this simple estimator is consistent in merge distortion. Modal Regression. Nonparametric modal regression [3] is an alternative to classical regression, where we are interested in estimating the modes of the conditional density f (y|X = x) rather than the mean. Current approaches primarily use a modification of mean-shift; however analysis for mean-shift require higher order smoothness assumptions. Using Quick Shift instead for modal regression requires less regularity assumptions while having consistency guarantees.

Conclusion
We provided consistency guarantees for Quick Shift under mild assumptions. We showed that Quick Shift recovers the modes of a density from a finite sample with minimax optimal guarantees. The approach of this method is considerably different from known procedures that attain similar guarantees. Moreover, Quick Shift allows tuning of the segmentation and we provided an analysis of this behavior. We also showed that Quick Shift can be used as an alternative for estimating the cluster tree which contrasts with current approaches which utilize proximity graph sweeps. We then constructed a procedure for modal regression using Quick Shift which attains strong statistical guarantees.

Cluster Tree Estimation Proofs
Lemma 5 (Minimality). The following holds with probability at least 1 − 1/n. If A is a connected component of {x ∈ X : f (x) ≥ λ}, then A ∩ X [n] is contained in the same component in C f (λ − ) for any > 0 as n → ∞.
Proof. It suffices to show that for each x ∈ A, there exists x ∈ B(x, τ /2) ∩ X [n] such that f h (x ) > λ − . Given our choice of τ , it follows by Lemma 7 of [2] that B(x, τ /2) ∩ X [n] is non-empty for n sufficiently large. Let x ∈ B(x, τ /2) ∩ X [n] . Choose n sufficiently large such that by Lemma 1, we where the last inequality holds for n sufficiently large so that τ is sufficiently small. Thus, we have f h (x ) > λ − , as desired.
Lemma 6 (Separation). Suppose that A and B are distinct connected components of {x ∈ X : f (x) ≥ λ} which merge at {x ∈ X : f (x) ≥ µ}. Then A ∩ X [n] and B ∩ X [n] are separated in C f (µ + ) for any > 0 as n → ∞.
Proof. It suffices to assume that λ = µ + . Let A and B be the connected components of {x ∈ X : f (x) ≥ µ + /2} which contain A and B respectively. By the uniform continuity of f , there existsr > 0 such that A + B(0, 3r) ⊆ A . We have sup x∈A \(A+B(0,r)) f (x) = µ + − for some > 0.
Choose n sufficiently large such that by Lemma 1, we have sup x∈X | f h (x) − f (x)| < /2. Thus, sup x∈A \(A+B(0,r)) f h (x) < µ + − /2. Hence, points in C f (µ + ) cannot belong to A \ (A + B(0,r)). Since A also contains A + B(0, 3r), it means that there cannot be a path from A to B with points of empirical density at least µ + with all edges of length less thanr. The result follows by taking n sufficiently large so that τ <r, as desired.
Proof of Theorem 4. By the regularity assumptions on f and Theorem 2 of [7], we have that Algorithm 2 has both uniform minimality and uniform separation (defined in [7]), which implies convergence in merge distortion.

Modal Regression Proofs
Proof of Theorem 5. There are two directions to show. (1) ifŷ ∈ M x thenŷ is a consistent estimator of some mode y 0 ∈ M x . (2) For each mode, y 0 ∈ M, there exists a uniqueŷ ∈ M which estimates it.


Multiagent System for Image Mining Multiagent System for Image Mining

The overdone growth, wide availability, and demands for remote sensing databases com‐ bined with human limits to analyze such huge datasets lead to a need to investigate tools, techniques, methodologies, and theories capable of assisting humans at extract‐ ing knowledge. Image mining arises as a solution to extract implicit knowledge intel‐ ligently and semiautomatically or other patterns not explicitly stored in the huge image databases. However, spatial databases are among the ones with the fastest growth due to the volume of spatial information produced many times a day, demanding the inves‐ tigation of other means for knowledge extraction. Multiagent systems are composed of multiple computing elements known as agents that interact to pursuit their goals. Agents have been used to explore information in the distributed, open, large, and heterogeneous platforms. Agent mining is a potential technology that studies ways of interaction and integration between data mining and agents. This area brought advances to the tech‐ nologies involved such as theories, methodologies, and solutions to solve relevant issues more precisely, accurately and faster. AgentGeo is evidence of this, a multiagent system of satellite image mining that, promotes advances in the state of the art of agent mining, since it relevant functions to extract knowledge from spatial databases.

Introduction
Technological advances have provided new ways to collect spatial data: satellites, radars, unmanned air vehicle, balloons, and many others. These instruments caused an enormous accumulation of images data on remote sensing databases for many reasons. These databases are the ones with the fastest growth due to the volume of spatial information produced all day long.
The systematic and intelligent analyses of the remote sensing images provide a unique opportunity for understanding how, when, and where changes take place in our world. Precious information exploited from spatial repositories has been promoting benefits on many areas, such as agricultural [1,2] (forecast of harvests and soil erosion), hydric [3] (use of water resources and verification of the water quality), urban [4] (urban planning and demographic inferences), forest [5][6][7] (monitoring deforestation and biomass control), limnology [8] (characterization of aquatic vegetation and identification of water types), meteorology [9] (weather and climate studies), air traffic [10] (information for safety in the air), and national security [11] (military strategic planning of operations and missions).
However, the manual analysis of huge databases is an extremely inconvenient task for human experts. Despite professionals such as physicists, meteorologists, and ecologists trained to analyze spatial data, the semi-automatic and intelligent interpretation of these data can be a useful tool to leverage the monitoring of the earth surface.
Data mining (DM) arises as solution to detect precious patterns semiautomatically and intelligently in huge databases. DM is defined as the nontrivial process of identifying valid, novel, potentially useful, and ultimately understandable patterns in data [12,13]. Image mining (IM) is also a challenging field which extends traditional data mining from structured data to unstructured data such as image data [14]. IM deals with the extraction of implicit knowledge, image data relationships, or other patterns not explicitly stored in the huge image databases such as remote sensing and medical database.
Despite the success in different applications, the research community of DM has dealt with some issues mining methodology, user interaction, efficiency and scalability, diversity of database models, and data mining and society [15]. The efficiency and scalability issue is particularly significant as the amount of data currently available are increasing rapidly day by day. Therefore, it is necessary to investigate new technological resources that improve some of these issues.
A multiagent system (MAS) is composed of multiple computing elements, known as agents that interact to pursuit their goals. Agents have software architecture for decision-making systems that are embedded in an environment. Consequently, this technology has been widely adopted in numerous applications to solve significant issues.
Agent mining is a new area under development that deals with interaction and integration of between data mining and intelligent agents, and aims to join resources to solve relevant problems that cannot be tackled by a single technology with the same quality and performance. This technology provides important resources, and promises to solve particular issues of both technologies involved.
In this chapter, we present an introduction about image mining, multiagent systems, and agent mining, as well as an overview of these areas. Besides that, a tool known as AgentGeo will be presented [16][17][18]. It is a multiagent system for satellite image mining that uses the agent resources to mine image data in remote sensing databases. AgentGeo improves the analysis and application of satellite image mining when compared to other systems. The agents leverage the process of image mining due to properties such as autonomy, interaction, reaction, and initiative. This system has been developed in Java and its functionalities are the creation, edition and selection of agents, selection and creation of the environment, and the use of agents to mine the satellite images. These agents can support many tasks of the image mining process, as well as improve the performance of the steps of preprocessing, transformation and feature extraction, and classification and evaluation.
The chapter is structured as follows: Section 2 discusses image mining, Section 3 describes multiagent systems, Section 4 presents the agent mining, Section 5 focuses on AgentGeo, and finally, Section 6 presents conclusion and mentions future research work.

Image mining
Data from computing systems are produced constantly, thereby causing the unbridled growth in the institutions, industries, and corporations databases. This explosive development is caused by several factors, including: internet versatility, reduction in the price of data storage devices, improvement of data collection tools, popularity of embedded systems, increasing of online work, among others. In addition, data are being made available in various formats such as video, text, image, and spreadsheet.
The data variety and volume are so immense that relevant information becomes hidden within databases. Unfortunately, it is difficult or even impossible for a human being to detect patterns handling huge and diversified databases. Several specialists such as economists, statisticians, forecasters, and communication engineers worked with the idea that patterns from data can be reached automatically, identified, validated, and used for various purposes [19]. Therefore, the need to assist the specialist in the extraction of knowledge from huge databases originated the knowledge discovery databases (KDD).
KDD is defined as the nontrivial process of identifying valid, novel, potentially useful, and ultimately understandable patterns in data [12]. This process is composed by the following steps selection, preprocessing, transformation, data mining, and interpretation/evaluation [12,13]. Data mining (DM) is a particular step in KDD process, where specific algorithms for extracting patterns are applied [12,13]. However, the term data mining has become popular in the database field, used by statisticians and data analyst like synonymous for KDD.
Data mining technology is application oriented and incorporates a variety of techniques, tools, and algorithms capable of extracting relevant information from a wide and diversified collection of databases. Image mining (IM) is a potential technology for data mining, and also a challenging field which extends traditional data mining from structured data to unstructured data such as satellite images, medical images, and digital pictures. Structured data patterns are different from unstructured data patterns. Extracted patterns from image databases are not easily interpreted and understood. Consequently, IM is considered more than just an extension of data mining. It is an interdisciplinary endeavor that incorporates knowledge of important areas such as machine learning, image processing, computer vision, data mining, database, and artificial intelligence [20,21].
In fact, IM is different from computer vision and image processing. The focus of these areas is extracting specific features from a single image, whereas the IM makes efforts for extraction patterns stored in the huge image databases. This implies in all aspects of databases such as the indexing scheme, the storage of images, and the image retrieval [20,21].
The image mining process is shown in Figure 1. Everything starts from an image database where data are being stored. There are often inconsistent data that need to pass through a preprocessing step in order to improve the level of database quality. Image processing techniques are applied on this step, which are mathematical operations to change the pixel values of images, such as filtering, histogram equalization, image subtraction, image restoration, and others [22]. In the transformation and feature extraction, the images undergo some transformations until identified the relevant objects present in these data. Then features from these objects are extracted, such as edge shape, texture, and length. Obtained such features, the mining step can be carried out using data mining techniques to discover significant patterns automatically or semiautomatically. These patterns need to be evaluated and interpreted by a specialist to obtain the knowledge, which can be applied to applications and can be useful on decision-making processes or on problem understanding.
Image mining process is analogous to data mining process. However, there are important differences between relational databases and image databases [20]. Some differences are as follows: • Domain dependency. Real-world activities belong to a given domain, and consequently have specific features and elements. In relational databases, the data values are semantically meaningful. However, at image databases, the identification of elements, their classes, and relationships are linked to the context itself, and a same image can have different information inherent to different domains.
• Spatial information (independent versus dependent position). In image databases, a simple image is composed by several elements (pixels). Each pixel is related to its neighbors, often forming a homogeneous region. Due to this, the image miners try to overcome this problem by extracting position-independent features from images before mining useful patterns.
• Unique versus multiple interpretation. In relational databases, the data values are easily understood. For example, field person is Paul; we already understood that the field stores a person's name. However, in image databases, an image data may reproduce ambiguous interpretations for the same visual pattern. For example, a simple intensity data can be seen like red, orange, or yellow.
Image mining is a promising and vast field, incorporating mature techniques. Despite the field is under development, there are techniques frequently used for object recognition, image indexing and retrieval, image classification and clustering, association rule mining, and neural network [23][24][25][26]. Besides that, image mining has become increasingly important die to its application in many areas such as health, meteorology, aerospace, agriculture, industry, air traffic, spatial research, among others.

GeoDMA: geographic data mining analyst
Satellite image mining, also known as remote sensing image mining, is an image mining process. Remote sensing image mining deals specifically with the challenge of capturing patterns, processes, and agents present in the geographic space, in order to extract specific knowledge for problem understanding or decision making related to a set of relevant topics, including land change, climate variations, and biodiversity studies. Events like deforestation patterns, weather change correlations, and species dynamics are examples of precious knowledge contained in remote sensing image repositories [27].
The spatial and multiband characteristics of the satellite images differ from the general category of image data. Therefore, remote sensing image mining demands specific image mining tools. GeoDMA is a toolbox for remote sensing image mining that arose based on methodology proposed by Silva et al. [28]. The software incorporates resources for segmentation, feature selection, feature extraction, classification, and multitemporal methods for change detection and analysis of remote sensing data [29].
GeoDMA works as a plugin of the software TerraView GIS [30], which provides the interface for the interpreter to visualize the geographic information stored in databases, to control the database, and also to display the objects' properties [31]. The image mining process in the GeoDMA is shown in Figure 2. This process is composed by five steps which described below:

Input data
This step is responsible for image selection. GeoDMA can only manipulate one image at a time. Therefore, it is necessary to define an image as input for segmentation process.

Segmentation
Segmentation is a process where an image pass through various transformations to detect its regions or objects. The level of detail to a segmentation process depends on the application purpose. This means that a segmentation process should stop when all important objects or regions of the application have been detected [29]. Therefore, segmentation is one of the most challenging tasks into digital image processing. The GeoDMA provides four segmentation algorithms as follows: • Region growing approach based on Ref. [32] • Segmentation approach based on Ref. [33] • Chessboard segmentation • Algorithm based on Ref. [34].

Feature extraction
This process aims to extraction attributes as well as spectral and spatial features of objects from the images. Spectral features relate all pixel values inside a region, therefore include metrics for maximum and minimum pixel values, or mean values such as amplitude, dissimilarity, and pixels mean. Spatial features measure the shapes of the regions, including height, width, or rotation [31].

Mining
Mining is the process where the algorithms to find a set of models (or functions) are defined, which describe and distinguish classes or concepts. The GeoDMA provides two ways for mining [31]: • Supervised classification using decision trees based on Ref.
[35] and • Unsupervised classification using self-organizing maps (SOM) [31]. Supervised classification can be divided in two process training and classification. Training is a process supervised by specialists, where data class to known objects is identified, and models to classification of objects are designed. Classification is a process where the build models are used to detect objects that are still unknown. Unsupervised classification is a process that searches for interpretable patterns in data and describe them forming regions known as cluster.
The search is based on spectral features such as variance, mean, and light intensity.

Evaluation
The output of GeoDMA is a thematic map. At image mining process, specialists should analyze whether the results are satisfying to application. In the decision tree classification model, the specialist should check whether the regions classified by the built models are valid. In the classifier SOM, generally the result produces more clusters than the desired patterns.
In this case, the specialist is responsible for label the patterns according to the application. However, if the results are not satisfactory in both process, previous tasks may be executed again [31].

Multiagent system
An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators [36]. Perception refers to input information received by agents at a certain moment. In general, the choice of its action at any instant may depend on the entire sequence of observed perceptions. These perceptions can occur through the physical world, via graphical interface, a collection of agents, the internet, or perhaps all combined [37]. The environment, which typically is both computational and physical, might be open or closed, and might or might not contain other agents.
Autonomy is the ability to analyze the environment and take their own decisions without the intervention of humans or other agents, controlling their acts and internal state. Social ability is the agent's ability to communicate with other agents or even with human beings using some kind of communication language. Reactivity is the agent's ability to respond in a timely manner, given a history of perception. Proactiveness (or initiative) refers to the agent's capacity of taking initiative in order to achieve their goals. These properties make agents of a technology capable of the following: cooperate in solving problems; share expertise; work in parallel on common problems; develop and implemented modularly; be fault tolerant through redundancy; represent multiple viewpoints and the knowledge of multiple experts; and mainly be reusable.
We can consider four basic kinds of agents: simple reflex agents, model-based reflex agents, goal-based agents, and utility-based agents [36].
• Simple reflex agents. Agents that select their actions based on current perception, ignoring the rest of the perceptual history.
• Model-based reflex agents. They are a simple reflex agent, but with some differences. These agents maintain some sort of internal state that depends on the perception history and thereby reflects at least some of the unobserved aspects of the current state.
• Goal-based agents. These agents know some sort of information about their goals. Based on this information and with internal state, these agents analyze and take their actions.
• Utility-based agents. These agents know some sort of information about their goals and have internal state. Besides that, there is a utility function that measures goal performance. This way, these agents choose actions to maximize its utility function.
Agents can learn new concepts and techniques, they can adapt to the needs of different users, they can anticipate the needs of the user, besides others abilities. During the last years, agents have become a powerful technology, which have been adopted in several applications as a solution to solve complex issues that cannot be solved by humans. In the area of remote sensing, for example, a complex task is to analyze remote sensing images, a human being is able to analyze a single remote sensing image, but analyze a significant amount of this data is unlikely because of limited human ability to reason and interpret huge information volumes.
However, when a simple agent cannot solve the problem, a MAS can be implemented. These systems have been studied since 1980s, but were only recognized in the mid of the 1990s. At that time, scientific and industrial interests raised due to the need of exploiting information and modern computing platforms, as well as distributed, open, large and heterogeneous ones [38]. A MAS is formed by two or more agents that interact between them to solve some specific problem. In general, the agents act on behalf of users with different goals. However, agents also may have the same goals, this is determined by the purpose of the system. The interaction in the system occurs through exchanging messages, and it is determined by the ability to coordinate, cooperate, and negotiate between agents [38].
Coordination is a property which aims performing some activity in a shared environment. The degree of coordination is determined by the extent to which they avoid extraneous activity by reducing resource contention, avoiding livelock and deadlock, and maintaining applicable safety conditions [39]. In general, we consider a relevant degree of coordination, when agent activities activities agents are well balanced inside of the environment as well as the operations are being distributed and involved among agents without any failure. There are some reasons why multiple agents need to be coordinated [40], which are as follows: • Their goals may be conflicting.
• Their goals may be interdependent.
• Agents may have different capabilities and different knowledge.
• Their goals can quickly achieve if different agents work together in a coordinated way.
Cooperation is coordination among nonantagonistic agents, while negotiation is coordination among competitive or simply self-interested agents [39]. At the MAS, agents can cooperate with each other to general goals of the system, or they can compete for their individual goals. Both features must be determined according to the general purpose of the agents into the application.
In the MAS, a key issue is how the agents will communicate. In particular, the communication among processes has long been an important research problem in computer science. In fact, concurrent processes need to be synchronized if there is a possibility that they can interfere with one another in a destructive way [41].
For example, being P1 and P2 two processes, which have access to some shared variable V, when P1 begins to update the value of V, P2 may act at the same moment, but without interfere in the P1 acts. Such communication among processes is like communication among agents. Among other reasons, agents communicate in order to coordinate actions more effectively, to distribute more accurate models of the environment, and to learn subtask solutions from one another [42]. This communication can be implemented through a determined programming language (for example, Java) or can be used an agent communication language like knowledge query and manipulation language (KQML) [41]. There are two types of communication in the multiagent systems [42]: • Direct communication: agents are able to communicate with each other using direct message exchange mechanisms between them. These mechanisms may be constrained in terms of throughput, latency, locality, and agent class.
• Indirect communication: consists of indirect transfer of information between agents. For example, when an agent wants to send a message to other agent, it relies on the mediating agent who is responsible for the exchange of information within the environment.
Multiagent systems are increasingly being implemented in several applications such as industry, distributed applications, applications for the internet, games, air traffic control, and teaching environment (e.g., distance education sites). MAS have become more and more important in many aspects of computer science such as distributed artificial intelligence, distributed computing systems, robotics, and artificial life. Some reasons to implement a MAS are • When the system is complex and the human being cannot or is unable to predict the behavior of that system.
• When it is expensive to keep a team of specialists working.
• When the activity involved put humans at risk.
• When the decision-making process requires performance, agents can solve the problem quickly using parallel processing.
• When it is necessary to ensure information privacy.

Multiagent system for image mining
Given the overview of multiagent systems and image mining, we have seen that the areas of agents and data mining emerged separately. Both independent research streams have been created and originally evolving with separate aims and objectives. The area of agents, for example, aims to study the autonomous and independent behavior of agents, and data mining, more comprehensively, dealt with the KDD process. Despite emerged separately, several similar aspects of these areas appear such as user-system interaction, human roles and involvement, constraints, dynamic modeling, life-cycle and process management, domain factor, and organizational and social factors [43].
Agents is a powerful technology, generally used to solve complex problems in distributed environments where agents can cooperate, coordinate, and communicate their activities in order to reduce the complexity of the problem. Agent research focuses on theoretical, methodological, technical, experimental, and practical issues and the means to handle system complexities [44]. Agent technologies have been contributing to many diverse domains such as software engineering, user interfaces, e-commerce, information retrieval, robotics, computer games, education and training, ubiquitous computing, and social simulation.
Data mining is an application-oriented technology that employs techniques, tools, and algorithms capable of extracting relevant information (or patterns) semi-automatically and intelligently from a massive and diversified collection of datasets. Data mining has been used in web mining services, text mining, medical data mining, meteorological data mining, governmental services, fraud detections, securities, and bioinformatics.
Agents and data mining deal with their specific problems and limitations. Both areas face critical challenges that the other technology might contribute. Agents can leverage the KDD process on data selection, extraction, preprocessing, and integration, and they are an excellent choice for peer-to-peer, parallel, and distributed computing. Agents can bridge the gap between humans and software systems by acting as interfaces that can sense and affect human-mining needs or multisource mining [44]. In the same way, the knowledge acquired through data mining processes provides more stable, predictable, and controllable models for dispatching and planning, and can be used for learning on multiagent systems.
Therefore, agents are elements that can leverage the data mining process, and data mining can contribute significantly to agent's area. A few years ago, researchers have studied means of joining forces between agent and data mining technologies. These studies have given rise to a new research field which became known as agent mining [44][45][46][47][48][49] or agent-mining interaction and integration [43,[50][51][52]. Agent mining is the most popular term.
Agent mining refers to the methodologies, principles, techniques, and applications for the integration and interaction of agents and data mining, as well as the community that focuses on the study of the complementarity between these two technologies, for better addressing issues that cannot be tackled by a single technology with the same quality and performance [44,49].
Agent-mining area is under development; therefore, some issues demands research on theoretical, technological, and methodological aspects. This area follows two fronts of research, which are • Agent-driven distributed data mining (otherwise known as multiagent-driven data mining, and multiagent data mining): studies ways to use agents to enhance data mining processes and systems. Agents can be used in data mining for different purposes such as agent-based data mining system, agent-based data warehouse, agents for information retrieval, mobile agents for distributed data mining, among others [49].
• Data mining driven agents: investigates issues related to the proper and formal representation of extracted knowledge models from data mining applications such as collaborative learning in multiagents, data mining driven agent learning, reasoning, adaptation and evolution, data mining-driven multiagent communication, planning and dispatching, data mining agent intelligence enhancement [52].
According to Seydim [53], in several steps for knowledge discovery, an agent can be used to automate the individual tasks, including data preprocessing, data mining as well as search for patterns of interest using learning and intelligence in classification, clustering, summarization, and generalization. For example, in the data mining step, an agent can perform automatic sensitivity analysis to determine which parameters should be used in learning. This would reduce the dependency of having domain experts available to examine the problem every time something changes in the environment. The great advantage of using agents in automation of data mining is indicated as their possible support for online transaction data mining [53].
This way, agents can also support and enhance the image mining process in some steps: • Preprocessing: When an image is added in a database or when it is defined the image database, an agent can perceive these events and automatically examine the data. If the agents perceive anomalies on these images, it can automatically use digital image processing techniques to deal with them. Agent acts can be taken based on rules built by a specialist. This reduces the dependence of having domain experts available for analyzing several images.
• Transformation and feature extraction: agents can be subordinated to transformation and feature extraction task. They can be trained to work together in a much faster way on data transformation, on detecting objects and on the segmentation process.
• Mining: agents can discover significant patterns automatically or semi-automatically in image databases. For example, on classification process, agents can be trained by specialists and can perform intelligently the classification process. Besides that, agents can specialize in one standard and the process may be carried out by several agents simultaneously. This certainly increases the performance and accuracy of the classification process.
• Interpretation and evaluation: knows itself that the evaluation of knowledge is realized usually performed by specialists. In general, specialists have experience in data analysis of a particular domain. In fact, agents can learn with specialists about a particular domain, that is, they can perform this task to support or even substitute the specialists.
Agent mining is a new area with huge opportunities that brings several advantages to multiagent systems, data mining, and machine learning, as well as new derived theories, tools and applications that are beyond any individual technology. The following section presents AgentGeo, a multiagent system of satellite image mining. The tool employs agent to leverage the image mining process, and uses the knowledge of image mining process to agent learning.

AgentGeo
AgentGeo emerged based on the ideas proposed by [16,17], which used agents to mine remote sensing images. This tool is being developed in Java language, and its initial version works along with GeoDMA and TerraView. This system brings advances in the state-of-the-art of multiagent systems and image mining, presenting relevant resources and precious functionalities. For example, this system implements functions such as creation, edition and selection of agents, selection and creation of the environment and use of agents for image mining. Moreover, it is capable of performing the classification process with multiple images at a time, differently from GeoDMA that is limited to only one image during the entire image mining process.

System architecture
The system architecture is shown in Figure 3.
The image mining process begins in TerraView, which provides the structure to insert and view the images. In fact, the database are created, and several images are inserted. GeoDMA is responsible for processing the data, therefore it receives a single image at a time, and segment and extract the characteristics of each one. These processed images are stored in the image database. AgentGeo connects to the same image database, and receives as input the processed data. So, the environment for the agents can be created, that is, the user defines the image process through GeoDMA. After that, the mining agents responsible for automatically mining these images are defined; that is an automatic process performed by agents and has as output the mined images. These images are stored in the image database, and can be visualized through TerraView. However, AgentGeo provides statistical data about the mining performed by agents, such the number of segments classified by agents. Each process presented may be performed several times.

Agent's structure
There are two agent types implemented at AgentGeo: the simple agents and a monitor agent. These agents have different properties, features, goals, functions, and behaviors. The simple agents are simple reflex ones, that is, they select their actions based on current perception, ignoring the rest of the perceptual history. Agent's perception occurs in the moment that a user defines the environment and selects the agents on the AgentGeo. Agents have a degree of autonomy, when they are pursuing their particular goals on mining process. The user is not able to forecast the agent actions, because they autonomously decide which goals to pursue.
Besides that, simple agents compete for their goals at the environment just looking to their own well-being, and when their goals conflict with other agents, a monitor agent is responsible for coordinating environmental disputes. For example, two simple agents "A" and "B" are in conflict because of resource at the environment. Soon, the monitor agent perceives the conflict, and takes initiative to finish it. Despite the monitor agent also be a simple reflex agent, we can notice that the behavior of the monitor agent is different from the simple agent.
The monitor agent is cooperative and it has the ability to communicate with other agents.
The communication at the environment occurs indirectly as shown in Figure 4. Simple agents only store information about their goals, whereas the monitor agent has the overview of the environment and knows the other agents. Thus, this communication architecture partially solves the communication, coordination, and negotiation problems and considerably reduces the complexity of the MAS.
The agents are implemented through a thread structure and they can perceive and act simultaneously in a certain environment. The simple agents are built by the user, and their internal structure consists of: a description (agent's name), a knowledge base (information about their goals obtained in the training step), and a metadata (extra information about the agent for the usability). This development has four steps as illustrated in Figure 5.
Any agent needs a knowledge base to reason about their acts. That base is built through the training phase carried out in GeoDMA and after that comes the segmentation and extraction of features. The agent's knowledge base is formed by a decision tree structure. Firstly, it is defined the image database, and the images are preprocessed on the segmentation and features extraction step. At training step, segment samples that are known to users referring to a specific class are selected by them, according to the agent goals referring to a specific class according to the agent goals. After that, the users can generate a decision tree using GeoDMA, which provides a resource to build it adapted for spatial data mining using the supervised algorithm C4.5 [35]. With the decision tree model generated by GeoDMA, the mining step happens automatically. That process is performed by means of thresholds referring to the spatial and spectral attributes of the segments present in the image. All the steps are performed several times in order to have a consistent model. If the user identifies that the results of the model are not satisfactory, he can return to previous steps and perform them again. Otherwise, the model will serve as the knowledge base for the agent, which integration step occurs when the user creates the agent within AgentGeo, that is, the user informs name, metadata, and the knowledge base to the agent.  All agents of AgentGeo are stored on"agentes.dat" file and can be used whenever necessary. For instance, consider the development of an agent that aims to detect water bodies. The image database is defined, and the images are preprocessed one at a time using GeoDMA. The users select several samples of water bodies and several samples of what is not water bodies. A decision tree model is gendered, and the mining process occurs at GeoDMA. After the user analyzed and evaluated positively the models, he can create the agent in AgentGeo.

Case study
In this section, we will briefly describe two case studies presented in Ref. [17]. The purpose is remote sensing images mining to detect exposed fields, vegetation fields, and water body patterns in Rio Grande Do Norte state, and water bodies patterns in 15 cities of the Ceara state, Brazil. Therefore, three agents were created to detect these patterns, using the methodology for agents construction presented in 5.2 Section.
The image database is formed by LANDSAT-8 satellite images, which are available at <https:// earthexplorer.usgs.gov/>. At first, image database is build using TerraView and satellite images are inserted. Second, the study region was delimited as well as the area of Rio Grande do Norte state and cities of the Ceara state using digital image processing techniques embedded in TerraView. Third, the images were segmented using the region growing algorithm implemented in GeoDMA, and the spatial and spectral features were extracted from the image database. Fourth, the AgentGeo is loaded, the environment is defined, and the mining agents, implemented in the tool, are selected. At last, the mining results are evaluated by visual inspection using Google Maps images.
This study shows that the methodology of AgentGeo is effective, reaching 92.66% of accuracy on first study and 95.04% on second study. It is important to make it clear that the results of the mining process would be the same if you used only GeoDMA for the mining process. However, the process would occur with one image at a time, which is different from AgentGeo approach, that is capable of mining multiple images, improving the performance of mining process, and keeping the results precision.

Conclusion
In this work, we proposed an approach on image mining, multiagent system, agent mining, and presented AgentGeo. In the image mining section, similarities and differences between image mining, data mining, and image mining process were presented. GeoDMA is a toolbox for remote sensing image mining. At the multiagent system section, concepts, proprieties, features, behaviors, structure, and applications about agents and multiagent system were described. Multiagent system for image mining section presented an overview about the new area known as agent mining; we presented and suggested improvements in the integration and interaction of agents and data mining, and of multiagent and image mining. Finally, AgentGeo was introduced, a multiagent system for image mining which uses agent mining to exploit remote sensing databases. This promising tool brings advances in the state-of-the-art in multiagent systems and image mining, due to relevant resources that leverage the image mining process.
As future work, we hope to integrate more resources into AgentGeo to provide operational advantages, optimization, and innovation. We concluded that agents can be used in several steps in image mining process, that is, we can use them and build modules at AgentGeo for the steps of preprocessing, transformation and feature extraction, and interpretation and evaluation. We can also expand the studies by creating other agents with different goals. For example, agents to road mining, to cloud mining, to deforestation mining, or to mine any spatial object.


extfieldsofstudy:
Mathematics
Computer Science

text:
Generalization Tower Network: A Novel Deep Neural Network Architecture for Multi-Task Learning

Deep learning (DL) advances state-of-the-art reinforcement learning (RL), by incorporating deep neural networks in learning representations from the input to RL. However, the conventional deep neural network architecture is limited in learning representations for multi-task RL (MT-RL), as multiple tasks can refer to different kinds of representations. In this paper, we thus propose a novel deep neural network architecture, namely generalization tower network (GTN), which can achieve MT-RL within a single learned model. Specifically, the architecture of GTN is composed of both horizontal and vertical streams. In our GTN architecture, horizontal streams are used to learn representation shared in similar tasks. In contrast, the vertical streams are introduced to be more suitable for handling diverse tasks, which encodes hierarchical shared knowledge of these tasks. The effectiveness of the introduced vertical stream is validated by experimental results. Experimental results further verify that our GTN architecture is able to advance the state-of-the-art MT-RL, via being tested on 51 Atari games.

Introduction
Reinforcement learning (RL) is an active research area of artificial intelligence (AI). It aims at making a computer, as the agent, take actions in an environment to maximize the cumulative reward. The past decades have witnessed the evolution of RL (Sutton and Barto 1998), especially the recent development of deep RL (DRL) . The recent DRL almost achieves human-level intelligence in mastering single task, for a small yet growing set of scenarios. Unfortunately, DRL is still in its infancy for handling more than one task, i.e., multi-task RL (MT-RL) (Caruana 1998).
For MT-RL, recent works have already achieved notable advances on inter-task transfer, showing the feasibility of transferring learned knowledge from one task to another. Actor-mimic (Parisotto, Ba, and Salakhutdinov 2016) and policy distillation ) leverage techniques from model compression to perform state-of-the-art intertask transfer. Beyond inter-task transfer, multi-task generalization has also been considered in MT-RL. For example, shared representations have been verified to be formable, Copyright c 2018, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. when facing diverse tasks (Borsa, Graepel, and Shawe-Taylor 2016;Romoff, Bengio, and Pineau 2016). The approach of progressive networks (Rusu et al. 2016) is insightful to transfer knowledge from the hidden layers of previously learned model to learn a never-experienced task. Yet, its multi-task ability is accomplished by storing all learned models and relying on inference to specify a corresponding model when switching to a specific task.
Despite all recent achievements for MT-RL, the multitask generalization is still limited, or requires all learned models to be stored and then be manually selected according to the task. To avoid such limitation, this paper proposes 1 a novel deep neural network architecture for MT-RL, which is capable of handling all tasks within a single model. It is worth pointing out that our architecture can be easily combined with the existing MT-RL algorithms, e.g., actor-mimic (Parisotto, Ba, and Salakhutdinov 2016), policy distillation , progressive networks (Rusu et al. 2016) and EWC (Kirkpatrick et al. 2017), as our work mainly focuses on the architecture of deep neural network for MT-RL. To be more specific, this paper proposes the generalization tower network (GTN), a novel deep neural network architecture for MT-RL. The proposed GTN is inspired by our finding in Figure 1, which implies that similar tasks in MT-RL share some hierarchical common knowledge, called hierarchical shared knowledge. As shown in Figure 1, we find that all shooting games 2 in Atari (Bellemare et al. 2013) need to shoot continuously, and some of them may further require shooting to targets or even shooting to valid targets. Thus, knowledge needs to be encoded in hierarchy for different tasks of MT-RL. Accordingly, our GTN architecture has both horizontal and vertical streams. Horizontal streams learn to extract multi-layer representations from 1 Code to reproduce this work is publicly available online for facilitating future research: https://github.com/YuhangSong/GTN. 2 For obtaining scores, we asked 10 volunteers aging from 18 to 26 to play these shooting games. All of them are naive to these games, avoiding the influence of prior knowledge. The volunteers were told to play the games only with three kinds of knowledge: I. shoot continuously, II. aim to targets and then shoot continuously and III. aim to valid targets and then shoot continuously. Scores of games are not displayed (by occlusion) for making knowledge valid. With each knowledge, they play three shooting games in a random order, each for 90 seconds.

SpaceInvaders Phoenix Carnival
SpaceInvaders aim to any target and then shoot continuously shoot continuously aim to valid target and then shoot continuously Figure 1: Hierarchical shared knowledge in shooting games. SpaveInvaders can obtain high scores by knowledge shoot continuously, since there are substantial and intensive targets. As for Phoenix, the targets are much fewer but are all valid targets, high scores are obtained with knowledge aim to targets and then shoot continuously. Carnival is much more demanding for the players, since there are both bad targets and good targets. Thus, we should aim to valid targets and then shoot continuously. Thus, all three shooting games share knowledge of shoot continuously. Both Phoenix and Carnival games have knowledge of aim to targets and then shoot continuously. Only Carnival holds knowledge of aim to valid targets and then shoot continuously.
high-dimensional input, whereas vertical streams generalize hierarchical shared knowledge. The functionality of the introduced vertical steams is two-fold: (1) When facing multiple tasks, low-level features may be shared in the vertical stream; (2) Some tasks can be learned by short streams, while others may be learnt by long streams. We further verify through experiments that vertical streams of GTN are capable of improving the performance of mastering multiple tasks within a single model for MT-RL. More importantly, experimental results show that our GTN approach is better than the state-of-the-art asynchronous advantage actor-critic (A3C) for MT-RL (Romoff, Bengio, and Pineau 2016) on 51 games of Atari, with higher scores.

Related Work
Deep Reinforcement Learning Taking advantage of recent success in deep learning (DL) (LeCun, Bengio, and Hinton 2015) and long-existing research in RL (Sutton and Barto 1998), DRL has made great progress, starting from deep Q-learning . Notable advances include double Q-learning (Van Hasselt, Guez, and Silver 2016), prioritized experience replay (Schaul et al. 2015), dueling networks (Wang et al. 2016), and asynchronous methods . Utilizing advances from these key achievements, A3C in  achieves the state-of-the-art performance in mastering RL tasks, reaching human level intelligence in various domains. However, in spite of the existing achievements, agents in above RL works are still limited to learning and mastering one task at a time.

Multi-Task Reinforcement Learning
MT-RL is crucial in the RL area, which learns more than one task at a time. In this regard, the ultimate goals for MT-RL can be classified as: (1) Inter-Task Transfer, i.e., the RL model learned for one task can help in learning other tasks; (2) Multi-Task Generalization, i.e., multiple tasks can be handled by learning a single RL model. Above topics have long been reputed as a critical challenge in many AI works (Ring 1994;Silver, Yang, and Li 2013;Taylor and Stone 2011). Recent advances in this topic can be generally divided into following two directions.
Inter-Task Transfer. Inter-Task transfer works primarily focus on how to transfer knowledge from the learned model to a new one, when facing a new task. Upon such transfer, the new learned models are capable of mastering more tasks, in a way that different tasks are learned one by one. However, the main challenge for inter-task transfer is catastrophic forgetting (McCloskey and Cohen 1989;McClelland, McNaughton, and O'reilly 1995;Ratcliff 1990), which means that the newly learned knowledge may completely break previous learned knowledge of old tasks. Some novel advances (Rusu et al. 2016;Fahlman and Lebiere 1990;Kirkpatrick et al. 2017;Terekhov, Montone, and ORegan 2015;Parisotto, Ba, and Salakhutdinov 2016) are produced to this issue.
Multi-Task Generalization. Multi-Task generalization (Borsa, Graepel, and Shawe-Taylor 2016;Romoff, Bengio, and Pineau 2016;Sermanet, Xu, and Levine 2016) generally concentrates on how to make an agent master several RL tasks only with a single learned model. Linking preknowledge towards any tasks, the agent is greatly confused by the diversity of state representations and strategies. In short, multi-task generalization in RL mainly aims at learning a generalized model across diverse tasks. In this direction, previous works have been proposed with novel models and algorithms that are more capable of generalizing and transferring with shared representations. For example, the latest literature (Borsa, Graepel, and Shawe-Taylor 2016) verifies the practicability of learning shared representations of value functions. However, it has been demonstrated that a completely shared model performs poorly (Romoff, Bengio, and Pineau 2016). Furthermore, to deal with the problem of task diversity, the agent in (Romoff, Bengio, and Pineau Global / local GTN 2016) is designed to have different hidden layers and output layers for each task, remaining shared convolutional layers across all tasks. However, it can only learn to master 3 Atari games. An additional network (Sermanet, Xu, and Levine 2016) has been designed to detect which task is the agent facing, and then to favor or select a corresponding stream in the model.
In summary, recent works have made great advances in MT-RL. However, they still use conventional deep neural network architecture, which does not consider hierarchical shared knowledge across multiple tasks. This leads to poor generalization ability in mastering diverse tasks, which can be solved by the proposed GTN architecture presented in the following.

Generalization Tower Network
In this section, we introduce our GTN approach for MT-RL, which learns to master multiple RL tasks in a single learned model. Specifically, the architecture of GTN is proposed at first. Then, we present how GTN is implemented to work on multi-task scenarios in an asynchronous way, so that all tasks go through the same GTN model without requiring any task labels. Figure 2 shows the architecture of GTN. We can see from this figure that the input to GTN is frame content from the game emulator, and the output is state value V and policy π. Here, π is the set of vectors with different size, corresponding to the action space of different tasks. As shown in Figure   2, GTN is a two-dimensional network, including horizontal and vertical streams. In the horizontal stream, representations features are extracted "layer by layer", the same as the traditional deep neural networks. To model hierarchical shared knowledge, the vertical stream is introduced in GTN to master multiple tasks, in which higher levels 3 learn more representations features than lower ones. As such, our GTN architecture is suitable for diverse tasks, which can be well mastered by different numbers of features. In the following, we present the horizontal and vertical streams of our GTN architecture, respectively.

Architecture of GTN
Horizontal streams with multi-layers. To present the horizontal streams of GTN, we take the horizontal stream at the level 1 in (Figure 2) as an example. The horizontal stream of GTN contains multiple convolutional layers, i.e., C (1,1) → C (1,2) →, ..., → C (1,n) →, ..., → C (1,N ) , to learn features from the high-dimensional input. Thus, the horizontal stream contains layers {1, · · · , n, · · · , N }. Finally, the convolutional layer is flattened into a vector with flatten layer (F ), followed by a single LSTM layer (L 1 ) to model the temporal correlation of RL tasks. For the LSTM chunk, we use the standard LSTM (Hochreiter and Schmidhuber 1997) which includes input, forget and output gates. Thus, the horizontal stream of our GTN mainly produces traditional representations features of deep convolutional neural network (Simonyan and Zisserman 2014) and learns tempo-ral information with LSTM, the same as (Hausknecht and Stone 2015). Similar structure holds for horizontal streams at other levels.
Vertical stream with multi-levels. Our GTN architecture includes the vertical streams with levels {1, · · · , m, · · · , M }. Specifically, after the first convolutional layer at level 1, i.e., C (1,1) , the stream of GTN splits into two sub-streams. One continues to go through the remaining horizontal layers in level 1, while the other is connected to level 2, acting as the input to level 2. As such, every level m uses the feature map from the first convolutional layer at level m-1 as input. This way, the vertical stream is formed in GTN. Then, the last LSTM layers of the multiple horizontal streams, i.e., L 1 , L 2 , · · · , L m , · · · , L M are combined together for yielding the final output. Let the activation of LSTM layer L m be a m . The activation of the concatenation layer (H) is obtained by the Rectified Linear Units (ReLU) operation ReLU = max(0, x), as follows, where {T 1 , · · · T m · · · , T M } are learned parameters, to concatenate M horizontal streams into a single layer. Note that T m ∈ R S×A , where S is the size of LSTM layer and A is the size of concatenation layer. The vertical stream aims to model hierarchical shared knowledge, making our GTN fit to different tasks.

Implementation of GTN in asynchronous method
To learn multiple tasks with a single GTN model, we develop an asynchronous method to implement GTN based on . Our asynchronous method is summarized in Figure 3. Specifically, there is one thread (thread 0) holding a global GTN and each of other multiple threads (thread 1, ..., i) holding a local GTN and a game emulator. Note that both global and local GTNs use the same architecture as depicted in Figure 2. In a single episode for each thread, the following procedure is conducted.
• Stage 1: The thread copies all parameters from global GTN to its own local GTN. • Stage 2: The local GTN interacts with the game emulator in the same thread and stores the experiences. • Stage 3: Once gathering enough experiences (i.e., meet terminal condition t max ), the thread updates all parameters in the global GTN via accumulating gradient.
At Stage 2, the local GTN and the game emulator interact with each other for several steps {1, · · · t · · · , t max }. At step t, GTN obtains observation o t from the game emulator, which is the frame content gray-scaled and down-sampled to 42 × 42. Then, observation o t is provided to GTN, together with the LSTM feature from the GTN at last step f t−1 . Afterwards, GTN produces a set of policies with different size, corresponding to the size-varied actions of MT-RL. Given policy π, action a t can be generated with standard deviation ε to ensure exploration. According to a t , the game emulator updates from o t to o t+1 , returning a step reward r t . At the end of step t, a set of experiences {o t , f t−1 , a t , r t } is stored for updating parameters at Stage 3. At Stage 3, we apply the accumulating gradient to update parameters as follows. We assume that θ and θ v are parameter vectors that generate policy π and state value V , in global GTN. In local GTN, θ and θ v are parameter vectors for π and V . Then, the accumulating gradients are calculated as In the above equation, R t denotes the discounted reward: where r i is the step reward, and γ (=0.99) is the discount factor for future rewards.

Generalization Analysis
Now, we analyze the generalization ability of our GTN approach across multiple tasks. In this paper, generalization ability refers to the performance of GTN, when mastering different numbers of tasks within a single learned model. Since our GTN introduces the vertical stream for learning multiple tasks, we investigate the generalization ability of GTN at different levels of the vertical stream. For such investigation, we measure the average perturbation sensitivity (APS) (Rusu et al. 2016), which indicates how much the final output relies on a specific level of GTN. To obtain APS, we first inject Gaussian noise N (0, 1) to a specific horizontal stream at one level for producing perturbation. Then, we measure the impact of this perturbation on the performance, using APS. In this paper, APS is measured by the percentage of score dropping, due to the noise perturbation. We compute the relative APS (RAPS) via dividing APS by the sum of the APSs of all streams, to investigate the relative dependency of different levels of GTN. Figure 4(a) shows the RAPS results of different levels in GTN, when learning 1 to 18 tasks. The results are obtained by running our GTN approach, averaged over all shooting games of Atari as list in Figure 5. From Figure 4(a), we can see that RAPS of high-level (e.g., levels 3 and 4) is increased at more tasks. This indicates that when learning to master more tasks, GTN turns to rely on higher levels. Thus, the proposed architecture of GTN, especially the vertical stream, is effective in improving the generalization ability for MT-RL. Figure 4(b) further draws the RAPS results of different levels versus training episodes, when running our GTN approach to handle all shooting games of Atari. One may observe that RAPS can be increased at more training episodes, for high-level in GTN (e.g., levels 3 and 4). By contrast, RAPS of low-level decreases alongside increased episodes. In a word, more levels in the vertical stream and larger number of training episodes are required for improving the generalization ability for MT-RL.

Experiment Setup
In our experiments, we manually divide the Atari games into 6 categories, according to the similarity of tasks. All tasks belonging to each category can be found from Figures 5 and 6. They cover 51 of total 57 Atari games, and the remaining 6 games are hard to be classified. Thus, our experiments were conducted on those 51 games.
GTN Setup. The GTN is simply set with 4 levels (M = 4) in vertical streams and 4 layers (N = 4) in horizontal streams. In our GTN, the architecture of each horizontal stream is the same as that of A3C , for fair comparison. Specifically, for all convolutional layers, the kernel size is 3 × 3 and the number of kernels is 32. The stride during convolution is set to 2. Besides, the size of the LSTM layers is all set to 288. For playing the Atari games, the minimal and maximal numbers of actions are 5 and 18, respectively. To master diverse tasks in a unified form, during the first 2 episodes, the agent detects and records the maximal step reward. Afterwards, every step reward is normalized by dividing this stored maximal step reward. We follow  to configure other unmentioned settings.
Evaluation Metric. To evaluate the multi-task generalization ability, we measure how much a multi-task agent achieves or exceeds the scores performed by the corresponding single-task agent. To this end, we measure the relative final scores (RFS):

RFS =
Score multi-task Score single-task , which eliminates the diverse scales of scores across multiple tasks 4 . Note that the same metric is also used in most recent works of MT-RL Parisotto, Ba, and Salakhutdinov 2016

Performance Analysis of GTN
Since the vertical and horizontal streams are the main contribution of the proposed GTN architecture, we analyze the performance of our GTN approach, with different numbers of levels (M ) and layers (N ). In the following, the performance of agent is evaluated on single task and multiple tasks, when M ranges from 1 to 4 and N varies from 1 to 6. Figure 7(a) shows the performance of GTN on single task, along with increased N and M . The performance in this figure is measured by RFS, averaged over all shooting games of Atari. We can see from this figure that when making GTN work on single task, the averaged RFS dramatically changes at different N and it can achieve 104% at N = 4 and M = 2. In contrast, M slightly improves the performance of our GTN on single task. This implies the horizontal stream is more effective in handling single task in RL. For multiple tasks, the averaged RFS values of playing all shoot-ing games are used for performance evaluation. Figure 7(b) shows the averaged RFS values of GTN on handling multiple tasks of all shooting games, with different N and M . We can find from this figure that enhancement of M from 1 to 4 improves the averaged RFS from 52% to 65% at N = 4, whereas N has little impact on the averaged RFS. In a word, the above analysis indicates that the horizonal stream in our GTN approach is effective in mastering single task, and the vertical stream is "good at" learning multiple tasks.

Evaluation on Performance.
This section compares the RFS results between the proposed GTN and conventional MT-A3C (Romoff, Bengio, and Pineau 2016), both based on . In particular, MT-A3C means that the asynchronous MT-RL framework is implemented with the same traditional deep neural network architecture as A3C, instead of our GTN. It is worth mentioning that we do not compare against progres- RFS RFS for multi-task 50%-52% 52%-54% 54%-56% 56%-58% 58%-60% 60%-62% 62%-64% 64%-66% sive network (Rusu et al. 2016). It is because progressive network (Rusu et al. 2016) requires manual inference to select corresponding stored model for different tasks, whereas our GTN approach requires no task labels and enables mastering multi-tasks in one model. In other words, it is unfair to compare our approach with (Rusu et al. 2016). Figures 5 and 6 plot the RFS results of MT-A3C and GTN for each game of different categories. We can see from these figures that GTN performs better than MT-A3C in all 6 categories, with up to 23% improvement (shooting games) in average RFS. Interestingly, the performance gap between GTN and MT-A3C is especially large for Shooting Games and Adventure Games, which have more tasks than other 4 categories. This indicates that our GTN approach is more suitable for MT-RL with larger number of similar RL tasks, thus showing the high generalization ability of our GTN approach. This also makes our approach practical in real-world application.

Conclusion
This paper has proposed a novel deep neural network architecture called GTN for MT-RL, which incorporates both horizontal and vertical streams within a single learned model. Different from the conventional MT-RL approaches, the vertical stream proposed in our GTN architecture enables better performance when dealing with multiple tasks. This is mainly motivated by the fact that multiple tasks may share common knowledge in hierarchy. Experimental results also showed that our GTN approach improves the performance of A3C in MT-RL, over 51 Atari games.


extfieldsofstudy: Business Physics

text:
An Optimization of Inventory Demand Forecasting in University Healthcare Centre

Healthcare industry becomes an important field for human beings nowadays as it concerns about one’s health. With that, forecasting demand for health services is an important step in managerial decision making for all healthcare organizations. Hence, a case study was conducted in University Health Centre to collect historical demand data of Panadol 650mg for 68 months from January 2009 until August 2014. The aim of the research is to optimize the overall inventory demand through forecasting techniques. Quantitative forecasting or time series forecasting model was used in the case study to forecast future data as a function of past data. Furthermore, the data pattern needs to be identified first before applying the forecasting techniques. Trend is the data pattern and then ten forecasting techniques are applied using Risk Simulator Software. Lastly, the best forecasting techniques will be find out with the least forecasting error. Among the ten forecasting techniques include single moving average, single exponential smoothing, double moving average, double exponential smoothing, regression, Holt-Winter’s additive, Seasonal additive, Holt-Winter’s multiplicative, seasonal multiplicative and Autoregressive Integrated Moving Average (ARIMA). According to the forecasting accuracy measurement, the best forecasting technique is regression analysis.

Introduction
Many companies do not know their demands and have to rely on demand forecasts to make decisions in production planning, sourcing and inventory management both in long and short term (Kerkkanen, 2010). According to Bon and Chong (2009), there are many factors affecting the frequent change of inventory's demand and those factors are trend, seasonality, and economic factors.Many organizations that utilized inventory optimization reduced inventory levels by up to 25 per cent in one year and enjoyed a discounted cash flow above 50 per cent in less than two years( Study of IDC Manufacturing Insight,2010). In fact, inventory optimization can better address demand volatility and supply variability and thus reducing the risk of both under stocks and overstocks for the organizations to optimize their inventory through demand forecasting. Healthcare in Malaysia is mainly under the jurisdiction of the Ministry of Health Malaysia. Currently, the nation practices a dual healthcare system and has good access to a blend of government and private healthcare services through a network of hospitals and clinics.In this research, a case study of University Health Centre in University Tun Hussein Onn Malaysia (UTHM) will be conducted. Health services UTHM started back in May 2002, when the Student Health Centre was a part of The Students Affair Office. The University Health Centre provides outpatient services as well as services to emergency cases during office hours. Under the 10th Malaysia Plan (2011)(2012)(2013)(2014)(2015), the government has identified healthcare services as one of the 12 National Key Economic Areas (NKEA) to generate revenue for the country. Healthcare is getting important and important even has becomes necessity for publics in the world.Demand forecasting is one of the most crucial issues of inventory management as forecasts form the basis for the planning of production, transportation and inventory levels (Korpela and Tuominen, 1996). Due to large forecast errors usually negatively affect companies' operational performance, forecast accuracy is often considered as a necessity (Danese and Kalchschmidt, 2011). Most of the time the healthcare provider will use their own judgement and experience to develop the decision on inventory demand. It means that there is yet have a proper and systematic way to forecast on the inventory demand. It is essential to apply forecasting methods in the aspect of inventory demand to avoid any wastage and lack of inventory situation happen. Besides that, since the actual demand is uncertainty the forecast also difficult to predict. Therefore, ongoing work should be carried out to improve the demand forecasting.

2.1
Forecasting Forecasting means estimating a future event or condition which is outside an organization's control and provides a basis for managerial planning (Kerkkanen,2010). Everyone requires forecasts; the need for forecasts cuts across all functional lines as well as all types of organizations (Hanke and Wichern, 2009) because it is useful in making management decision. According to Panneerselvam (2005), in any organizations, all the functional managers will base their decision on the forecast value. Many companies rely forecasting heavily in decision making for production planning, sourcing and inventory management. Hence, forecasting is used by companies to determine how to allocate their budgets for an upcoming period of time. This is typically based on demand for the goods and services it offers, compared to the cost of producing them.

2.2
Healthcare Forecasting Forecasting demand for health services is an important step in managerial decision making for all healthcare organizations. This task, which often is assumed by financial managers, first requires the compilation and examination of historical information. Although many quantitative forecasting methods exist, four common methods of forecasting are percent adjustment, 12-month moving average, trend line, and seasonalized forecast.Healthcare financial managers who want to project demand for healthcare services in their facility should understand the advantages and disadvantages of each method and then select the method that will best meet the organization's needs.

2.3
Demand Forecasting Demand is the need for a particular product or component and it can come from a number of sources (e.g., customer order or producer's good) (Armstrong, 2000).Demand forecasting means estimate of most likely future demand for product under given conditions. Based on the study by Kerkkanen (2010), one of the fundamental managerial tasks is demand forecasting.Demand forecasting is the area of predictive analytics dedicated to understanding consumer demand for goods or services. That understanding is harnessed and used to forecast consumer demand. Knowledge of how demand will fluctuate enables the supplier to keep the right amount of stock on hand.

2.4
Forecasting Techniques A forecasting technique is a procedure for computing forecasts from present and past values (Chatfield, 2000). According to Lawrence, Klimberg, and Lawrence (2009), forecasting techniques can be divided into two broad categories that are both qualitative and quantitative. Qualitativetechniquesareprojectionsbasedonjudgment,intuition,andinformed opinions,andtheyaresubjectivebynature (Kerkkanen ,2010). Examples of qualitative forecasting techniques include jury of executive opinion, Delphi method, sales force composite and consumer market survey.Quantitative forecasting techniques are used to forecast future data as a function of past data; they are appropriate when past data are available. Examples of quantitative forecasting techniques such as single moving average, single exponential smoothing, double moving average, double exponential smoothing, regression , Holt-Winter's additive, Seasonal additive, Holt-Winter's multiplicative, seasonal multiplicative and Autoregressive Integrated Moving Average (ARIMA).

3.1
Research Design According to Marican (2006), research design is like a map that clearly presents how a research will be carried out. Research design is the logic that links the data to be collected and the conclusions to be drawn to the initial question of study (Yin ,2009).The research design may be classified in three broad categories depending on objectives and types of research which are exploratory, descriptive and causal (Hair et. al., 2003). This study was designed as a descriptive research. A descriptive research may be classified as both qualitative and quantitative.

3.2
Data Collection This research will involve both qualitative and quantitative method for data collection. For qualitative process, the researcher will ask the problems existed in managing inventory demand in pharmacy, University Health Centre UTHM. For quantitative, the secondary data regarding inventory demand will be collected for data analysis.The most recent 5 years of high demand medicine like Panadol 650mg will be collected so that the forecasting techniques can be applied to analyze the data using risk simulator software and hence the best techniques can be selected to optimize the inventory demand.

4.1
Data Analysis Background Data analysis is one of the important elements of the research. After the data being collected, analysis data process will be started in order to determine and find out the result. The research data will be analyzed using Risk Simulator Software. Quantitative analysis was used for this study. Data analysis was committed once the historical data collected from University Health Centre.Before analyzing the data, the historical data were collected from University Health Centre, UTHM. The collected data was inventory demand regarding Panadol 650mg beginning from January 2009 until August 2014.Analyzing started by, firstly, forecast time horizon of collected data will be determined. It is an important preliminary step to selection of the forecasting methods to be used. The determined forecast time horizon was 1 month, a short range forecast. Then, time series plots of the data were constructed and visually inspected the data patterns. After the identification of the data pattern, the study proceeds to select and implement the appropriate forecasting methods based on the data pattern. Finally, the forecast accuracy measurements with the lowest values were selected as the best forecasting method.

4.2
Identification of Data Pattern A time series are likely to consist one or more of the data patterns such as trend, cyclical, seasonal and horizontal component.  For regression analysis, the error measurement for RMSE was 1334.7793. The forecast using this technique will indicate continuous incremental pattern.

Identification of best forecasting technique
The four forecast accuracy measurements used were RMSE, MSE, MAD, and MAPE. The smaller the forecast error is, the more accurate the forecasting methods. Regression analysis technique with RMSE= 1334.7793 was in first ranking as it showed the lowest RMSE value. Next was double moving average technique with RMSE=1376.7254 in second ranking. The single moving average technique in third ranking with RMSE=1406.8824 followed by double exponential smoothing technique with RMSE=1413.5199 the forth ranking. The fifth rank was single exponential smoothing technique with RMSE=1501.6296. The sixth ranking was ARIMA technique with RMSE= 1666.1652. Furthermore, the seventh ranking and eighth ranking were Holt-Winter's additive and seasonal additive with RMSE=1771.9117 and RMSE=1793.8089 respectively. Holt-Winter's multiplicative with RMSE=1818.6557 was in ninth ranking whereas the last ranking was seasonal multiplicative with RMSE=1849.2047.It can be concluded that the best forecasting techniques to optimize the inventory demand is regression analysis.

5.1
Discussion of Findings The first research objective is to identify the possible forecasting methods to be applied for inventory demand. In order to achieve the objective, the data pattern needs to be identified first. There were 10 forecasting techniques have been selected for the Panadol 650mg as required for demand forecasting analysis.The second objective is to optimize the inventory demand using the best forecasting method. Once the forecasting techniques are completed to be used for demand forecast process, the forecasting accuracy measurements will be compared among the ten forecasting techniques. The best forecasting methods for the Panadol 650mg is regression analysis.

5.2
Contribution For University Health Centre With the recommended forecasting method, the University Health Centre is able to produce more accurate forecast which can aid in making precise or reliable decision for the Panadol 650 mg inventory planning.The research can contribute to University Health Centre by assist the pharmacy using statistical forecasting techniques in a better inventory planning and avoid any overstocks or under stocks problems. The researcher recommends that more inventories should involve in demand forecasting process. This is a way to ensure that more accurate data of inventory demand quantities can be obtained to optimize the overall supply chain for the medicine.Further study should be carried out as well for other field like manufacturing industry. Manufacturing industry really need more accurate forecasting for their production to maximize the profit and ensure safety inventory always existed.

5.4
Conclusion In a nutshell, the research objectives were achieved. The data pattern was trend and ten forecasting techniques were used. Among the ten forecasting techniques, the best technique which was regression analysis with least forecasting error provided. A better inventory management through forecasting techniques can be implemented and hence improve the quality and performance in a long term period for University Health Centre.