Web Usage Mining for Comparing User Access Behavior using Clustering

Motivated by the practical needs in improving the access to information over the internet with comparison between the users, many authors have given their most time on studying and designing the framework for the application of data mining like web mining, text mining and so on. Now it is the time to study data mining in systematic manner with few foundational measures related to the implementation of web mining through proposed low level prototype WebUMining. In this study, authors worked on data mining techniques applied successfully for suggesting the web usage mining to the organizational and authority to improve their website by comparing the user access and access behavior with result discussed and analysis. Authors used datasets from reliable repositories, which applied with algorithm proposed in WebUMining model. These datasets treated as training data for the all proposed algorithms. This study of data mining has primarily focused on the mining algorithms which are used in web usage mining.


Introduction
Web mining has been implemented and developed into several research areas to satisfy the needs of website developer and trend analysts. By Etzioni et al [1], the term web mining is prepared to symbolize the use of data mining techniques for automatically discovering hidden information from the data available over internet in the form of web content, website structure and web usage. The research paper explains the algorithm based on predictive approach of data mining technique used for web usage mining for comparing access behavior. When applied to web access log data for comparing user access behavior, clustering technique used to group the accesses based on the characteristics. The main advantage that Clustering has it makes group of common characteristics like user"s interest, behavior together to classify different user requests. In this research work, authors proposed an algorithm named ClusterView based on clustering technique.
Clustering can be supervised data mining [2] or unsupervised data mining as given in [3] means that it does not rely on predefined training itemset or semisupervised data mining proposed in [4]. Clustering can be of two types either distance based or of model based. In distance based clustering technique, it involves determining distance computed between pairs of data items and then creating group of similar items putting together into clusters such as partitioned clustering and hierarchical clustering. Partitioned methods create partitions of data items into K groups and it is represented by k-means algorithm. A hierarchical method used to build hierarchical group (or set) of interleaved clusters with the top level cluster containing a single cluster of all data item. In model based clustering defined by Zhong et al and Ghosh et al [5], the model type is proposed as the model structure that can be determined by model selection techniques and parameters estimated using maximum likelihood algorithms such as Expectation Maximization (EM) technique. In proposed ClusterView algorithm, authors used partitioned and hierarchical clustering method to compare the user access behavior on internet. Clustering can also be used in data preprocessing task for classification technique that could operate on the detected clusters.

Implementation of Proposed Algorithm
In the proposed algorithm, implementation is approaches towards a representation of the process analyzed by clustering in order to discover patterns that used for preparing comparison model based on the characteristic of user who accessed the particular website with respect to attributes of the user showing the interest or attributes of the context in which the activities taken place.
The first is more relevant in this scenario, the focus is on the identification of need by experts while developing the website. In proposing the algorithm, authors prepared clustering model implemented with WebUMining prototype in the combination with simulated monitored data that usually relates to the activities themselves. The proposed technique involves a dynamic manner clustering process controlled in order to function towards different goals. Thus, we not only have to determine these goals but additionally also need to establish metrics that can be used for this kind of dynamic control. This is the purpose of the clustering model to provide data sets containing a certain number of data items. The analysis process varies as required for visualization, complying with the respective clustering goal. In implementation of proposed ClusterView algorithm the authors studied the following key aspects that put together in the implementation.
a. Entropy is often associated with the certainty or uncertainty as discussed in Tribus et al and McIrvine et al in [7] it is assigning to an event with the order or disorder in a system which is, however, discussed controversially. Although a universal definition for sure of the term entropy not exist as there is common valuation in its implications

Proposed ClusterView Algorithm
To implement the ClusterView algorithm, authors applied partitioning algorithm to finds groups of strongly related (or correlated) pages by making partition according to its connected items those have the common characteristics that used for comparing the web access behavior using web access log data stored at server side. At first proposed algorithm performs Depth First Searching techniques in the web access log file that brings all related component in to matrix M by searching for the file. Once the item found, the algorithm checks if there are any items which not get visited or considered. It means that a related item has been split and therefore it needs to be identified to get into M. To repeat it, the searching technique again applied by starting from one of item that not visited. Proceeding further, in the proposed system authors used the two key aspects as discussed earlier, these are entropy and variance which are implemented in the proposed algorithm by counting the time spend on requestedURL and the occurrence in the manner of the number of users used a particular page. These two considerations used in identifying the relation between each pair of webpages. Therefore authors proposed a formula to bring all those pages together which are accessed maximum number of times with the maximum number of time spent on it. This formula is developed with statistical experiments that used the harmonic mean of time spent and the occurrence foe each page as given below: Let us take this formula for counting support of each webpage in the web access log file. Considering a worst case, when all the URLs are in the same cluster, the output generated by the algorithm will have execution cost of this algorithm it the linear form of data available in the web access log data. Before putting items into the clusters for getting navigational pattern profile, the generated clusters are then ranked based on data available in the matrix M.

Pseudo code and Flowchart
The proposed algorithm at first scans each web access log entry from web access log one after and then stores it with assigning occurrence of each item to 0. In this context, the item is the webpage that accessed by user during particular session. These items at first get stored in the list L, while reading the items from itemset, the matrix M with items that satisfies the formula as discussed above. This matrix M is then loaded with items those are present in the web log file with the time taken by user to spend over webpage and the occurrence of that web page. After completing the data satisfying formula from web access log data, this item in matrix M is get assigned to the current item for cluster C.
After looking at each entry for item into matrix M, then next is to scan for the items those not satisfies the defined formula for minimum support. For making clusters with respect to the user defined minimum support count taken as input. Perhaps, the items with support that not matches the item available in the matrix M are then get searched with the help of depth first searching technique so that it returns the item which are below the user defined minimum support and then algorithm creates cluster for this items. If the occurrence for an item goes more than defined support, this element belongs to the cluster where cluster size is given by user as an input for which website administrator is looking for.
The proposed algorithm is provided with the following pseudo code hence the authors are intends to have the implementation of the proposed algorithm in the low level prototype model.
Algorithm Name: ClusterView Input: Web access log file after data cleaning and preprocessing. Minimum Support measure Minimum Cluster Size Output: List of Clusters. While implementing the proposed ClusterView algorithm, initially there was no method to consider a target page that can be used as centered for the cluster while comparing the user access and to make the cluster depending upon the common characteristics. And hence, the most accessed page taken as the target page. The output of the proposed ClusterView algorithm is in the form of a text file containing the list of clusters with respect to the support measure given as input from the user. In this format the results are then be easily represented by visualization software. Effectiveness of the proposed ClusterView algorithm is implemented and evaluated through comparing processing time and number of patterns generated during the web access log data submitted as input to algorithm so that it can be useful for making comparative analysis of the data.

Flowchart
Following is the flowchart of the proposed ClusterView algorithm as discussed above. The flowchart contains all the keywords that discussed during implementation of the algorithm discussed earlier. The flowchart gives the idea about the flow of data and the actual process implemented in the proposed ClusterView algorithm. The algorithm is then implemented with JAVA and it is one of the algorithms of WebUMining model used for web usage mining for comparing user access behavior.

Visualizing Result
In this section, authors worked on the web access log data "NASALog.arff" of NASA"s Kennedy Space Centre"s website available at the archive.org. On this input file, experiment performed to find out the clusters base on the web pages and the maximum times spend over that webpage. The proposed ClusterView algorithm can also be used to compare and for predicting user access behavior through web access log data and it is implemented in WebUMining model. During experiment, the focus is on looking and searching for the webpages that are accessed by the users who had the common characteristics about their behavior access from web log data. Then this file is converted from text file to arff file format using WEKA as shown in figure 1 given below: Input web access log data file for ClusterView algorithm The log file produced as shown in the figure 6.1is then filtered so that it only contains the fields which are IPaddress, sessionID considered from the webpage time which is accessed maximum number of time by the users and the webpage that requested maximum number of times as requested URL. Figure 6.2 shows the filtered file prepared with WEKA containing the fields as discussed above.
Input web access log data file after filtering Result from the ClusterView algorithm achieved as shown and the visualization is represented using the same technique as authors visualized the AssociationRule algorithm to represent the associated webpages. With the minimum cluster size 4 in that the numbers represent the webpages that are accessed by the users and we can simply compare these after looking towards the each cluster combined in between [ ] (square brackets) as shown. The web pages that form part of a specific cluster are then Listed with the specific cluster as shown in class with some colored similar to show their class.

Result of WebUMining using ClusterView algorithm
After getting the results with respect to the user"s common characteristics based on the user access the research is showing the clusters that makes each access to the requested URL from multiple users during particular session.  Table 2 shows the cluster for IPAddress that spend the maximum time on the particular page. The cluster 0 represents the common items containing the IPAddresses of the user who followed a sequential path and during that they made the visits to more than one webpage in a session. While Cluster two shows the class of those IPAddresses who visited the webpages at least once during surfing the internet. During comparison between the clusters depending upon the access made by them is represented in the manner of percentage against the total percentage of users who accessed the webpages. This clusters represented with the numbers 0 and 1 so that it will be easy for website administrator to compare the users those are accessing the website more than once and creating sequential navigational path during access to website. The items in cluster 0 contain 32% of the overall access made to the website while cluster 1 contains 68% total   Table 3 represents the clusters created on RequestedURL information of the web access log file. The cluster 0 represents the class of those webpages which are accessed more than once while cluster 1 represents the class that contains the webpages those are accessed at least once. Figure 6.6 shows the groups of the filtered information which used for clustering as discussed and shown in the figure 6.1. The visualization of these information and the generated results are shown in the following figures.
Result of WebUMining using ClusterView algorithm The figures given below are the graphical representation of the clusters created with the proposed ClusterView and visualizes with the WEKA software for interpretation. Figure  A shows the cluster with respect to IPAddress information whereas Figure B shows the cluster with respect to the RequestedURL information from the filtered web access log data. The result can be interpreted and determined by the requirement criteria when given as inputs after selecting ClusterView algorithm. At the initial stage of the visualization, it involves the visualization software and then for specific form result authors used WEKA and SPSS software for same. These software provides the advantage of being accessible for any process of analysis over log data hence it makes sensible for making decision about the access to internet so that for any type of comparison since it is in demand to use for web usage mining process.

Recommendations and Limitation
The aim of this research work is in using the proposed WebUMining model in order to suggest the comparison regarding the user access to the particular web site. In order to do this, WebUMining model used to process the web usage data from three different repositories to determine the changes or improvements needed to made a particular webpage accessible to the user. This web access log data is processed with ClusterView algorithm. In proposed clustering based ClusterView algorithm, authors recommending that if the website is design, build and implemented with what the user want and what link the user accessing then website administrator can made that improvements in their website so that comparing user will make the access easy to the website.
A limitation of WebUMining is that no support is provided for automatically recommendation of a particular web site depending upon the user access and on other hand by comparing the user access behavior, web administrator cannot take decision whether to provide the next webpage in the path or not through website design. As the web site structure is dynamic in the form of there hierarchy of the webpage attached to one another and hence due to the newly attached link varies the accesses to the different webpages those involves in sequential path and this creates the result that cannot give the comparison. For this variation in the input information, the comparison based on user"s behavior over website needs to have sufficient domain knowledge regarding website while access based user behavior being compared and analyzed.

Conclusion
This research work proposed by the authors demonstrate a low level prototype called WebUMining which implemented with the proposed ClusterView Algorithm which can be used to compare the user access behavior using web access log data through data mining techniques. The proposed algorithm is implemented using Java and WEKA. For visualization of the result we have used the SPSS and RapidMiner software.