Web site traffic analysis is the process of collecting and analyzing data about a web site's traffic to determine the web site's usage patterns and help measure the site's performance and effectiveness. The analysis may examine the total number of visitors to a site, the portions of a site visited, the duration of the visits, and the details of the interactions with the site, such as search queries generated by the visits. Web site traffic analysis is important to the effective functioning of web sites. By some estimates, the business of web site traffic analysis may gross over a billion dollars a year.
Web site traffic analysis may be performed by creating logs with entries for each Hypertext Transfer Protocol (HTTP) request or hit to the web site server. A log entry may be used to generate information about the number of hits, visitors, visitor duration, visitor origin (sub domain, referral link), visitor IP address, browser type and version, platform, cookies, and details of interaction with the web site. Web site traffic analysis may also be performed by creating records of the links or Uniform Resource Locators (URLs) visited at a web site.
Web servers may utilize a portal architecture. In this architecture, multiple applications called portlets are integrated into a single, customizable application called a portal. Content provided by multiple portlets may be displayed on a single page, and a user may interact with an individual portlet on a page without necessarily affecting the state of other portlets on the page. The portlets may be local, run by the web server, or remote. A remote portlet for a portal may be operated by a different enterprise than operates the portal. For example, the portlet may provide weather information to a large number of unrelated portals.
Traditional methods of web site traffic analysis may fail to capture critical information about web site usage of portals. To understand the usage of a portal site, the following data may prove useful for analysis:
the name of the portal page being displayed,
the names of all the portlets on the page,
the states of the portlets on the page (normal, minimized, maximized, etc.),
miscellaneous data about state internal to and specific to each portlet on the page, and
the user id of the user using the portal.
The data listed above may prove unavailable under traditional methods of web site traffic analysis. The web pages generated by portal servers may, for example, not reveal that the pages are portal pages. The pages may consist of Hypertext Markup Language (HTML) pages which contain fragments corresponding to the portlets. The web servers may not place easily extractable information into the page HTML regarding the portal environment, the portal page, or the individual portlets. The portlets themselves may be presented as nested tables with no identification. It may, therefore, prove difficult to determine from the page the details of the interaction between a user and an individual portlet. Further, reliance on records of server URLs may prove problematic. The server URLs may be highly dynamic, reflecting the changing content of portlets in response to user actions. Thus, a logged URL may no longer exist when an analysis is performed. Further, the URL may not expose all relevant information about the state of the page being viewed. Moreover, relying on the server to gather data about web site traffic may miss revisits to cached pages. These pages may be viewed again by retrieval from the cache without making an HTTP request of the server.
In addition, under traditional methods of web site traffic analysis, the integration of the collected data may prove difficult. The data about web site traffic may be written to multiple log files without a common identifier to tie them together. Further, the generation of server-created logs by a portal may consume large amounts of the portal's storage or may require intensive computing resources from the portal. Further, a portal may not deliver the logs to the web site traffic analysis provider in a timely fashion.