The Influence of Perceived Organizational Injustice Towards Workplace Personal Web Usage and Work Productivity in Indonesia

Workplace personal web USAge (WPWU) is an employee’s activity in using internet for non-related task during working hours. It is considered a counterproductive behavior when done excessively because it can interrupt employee’s productivity, but it can increase creativity and eliminate boredom when used in a rational amount. The objective of this study was to prove whether perceived organizational injustice had influence on WPWU which affected work productivity. A total of 222 respondents working in various industries were gathered through web-survey. By using multinomial logistic regression analysis, this study found that high level use of internet for unrelated jobs between 2 to 4 hours a day was influenced by respondents’ perception of not getting fair treatment and incentive for being good performer, which then caused them to perform very low completion of tasks. There were two contrasting views regarding this result; organizations considered it as deviant behavior because it reduced employees’ performance whereas employees regarded it as just short breaks to get rid of stress. Hence, this finding suggested that companies should redesign its internet policies to accommodate “Work-Life Blend”; blending work and personal lives, as a consequence of cultural shift in the era of globalization and new technologies.

nevertheless, many studies considered that wPwu as a negative behavior (lim, 2002) and could interrupt employees' performances because taking up time that should be used to accomplish the job (coker, 2011). This was in line with the studies conducted by Malachowski (2005) which showed that wPwu had an effect on decreasing company performance. coker (2011) also found that an employee who committed excessive wPwu (more than 12% of total working hours) would have a decline in work productivity, while hilts (2008) stated that WPWU have a negative influence towards work productivity. a number of studies found that one of the factors driving employees in committing wPwu was related to the perception on how superiors gave appreciation and fair treatment in the workplace (lim & Teo, 2005) or also called inappropriate perceived organizational justice. studies conducted by blau, yang, and ward-cook (2006) also found that employees who were unfairly treated in the organization tended to commit wPwu which was supported by the finding of de Lara (2006) on the negative influence of wPwu towards interactional justice, a dimension of organizational justice.
Indonesian internet users in 2012 had increased to 61.1 million from 42.2 million in 2010 (MarkPlus Insight as cited by eMarketer, 2013a) in which the users not only consisted of young age (15 to 19 years) but also older age group (30 to 50 years) (lukman, with the increase number of internet users everywhere in the world, companies have realized the internets' potential as a new form which leads business through different ways and also as a tool to accelerate business performances (lim, 2002). This is because internet has played important roles in reducing costs, shortening production cycles, and promoting their goods and services in more effective way (anandarajan, simmers, & Igbaria, 2000). however, with easier access and plentiful of information available in the internet, it was found that employees spent at least one hour a week or less to perform activities that were not related to the job during working hours (39%), followed by 2 hours a week (29%), 5 hours a week (21%), and 10 hours or more (3%) (gouveia, 2012). This activity was considered working violation by the organizations because employees were using internet and mobile technology during working hours for personal purpose (lim, 2002;Mastrangelo, everton, & jolton, 2006). The term used for this activity was workplace personal web usage (wPwu) (anandarajan, simmers, & Igbaria, 2000), or some other various terms that frequently used in different studies were cyberslacking, cyberloafing, cyber deviance, and internet abuse (kim & bryne, 2011).
Previous studies found that wPwu had both positive and negative effects on business performance. aside from improving employees' performances, coker (2011) cited that some positive effects for allowing employees with rational amount of wPwu were escalating job satisfaction or creativity or reading blog sites, sending short messages (instant messages), reading adult-oriented sites, private investment sites, and online auction sites (kim & bryne, 2011). as explained by lim (2002), wPwu is seen as an activity that interferes with work and does not provide benefits to the company because of its effect on income reduction through decreasing productivity (Malachowski, 2005).
In this study, work productivity is described as individuals' effort which can be measured and contributes to output produced by organizations (huang, 2008). work productivity associated with internet usage requires clarity which activities are productive and are considered to be unproductive (welebir & kleiner, 2005). Many controversies have arisen whether internet usage enhanced or inhibited productivity. Powell (2010) revealed that employees could be inspired when online and ultimately became more productive; on the other hand, lim (2002) found that employees became unproductive by using the internet. organizational justice is generally defined as personal evaluation about how fair managerial conduct based on the results, procedures, and interactions within the organization (colquitt, greenberg, & zapata-Phelan, 2005). There are three factors of organizational justice. first, distributive justice is concerned with an individual's perception on the amount of his/her input to the organization relative the amount of output he/she receives from the organization (ensey, 2012).
second, procedural justice refers to employees' perception of fairness to 2012), the age group of productive people. based on the 2012 yahoo! net Index study, Indonesian internet users access via office was 20% in 2012 from 19% in 2010, while via mobile phone was 62% in 2012 from only 22% in 200922% in (eMarketer, 2013b. Moreover, regardless of whether Indonesian internet users were using smartphone or desktop, they would use internet for social networking and entertainment as their primarily activities, aside from for searching information and email (eMarketer, 2013b). furthermore, the number of people using smartphones in Indonesia had also increased to 26.3 million in 2012 from 11.7 million in 2011 (eMarketer, 2013c). It is predicted that by the end of 2013, there would be 41.6 million smartphone users in Indonesia.
These conditions had increased employers' concern because wPwu has affected office activities. Workers preferred to use internet for non-related works instead of doing their job responsibilities which affected job performance (abi, 2010). Therefore, the objective of this paper is to examine whether perceived unfair organizational treatment could cause excessive amount of wPwu which affect their work productivity.

LITERATURE REVIEW
workplace personal web usage (WPWU) is defined as all voluntary activities in using organizations' internet access during working hours to open the sites that are not related to work and to check personal e-mail by the employees (lim, 2002). examples of wPwu activities are online shopping, playing online games, writing

RESEARCH METHOD
The research was conducted within a period of february to March 2013 (cross-sectional design) by using survey approach. web-based questionnaire was utilized to collect quantitative data via Google Spreadsheet application. This study used purposive sampling method in which respondent should be an employee who work in the organization and have internet access in the workplace.
Invitations to participate in the survey were sent to more than 500 potential respondents by means of e-mail, Facebook, and Twitter. using web-based questionnaire can be used as a filter to check whether the respondent has access to internet aside from shortening the period of data collection. however, there might be several possible limitations such as the same respondents could submit more than one response because of bad internet connection, respondent could falsify data entry, and also other than the target respondents could also submit responses. data were analyzed using descriptive and multinomial logistic regression analysis.
descriptive analysis was used to illustrate the means of perceived organizational justice, level of workplace personal web usage (wPwu), and work productivity. Multinomial logistic regression analysis was performed because work pro-the processes by which rewards are distributed or decisions are made in the workplace (berry, ones, & sacket, 2007). Procedural justice establishes certain principles specifying and governing the roles of participants within the decision-making processes such as free of bias, accuracy, consistency, correctness, and representative groups (cropanzano, bowen, & gilliland, 2007). for example, an employee may feel procedural injustice if they perceive that their supervisor provides bonuses to those employees he or she likes rather than on the basis of productivity or merit (ensey, 2012).
The last factor is interactional justice which is perceived individuals' justice about interpersonal communication within organization (Thorn, 2010) that depends on honesty, justification, respect, and courtesy. Interactional justice emerged as an important component of organizational justice due to its emphasis on the 'human' element in the organizational context (devonish & greenidge, 2010).
This study did not analyze organizational justice based on the three dimensions because employees perceived organizational treatment fairness as a combination of results, procedures, and interactions justice when influencing the level of wPwu and hence affect their work productivity. The research model can be seen in figure 1. figure 1. research Model ewPs consisted of 4 dimensions, namely attendance, work quality, performance capacity, and social/emotional factors. There were 37 activities which represented work productivity. In addition, ewPs used a 5-point likert scale in which 0 was never, 1 was seldom, 2 was sometimes, 3 was often, and 4 was always, therefore it had score range between 0 (lowest score) to 148 (highest score).

Workplace personal web usage
workplace personal web usage (wPwu) was based on the questionnaire developed by coker (2011). It consisted of frequency and duration on each of the 20 types of wPwu activities that had been adapted to the context of this study (see Table 1). ductivity and wPwu have more than one category.

Instruments
This study used three instruments, namely work productivity, workplace personal web usage, and organizational justice. Prior to data collection, the instruments were translated from english to bahasa Indonesia and back translated to english. validity and reliability tests were also conducted.

Work productivity
work productivity was measured by endicott work Productivity scale (ewPs) developed by endicott and nee (1997) to assess the behavior and attitudes that could affect performance (α=.92). ewPs is a patented measurement tool so its usage requires permission. for the purpose of this study, permission to use questionnaire was asked and obtained from endicott and nee via personal email.

RESULT AND DISCUSSION
Three filter questions were used to elicit the target respondents, whether respondents were (1) working in an organization, (2) not an entrepreneur, and (3) using computer and internet facilities in the workplace. from 297 respondents who were willing to participate, only 222 respondents which could be further analyzed. out of the 75 invalid data, 28 persons did not have job (9.4%), 18 persons worked as entrepreneur (6.1%), 18 persons had incomplete answers (6.1%), and 11 persons had no access to use internet facilities at the workplace (3.7%).
respondents worked in educational institution (14.9%), followed by financial service (12.2%), banking (10.8%), public sector (9.9%), and oil & gas (9.9%). ownership of the organization was mostly private-owned (41%), state-owned (27.9%), and foreignowned (23.4%). The total numbers of employees in the organization were more than 1,000 employees (31.1%) and 51-100 employees (11.7%). Table 3 showed that majority of respondents (74.3%) performed low The amount of time wPwu was done by respondents was the duration of each wPwu activity in minutes multiplied by the frequency of each wPwu activity undertaken by respondents within a week. Then, multiplication result was divided by 60 minutes to obtain the amount of wPwu time in hours (1). The amount of wPwu time in hours (1) was divided by respondents' total working hours in a week and multiplied by 100% to obtain the percentage of respondents' wPwu score (2).
To identify the level of internet usage for personal purpose, wPwu scores were divided into four categories: low wPwu (0-10%), moderate wPwu (10.01 to 25%), high wPwu (25.01 to 50%), and very high wPwu (> 50% of working hours in a week using internet for personal reasons).

Organizational Justice
Indicators developed by Moorman (1991) were used to assess employees' perception on organizational justice. These indicators had three dimensions, namely distributive justice (α=.92), procedural justice (α=.94), and interactional justice (α=.90) which consisted of 18 questions (see Table 2). This questionnaire used a 5-point likert scale in which 1 was strongly disagree, 2 was disagree, 3 was neutral, 4 was agree, and 5 was strongly agree.
To discover the perception of organizational justice, its mean scores were Table 4). although 20.3% of respondents had high work productivity, it was outnumbered by the respondents with low to very low work productivity (29.8%).
descriptive analysis was also performed on each dimension of organizational justice. using 5-point likert scale, distributive and procedural justice had the same mean score (3.33); while interactional justice had mean level of wPwu ranging from 0-10 percent of their working hours or a maximum of 4 hours out of 40 working-hours a week. but there were 14 respondents (6.3%) who had high and very high level of wPwu (used internet for personal purposes between more than 10 to 20 hours a week and more than 20 hours a week).
half of the respondents had a moderate level of work productivity (see   There were 10 respondents who belonged to high level of wPwu (4.5%). These respondents used internet for personal usage between 10 to 20 working-hours in a week or 2 to 4 hours a day.
Majority of the respondents worked in education institutions (14.90%) and financial services (12.20%). More specifically, they worked in three job divi-  Table 5). with a mean score of 3.40, organizational justice was considered moderate; meaning most of the respondents perceived their organizations treated them fairly enough.
To analyze the objective of this study, multinomial logistic regression was performed. Table 6 showed that WPWU had significantly negative influence (-1.75) towards very low work productivity category. This result meant that the longer employees used internet for personal purposes, the lower would be the employees' work productivity. In this study, there were 3 respondents (1.4%) who had very low work productivity.
from Table 7, it was found that organizational justice had significantly negative influence (-0.70) on high workplace personal web usage (wPwu). Meaning, the lower employees perceived they were treated fairly, the using 5-point likert scale ranged from 1 = strongly disagree to 5 = strongly agree and access information. some tasks that were accomplished by using internet were sending emails, inputting online data, downloading data, having teleconference, clarifying clients' data or customers' needs, and searching for information regarding vendors' background. That is why some of them used computer for more than 8 hours in a day (39.2%).
This study indicated that workplace personal web usage (wPwu) had significantly negative influence on a very low level of work productivity. This finding supported the previous studies conducted by coker (2011) and hilts (2008). study of coker (2011) showed that excessive wPwu could reduce work productivity, while hilts (2008) found that wPwu was used as a way to escape from the work they supposed to do.
The 3 respondents with very low productivity worked in organizations with no restrictions on internet usage. Thus, they frequently took this opportunity to watch or download movies. Moreover, half of the respondents had moderate work productivity and 28.4% had low work productivity which was measured through their level of attendance, work quality, performance capacity, and social emotional factors.
descriptive analysis of wPwu showed that majority of respondents (74.3%) had low level of wPwu or only less than 10% of their total working hours. It meant they spent a maximum of 4 hours using internet for non-related work in a week. This low level of wPwu was most likely because majority of the respondents accessed internet during working hours dealt with employees or training providers, and sales & Marketing division needed to contact customers and other related divisions. This was confirmed by 71.60% of respondents who had to interact with clients, vendors, and consumers. In addition, respondents who worked in education institutions needed to interact with students and academic staff as well as seeking additional teaching readings. Those complaints were reflected on respondents' comments which stated that there was lack of socialization on internet policies especially on the purpose, consequences or penalties of internet violation. some comments were: "Internet policies were not clearly socialized yet" (respondent 159, assistant manager in accounting & finance division) and "there were internet policies in the organization, but it was not socialized well" (respondent 219, sales & marketing staff). while an hr staff from local private manufacturing company complained that "Basically these policies existed but many employees did not understand about the purpose of blocking certain websites and the risk of opening those blocked sites" (respondent 181).
organizations which blocked several websites should be prepared that this policy might be ineffective since employees would simply use their own smartphones, tablets, and laptops to access personal websites during working hours (gouveia, 2012). This ineffective policy was stated more clearly in kPMg International report, "by restricting or blocking internet access, many employees tend to move their activity to their own personal devices which are often less secure and completely unmonitored" (gouveia, 2012). In this study, most of the respondents used office laptop/desktop (42.8%), followed by personal laptop (15.8%), smartphone and office laptop/desktop (14.4%), and smartphone, personal laptop and office laptop/desktop (13.1%) when accessing (72.50%) in relation to task completion (42.8%). The organization ownership of respondents was local private-owned (41%) and state-owned (27.9%), which usually had strict regulations on the internet usage. Most of the organizations (68.5%) had blocked several sites during working hours, such as pornographic, social networking (e.g. Facebook, Twitter), online games, and general e-mail (e.g. Yahoo, MSN, Gmail). This regulation was supported by the comments of two respondents: "fair, just close websites that is really not related to work" (respondent 65) and "it is quite wise to restrict on certain sites during working hours" (respondent 146).
The reasons why many organizations applied limitation to internet access could be caused by several studies claiming that workers spent most of their working hours for personal internet usage. for example, primarily activities of Indonesian internet users were social networking and entertainment, aside from searching information and email (eMarketer, 2013b). furthermore, according to gouveia (2012) around 64% of workers being surveyed visited non-work related websites every day during working hours.
nevertheless, this blocking regulation was complained by some divisions such as sales, Marketing communication, distribution, and general affairs. The regulation had hampered sales & Marketing division to monitor market share positions, marketing strategies of competitors, and seek creative ideas. Moreover, general affairs division had difficult time browsing different vendors' price lists, while dis-ents who had lower satisfaction with supervisors' treatment at workplace were likely to spend more time using internet for non-related works. high wPwu was performed by ten respondents (4.5%) who spent between 10 to 20 working hours in a week for personal internet usage. Meaning, they used internet for unrelated jobs for 2 to 4 hours a day because they felt they had too much workload compared to their co-workers, their compensation was not comparable with the job responsibilities they did, they received unfair treatment during interaction with boss, and there was no incentive for being good performers.
according to coker (2011), excessive internet usage from personal purpose which could reduce employees' productivity was more than 4 hours in a week. Thus in this study there were 25.7% of the respondents who were classified as excessive internet users because they used internet between more than 4 to 20 hours or above in a week. This condition was disclosed by one of the respondents who stated that "some employees take more time in using internet for things that have nothing to do with their job" (respondent 219). although only one-fourth of the total respondents were classified as excessive internet users, further analysis was done to identify the reasons they wasted a lot of time during working hours.

CONCLUSION
This study found that moderate perception of organizational justice caused high level of internet use for personal purpose which resulted in very low work productivity. high level use of internet for unrelated jobs between 2 to 4 hours a day was caused by respondents' perception of not getting fair treatment and incentive for being good performer, which then caused them to perform very low completion of tasks.
This finding had produced two contrasting views. organization considered workplace personal web usage (wPwu) as a counterproductive behavior, while employees viewed wPwu as one way to eliminate boredom.
To overcome this condition, organization could implement internet policy to block several sites unrelated to completion of tasks. but top management-accompanied by IT and human resource departments-should socialize the objective of the policies, belonged to generation y or millennials, who were born between 1977 and 1997 (Meister & Dahlberg, 2012) and have been using computers, mobile phones, the internet, social media tools and other technologies since childhood (friese & jowett, 2013).
gouveia (2012) stated that these workers refused to be called wasting time when using internet because they were just taking short breaks after having to go through activities which they claimed to be wasting their working hours. Those activities were: having to attend too many meetings (47%), dealing with office politics (43%), fixing other peoples' mistakes (37%), and coping with annoying coworkers (36%) (gouveia, 2012). by checking their social networking or entertainment sites and personal emails, they believed they would actually be more productive than if restrictions are placed on them. This finding supported the studies cited by coker (2011) that wPwu could decrease boredom or exhaustion (oravec, 2002) and escalate job satisfaction or creativity (reinecke, 2009). based on the above analysis there were two opposing point of views. on one side, organizations considered employees using internet for personal purposes as deviant behavior because it could reduce work productivity. on the other side, employees believed that their personal internet usage was just short breaks to boost their work productivity especially if unfair treatments existed in the organization. according to karzan, the ceo of kenexa, salary.com's parent company (as cited by gouveia, 2012), the best way for companies to deal with employee's in-

Suggestions for Further Studies
although this study provides new insights, there are some limitations. In this study, majority of the respondents worked as staff. Therefore, future studies should gather data from respondents with diverse job positions in order to obtain more representative findings and better description about the actual workplace condition.
In addition, further studies should distinguish between internet-intensive companies and non-internet-intensive companies in order to provide more extensive and in-depth suggestion in addressing the use of the internet for personal activities and employees' productivity in accordance to the conditions of the company.
consequences of opening prohibited websites to the company, penalties of violating the policies. Monitoring should be done on a regular basis as to protect data security.
nevertheless in the current information and communication technology in the globalization era, in which most jobs require a computer or laptop with internet access, organizations should begin recognizing the value of a "work-life blend", blending work and personal lives every day. rather than focusing on how wPwu could distract work productivity, organizations should start fostering productive working culture by giving challenging job with clear target and worthy incentive.


Use of web mining in studying innovation

As enterprises expand and post increasing information about their business activities on their websites, website data promises to be a valuable source for investigating innovation. This article examines the practicalities and effectiveness of web mining as a research method for innovation studies. We use web mining to explore the R&D activities of 296 UK-based green goods small and mid-size enterprises. We find that website data offers additional insights when compared with other traditional unobtrusive research methods, such as patent and publication analysis. We examine the strengths and limitations of enterprise innovation web mining in terms of a wide range of data quality dimensions, including accuracy, completeness, currency, quantity, flexibility and accessibility. We observe that far more companies in our sample report undertaking R&D activities on their web sites than would be suggested by looking only at conventional data sources. While traditional methods offer information about the early phases of R&D and invention through publications and patents, web mining offers insights that are more downstream in the innovation process. Handling website data is not as easy as alternative data sources, and care needs to be taken in executing search strategies. Website information is also self-reported and companies may vary in their motivations for posting (or not posting) information about their activities on websites. Nonetheless, we find that web mining is a significant and useful complement to current methods, as well as offering novel insights not easily obtained from other unobtrusive sources.

Introduction
Enterprises use their publicly-viewable websites for a variety of reasons, including promoting their products and services, directly selling those products and services, presenting information about their development, capabilities and credentials, documenting their achievements, and expanding their customer base, especially in export markets (Fisher et al. 2007). Enterprise websites often also contain valuable information about the company's location(s) and facilities, specifications of products and services offered, the orientation and attitude of the firm, key personnel, and strategies and relationships with other firms and organizations.
The ever-growing amount of information that is available through enterprise websites offers significant opportunities for researchers. With the understanding that websites are self-reports, website information has advantages in that it is readily and publicly available, is cost-effective to obtain, and can be extensive in terms of coverage and the amount of data contained. In an era where corporate response rates to voluntary academic research questionnaire surveys are frequently rather low, those same corporations maintain a relatively high website presence. For example, just under three-quarters of UK companies with at least one employee indicate that they maintain a website-a figure that rises to 85 % for companies (all sectors) with 10 or more employees and 91 % for manufacturing companies with 10 or more employees. 1 We recognize that the information that is available on enterprise websites is not standardized, varies according to the company andhow it wishes to present itself, and is typically in an unstructured format. Yet, notwithstanding these and other caveats, we suggest that the data that can be found on enterprise websites is an additional and important source of information and intelligence, particularly in addressing questions related to innovation where other data sources are less effective in gathering sufficient and relevant information. Technical advances in handling and analyzing unstructured data as well as current interest in the use of ''big data'' in discovering patterns and trends make it timely to investigate the appropriateness of using enterprise websites as a data source. However, while there are significant benefits to using website data through methods such as web scraping or web mining in innovation research, the literature on the use and validity of these approaches is relatively underdeveloped. This article aims to address this issue by analyzing the usefulness of website data in comparison with other data sources. We explore a web-derived dataset to discuss methodological issues related to the processes of conceptualizing, retrieving, structuring, cleaning, manipulating and interpreting website data in understanding company innovation strategies, focusing on enterprise research and development (R&D) activity.
In the next section, we review the available literature on the use of website data in innovation studies and wider social science applications. This is followed by discussion of our sample dataset and the methodologies used to obtain and analyze the data. In the ensuing section, there is an empirical demonstration of how the web mining process is operationalized in identifying R&D activities, as well as how this data correlates with other data sources. We then present a conceptual discussion of the relative qualities of website data for evaluating the enterprise R&D activities in comparison with other data sources. A discussion of the results and conclusions, as well as limitations, is contained in the final section.

Literature review
Researchers often select methods such as web mining due to their ''unobtrusiveness''. Webb et al. (1966) first coined the term ''unobtrusive measures'' in reference to methods of data collection that do not require direct contact with research subjects. Conversely, obtrusive methods can be regarded as those that require direct contact with the population studied.
These methods are each suitable to different circumstances, depending on what is being studied. For example, the opinions and beliefs of individuals are often best explored through interviews or questionnaires-obtrusive methods. However, if the research concerns real actions and behaviors, these may best be observed from a distance-using unobtrusive methods. Unobtrusive methods are a way of collecting data about a subject without their direct knowledge or participation (Cargan 2007). Unobtrusive methods can be less expensive in that they do not involve the costs of training and placing researchers in the field and following up directly with respondents. Additionally, as Lee (2000) discusses, one major advantage of using ''non-reactive'' approaches (Webb et al. 1981) is that they avoid problems caused by the researcher's presence. In the case of obtrusive methods, the respondents are aware of the researcher and may alter their response to these research methods in light of this. Unobtrusive methods are also not limited to those who are accessible and cooperative (Webb et al. 1966). Lee (2000) also outlines the opportunity that internet data presents in unobtrusive research.
In the field of innovation studies, there has long been the use of a combination of obtrusive methods (such as innovation surveys of firms or business case studies) and unobtrusive methods (such as analyzing databases of patents and publications). More recently, innovation researchers have demonstrated increasing creativity in developing more diverse unobtrusive methods, many of which use website or social media data. Robson (2002) distinguishes between the more-traditional unobtrusive approaches already discussed and another-content analysis. The author describes content analysis as that conducted on a written document, such as books, letters and newspapers. However, we can see how this can be extended to analyzing the textual content of a website, through the process of web mining. Robson states that such an approach is different from other unobtrusive methods as the observation itself is indirect (i.e. there is no need to observe the participants -in our case, a group of companies-directly).
There are three general categories of web mining (Miner et al. 2012). Web content mining involves the analysis of unstructured text data within webpages to extract structured information. Web structure mining focuses on analyses of the hyper-linked structure of a set of webpages, typically using methods of network analysis. Web usage mining is the data mining process involving the usage data of webpages. All three types of web mining have been used in innovation studies.
An example of web structure mining in innovation studies is offered by Katz and Cothey (2006) who investigate relationships between the internet and innovation systems by utilizing website-based indicators from webpage counts and links. Another instance of web structure mining is from van de Lei and Cunningham (2006), who employ website data in a future-oriented technology analysis, where it is used to identify existing networks that are concerned with technological change. In this research, a web crawling process is used to identify linkages between nanotechnology web portals, creating a network of activity between parties across many sectors. Ladwig et al. (2010) use web structure mining to study the landscape of online resources in emerging technologies by identifying the top search terms and resulting top-ranked webpages from Google. Similarly, Ackland et al. (2010) use web crawling to capture hyperlinks: examining the relationships between, and prominence of, actors engaged in nanotechnology. The use of metrics based on web presence in measuring scientific performance (''webometrics'') has widely been discussed in science policy literature (see Thelwall (2012) for an overview). Webometrics approaches use both web structure mining and web usage mining.
More recently, innovation scholars have been applying web content analysis in their research. Veltri (2013) carried out semantic analysis on 24,000 tweets from Twitter to understand the public perception of nanotechnology. Libaers et al. (2010) examine keyword occurrence in company websites from a cross-industry sample of small and medium-size enterprises to identify commercialization-focused business models among highly-innovative firms. Hyun Kim (2012) conducted both web-content and web-structure analysis of nanotechnology websites across the ''Triple Helix'' (Etzkowitz and Leydesdorff 2000) of university, government and enterprise relationships. The former allowed the author to discern different lexicons from three sectors, while the latter offered an understanding of which organizations played key roles in the development of an emerging technology.
Two recent studies are notable for examining the commercialization of emerging technologies by small and medium-sized firms through web content analysis. Youtie et al. (2012) examine current and archived website data of nanotechnology small and medium-sized enterprises, with a particular focus on the transition of such technologies from discovery to commercialization. The authors note the problems of coverage, timeliness, and response rate in commonly used sources of information such as patent databases and surveys in understanding enterprise innovation in rapidly transforming domains. A new approach-one which uses current and archival website data-is proposed. This method involved identifying and mining content information found on the websites of a pilot sample of 30 small and mediumsized enterprises from the United States, then analyzing the unstructured data in order to draw findings. The authors note that smaller firms tend to have smaller websites, therefore making the web mining process and subsequent analysis more manageable in such cases. From their analysis of the website data, the authors were able to identify the occurrence of various innovation stages and production transitions in the development of their sample of enterprises. The paper also discusses the role of government research grants and venture capital investment in bringing a technology to market.
The second study by Arora et al. (2013) undertakes a similar web content analysis method to examine the activities of small and medium size enterprises in the US, UK and China commercializing emerging graphene technologies. The authors again discuss the limitations of conventional methods, including issues of response rate and bias in surveys, and coverage and time lag in bibliometric and patent data. The study employs a web crawling technique of searching for keywords across all webpages of the sample firms' websites. This allowed the authors to not only draw conclusions on the degree of innovation employed by the sample firms but also the extent to which these activities were globalized and in partnerships, using such analysis to characterize three different types of emerging technology SME.
Web mining has also been used in other areas of social science. AleEbrahim and Fathian (2013) develop a method to summarize customer online reviews from websites. Al-Hassan et al. (2013) investigate whether the North American Industry Classification System code (NAICS) effectively shows the true industrial sectors of Fortune 500 firms by analyzing their websites. Battistini et al. (2013) present a technique to map geo-tagged geo-hazards, such as landslides, earthquakes and floods, by analyzing online news. Hoekstra et al. (2012) investigate the feasibility and desirability of the automated collection of official statistics, such as consumer price index, from websites. There is a stream of publications concerning the mining of political opinions from websites, forums and social media . There are also attempts to use web mining in health research: for instance content mining of website discussion forums to detect concern levels for HIV/AIDS (Sung et al. 2013) and mining social media to discover drug adverse effects (Yang et al. 2012).

Research questions
In this study, we develop and operationalize a web content mining process to assess the extent and character of R&D activities in a sample of small and medium sized enterprises. We discuss the specific process of extracting keywords for this purpose and explore how the final results are sensitive to the procedure used. We also compare and contrast the effectiveness of website data to other data sources in studying the innovation process. In summary, the key research questions of the study are: • How can web content mining be operationalized to study business R&D activities?
• How sensitive are the results to the web content mining procedure followed?
• How do website-based R&D activity indicators compare with other conventional R&D indicators? • What are the relative advantages and disadvantages of website data over other data sources for understanding enterprise-level R&D activities?

Data and methodology
The data we use in this study is derived from the project on Sustaining Growth for Innovative New Enterprises, which is examining the determinants of growth in green goods small and medium-sized enterprises (Shapira and Harding 2012). Green goods firms are enterprises who are involved in the production of manufactured goods whose outputs benefit the environment or conserve natural resources. These firms, who produce goods for use in renewable energy, environmental control, and other low carbon applications, typically develop and employ a range of innovative technologies. The project is focusing on small and medium-size enterprises, started in the last decade or so, to probe how such firms grow, what strategies and relationships are employed, and what is the role of external policy and regional resources in supporting growth. To address the limitations of conventional approaches for identification of firms (such as government statistical industrial classifications, which generally lag in developing classifications for emerging new technology sectors), the project developed a comprehensive set of green goods search terms, applicable across sectors, which identified qualifying firms when extracted through textual searches of the business descriptions (for full details of this method, see Shapira et al. 2014). Following a text mining search of the FAME (2014) enterprise database (which incorporates data from UK official company registrations), and subsequent manual validation, a sample of 296 UK-based green goods firms was identified. For the timeframe of 2004-2012, multiple items of data were combined together for each identified green goods firm, drawing from the following data sources: 1. The FAME time series enterprise database for financial information, including R&D expenditure (FAME, 2014). 2. The Technology Strategy Board (TSB) funded R&D projects database, which includes the extent to which identified firms receive support from the UK government for their R&D activities. 3. Publications and patents from the Scopus and Derwent databases. 4. Firm websites (current and archived websites).
A set of keywords was developed for the variables that were to be measured by the web content mining process, subsequently arranged into variable sets and subsets. These included manufacturing strategy (products, manufacturing intensity, customization, greenness); linkages (universities, partnerships, membership organizations, regional/extra-regional links); investment strategy (venture capital, investment); policy influence (regulation); and R&D activity (R&D, research, development, product development, technology development, and related terms). About 25 of the firms are also the subject of case studies to compare insights gained from web and other unobtrusive data sources with those obtained from interviews.
In this paper, we concentrate particularly on variables related to R&D, obtained from the different data sources mentioned above and compare them to indicators derived from the firm websites. From the FAME database, we use company reported R&D expenditures. The TSB database provides us with R&D financial awards from UK governmental sources. Publications and patents for the companies are drawn from the Scopus and Derwent databases. Additional information about company R&D activities is derived from firm websites.

Operationalization
This section discusses the operationalization of the web content mining process used to understand the R&D activities of the firms in our dataset. We discuss the mechanics of the data collection retrieval, preparation and cleaning. The procedure to transform the unstructured website data into structured data is also discussed, as well as the sensitivity analysis conducted on a number of keyword sets.

Operationalization of web content mining
After identifying target firms, the first step in the process is web crawling, an automated process used to capture web pages for subsequent analysis. The web crawling process was conducted in IBM Content Analytics (ICA). We started with a seed list of website domain addresses available in the FAME database. We manually checked and cleaned these website addresses against misspelling and address changes, dropping dead links and adding a small number of new addresses. The resulting seed list of website addresses was fed into the ICA crawler. Once the crawler started, it progressively advanced from the initial seed website address to all other subdomains and webpages on that website, capturing all accessible text. This included text residing within HTML (HyperText Markup Language) as well as other text-based files such as Portable Document Format (PDF) and word processing (DOC and DOCX) files. At the time of the analysis, 2012 was the current or live year. Web pages for the years 2004-2011 were accessed through the Internet Archive Wayback Machine (http://archive.org/web/)-an online archive of past instances of websites. One limitation of using the Wayback Machine to access archived legacy websites is that the Wayback Machine tends to have better coverage of larger websites (which are inclined to be associated with larger firms). Additionally, Wayback Machine coverage is better in some years than in other years. At the end of this stage, rules were set to exclude certain web addresses from the collection. This process encompassed excluding irrelevant sub-domains of a website, which was useful in isolating an English-language version of a website (e.g. allowing www.companyname.com but excluding fr.companyname.com) and in restricting irrelevant sections of large websites (e.g. allowing www.companyname.com but excluding forum.companyname.com). For 2012, we identified 237 firms with websites, out of our initial sample of 296 firms -equivalent to a response rate of 80.1 %. The number of firms with web sites diminishes for earlier years. The number of firms with websites, number of webpages and total number of phrases for each year is presented in Table 1.
Once crawling was completed on all webpages within a collection (i.e. year), the data collected in that collection was indexed and structured into a number of fields. The data was exported from ICA as a series of XML (Extensible Market Language) files that were both human and machine readable. The XML files were then imported into Vantage-Point-a software tool for text mining and analysis. In the course of carrying out this process, a single year was crawled a number of times, as and when problems emerged with particular web sites. In such cases, these datasets were merged and duplicates were removed, resulting in a single and complete dataset for each year. From here, the data was cleaned-a laborious but important part of the process. A number of errors were flagged and corrected, for example: webpages that were not assigned to a firm; webpages not in the English language; duplicated webpages; and irrelevant or non-pertinent webpages, such as customer forums or non-germane sub-domains.
Following this cleaning process, our variables were extracted from the dataset, having first being imported from a group of text files. A further, final cleaning process was required at this stage, for cases where false positives were captured by the extracted variables. For example, the word machine (included as part of a keyword set measuring manufacturing intensity) delivered false positives from the past year collections that were drawn from the Wayback Machine (as Wayback Machine included a header for all webpages). We developed a procedure to exclude such references. Upon completion of this final cleaning process, the dataset was exported as a structured data set. The output was a table of firms against variables (either individual keywords or variable sets). Both the number of occurrences, with counts of multiple instances on the same webpage, and the number of webpages were tallied. The total number of words and webpages for each firm within the dataset was also captured to allow for a normalization process. The data set was then structured and ready to be imported to STATA-a widely used statistical analysis software package.
In STATA, we experimented with a number of conversion and normalization operations. We observed that there was a high degree of variety in terms of the size of websites. Some firms had websites with only a few webpages as a placeholder, whilst others built extensive websites with thousands of webpages. Therefore, normalization of the variables captured was required. After trying both the number of webpages and the number of noun phrases for normalization of variables, we decided to use the latter as the former was not suitable to the variation of the number of webpages across the sample. We then conducted a stock-flow conversion process. The number of keyword occurrences is a flow variable and may not be suitable to be used in conjunction with flow variables in time series data. Therefore, we created flow variables by using the year-on-year percentage changes (Fig. 1).

Defining the keywords for R&D activity
To capture R&D activity, we conducted a simple rule-based variable analysis, structured into six different variable sets. This is summarized in Table 2 in Boolean format. These definitions of R&D activity emerged through an iterative sensitivity analysis. Starting with rndweb1 variable, which simply identifies all webpages that include ''research*'', we examined the results by manual inspection of the text on a sample of individual webpages and determined that it was far too limited. As a result, we iteratively added a further two keywords to subsequent variables. The addition of ''development*'' (to variable rndweb2) Source: Website analysis of sample of 296 UK-based green goods small and medium-sized enterprises (see text for details) Fig. 1 Web content analysis process and ''R&D'' (to variable rndweb3) made a significant difference to our counts, particularly in the case of the former, although this was far from complete. We further examined a sample of webpages that had been captured by these additional keywords, which offered an insight into the words and phrases that appeared on the same webpage as these variables. As a result, a number of additional keywords were added to a fourth variable (rndweb4): ''lab*, scientist*''. However, it was apparent that this variable was also capturing many false positives, particularly in relation to the keyword ''development''. For instance, in our sample of green goods companies, there were a sizeable number of occurrences of ''property development'' that had no relationship to research and innovation activities. A further iteration (variable rndweb5) was undertaken, whereby rather than taking all instances of ''development*'', we instead took only those where ''development'' appeared in the same sentence as ''research'' (a nearby phrase analysis). This again had a significant impact upon our counts for the keyword set. An examination of the rndweb5 variable, where ''development'' is captured nearby ''research'', revealed that whilst this eliminates the capture of false positives of irrelevant ''development'' variants (e.g. ''property development''), a number of relevant phrases were also left out in the process. Therefore, as a final step we developed the rndweb6 variable, in which we explicitly defined a number of relevant instances that included the keyword ''development''. This variable resulted in an optimum balance between capturing the essential concepts and keeping false positives to a minimum.
The six different R&D activity keyword sets performed differently in terms of results (Fig. 2). The average values for each keyword set across all firms and all years vary with a change in keywords. Furthermore, combinations of transformations (i.e. stock or flow treatment) and normalizations (i.e. noun phrases or webpage normalizations) changed the results significantly. An important conclusion from this process is that care needs to be  taken in using simple keywords when mining web text, as false positives will likely be introduced. A more robust process is to ensure that keywords are used in the intended context, and to implement this with key phrases, context searches, and exclusion terms.

Findings
This section presents findings and results from our analysis. This includes the relationship between the website-based R&D activity and other R&D indicators. We also discuss the relative qualities of different data sources in studying R&D.

Results for R&D performance
We found that far more companies in our sample report on their websites that they undertake R&D activities than would be suggested by looking only at conventional data sources. For our sample of firms (N = 296), about 15 % patented at any time between 2004 and 2012, around 5 % published, around 17 % received an R&D grant and around 20 % reported R&D expenditure (according to enterprise reports captured by FAME). In contrast, the rndweb6 website-based variable finds that nearly 70 % of firms in the sample report some kind of research or development activity (including scientific research, technology development, and product development   Table 1 for N each year. See Table 2 for keyword definitions of R&D variable labels over 9 years, patents cover 5.63 %, publications 1.24 %, grants 3.24 % and R&D expenditure data 7.02 %. However, the website based R&D activity variable (rndweb6) encompasses 34.12 % of all observations (Table 3). While all five variables for R&D show the same phenomenon, the correlation between them is not highly significant. We conducted a simple pairwise cross correlation test at 95 % confidence level between these variables. The results show that there is a degree of correlation between R&D expenditure and grants and patents, although these are not correlated with publications. All four non-website-based variables are not correlated with the website-based variables at the 95 % significance level. However, website-based variables highly correlate with each other, due to the highly intersecting keywords sets used for these variables (Table 4). Overall, these results suggests that the web-site variables are providing additional information, and that many firms who report R&D-like activities on their websites do so without also seeking patents, publications, or R&D grants. The additional R&D website information presented by companies does not represent trade secrets, since by definition this is information that is not readily made public. Rather, it can be viewed as R&D that is company-sponsored, which the company cannot or is unable to patent, and which is likely to involve product and technological development rather than scientific research oriented towards publication.
Comparing website data with other data sources Website data has different characteristics compared to other data sources. The fundamental difference stems from the fact that website data is unstructured (i.e. there is no data schema). In contrast, financial databases most often provide structured data, in which variables are uniformly defined across all observations. Publications and patents are semistructured in the sense that the data includes some structured variables, as well as some other variables consisting of text entries without a schema. The challenge of web content mining often lies in the process of structuring the unstructured data by creating a schema and processing the data in order to fit this schema. We use and adapt the data quality framework from Batini and Scannapieco (2006) to compare the relative advantages and disadvantages of website data over other data sources, using the particular example of R&D activity. There are twelve data quality dimensions in Batini and Scannapieco's framework. The relevance of these data quality dimensions to four different sources in our dataset are discussed below and summarized in Table 5.

Completeness (Coverage)
Website data promises advantages in terms of coverage of the population, as the majority of firms have websites. Financial, government support and publication and patent databases typically have lesser coverage. This is particularly important when research involves smaller size firms, which are not obliged to provide official financial data. Similarly, these smaller firms tend to publish very rarely and only a small percentage of them are likely to be supported by government grants. As with other data sources, website data can also cover multiple years, for example by using the Wayback Machine (with the limitation that use of the web first began to grow in the mid-1990s, with companies beginning to use web sites from the late-1990s and early 2000s onwards). In our sample, website data covers around five times more observations than financial data, 10 times more than administrative grants data and around seven times more than patent data.

Accuracy
Website data are self-reports, although this is also the case in one way or another for most other data sources. Keeping this in mind, website data has the potential to give a fuller picture in representing the broad extent of R&D activity. R&D expenditure data from the FAME database and the R&D support data from the TSB database are inputs for R&D activity. While this is often used as the main indicator of R&D activity, R&D inputs might not show the full extent of R&D activity, as some activities might be unfunded or firms may not keep a full account of their R&D expenditure (Kleinknecht et al. 2002). Similarly, patents and publications are output indicators of R&D activity and might miss the full extent of R&D activity. Only a small proportion of R&D activity might result in patent and publication outputs and even when they do so, firms may choose not to publish or patent for strategic reasons. (For a full discussion of advantages and weaknesses of innovation indicators, including R&D expenditure and patents and publications, see Kleinknecht et al. 2002). In contrast, website data appears to show R&D activity that is mid-process, downstream or customer-oriented, as firms have inherent marketing motives to report such activities on their websites. Again, noting that website data is self-reported, there is the possibility that firms might over-represent their activities in their websites (for example, claiming new product developments that are perhaps neither new nor innovative).

Currency
Website data is potentially more current than other data sources. Whilst some firms use websites as placeholders and update them infrequently, most firms tend to add information to their websites constantly and delete information that they think no longer represents their firm (although many older or deleted pages can be retrieved via the Wayback Machine). In contrast, financial data is often already outdated at the time of its release. For instance, firms in the UK are required to submit their accounts to Companies House within Table 4 Pairwise 95 % significant correlations between website-based and other R&D variables 6-9 months (depending on the legal status) of the accounting reference date. Moreover, it usually takes a further few months for this data to be reported in financial databases such as FAME. Similarly, publications, and particularly patents, often take a long time before they are publicly available.

Volatility (frequency)
The relatively higher currency of the website data is also related to the frequency of change (volatility). Website data, as a category, is refreshed every day, as opposed to reported financial data, which changes every 12 months. Patents and publication data and government support data change relatively more frequently than financial data but less than website data.

Consistency
Agreement between the components of the data is an important data quality dimension.
There are issues with all of the data sources we discuss. International guidelines and principles are promulgated on what to include in R&D expenditures (see, for example, the Frascati guidelines on the measurement of science and technological activities in OECD 2002). However, accounting practices differ, particularly among smaller firms. This introduces a certain bias in making the comparison of R&D expenditure between different firms. Similarly, patents and publications vary greatly in terms of quality, which in turn make comparison problematic. R&D grants data is more consistent because of government procedures and regulations. Website data is the least consistent of all the sources considered as the motivations for posting information vary greatly between different firms. Companies may vary in what they choose to disclose and in the way that they report information on their web sites. At the same time, the public nature of websites allows false information to be exposed (and this would not be helpful for firms that seek to maintain their business reputations).

Interpretability
There is a wealth of literature explaining the meaning of enterprise financial data and government grant data. Patent and publication data are also structured and the components of this data are straightforward to understand. However, website data is often difficult to interpret and is sensitive to methodological choices. As we discussed earlier, changes in keywords and search methods used to capture R&D activity do affect the results. Care needs to be taken in methods used to make sense of website data, and it is generally useful to build in multiple iterative steps to review and refine the procedures used.

Accessibility
Enterprise financial and government support data is often more difficult to access than patent, publication or publicly available website data. In our case, UK government grant support data was recently made available but in many other countries this kind of data is not available to the public. Proprietary databases (such as FAME or the Web of Knowledge) require subscriptions (although universities often subscribe to such databases and make them available for affiliated researchers and students). Adapted and extended from Batini and Scannapieco (2006) b Stars qualitatively denote the relative performance of data sources for data quality dimensions (three stars indicate relatively superior performance, while relatively inferior performance is denoted by one star). Comparison is made for the UK

Handling
Website data is more difficult to handle as it is initially in an unstructured form, whilst the semi-structured nature of patent and publication data makes the analysis relatively easier. The methodology outlined in previous sections demonstrates that handling website data is at present a complex, multistep process. Financial and government support data is the easiest in terms of analysis due to established methods and procedures.

Amount
Websites include large amounts of information, in contrast to financial and government support data, which often include a limited number of well-defined variables. Patent and publication data is also sizeable compared to financial data but still relatively smaller than website data. As illustrated in Table 1, which shows the number of websites, webpages and noun phrases in our data set, in 2012 our dataset included over 50,000 webpages and 10 million noun phrases. The substantial quantity of website data provides opportunities in terms of flexibility (as noted below) in what can be analyzed, although sheer size can present problems of data management and operationalization.

Flexibility
Due to its unstructured nature, website data is adaptable to a range of purposes. Different keyword sets and variables can be extracted from websites depending on the research goals. Semi-structured patent and publication data is less flexible than the website data, whilst structured financial and government support data is the least flexible. In our dataset, both financial and publication and patent data is used to investigate the R&D activity.
There are other categories of variables, including those related to manufacturing strategy, linkages, investment strategy and policy, which can be mined and analyzed.

Conclusions
Website data is finding increasing applications in social science, including in innovation studies. Website data has potential advantages over other data sources on aspects such as coverage, currency, accessibility, quantity and flexibility, whilst there are potential issues about its consistency, interpretability and handling.
In this article, we discussed a particular case in which we operationalized website-based variables to understand R&D activity, amongst others. The processes of data retrieval, preparation, cleaning and analysis for web content data are more complex compared to conventional data sources. Care in the interpretation of the website data is particularly important. As shown in the paper, approaches to searching that are too simplistic and overlook context can lead to false positives or incorrectly dropping instances. Studies using website data should conduct extensive sensitivity analyses and also make their methodology transparent.
However, once these steps are performed, website data promises valuable complementary and new insights. We established that for R&D activity the correlation between website-based variables and non-website-based variables was not significant. This reflects the different facets of R&D activity shown on websites when compared with other conventional R&D data sources. Website data frequently highlights downstream or customer-focused aspects of R&D processes, particularly those related to technology and product development, as firms announce their new R&D projects and activities on their websites. In contrast, the annual R&D expenditure for firms, which is the financial data for R&D obtained from the FAME database, essentially gives information about one of the inputs for R&D. Furthermore, financial data is annually aggregated data, as opposed to website data, which tends to show discrete individual R&D activities. R&D expenditure is not the only input to the R&D process and subsequently to innovation (Smith 2005), thus a lack of correlation between R&D expenditure and the R&D process indicated by the website data might also be due to the relative importance of non-financial inputs to the R&D process of the firms in our sample. The same logic applies to the R&D grants provided by the government, as they are another indicator of R&D expenditure and thus an input indicator as well.
The website-based indicator of R&D is also not significantly correlated with patents and publications. Patents and publications show important formal results of R&D activities but not the other complementary processes that bring the results of formal R&D activities to the marketplace (and which are often not well-captured by conventional data sources). It is well documented that there are a number of factors that influence patenting and publishing behavior and firms may decide not to patent or publish the outputs of their R&D activity. Firms may also choose not to place all details of their R&D activities on public websites, but they can generally describe what they are doing for the benefit of customers (real and potential) without revealing extensive technical details to their competitors. This helps to explain the lack of correlation between the website-based indicator of R&D activity and patent and publication based R&D output. The lack of correlation between different conventional indicators of R&D, their relative strengths and weaknesses, and the particular biases associated with them is already well documented in the literature (see, for example, Kleinknecht et al. 2002). The repeated conclusion in many studies is to recommend use of indicators that are appropriate to the purpose, context and particular issues being investigated. Our conclusions augment the previous literature by explicating a framework of relative strengths and weaknesses of website data. This can help in making choices about which data sources and approaches to use in a particular study.
We conclude that website data is promising both as a complementary and an additional source of enterprise information that is useful in studying innovation. There are caveats and limitations to interpreting enterprise website information, and the handling of large website datasets presents its own set of issues. Using website data needs particular technical skills, including skills which are different from those used in the handling of conventional and well-established data sources. In the near future, improved software and more easily manageable searching and analytical methods may make it easier to handle large-scale data mining of web and other online sources. However, researchers in innovation studies (as in other fields) should continue to experiment with using website data: this exploration is likely to open up new ways to generate enhanced understandings of complex innovation processes.
Open Access This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.


PREDICTING FREEWAY PAVEMENT CONSTRUCTION COST USING A BACk-PROPAGATION NEURAL NETWORk: A CASE STUDY IN HENAN, CHINA

mark.king@qut.edu.au Abstract. The objective of this research was to develop a model to estimate future freeway pavement construction costs in Henan Province, China. A comprehensive set of factors contributing to the cost of freeway pavement construction were included in the model formulation. These factors comprehensively reflect the characteristics of region and topography and altitude variation, the cost of labour, material, and equipment, and time-related variables such as index numbers of labour prices, material prices and equipment prices. An Artificial Neural Network model using the Back-Propagation learning algorithm was developed to estimate the cost of freeway pavement construction. A total of 88 valid freeway cases were obtained from freeway construction projects let by the Henan Transportation Department during the period 1994−2007. Data from a random selection of 81 freeway cases were used to train the Neural Network model and the remaining data were used to test the performance of the Neural Network model. The tested model was used to predict freeway pavement construction costs in 2010 based on predictions of input values. In addition, this paper provides a suggested correction for the prediction of the value for the future freeway pavement construction costs. Since the change in future freeway pavement construction cost is affected by many factors, the predictions obtained by the proposed method, and therefore the model, will need to be tested once actual data are

Introduction
Henan Province is located in the centre of China. Its area is 167 000 km2, it has a population of 99 mln and contains 18 cities ranging in size from 1.5 mln to 11 mln people. In 1994, Henan's first freeway was built from Zhengzhou to Kaifeng. In 2007, the total length of the freeways in Henan Province was 4556 km, and by 2020, it is expected that the total length will reach 6280 km.
In the past 10 years, the freeway network in China has developed very quickly and the total investment has been huge. However, in some freeway projects, the final construction cost is higher than the estimated cost at the detailed design stage, which in turn is higher than the conceptual cost at the preliminary design stage. In the context of government financial accountability practices in China this presents challenges; any deviation is likely to be queried, and the Secretary of the Provincial Transportation Dept or a senior official in the department will often have to defend the increased costs publicly or in the state legislature. As a result, the legislature and the public will have perceptions of incompetence and erosion. A more accurate cost estimation process for freeways would therefore contribute to greater public and government confidence in the operation of infrastructure planning and development agencies, as well as contributing to more efficient budget processes.
Researches have indicated that projct definition in the early planning process is an important factor leading to project success (Le et al. 2010;Scott-Young, Samson 2008;Thomas, Fernández 2008). To prepare reliable budgets for freeway construction programs, road authorities must have accurate estimates of future funding allocations they are likely to receive, and future project costs for long term infrastructure programs. While future funding is obviously never known with a great deal of certainty, it is more often the inaccurate estimation of project costs that causes greater disruption to the execution of construction programs.
Various critical factors must be identified to estimate construction costs effectively. Several studies have set out to identify relevant factors, ranging from generic management and financial factors through to those that are specific to the industry under consideration. Stoy et al. (2008) identified quantitative cost factors such as absolute size, construction duration, and compactness as influence factors for good bidding information. Liu et al. (2011) found uncertain factors such as meteorological factor has a great uncertainty in the construction schedule of hydropower construction. Pinto and Mantel (1990) identified the ten critical factors such as project scope, management goals, time planning and management, communication with owner, etc. In a study conducted in Newfoundland, Hegazy and Ayed (1998) found that season, location, type of project, contract duration, and contract size had a significant impact on an individual contract cost. Wilmot and Cheng (2003) described future construction cost in terms of predicted index values based on forecasts of the price of construction labour, materials, and equipment and the expected contract characteristics and contract environments. In a building construction study conducted by Cheng et al. (2009b), ten key quantitative factors were identified in the planning stage of projects. Six were quantitative: floors underground, total floor area, floors aboveground, site area, the number of households and households in adjacent buildings; and four were qualitative: soil condition, seismic zone, interior decoration and electromechanical infrastructure,. Thus, examination of the literature shows that a wide variety of factors have been found to influence construction costs. Factors such as those described above have been used in models of construction costs, but the models rarely attempt to use a comprehensive set of factors. In part, this is a consequence of the methods used for estimation. Shi, Li (2008) integrated rough sets (RS) theory and Artificial Neural Network (ANN) to forecast construction project cost. To overcome cost overruns in projects, some methods such as Probabilistic Simulation (Chou et al. 2009) and Support Vector Machine (Cheng et al. 2010;Chou 2011) have been used to develop appropriate cost models for predicting the expected project cos.
On the contrary, regression analysis represents a traditional approach (Khosrowshahi, Kaka 1996), an inherent disadvantage of which is its requirement of a defined mathematical form for cost functions, i.e. the nature of the relationships between variables must be assumed at the outset. In addition, such traditional methods of estimating project costs are hampered by the large number of important variables and the interactions between them. In addition, some of the variables that influence construction costs, such as the cost of labour, equipment, and materials, are usually highly correlated with each other, resulting in multicollinearity in the model when more than one of them is included. Thus, traditional methods are limited in their potential applicability to the estimation of construction costs.
As a comparatively new method, Neural Network (NN) models have no implicit functional form and therefore have greater freedom to fit the data than do regression models. It is therefore possible that the greater flexibility in the relationship between input and output variables in NN might translate into a better model than that achieved with regression analysis. One purpose of the research reported in this paper is to use NN to identify a better model. Some researchers have employed NN models to estimate the construction costs of individual projects (Ji et al. 2009). By combining NN and fuzzy logic, Boussabaine (1999), Boussebaine and Elhag (1999) developed neurofuzzy systems to estimate the construction cost and project duration of individual building projects. Wilmot and Mei (2005) developed a NN model to estimate highway construction cost escalation over time. Cheng et al. (2009b) developed an evolutionary fuzzy neural inference model to estimate costs at the concept stage. Ma et al. (2012) propose to modify the existing model (a single cost for cost-sensitive neural networks), the traditional back-propagation neural networks (TNN), by extending the back-propagation error equation for multiple cost decisions. Yip et al. (2014) presents a comparative study on the applications of general regression neural network (GRNN) models and conventional Box − Jenkins time series models to predict the maintenance cost of construction equipment.
Furthermore, hybrid models (combining NN and other approaches) have also been developed to estimate construction costs. Hegazy and Ayed (1998) used NN to develop a parametric cost estimating model for highway projects, with optimal NN weightings optimized by genetic algorithms. Kim et al. (2005) applied hybrid models of NN and genetic algorithms to residential building cost estimation in order to predict preliminary cost estimates.
These studies indicate that NN and NN hybrid models have been used instead of traditional methods to estimate the cost, duration, and other features of construction project costs, including highway construction projects. However, it is also clear from the limited literature that NN models have usually been used only for individual construction projects, rather than investigating the overall cost of construction across a range of projects, and examining how their cost alters over time. This approach has an inherent limitation, i.e. that the models developed are relevant only to the case studied, and will therefore not be readily generalizable to other projects. And the models discussed also lack relevance to similar projects undertaken some time later, as the model is specific to a particular time as well, whereas some of the important variables are changing over time in ways which they are modelled. The objective of this paper is to address these issues by developing a NN model based on a range of freeway pavement construction projects and taking temporal factors into consideration. In particular, this research will apply a back-propagation (BP) NN model to predict design cost estimates for freeway pavement construction projects, using historical data on freeway construction projects in Henan Province as a case study of the application of the approach.

Influential factors analysis
The first step in developing the model is to identify which factors influence the costs of freeway pavement construction, so that they are considered for inclusion in the model. These factors have been categorized below as location, resource or time factors, though they also incorporate other variables, e.g. location is related to altitude and topography, both of which influence pavement construction costs in Henan. While the list of potential influencing factors is quite lengthy, a balance needs to be found, such that the number of factors is sufficient to provide adequate forecasts of costs, but not too large for practical application in a management setting. There is no clear guideline as to what the ideal number of factors should be. In this study, it was judged that the nine factors described below (two location, four resource and three time-related factors) should provide a more comprehensive basis for modelling and forecasting than has previously been the case, without creating disproportionate information needs.

Location factors
The freeway construction projects were located across Henan, which is characterized by differences in climate, geology, and topography that might be expected to have an influence on the cost of the projects. These characteristics tended to vary together, so that it was possible to define just three regions based on climate, geology and topography. The region factor is given in Table 1.
An important practical issue for highway construction is the amount of variation in altitude along the road, as greater variation increases costs. This is related to topography, which is taken into account in a broad sense in the regional categories above, but the degree of variation between individual projects pointed to a need to develop categories at the project level. Variation in altitude was therefore divided into five categories from "very small", which described roads that were essentially flat, to variations of between 450 m and 800 m. The variation in altitude factor categories (B1 to B5) are listed in Table 2, along with an indication of where the projects for each category took place, the range of absolute altitudes which applied there, and the freeway contracts which fell into these categories.

Resource factors
Labour, material and equipment are the main resources for a construction project. For simplicity, this study randomly selected five cases as an example to illustrate pavement construction cost components as shown in Table 3. Material costs constituted nearly 85% of pavement construction costs and the equipment costs constituted nearly 12%.
A construction project usually requires more than 100 types of material. The components of pavement material costs of the five cases are shown in Table 4, with the largest four components listed separately. Taken together, the two largest components (concrete and asphalt costs and stone costs) accounted for approximately 76% of material costs. For simplicity, the authors proposed to use the costs of crushed stone (diameter 4 cm) as a proxy for stone costs. The quantity of stone category material is written in the following form: , where N stone − the quantity of stone category material; A similar process is used to quantify an equivalent amount of Calx (calcium oxide) as a proxy for concrete and asphalt material costs. The quantity of concrete and asphalt category material is written in the following form: , where N Conctere And Asphalt − the quantity of concrete and asphalt category material; N j − the quantity of material j; p jo − the bid price of material j at time o; P calxo − the calx price at time o.   − the price of using equipment item i at time k; − the price of using equipment item i at time o. As with the materials costs, this study did not base INEP calculations on all equipment items, selecting only major items which together accounted for more than 80% of all equipment costs.

Data for model development
Data were obtained for freeway construction projects contracted by the Henan Transportation Department during the period 1994−2007. Some nonstandard design and construction projects were removed from the data base. The effective data consisted of contractual information on 88 projects, all of which were four lane divided carriageway freeways with 120 km/h speed limits. Pavement construction cost factors for a sample of the projects are shown in Table 5.
Eighty one of the 88 projects in the data set were used as a training data set, which was designed to comply with the following criteria for minimum size and proportion of total data set.
The minimum training set for the NN is written as follows: , where m − the number of the factors of a BP NN; n − the possible value of each factor (Shi 1995); N − the number of combinations of all possible values of the m parameters. The training sample set is considered incomplete in terms of solving the problem without a sample equal to or greater than N. The problem in this paper has 9 factors. The location factor has three possible values; the altitude factor has five possible values; the labour cost per km is simple and does not involve categories, so it has only one possible value; the two largest components of the cost of materials are "concrete and asphalt" and "stone", so it has two possible values; and the cost of equipment is simple and therefore has one possible value. The influence of "other costs" on the overall cost of pavement construction is not significant compared with other resource costs; it is assigned a value of 1 in our calculation. The possible values of the Index Number of Labour Prices (INLP) and Index Number of Equipment Prices (INEP), which correspond with the labour and equipment resource factors, are both 1. As the possible value of different types of materials has been taken into account in the materials resource costs, the value of the Index Number of Material Prices (INMP) is set at 1 to avoid recalculation. From the discussion above, it can safely be concluded that the minimum size of the training set for the BP NN used in this paper was 30. In theory, the more training samples, the better, but in practice there are limitations on the number of road segments available. 88 samples are gathered; most researchers will select 90% of them as training samples and use the remainder for testing. This research selected 81 of the 88 samples as training samples and used the remaining 7 for testing.

Artificial neural network models for construction cost estimation
Artificial Neural Networks (NNs) were selected to model the pavement construction cost. ANNs are versatile because of their highly distributed parallel structures and adaptive learning processes (Cheng et al. 2009b;Raab et al. 2013;Šliupas, Bazaras 2013;Wilmot, Mei 2005). Of the many structures available for NNs, the multilayer feed-forward network was chosen for this study because such networks have the ability to deal with complex systems and yet are relatively easy to construct (Hegazy, Ayed 1998;Hunter et al. 2012;Ji et al. 2009). To train the model, the back-propagation (BP) learning algorithm was used because it has strong classification and generalization capabilities (Cheng et al. 2009a;Li, Chen 2012;Xiaokang, Mei 2010). The form of neural network used in this study is common in civil engineering applications.
In theory, a three layer BP network consisting of an input layer, n input variables are mapped to m target output variables in a hidden layer and an output layer. Therefore, the general form of the neural network models used in this study is represented as the simple three layers shown in Fig. 1.
The number of neurons in the hidden layer is difficult to ascertain and is normally found by experiment and experience.
The number of neurons in the hidden layer is directly related to the requirements of the problem and the number of neurons in the input or output layer. If the number is too small, there will be insufficient information acquired by the network to resolve the question; if there are too many neurons, it will increase the number of iterations of the network, thus extending the training time and reducing network generalization, thus decreasing predictive power.
First, the number of neurons in the hidden layer is determined using empirical formulae during the design of the network. Second, the network is trained using different neuron numbers. Finally, the optimal number of neurons is obtained by comparing the operating results. The general empirical formula used to determine the number of neurons in the hidden layer (Hirose et al. 1991;Sheela, Deepa 2013) where i − the number of hidden neurons; n − the number of input neurons; m − the number of output neurons; a − a constant and 1 < a < 10.
According to Kolmogorov's theorem, if the number of neurons in the input layer was n, then the number of neurons in the hidden layer is 2n + 1. i is written as: And i is written as: , (9) where n − the number of input neurons; i − the number of hidden neurons. In this study, the max and min number of hidden neurons (i_max, i_min) was determined by (7), (8) and (9), while training the network from the min to max increased the number of neurons by one. The optimal number of hidden neurons was selected by convergence data and training error using the operating results of different neurons number.

Pavement construction cost model development
Nine neurons were used in the input layer. These arose from the construction cost factors identified earlier and shown in Table 5 (region, variation in altitude, labour costs, stone costs, concrete and asphalt costs, and equipment costs, INLP, INMP and INEP).
The min and max determined by (7), (8) and (9) were 4 and 19. The training error and testing error that varied through different numbers of neurons are listed in Table 6. According to the changes of training step and training error listed in Table 6, the training error gradually decreased with the increase of the number of hidden layer neurons, but it rebounds when the number was 17 to 19. In summary, the optimal hidden neurons number was 16.
Only one neuron appeared in the output layer, representing pavement construction cost.

MATLAB program
The MATLAB software package was used to estimate the neural network models. The MATLAB training function for BP network has training functions traingd, trainrp, traincgf, trainscg, trainlm, trainbr and so on. Each has its own characteristics but no single function is adapted to the training process in all cases (Adeli, Wu 1998;Minli, Shanshan 2012). There are also many improved BP algorithms  such as the algorithm with adaptive study velocity and the additive momentum which is implemented using Matlab function 'traingdx' , the gradient descent with momentum function that is implemented using 'traingdm' , and the gradient descent adaptive function which is implemented by 'traingda' etc. The training data set was used to map the input variable pattern to the target output pattern and minimize the error by adjusting the weights of the network links in an iterative process. Training was set to stop after 7000 iterations or until convergence of the root mean square error (RMSE) to a value less than 0.01.
Observing the changes of training step and training error obtained by different training functions determined the number of neurons in each layer of the network. The result showed that 'trainbr' was the best function, as its testing error was the minimum; even 'trainlm' and 'trainrp' had a small training step, but their testing error was relatively large; 'trainscg' , 'traincgf ' and 'traingd' showed much worse results; 'traingdm' and 'traingd' showed the worst results. In short, 'trainbr' is chosen as the training function for the network.

Model testing
A random selection of 81 freeway cases were used as a training data set for the neural network model and the remaining 7 freeway cases were used as a testing set on which the performance of the NN model was evaluated. The testing set projects were Shang-Zhou 4, Shang-Zhou(SQ)02, Yong-Bo A4, An-Nan, Daguang-Xin 8, Ji-Jin and Feng-Nan 08.
The NN model was programmed in MATLAB, with each run producing a slightly different result. The results of 10 runs on the testing set are listed in the Table 8. The statistical measure mean absolute percentage error (MAPE) was used to measure the performance of the models. The MAPE of the seven test cases varied from 0.048% to 2.24%, and the mean MAPE was 0.67%. The implications of this value for the accuracy of cost estimates are discussed below.

Predicting
The model was used to forecast the change in future freeway pavement construction costs based on predictions of input values.
Input variables such as labour, N stone , N Concrete and Asphalt , and equipment costs utilized average values observed between 1994 and 2007. The next two variables, variation in altitude and region, were taken from Tables 1 and 2 based on the specific location and characteristics of each contract. The other three variables , , and were calculated using (3), (4), and (5) and were based on forecasts of future GDP. The resulting values are listed in Table 9.
Using these values, the freeway pavement construction costs predicted by the model for 2010 are shown in Table 10.

Accuracy of predicted costs
As noted above, the MAPE is around 0.67%, which needs to be taken into account in the predictions made by the model. The MAPE is used to calculate an expected range within which the actual future cost is expected to fall. MAPE is written in the following form: .
Hence, CReality is expressed as , , where and are the correction coefficients. Therefore, the range of future freeway pavement construction costs is written in the following form: . (13) Using (13), the range of future freeway pavement construction costs in 2010 is shown in Table 11.

Discussion
The paper presents prediction of the construction cost of freeway pavement in Henan, China using an Artificial Neural Network. It seems to be informative and to provide accurate forecasts. However, the following issues need to be taken into account: -there are more than 9 factors that influence the costs of freeway pavement construction -increasing the number of factors would give a more accurate model, but a greater sample size might also be needed; -for NN, in theory, the more training samples the better, but in practice there are limits to numbers of road segments available; -the price of product is changed suddenly due to the international economic situation; -the model will need to be tested using actual data. In short, there is still some distance to go in order to pursue this approach as an engineering application, but in the meantime it is useful for the Secretary of the Provincial Transportation Department or a senior official in the department to adopt as a reference.

Conclusion
1. This paper has explored a new approach to the estimation of future freeway pavement construction costs by using a Neural Network trained with real data from 81 construction projects and incorporating a more comprehensive set of factors than is typically employed. Data were obtained from freeway construction projects let by the Henan Transportation Department during the period 1994-2007. The data consisted of information on 88 freeway contracts. Data from a random selection of 81 freeway cases were used to train a neural network model and the remaining data were used to test the performance of the Neural Network model. Finally, the likely range of pavement construction cost of three freeways in 2010 was predicted.
2. The factors used in the Neural Network model in this study reflect the characteristics of location (regionwhich incorporates differences in climate, geology and topography -and variation in altitude along the constructed road), resources costs (labour costs, and proxy costs for stone, concrete and asphalt, and equipment), and time-related changes dependent on indexation costs of labour, materials and equipment (Index Number of Prices, Index Number of Labour Prices and Index Number of Equipment Prices, respectively). Neural Network models have usually been used only for individual construction projects, rather than investigating the overall cost of construction across a range of projects, and examining how their cost alters over time, so that this approach represents a new way of addressing the problem of predicting future pavement construction costs.
3. A question which arises is the generalizability of the results, however this is relevant to the specific model derived rather than the process described. While the Neural Network was developed using data from Henan Province, the principal factors and the applicability of the Neural Network process are transferable to other locations. The nine factors used in this study will not all be applicable in another location, and this would need to be determined at the outset through consultation with the relevant agencies and experts.
4. The general form of the neural network model used in this study was three layers and for training, the Back-Propagation learning algorithm was used. In addition, the MATLAB® software package was used to estimate the neural network models, utilizing the training function 'trainbr' with characteristics of adaptive study velocity and the additive momentum method. Again, alternative approaches are tested and other software packages used. 5. One limitation which requires further testing relates to the success of the model's predictions in practice. This study has shown how to develop and train the model, and has tested how consistent its predictions were when applied to a different set of cases, but no attempt was made to test the accuracy of the predictions in practice. This requires a longer term study with greater amounts of data.