The eﬀectiveness of RD support in Italy. Some evidence from matching methods

In this study several matching procedures have been used to evaluate the impact of public R&D support received by Italian manufacturing firms over the three-year period 2004-2006. Data are from the Capitalia-UniCredit survey and estimations refer to a sample of 605 treated firms (untreated are 2414). The evidence is mixed and depends on the objective-variable under consideration. As far as the total amount of R&D investments is concerned, the role of public support to innovation is positive and significant, while no impact has been found when considering the R&D intensity and the share of sales due to innovative-products. These differences in results are quite regular, whatever the matching method applied in the evaluation.


Introduction
There are two main arguments to explain the low level of private R&D investments. The first refers to the appropriability of basic research. If technology is a quasi-public good then the incentive to invest will be reduced because each firm will try to take advantages from the innovative efforts made by others. The final outcome is a level of innovative activities which is lower than that desirable at an aggregate level (Arrow 1962). The second element influencing R&D investments relates to capitalmarket imperfections. The risk of research leads investors to increase the cost of financing innovation and, as a consequence, tends to reduce the amount of research made by the private sector. This is particularly true for Italy, a country with a low propensity to innovate due to specific characteristics of its industrial sector which is dominated by small firms and by firms operating in low-tech sectors.
These considerations help to understand state intervention in favour of R&D activities. Any innovation policy is aimed at making up for the difference between social and private returns on R&D innovations and ensuring financial facilities to innovators, particularly in the first stage of the innovation process. While the initial objective of R&D policy is to increase the amount of innovative activity, the general scope of any research and innovation policy is to strengthen the position of each country among the leading knowledge and competence-based countries. In other words, public support for private R&D is a good policy option per se because increasing technological potential through sizeable investments should lead to innovation and, ultimately, growth in an economy. This is basically the mission of many R&D programmes, such as, for example, Europe 2020 which is part of the EU's growth strategy to promote a more competitive economy in the coming years. With regards to the theme of this paper, it is of value to point out that, among many other objectives, Europe 2020 fixes at 3% the proportion of the EU's GDP to be invested in R&D up to 2020. According to the EU commission, this is a pre-requisite to have a smart-growth which is based on more effective investments in education, research and innovation. As mentioned before, the level of actual R&D efforts is lower than the optimum and very far from 3%. For instance, in Italy, R&D investments were 1,26% of GDP in 2010, while the average of the EU-27 was around 2% (the intensity was more than 3% in some Nordic countries (Finland, Sweden, Denmark) and more than 2% in Austria, France, Germany and Slovenia. However, compared to the early 2000s, Italy has increased its innovative efforts by about 20-25 basis points from, R&D investments of just over 1% of GDP in 2000.
However if, and to what extent, the objectives of R&D programmes have been achieved is an empirical issue to be addressed through an evaluation study. This paper analyses the effect of the innovation policy from which a sample of Italian manufacturing firms benefitted from 2004 to 2006. With this goal, the literature is followed and an ex-post evaluation is carried out by using the counterfactual approach, which -through different methods -permits the measurement of what would have happened without the policy. In order to assess the impact of Italian R&D policy support at firm level, the matching techniques are applied, just as in Almul and Czrnitzki (2003), Czrnitzki and Licht (2005), Herrera and Heijs (2007), Duguet (2004), Gonzalez and Pazò (2008).
Data used in this paper are from the survey carried out by Capitalia-UniCredit (2008) and cover the years 2004-2006. This source allows precise identification of whether a firm has received a policy support within R&D programmes or not. The possibility of distinguishing the two groups of treated and untreated gives an advantage in that the analysis does not suffer from the potential bias of other sources of public funding, as would be the case if we only paid attention to a specific scheme without being able to control for the presence of other policies. In this, the paper is similar to many other studies. However, this is not without cost. Indeed, knowing whether a firm participates or not in a programme impedes to assess the role of the different policies implemented in favour of private innovation in Italy. Therefore, the results are meant to be the average effect of overall R&D policies adopted in Italy in the period 2004-2006. We find that R&D policy has been effective in increasing the amount of R&D investments made by firms, although the effect disappears when considering the intensity of innovative efforts. There is similar inconclusive evidence with regard to the impact of R&D policy on the capability of firms to sell innovative products.
The paper is organised as follows. The next section presents a brief break-down of the sample of firms used in the empirical analysis. Section 3 describes the methodology, while section 4 looks at the results and concludes..

The sample of Italian manufacturing firms
This analysis uses data from the survey carried out by Unicredit-Capitalia in 2008 for the years 2004-2006. This survey comprises standard balance sheets and collects a great deal of qualitative information on firm characteristics for a sample of about 4,500 Italian manufacturing firms, including all firms with more than 500 workers and a representative subsample of firms with more than ten workers (the stratification used by Unicredit-Capitalia considers location, size and sector). With regards the objective of this paper, the Unicredit-Capitalia survey comprises information regarding firms' R&D investments in 2004, 2005 and 2006, the value of innovative products as a share of total sales (averaged over [2004][2005][2006], the type of innovation introduced and whether firms benefitted from public support for R&D activities over the period [2004][2005][2006]. There is no information on the source of financing, i.e. whether the support was activated as part of local, national or European projects -or on the amount of funding received. Table 1 presents the sample of firms, classifies firms by size and distinguishes the innovative from the non-innovative firms and those receiving R&D public support. The entire sample is comprised of 3,019 firms, many of which are small (the proportion of firms with sales below 5 million euros in 2004-2006 is 26%). The share of firms with a value of sales below 50 million euros is about 88%. Only 2.19% of firms are big. This distribution is roughly repeated in the case of supported firms, which number 625, that is to say 20% of the entire sample. Thus, the sample is formed of 605 treated and 2,414 untreated firms. From data displayed in table 1, it emerges that the majority of firms (2,445 out of 3,019) are noninnovators, in the sense that they did not introduce any product/process innovation over the years under scrutiny. The imbalance of firms in favour of noninnovators is also found when considering the subsample of supported firms. In this case, in fact, 483 out of 605 firms received public aid over the threeyear period 2004-2006, but they introduced no innovation. In the group of supported-non innovators, the distribution by size indicates a large presence of small firms: 283 out of 483 firms -that it is to say 60% of the sub-sample -register sales of less than 10 million euros. In brief, the sample is dominated by non-supported firms, no-innovators and small-sized firms.

The empirical setting
This paper aims at assessing the effect of R&D support on the innovative activities of Italian firms. For each firm, we observe the amount of R&D investment and the sales of innovative products as a share of total sales. To truly know the effect of R&D policy, it is necessary to compare the observed outcome (the so-called factual outcome) with the outcome that would have occurred had that firm not benefitted from public support (the counterfactual outcome). The latter is unobservable and, therefore, represents an evaluation problem. The issue is to provide an estimation of the counterfactual which allows a calculation of the policy-effect. In order to evaluate the counterfactual, this paper refers to the literature on non-experimental methods because of the non-randomness of the assignment of firms to the groups of beneficiaries. To be more precise, the matching methodology is used. The empirical analysis takes place in two steps.
First, the study identifies a group of untreated firms which are as similar as possible to the treated. Initially, the analysis deals with the curse of dimensionality (firms may be similar in a given dimension but different in others), which we address by using the propensity score. The matching between treated and untreated is carried out by using an index of the probability of being treated, known as the propensity score. In this sense, the propensity score forms the basis for the match. This approach addresses the problem of a non-random assignment to the programme and controls the groups of treated and untreated by comparing observations that are similar concerning their characteristics. The idea of considering observable variables is to eliminate the initial differences between treated and untreated so that the assignment to the programme is random. This is known as the Conditional Independence Assumption (CIA) and ensures to control for sample selection bias. In other words, if the CIA holds, then the differences between treated and untreated can be attributed to the R&D programme. In order to test the quality of results, different goodness-of-fit measures are used and some robustness checks are performed.
In the second step of the analysis, the group of matched-untreated firms is identified and the ATE, that is to say the effect of R&D policies on the treated, is measured. As shown below, in order to test the robustness of the effect, matching has been carried out in different ways.

The participation model
Firms' participation in the R&D programme is modelled as a binary choice, where the dependent variable takes the value of one if the firm is in the group of beneficiaries and zero if it does not actually benefit from any support. The propensity score is obtained as a conditional probability of being in the programme, provided a set of firm's characteristics. Following the related literature, a broad variety of variables that might have an influence on a firm's decision to participate in a R&D programme is used. The variables are the following: firm size and firm age, total debts, cash flow, number of patents, a dummy equal to unity if the credit obtained is less than that the firm required at the market interest rate, and a measure of human capital. In estimating the probit equation, we also control for sector and South effects. In order to control for sectoral heterogeneity which might be influential in determining the probability of participating in the programme, we include dummy S4 which is unity if the firm operates in science-based sectors and zero otherwise, that is to say the dummy is zero if the firm belongs to one of the other three Pavitt sectors (supplied-dominated sectors, scale intensive sectors, specialised suppliers). We also use the variable South which is a binary variable equal to unity if the firm is located in the South of Italy and zero otherwise. The variable South is supposed to capture the non-observable differences between the Centre-North and the South of Italy.
Results are displayed in table 2. Despite our interest is in understanding the effectiveness of the policy, it is useful to briefly discuss some results. It is interesting to point out that the probability of being treated is negatively dependent on the age of the company, in the sense that the probability of participating is higher for young firms than older ones. The evidence shows that size does not affect the probability of being treated, while significant impact has been found with regards to cash flow: the higher the internal finance, the lower the probability of participating in the R&D programme. Although the statistical significance is not high, similar results emerge for the effect of debt on the probability of participating. Finally, being located in Southern Italian regions reduces the probability of participating in a public programme of support for private R&D activities ( Table 2).
The usefulness of considering a participation model is that it helps to balance the distribution of firms' variables across the two groups of treated and untreated firms. In this respect, the balancing property of the propensity score must be satisfied before proceeding with the matching of firms (Rosenbaum and Rubin 1983). This condition requires that firms with the close propensity score must have the same distribution of observable covariates independently of the treatment status. This ensures that, for any propensity score value, the assignment to treatment is meant to be "random" and, therefore, treatment and control units are observationally identical, on average. Put in a different way, the balancing property says that treated and controls are close in terms of observables and, therefore, that matching is possible. From a practical perspective, the test is an iterative procedure. It requires treated firms to be divided into a number of strata according to their estimated propensity scores. When, in each stratum, the regressors of the probit model do not differ between treated and untreated, then the participation model may be considered adequate to balance firms' characteristics. In the case of imbalance, even in a single stratum, the second step is to consider strata more finely. It might be that the property is not satisfied whatever the strata. This leaves room for modifying the participation model through the introduction of higher-order terms (squares, cubes etc.) for particular variables or for interactive terms. Hence, this test can provide a useful diagnostic with regard the specification of the participation model. The model specification presented in table 2 is that which guarantees that the balancing propriety is satisfied.
After it has been verified that the participation model satisfies the balancing propriety, the next step is to select the outcome-variables of interest so as to determine their average values between treated and untreated-matched firms and to use the difference in average as a measurement of the impact of R&D programme. In order to pursue this goal, firms must be matched and this is done by referring to different procedures, as is shown in the following section.

Number of firms to be selected: matching at work
The first matching is performed through the nearest neighbour method (henceforth NNM), where each treated firm is matched with the untreated firm with the most similar propensity score. The selection of controls has been carried out with repetition. This means that the possibility of assigning the same nontreated to more than one treated was not excluded a priori. Table 3 indicates that the number of units included in the untreated control group is 399, a value lower than 481, which is the number of treated firms used in the matching under the restriction of common support.
Through NNM, it is possible to match a treated with a control with a very different value of its propensity score, because it is the closest among those available. This caveat is overcome by fixing a minimum distance between the two propensity scores to be matched. This is the idea of radius matching, which ensures that for each treated, the group of controls is comprised of all the firms whose propensity score has a distance that is less than or equal to a certain "radius" (which will tend to be very close to zero: for example 0.01). Results may be sensitive to the size of the radius that is the basis for matching: the smaller the radius, the more difficult it is to find a match within that range and this will lead to a greater number of cases failing the support requirement. In this study, three radii are considered, 0.1, 0.05 and 0.01. Table 3 shows that there are always more than 1,900 matching-untreated firms. It can also be noted that the number of treated used in the matching is 480 when the radii are 0.05 and 0.01. This means that only one firm is excluded because there is no control whose propensity score is within the interval defined by those radii.
The third approach used in matching the firms is the stratification method according to which the propensity score is divided into strata, in such a way that, within each strata, the treated firms and the untreated firms have the same propensity score, on average. As table 3 shows, the stratification method uses 481 treated firms and 1,912 untreated firms.
Finally, firms are matched by using kernel matching, through which all untreated firms are used as controls. This means that all the information available is used in the analysis, as all firms -treated and non-treated -are included in the estimate of the treatment effect. Under the common support requirement, 481 firms are used by the kernel matching as participants and 1,912 as nonparticipants in the R&D programme.
Before passing on to the discussion of the treatment effect, it is essential to verify if the matching is appropriate. The importance of whether the participation model satisfies the balancing condition has already been discussed. A further confirmation of the appropriateness of the matching comes from comparison of the estimated propensity scores across treated and controls. This comparison provides a useful diagnostic tool to evaluate how similar treated and controls are and, therefore, how reliable the estimation strategy is. Figures 1 and 2 display the estimated propensity scores before and after matching. In figure 1, the kernel of propensity score for treated firms is compared to that obtained for the sample of the potential group of control, while, in figure 2, the focus is on the group of firms used as controls and obtained applying the NNM. As can be seen, the matching works well given that the overlapping of distribution improves moving from figure 1 to figure 2. As the graph of the propensity score distributions after the matching shows, both groups of firms are well balanced with respect to the propensity score.

Discussion and conclusion
Previous sections mention that the paper is aimed at evaluating the impact of R&D policy support received by a sample of Italian firms over the 2004-2006 period. Due to the lack of other information about the type and amount of funds received by these firms, the only research question that we can ask is whether the subsidized firms perform better than firms that did not receive any support. As in all evaluation programs, the next and final step regards the identification of the outcomevariables, that is the variables that R&D policy is expected to change. Just as with many other papers in this field of research, the outcome variable selected is the R&D investments made by firms. The Capitalia-UniCredit survey reports on the annual amount of money spent on R&D investment by firms. We utilise R&D expenditure in absolute terms and as R&D intensities (R&D expenditures as a share of total sales). While R&D investment is an input of the innovative process carried out by firms, there may also be interest in investigating the effect on output of innovative effort. In this respect, we consider the proportion of sales which is due to innovative products.
Results of the calculation of ATE obtained through matching methods are presented in table 2, where, for each variable-outcome and each method, we summarise the number of treated and controls, the average of the variable for treated -labelled Y(1)and untreated -labelled Y(0), the mean difference, Y(1)-Y(0), and the value of t-statistics obtained with bootstrap. In panel A, the results are those under the common support condition, while this condition has been relaxed in panel B.
The calculation of counterfactual outcome differs across the methods. In the case of NNM, it is the average value of the output-variable for the untreated, while, in the case of the radius method, it is obtained by averaging the outcome-variable of controls belonging to the radius. The stratification method uses two steps. Firstly, the procedure determines the mean difference between Y(1) and Y(0) in each stratum. It is like a policy effect within each stratum. The overall impact is calculated as the weighted-average of differences in the various strata, where the weights are the number of treated firms in each strata. The idea is to give more weight to the strata with many treated firms. With regard to the kernel matching, the counterfactual Y(0) for each i-th participant is determined as a kernel-weighted average of the outcome of untreated, where the weight of each j-th untreated is in proportion to how close its propensity score is to that of the i-th treated firm.
Let us now present the main results of the analysis. When considering the NNM estimator and the common support requirement, it is found that the average R&D intensity is 0,88% for the 481 subsidised firms and 0,49% for the group of 399 treated. Thus, the resulting effect is about 0,4% and is statistically different from zero even after bootstrapping. However, this evidence is not robust to the procedure used for matching. Indeed, the average treatment effect remains positive and amounts to about 0,15% with any method other than nearest neighbour, but its significance is lost. Similar results have been found without the common support condition. Based on this, it may be concluded that public R&D policy does not generate a significant positive effect on the R&D efforts (measured through R&D intensities) of Italian manufacturing firms. Table 2 also displays estimations of the average treatment effect on the level of R&D investments. In this case, the picture changes drastically. With regards the effect obtained using the NM method, it is found that, on average, the treated firms invest about 969 thousand euros in R&D, while the untreatedmatched firms invest 273 thousand euros. This difference represents the ATE which amounts to 696 thousand euros. More importantly, this difference is statistically diverse from zero, as the bootstrapped tvalue is 2.9. For a robustness check, it is necessary to look at the other estimations. It emerges that the set of untreated is comprised of more than 1,900 firms and the resulting average of R&D investments is always more than 400 thousand euros. This implies that the mean difference decreases (ranging from 535 to 576 thousand euros), but, above all, that ATE is always statistically significant. The same applies when the common support hypothesis does not hold.
To sum up, the impact of R&D support depends on the method used to measure the R&D efforts.
When the outcome-variable is expressed in absolute terms, that are the value of R&D investments, the evaluating analysis yields a positive robust significant effect of incentives, while no conclusion can be drawn when using the intensity of R&D activities.
The impact on inputs is useful to the extent that it is accompanied by a similar impact on the outcome variables of the innovation process. Assessment of the effects on output responds to the question of whether the results that are observed are due to the subsidy, and to what extent they would have been obtained even in the absence incentive. In this sense, we consider the sales on innovative products (expressed as the percentage of total sales). Data refer to 2006. The analysis shows that, on average, innovative products account for about 8% of firms' sales. While this amount is slightly higher for untreated than for treated, the difference is not statically significant, whatever matching procedure is used.  The final number of blocks is 9 -This number of blocks ensures that the mean propensity score is not different for treated and controls in each blocks. The balancing property is satisfied.