Public R&D Policy Impact Evaluation:Propensity Score Matching and Structural Modeling Estimations

Abstract

This dissertation is about public research and development (R&D) subsidies to support private firms doing innovative activities and quantitative impact evaluation of the policy on total factor productivity (TFP) change and additional R&D effort. Public R&D subsidization as a public R&D policy, beside different types of public interventions, has been widely used by governments to stimulate private R&D. These policies aim to fill the gap between the private and social rates of returns by encouraging business enterprises to spend on additional R&D, produce more innovation output and inventions, or change their innovative behavior. These changes can be carried out either individually or in collaboration with other entities. One ultimate goal of R&D policy is increasing the total factor productivity and relative performance both at firm and aggregate levels. This study deals with direct place-based public R&D subsidies and empirically measures the effects of this type of public incentives on productivity growth and R&D input additionality. In order to evaluate the policy effect, a quasi-experimental counterfactual setting for subsidized (treated) and non-subsidized (non-treated) firms can be framed thank to the characteristics and mechanism of the local R&D program in the Province of Trento in Italy. The average treatment effect of the policy on target variables is measured for subsidized units (Average treatment effect on treated: ATET) and for the whole population of the firms (Average treatment effect: ATE), using techniques capable of tackling the problems of endogeneity and selection bias which arise in empirical evaluation studies. Propensity score matching (PSM) and structural modeling methodologies are used to measure the effects of the R&D subsidies on target variables, TFP change and additional R&D expenditure, respectively. The former approach is non-parametric and does not assume a functional form for the effect of policy on R&D and productivity change, while the latter models the optimizing behavior of the firm (agent) and the public agency, searching for an equilibrium in a pre-determined game theoretical framework. Although the PSM method takes advantage of no pre-defined structure assumption, however and in contrary, the structural model with simultaneous equations, takes into account the effect of unobservables on subsidized firms’ selection procedure, beside R&D spillovers effect. In order to design the evaluation framework to estimate ATE and ATET on the target variable of interest (TFP change), we have built a firm-level panel dataset (maximum 5 years of information) constructed by combination and merge of datasets related to public (provincial) R&D policy, firms’ characteristics, firms’ R&D activities and TFP change measures. The time span of the dataset allows us to capture the effects in both short-term and long run, consequently tracing the short and long term effects of the R&D program. This helps us to consider the usual longer effect lag an innovation policy entails, specifically on a target variable such as productivity and a treatment such as innovation incentive, which the effects may take time to be realized in comparison with other types of outcomes and investment policies. The dataset represents the outcome of a long process of combining and merging various datasets related to firms’ financial statements and balance sheet (AIDA: Italian company information and business intelligence) and APIAE’s R&D policy information provided by ISPAT. TFP change and its decompositions, technical efficiency change and technological frontier change are realized using Malmquist Data Envelopment Analysis (DEA) method. DEA takes a system approach towards the firm as decision making unit (DMU) and only applies the input(s) and output(s) measures to calculate the relative (in)efficiency of the firms. Malmquist method based on index theory, captures the (in)efficiency change and the technological frontier movement within a time interval. TFP change measures are calculated by CRS output-oriented DEA dual model using a new package introduced in STATA software and merged into the reference dataset described previously. To sum up, after the formation and construction of dataset by combining and merging different datasets, treatment effect analysis is carried out using PSM nearest neighbor and kernel estimators. The balancing property satisfaction on pre-treatment observable factors (age and size in our setting) is primarily investigated and propensity distribution graphs have been also provided. Taking into account the dataset features, R&D subsidies effect is measured for manufacturing and ICT industries (using 4 techniques to measure both ATE and ATET), beside low-medium technology and high-tech industries classifications (Both ATE and ATET). Moreover, the subsidies effect on TFP measures have also been measured for different categories of selection procedures. Results show heterogenous and mixed effect of R&D subsidies based on different settings of evaluation (sectors and selection categories), targeted outcome, PSM method (different PSM algorithms for nearest neighbor and kernel) and time of the effect (short-term or long run). The complete results have been discussed in detail in the related sections in chapter three. To address the effect of unobservable factors, beside spillover effect on R&D subsidies allocation and the effect on outcome, a structural model is estimated using a cross-sectional dataset. The dataset is formed by merging R&D policy-related (linked to Provincial Law LP 6/99 enforced by provincial agency for the promotion of economic activities :APIAE) dataset and firms’ determinants provided by ISPAT (Statistical institute of Province of Trento).This approach complements the drawbacks due to estimation using PSM methodology. However, the pre-defined functional form for equations is a limitation of this approach. The structural model applied includes application decision, selection (subsidies allocation) and R&D investment equations to be turned into econometric equations for empirical estimation. The context and dataset features allow for different empirical modifications with respect to the benchmark model applied. The results determine the effect of firm (project) characteristics on all stages of the subsidization game. Size, age, exporting status, board size and sector are main factors being investigated. The results show not only there is no additional R&D expenditure, but also some crowding out of subsidies occurs. The base model is determined in such a format which makes it possible to evaluate the spillover effect and spillover rate of R&D spending as well. The results show that on average half (50%) of each euro spent on R&D spill overs. The results shed light on the effects and impacts of a place-based R&D policy on TFP change ,R&D additionality and spillovers, while suggesting policy implications to the local public authorities. Furthermore, the design and process of impact evaluation using two different complementary approaches in a new context on a different target variable (TFP change in addition to classical input additionality variable) can be referred and applied in any policy evaluation related studies. In the following, chapter one deals with the theoretical and empirical reasons for the existence of different public R&D policies based on Schumpeterian growth theory, spillovers effect and tackling market failure. It further provides a review of R&D and innovative activity indices at different levels of analysis (regional, national and international) and reviews the literature of empirical innovation policy evaluation studies related to the effect of R&D policy on additionality. The review concerns both micro and macro perspectives in approaching public R&D policy and the impact of the policy on additionalities and TFP growth. Studies in R&D policy usually concern either the macro growth accounting approach and measure the effect of R&D policies on aggregate growth indices regardless of pointing out to micro foundation effects leading to the aggregate level changes or they only focus on the micro-econometric firm-level evaluation without addressing the relationship between firms’ additional R&D activities and the economic growth. Moreover, the R&D activities expenditure and growth indices at international, EU, national (Italy) and regional (Trento Province) have also been briefly pointed out and tracked over time, to realize the practical importance of R&D incentives. In order to introduce and spot the areas this research addresses, the traditional market failure and the logic and reasons behind public R&D policies (aimed at increasing positive externalities and R&D spillovers) and subsequently different innovation policy instruments and their interaction with firms’ R&D decision making have been reviewed. This provides a comprehensive perspective over the importance and the forms of R&D policies. Chapter two primarily discusses about the effect of R&D subsidies on TFP change. The discussion addresses the relationship between R&D and total factor productivity (TFP) as a channel which subsidies may affect TFP. In addition, other channels and interactions which can explain the effect of R&D subsidies on TFP change and the components of TFP change, will be investigated and discussed. Afterwards, in line with the review of the previous chapter, the empirical literature of studies dealing with evaluation of the effect of R&D subsidies on TFP (as a different outcome variable from additionality variables discussed in the previous chapter) will be reviewed. This theoretical background helps us to shape the R&D policy evaluation framework to investigate the direct casual impact of R&D subsidization policy on target outcomes including TFP change (Chapter 3) and R&D expenditure (Chapter 4). Finally, taking into account the evaluation framework, we hypothesize the research questions based on the theoretical concepts and literature review discussed through the previous and current chapters. Chapter three measures the effect of the provincial R&D subsidies on technical efficiency and technological frontier change as the decomposing elements of productivity change. It empirically measures the impact of R&D subsidies on productivity change using counterfactual treatment effect analysis. Malmquist Productivity Index (MPI) based on the non-parametric method of Data Envelopment Analysis (CRS output-oriented dual DEA model) is applied to measure the productivity change and the disentangled elements of productivity change. The chapter has contributed to the literature in some different aspects. The main focus of this chapter is measuring the effects of R&D subsidies on decomposing elements of TFP, technical efficiency and technological frontier change. In the whole literature, there is only one other similar work in which the effects of R&D subsidies on TFP decomposed components have been assessed. There are few other papers in which they measure the impact of other type of investment subsidies (mainly capital subsidies) on targeted variables of TFP decompositions. However, they all use a parametric approach to measure the TFP components in contrary to non-parametric Malmquist DEA method applied in our study making no predefined assumption about the production function. The subsidies effect evaluation is implemented using both PSM nearest neighbor and kernel methods (to check the robustness) to measure the average treatment effect of R&D subsidies on subsidized (treated) and all (the population) firms labeled as ATET (average treatment effect on treated) and ATE (Average treatment effect), respectively. The analysis is mainly carried out in manufacturing and ICT sectors as two main sectors in which R&D incentives allocations occur. The elaboration on classification of firms in different main industries based on ATECO 2007 system of firms’ economic activity coding has been carried out. It has also been defined and described in detail how 6-digit industry code is categorized into sectors. Another important feature of this study different with a considerable share of empirical literature, is construction of a panel dataset on subsidies allocation and firms’ characteristics which allows us to capture the effect of the policy both in short term and long run (maximum of 5 years). Moreover, the effect of the evaluation based on two different types of selection and allocation procedures (automatic, evaluative (combined with negotiation method) is implemented. The limitations of this chapter imposed by the methodology used, are excluding the effect of unobservable factors on selection process and not taking into account the spillovers effect. Consequently, Chapter’s four structural modeling puts effort to overcome these restrictions and suggest a complementary approach. Chapter four empirically estimates an equilibrium oriented structural game model to investigate the relationship between firms’ characteristics with application cost (application decision equation), spillover rate (subsidization equation) and R&D investment (investment equation). The chapter reviews, modifies and estimates a structural model describing the mechanism through which the R&D subsidization policy influences R&D activity and the R&D spillover rate. The empirical contribution of this chapter is proposing a simplified model of a reference 4-staged game model with a Nash Bayesian Equilibrium (NBE), based on the contextual setting of the region under study and data availability. The advantage of using this structural model is the ability to assume spillovers effect. This optimization approach relaxes the incapability of evaluation approach used in Chapter three to assume the presence of spillover effect due to the violation of Stable Unit Treatment Value Assumption (SUTVA). Moreover, chapter four takes into account the effect of unobservables on selection procedure and targeted variable, while in chapter three the unobservables are assumed uncorrelated with selection (subsidy) variable and the outcome. Nevertheless, opposed to structural modelling, the method used in chapter three does not assume any parametric form to evaluate the impact. Hence, chapter three and chapter four complement each other in measuring the impact of R&D policy on targeted variables. The empirical evaluations and results of both final chapters are explained and concluded in the related essays. Moreover, the features and contribution of chapters will be restated in the abstract at the beginning of each chapter.