Randomized experiments are considered the gold standard. How does it work? First, you divide the population, these black mice, into sub-populations, that receive the treatment – the red mice – or that do not receive the treatment, they serve as the control group – the blue mice.
A random assignment provides us with the needed counterfactual – again not at the individual level. We can only make comparisons at the average. Randomization makes sure that the assumptions of the regression model hold, and the interpretation of regression results becomes straightforward.
Why is that not used all time? Because it is oftentimes infeasible. First of all, the costs for these experiments are quite high, directly or indirectly speaking, and there might be ethical considerations that simply make it impossible to run an experiment, especially in social sciences and a couple of other reasons.
How can we think about the identification using more data? Here we have an example of cross-sectional heterogeneity in panel data. So, there is firm A, the one with the yellow dots, firm B, the one with the blue dots, and there is firm C with the red dots.
What happens if you treat them as the same observation in, what is called a pooled OLS model, though you know that they are different firms? You get this black line here and you are discarding these unobserved firm effects. They are then included in the error term.
The consequences are serial correlation between the error terms and if there is a correlation between x and these firm effects, the , the firm specific intercepts, then the error term becomes correlated with x. And this bias estimates of and you get incorrect standard errors. So quite obviously, if we look at the graph, the black solid line is a very inaccurate picture to describe the relationship between x and y.
How do we manage to get rid of these estimation bias? We can use, what is called, fixed effects or random effects. The idea of fixed effects is that you are allowing for an intercept for each group.
So, you give firm A, firm B and firm C their intercepts. And therefore, you get also different slope parameters. The way you do this is that you create a dummy variable for each group and estimate the by least squares.
That variable is either 1 or 0. So in our regression model, you are adding dummy variables for the firms. Or you can subtract the group-mean of y from the Then you can get rid of everything that is unobserved.
But this will only work with a one panel dimension. Random effects are slightly more involved. Here you assume these group specific intercepts.
They are distributed in some hyper population and then it has some kind of distribution, say it is normally distributed. And then we estimate the parameters of that distribution – we estimate the mean c and the standard deviation from data. Finally, estimate for each group as if they were drawn from this normal distribution .
What are the pros and cons of random effects vs. fixed effects? While Random effects allow to include group level predictors that are constant within groups, for example in the industry.
Hence, it needs less data than fixed effects, because you only need to estimate two parameters, the mean and the standard deviation of that hyper population from which you draw your random effects. As with Fixed effects, you need to estimate as many parameters as there are groups, actually the number of groups minus 1. In contrast to Fixed-effects, Random-effects assumes that there is no correlation between x and these intercepts.
As this is often hard to justify, people typically prefer Fixed-effects over Random-effects. Also, fixed effects are kind of easy to think about and easy to implement. Therefore, it is widely used in practice.
Let’s talk about Identification using quasi experiments. A consistent estimator can be based on instrumental variables The idea is to find a variable that is correlated with , but is uncorrelated with the error term . The problem here is that good instruments are typically hard to find.
You can think of them as suggested by economic theory, they can be lagged variables that instrument for current values under some assumptions, and there can be quasi-experimental variation. Here is an example of that: In Galasso and Schankerman, 2015 They asked whether patent rights facilitate or impede follow-on innovation. Here we have an obvious endogeneity problem: Technologies with greater commercial potential are both more likely to be protected by patents and to be an attractive target for follow-on innovation.
The Set-up in the paper is to estimate the effect of removing patent rights by court invalidation on subsequent research, as measured by later citations of the patent. The Identification strategy is fairly clever. Some judges are stricter than others, and therefore invalidate more patents.
But the way the court system, at least in the U. S. , works is that it randomly assigns judges to cases.
So, you can randomly have a stricter judge in some cases. The Results are there are 50% more citations after invalidation; an effect stronger in electronics than in biotech; stronger for small firms. What they estimate is the number of citations that a patent gets as a function whether it has been invalidated and a bunch of other stuff.
And they instrument this variable invalidated, the x variable with an instrumental variable z, which is in this case, a propensity of whether the three judges, that are randomly assigned to the case, will declare the patent invalid. They construct this variable based on historical court decisions. Finally, Identification using natural experiments.
One very useful tool is difference-in-differences. The Identifying assumption is, that whatever happened to the control group over time is what would have happened to the treatment group in the absence of the treatment. Sounds complicated but is very simple as you will see in a minute.
So, you have a control group that sort of develops over time. Represented by the black dots here. And then you have a treatment group that also develops somewhat over time.
Shown by the green dots. Next, there is the treatment happening, which transforms the green dots into the red dots. And that is what we are interested in: Trying to measure how big is this treatment effect.
With the tool Difference-in-differences, we estimate the dashed line here and we use the slope of the control group to predict what the treatment group would have looked like, has the treatment not happened. So why is it called difference-in-differences? Because we, first, need to take care of the difference between the treatment and the control group.
That is this stable constant difference between the black dots and the green dots. Then we take the difference for and after. And then, we take the difference-in-differences to arrive at the treatment effect.
Difference-in-differences is a simple two-period, two-group comparison. It is incredibly useful in combination with randomization, matching, or regression discontinuity. Regression discontinuity is random discrete jumps in .
Kretschmer and Peukert, 2020 This is an example where we used this tool in my research. Our question in this paper was: Do music videos increase or decrease sales of recorded music? The Set-up is that we observed weekly sales of thousands of songs.
There has been a licensing dispute that causes YouTube to block all music videos in Germany. So that is the natural experiment here. For the Identification strategy, we compare sales of songs with music videos – that is the treatment group – to sales of songs without music videos – that is the control group, just before and after the blocking.
So, some of these songs now don’t have music videos anymore and the other songs don’t care about this licensing dispute, because they never had music videos in the first place. And when we do this before and after comparison between the treatment and the control group, we can show that without music videos on YouTube, sales decrease by 10%.
