Reddit reviews Mostly Harmless Econometrics: An Empiricist's Companion

>Empirical methodology is about running regressions in order to establish causal or at least predictive relationships within the dataset.

Perhaps this is what empirical rigor means in practice, but the view that this is what empirical rigor should mean is ultimately untenable.

Josh Angrist might re-assert /u/OMG_TRIGGER_WARNING's question like this: it doesn't matter if X predicts Y almost with certainty, if tomorrow some policy change will cause the relationship to fall apart entirely. Causality is more important than correlation, because causality is the only true test of an actual economic model. Moreover, causality isn't something that you get from matching your data with some DSGE equations, finding p<.00001 with Newey-West standard errors, then passing a Hausman test. Unless you have a plausible quasi-experiment with a tight chain of causality, you have nothing except a statistical relationship. You can't even identify a diagram like X -> Y -> Z -> X.

There is a sort of nihilism in that worldview. If someone makes a valid criticism that breaks your chain of causality, there's no honest response except to ask for a suspension of disbelief. When all's said and done, you're not allowed to believe anything except local average treatment effects (LATEs) from randomized experiments. I don't see this as a useful standard to hold every single piece of empirical research to, because it's unreasonably demanding.

That's why I would agree with your general response, since I think macro is useful. This is because of one of the other reasons you've mentioned - there seems to be a sort of stationarity in the data where predictive relationships remain stable for a while. That's where I permit some suspension of disbelief. I think that makes me relatively lax, but I don't see a better alternative to answering the kinds of questions macroeconomists and policymakers need to ask. I might rephrase your answer to OP's question like this: macro is useful if we're OK accepting a lower standard for what constitutes useful information. There is use for statistical relationships which we hope will continue into the future but which aren't, currently, causally founded.

This is definitely the right sub. A few notes:

1. You can approach applied economics papers with just a semester of economic statistics / econometrics and a semester of intermediate theory. Applied papers can be either micro or macro. For example, at this point you should be able to comfortably read Mankiw, Romer, and Weil (1992).
   
   Most applied economics paper boil down to I ran a regression of Y on X. The key question you have to ask yourself is, "Do I buy their identification?" To assess such claims, you don't need much more than a semester or two of econometrics and critical thinking. Sure, some of the more obscure estimators might be beyond your ability, but the broad swath of applied papers boil down to some kind of instrumental variables regression.
   
   A somewhat more advanced paper that should still be readable to you is Gali and Gertler (1999).
   
   The game here is: write down your model to be estimated; argue that you have a good identification strategy; show us the tables of coefficients; argue that your results are robust; tell me why I should care. The trick is clean identification and economically & statistically significant results.
   
   Resources: your companion here should be Mostly Harmless Econometrics.
   
   
2. Then there are theory papers, which can be micro or macro. These will generally set up an economic model and prove some results analytically. The limiting factor in reading them will be your mathematical maturity: here is where the real analysis and topology come into play. A typical theory paper (that you cannot probably read comfortably) might look like Mas-Colell (1975).
   
   The game here is: write down a model; derive some mathematical results (typically about certain partial derivatives, cross-elasticities, or existence of equilibria); tell me why I should care. The trick is to write down a model that makes sense and bring results that are applicable, either to other theoretical problems or that can be applied to practical problems.
   
   Resources: MWG is probably good preparation for these papers, at least in form.
   
   
3. Third, there are computational papers. These tend to be macro, and within macro tend to be oriented around business cycle analysis. Here there is a lot of assumed background knowledge: macroeconomists expect to see models written down in particular ways and have certain preconceived expectations of what your "results" should look like. Again, I don't think these are approachable right out of the undergrad curriculum. I certainly could not follow the arguments of Ireland (2004) when I was an undergrad. I could barely work through the first few pages of Clarida, Gali and Gertler (1999).
   
   The game here is: write down a model; solve it approximately near the steady-state; simulate the model; show me some dynamics around the steady-state; show me how the model reacts to shocks; tell me why I should care. The trick is: write down a sensible model that captures the phenomena of interest, and show how the economy reacts to the shocks that you hit the economy with. Sometimes you want to show how policy can counteract those shocks.
   
   Resources: while reading a few chapters of Sargent & Ljungqvist is nice, there is still a lot of assumed background when reading macro that makes these papers a bit forbidding if you haven't had graduate training in the subject. Yes, I find it deplorable, but that's how the profession has evolved.
   
   A fantastic place to start in macro is Models of Business Cycles. You can probably read it if you know a little calculus and have taken a course in intermediate macro.
   
   By the way, all of the papers I've linked to are "classics" and are worth perusing, even if you don't fully grasp what's going on in them. I'm biased, so you've gotten a sampling of macro papers. Let me know if you want details/context on any of the papers I linked to.
   
   My deepest apologies if, in these summaries, I have offended the sensibilities of my applied and theoretical brethren. I'm stepping a bit out of the bounds of my field (money & macro). :)
   
   (Anyone know of good "classics" in applied micro that are readable? Card-Krueger (1994) is probably readable, as is Angrist (1990). But I'm not familiar with the classics in labor and IO.)

Both time series and regression are not strictly econometric methods per se, and there are a range of wonderful statistics textbooks that detail them. If you're looking for methods more closely aligned with econometrics (e.g. difference in difference, instrumental variables) then the recommendation for Angrist 'Mostly Harmless Econometrics' is a good one. Another oft-prescribed econometric text that goes beyond Angrist is Wooldridge 'Introductory Econometrics: A Modern Approach'.

For a very well considered and basic approach to statistics up to regression including an excellent treatment of probability theory and the basic assumptions of statistical methodology, Andy Field (and co's) books 'Discovering Statistics Using...' (SPSS/SAS/R) are excellent.

Two excellent all-rounders are Cohen and Cohen 'Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences' and Gelman and Hill 'Data Analysis Using Regression and Multilevel/Hierarchical Modelling' although I would suggest both are more advanced than I am guessing you need right now.

For time series I can recommend Rob Hyndman's book/s on forecasting (online copy freely available)

For longitudinal data analysis I really like Judith Singer's book 'Applied Longitudinal Data Analysis'.

It sounds however as if you're looking for a bit of a book to explain why you would want to use one method over another. In my experience I wanted to know this when I was just starting. It really comes down to your own research questions and the available data. For example I had to learn Longitudinal/fixed/random effects modelling because I had to do a project with a longitudinal survey. Only after I put it into practice (and completed my stats training) did I come to understand why the modelling I used was appropriate.

ORLY.

No, I provide data (Bernanke's book) that proves that there is clearly causation, and that lowering deflation helps get out of recessions. If you want more data, look it up yourself. You'll be damned to find any evidence that deflation helps the economy.

I shouldn't even have said "correlation is not causation." My mistake. The proper thing I should have said is "now we have to figure out the causality." See, when you have correlations like the Phillips' curve, it's maddeningly obvious that there is causality; you just don't know where it is exactly. Here's a great quote from a great econometrics book:
>Less thoughtful observers fall back on the truism that "correlation is not causality." Like most people who work with data for a living, [economists] believe that correlation can sometimes provide pretty good evidence of a causal relation, even when the variable of interest has not been manipulated by a researcher or experimenter.

In other words, correlations are good evidence for causality. Which makes perfect sense, actually.

If anything, the author of this article has committed a huge case of "correlation is not causation," not me. He's taking a sample size of three (US, France, Germany), and he's making the case that Germany's and France's slightly faster recovery is attributed to one thing when there are not only literally thousands of variables at play, but a million cases in history where deflation clearly does not result in economic well-doing.

Anyway, the causality happens to be both ways according to the last half century of economic research. The model used to explain this is the "aggregate supply and demand" model. Basically, 99% of things that increase inflation also usually increase unemployment, and vice versa. For example, increasing the money supply is inherently inflationary, and an economy-wide demand for employees inherently increases employment... but they're both considered rightward shifts in aggregate demand, i.e. they both decrease unemployment and increase inflation.

Sigh... this is econ101, dude.

The Card paper is old but the Peri and Yasenov paper does have clean data and code available here.

Also has the raw CPS mariel data and codes on how to construct the synthetic controls. My guess is that with the raw CPS data you should be able to get at Card's results too.

Knock yourself out jan. Pick up an econometrics textbook while you're at it too. You might actually learn something in the panel data section (if indeed you are capable of such a thing at all).

Please report the findings of your "investigation" for our viewing pleasure.

Thanks!

Mostly Harmless Econometrics. Explains things at an undergraduate level but still a good resource for looking back on at all levels. (I have a PhD in Econ and have this on my shelf.)

I'm a Political Methodologist; I'm happy to give you some help. It would be useful to know what your mathematical background is, and what sort of things you're interested in doing. You have to understand, to me, this question is a little bit like "I'm interested in American Politics; suggest an introductory text, please." There's a huge variety of stuff going on here, it's hard to know where to start.

Do you want to be able to read statistics wrt PoliSci? Or are you interested in figuring out how everything works, so that you can create / replicate?

If you want something very undergraduate centric, my undergrad research methods class used the Kellstedt and Whitten book. It was fine, but obviously very rudimentary. It will get you to understand some of the big picture type stuff, as well as some of the simple statistical nuts and bolts you'd want to understand. This class also used the everpresent King, Keohane and Verba text, which is oriented around qualitative work, but Gary King is the foremost quantitative methodologist in the discipline, so it's still pretty good (and "qualitative" certainly doesn't mean "non-rigorous" -- it's cited a lot because it really delves into deeply into research design). That said, I don't remember a whole lot about this class anymore, and I haven't looked in these books for ages. My feeling is that both of these books will probably be close to what you're looking for -- they're oriented around intuition and identifying the main issues in inference in the social sciences, without getting too bogged down in all of the math.

That said, if you have more math background, I'd suggest Mostly Harmless Econometrics which is often used as a first year graduate level quant methods book. It's absolutely fantastic, but it isn't easy if you don't have the math background. It may also assume some preexisting rudimentary probability or statistical knowledge. I'd also suggest the Morgan and Winship. These two books are structured more around causal inference, which is a subtle reframing of the whole "statistics in the social sciences".

For more nuts and bolts econometrics, Baby Wooldridge is one of the standards. I think it's pretty often used in undergrad econ classes.

In general, though, statistics is statistics, so if you want to learn it, find an appropriate level of statistics/econometrics book.

Take a look at those books in your library/online/etc and see if any of them are what you're looking for.

Mostly Harmless Econometrics

and

The Signal and the Noise

are my recommendations for an introduction into more advanced topics in econometrics. If you want more of a textbook Th3Plot_inYou's suggestion is good (I still have mine from my class).

Edit: Signal and the Noise is more theoretical about forecasting in general.

Mostly Harmless Econometrics, Joshua Angrist and Jorn Steffen Pischke

Game Theory Evolving, Herbert Gintis

Microeconomics: Behavior, Institutions, and Evolution, Sam Bowles

The Company of Strangers: A Natural History of Economic Life, Paul Seabright

Mostly Harmless Econometrics is a classic. It gets into regression, but also design with inference in mind. Combine that with ISLR and you should be set!

I would highly recommend the following book: Mostly harmless econometrics

http://www.amazon.com/Mostly-Harmless-Econometrics-Empiricists-Companion/dp/0691120358/ref=wl_it_dp_o?ie=UTF8&coliid=IVHBGLQH4VJ3I&colid=21QCK3AR703JR

It is very problem driven book and will help build up your knowledge base to know what models are appropriate for a given situation or dataset.

You will then need to start practicing in a statistical program to gain the practical skills of applying those models to real data. Excel works, but I don't know a good book to recommend to guide you through using excel on real problems.

I recommend Stata to new data analysts and have them pick up "microeconomics using stata"; once they've worked through these two books they get excited and start grabbing data all over and begin running models, its exciting to watch new data modellers apply tools they're learning. R is free and open source but is more difficult to learn, if you're willing to ask there are tons of people willing to help you through R.

I'm not sure about an online course, but I can recommend some econometrics textbooks.

Goldberger's "A Course in Econometrics" is well written and covers a lot of important ideas. I especially like his treatment of residual regression in chapter 17 (I think): https://www.amazon.com/Course-Econometrics-Arthur-S-Goldberger/dp/0674175441/ref=sr_1_1?ie=UTF8&qid=1465847395&sr=8-1&keywords=goldberger+econometrics

Many people teach regression as minimizing the squared residual from a linear model. While that's a correct way to think about it, in my opinion it is easier to understand regression as performing matrix algebra on a data-generating process. That is, a linear model says that x causes y according to

y = xb + e

where y is an observed column vector of length n (for number of observations) x is an observed matrix, possibly including a constant, e is unobserved, and b is a parameter (vector) to be estimated. Well, just do algebra on it.

you want to "move" x to the left-hand side, but x doesn't have an inverse. Instead, multiply both sides by the transpose of x, which is x', and then you have x'x in front of b. If this can be inverted, then multiply both side by it's inverse. (x'x)^-1 x'x cancels, yielding

(x'x)^-1 x'y=b+(x'x)^-1 x'e

if (x'x)^-1 x'e=0, then you have just solved for b. In expectation, this is true under the OLS assumptions, and as the sample gets large, it is approximately true in sample. This is why OLS can recover b if the error is orthogonal to x. If not, then OLS gives you biased estimates of the causal parameter b.

Regression algebra is indeed quite simple. This makes regression algebra satisfying -- you are doing something extremely powerful without requiring comparably sophisticated mathematical technology.

Anyway, Goldberger's treatment of regression algebra really clicked for me, especially making sense of residual regression (why "all else equal" makes sense). You don't need to read every chapter. Chapter 17 works pretty well on it's own, for example. But the other stuff is useful as well.


"Mostly Harmless Econometrics" is not too hard to read without coursework forcing you to focus: https://www.amazon.com/Mostly-Harmless-Econometrics-Empiricists-Companion/dp/0691120358/ref=pd_sim_14_6?ie=UTF8&dpID=51qgNUMbyXL&dpSrc=sims&preST=_AC_UL160_SR104%2C160_&refRID=NY8XZVBAX0ZHXXV69SAT

You might as well get Wooldridge's graduate level textbook on panel data econometrics -- you'll probably need to buy it in grad school anyway. It's hard to make sense of until until you've been forced to work through a lot of the math. After your first quarter or two of graduate level course work you should be comfortable enough with the material to teach yourself anything in this textbook. Before that though and you might not have the discipline or background to make heads or tails of this: https://www.amazon.com/Econometric-Analysis-Cross-Section-Panel/dp/0262232197/ref=sr_1_7?s=books&ie=UTF8&qid=1465847582&sr=1-7&keywords=wooldridge

A good bet would be "Mostly harmless Econometrics"

http://www.amazon.com/Mostly-Harmless-Econometrics-Empiricists-Companion/dp/0691120358

Not overly theoretic and very focussed on practical applications.

I really like applied stats but think a good understanding of stats theory is important for any researcher. A good "litmus" test, I think, would be reading Mostly Harmless Econometrics (you can probably find a place to check the book out for free). It isn't about health statistics at all but it is a very good "applied" theory book. If you get into the first bit and go "this isn't for me," that is fine and epi probably won't be a problem. If you go "this is interesting," then you might be worth looking at doing the required pre-reqs for the MS biostats program.

Relatedly, check out this free biostatistics bootcamp on Coursera. Check out the first few weeks of lectures and if you decide that the stat theory is more than you care for, epi is a good place.

Epi is a good field, don't get me wrong, but if you are interested in statistics, then it might not be a great fit. MHE and a few of those lectures might be very helpful in deciding if you are at all unsure of how you lean.

/u/sumant28 gave a solid piece of advice above. Of these two, normally I'd recommend Wooldridge over MHE anyday, but now considering that a lot of health economics is treatment-effect stuff, you might want to pick up MHE.

e. that's Mostly Harmless Econometrics just in case you didn't know this.

> Or that communism creates starvation (joke)

I don't think this is a joke. While causal designs would be difficult to apply, the spatio-temporal correlation is hard to ignore.


>Regarding causality- as you know that’s nearly impossible to prove in the social sciences.

Actually, these days the application of designs and approaches that provide strong support for causal claims have become quite prevalent. Some standard references-



1

2

3

4

good framework reference or a slightly heavier read

and the old classic


In fact, the Nobel prize in economics this year went to some people who have built their careers doing exactly that

It's actually become quite hard to publish in ranking journals in some fields without a convincing (causal) identification strategy.


But we digress.


>We will never be able to do an apples to apples study between heterosexual and homosexual child rearing for some of the reasons you mentioned above. (Diversity of relationship styles, not both biological parents within gay/lesbian couples)

In this case it isn't far fetched at all. The data collection for the survey data used in the study you linked could just as easily have disagregated the parents involved in same sex romantic relationships instead of pooling them. If I understood correctly, the researcher had obtained the data as a secondary source, so they didn't have control over this.

Outcomes for children in the foster care system are well studied, so one could in principal easily replicate the study comparing outcomes between children in the foster care system and those adopted into homes shared by stable same sex couples (you couldn't likely restrict it to married same sex couples, though, because laws permitting same sex civil marriage are too recent to observe outcomes).

>My bottom line-that I don’t see many disagree with if they are being intellectually honest, is a stable monogamous heterosexual family structure is the best model for immediate families. Or would you disagree?

But that's not the question at hand, is it? What we are interested in here is comparing kids bouncing around the state care system to those adopted into homes with two same-sex parents in a stable relationship.

That is exactly my point. The comparison you propose is uninformative relative to the question of permitting same sex couples to "foster to adopt". Because the counterfactual for those children is not likely to be a "stable monogamous heterosexual family". It is bouncing around the foster care system.

Beyond intermediate texts, my classes ended up just reading papers from econ journals. You may want to pick up an econometrics text, get familiar with the methods, then read papers (here is a list of the 100 most cited).

I wrote my opinions on econometric textbooks I've used for another reddit comment, so I just pasted it in below. If you get into it, I'd recommend reading a less rigorous book straight through, then using a more rigorous text as reference or to do the practice stuff.

Less Mathematically Rigorous

• Kennedy - survey of modeling issues without the math. More about how to think about modeling rather than how do it. Easy to read, I liked it
  
  
• Angrist - similar to Kennedy, covers the why & how econometrics answers questions, very little math. Each chapter starts with a hitchhikers guide to the galaxy quote, which is fun. Just as good as Kennedy
  
  
• Long - this book is more about just "doing stuff" and presenting results, absolutely non-technical, but also dodges the heavy thinking in Angrist & Kennedy so I wasn't a big fan
  
  
• King - covers the thinking of Angrist & content of Maddala. It is more accessible but wordier, so give it a go if Kennedy or Angrist are too much. It is aimed at Poli Sci rather than econ.
  
  Middle of the Road
  
  
• Gujarati - I used this for a class. It wasn't hard to follow, but it mostly taught methodology and the how/why/when/what, and I didn't like that - a little too "push button" and slow moving.
  
  
• Woodlridge - a bit more rigorous than Gujarati, but it was more interesting and was clearer about motivations from the standpoint of interesting problems
  
  
• Cameron & Trivedi - I liked the few chapters I read, the math is there, but the methodology isn't driven by the math. I ddin't get too far into it
  
  More Mathematically Rigorous
  
  
• Greene - lots of math, so much it was distracting for me, but probably good for people who really want to learn the methodology
  
  
• Wooldridge - similar to Greene, you need a solid understanding before diving into this book. Some of the chapters are impenetrable
  
  
• Maddala - this book is best for probit/logit/tobit models and is somewhat technical but dated. My best econometrics teacher loved it

You could try "Mostly Harmless Econometrics" by Joshua Angrist

Edit: https://www.amazon.com/dp/0691120358/ref=cm_sw_r_cp_apa_R1VFAbRYNTEV9

I don't understand how my example of spurious correlation among randomly generated numbers doesn't already meet that burden. That's a data generating process that is not causal by design but produces your preferred observed signal.

Your additions of "repeated", "different times" and "different places" only reduce likelihood of finding a set with your preferred signal (or similarly require checking more pairs). There's literally a cottage industry around finding these funny noncausal relationships http://tylervigen.com/page?page=1

If you're imagining something more elaborate about what it means to move "reliably" together, Mostly Harmless Econometrics walks through how every single thing you might be thinking of is really just trying to get back to Rubin style randomized treatment assignment
https://www.amazon.com/Mostly-Harmless-Econometrics-Empiricists-Companion/dp/0691120358

Yes but the question is why investor 1's should be significantly different than investor 2's in regards to their covariates. If you have issues with the methodology of the paper, you have an issue with every single pharmaceutical study ever run.

In addition, the issue isn't really whether investor 1's and investor 2's balance (though they should and do), it is whether there is a significant difference within investor 2's. Category B and C are being compared to the category A baseline, which captures the possible prior-networking effects you mention.

>The follow up survey relies on honesty

To a degree yes. When the vast majority of your responses are weighted 90%+ in one direction or the other? Probably no issues there.

>something that people with some experinece and some sense already know

Yes, but there's the question of quantifying it. It doesn't strike you as interesting that knowing a friend owns an asset vs. knowing they want to buy it vs. knowing they bought it and currently own it have 3 very different levels of influence on your purchase decision?

EDIT: Since I think we're rapidly approaching the point of intransigence, I will just say this: I think you need to give the field as a whole a bit more credit. I know that's a tough ask considering your opinion that the field as a whole is bunk. I'd ask you to consider that most of the economists that I've met are seriously intelligent, with a great grasp of graduate level mathematics, statistics and of course, economics. Many share your views about basic fallacies in the field, others don't. The long and short of it is: they spend months of their lives thinking about these topics. Don't get me wrong, there are bad papers and there are bad economists. That said, most papers published in top journals are actually pretty good, and are actually pretty interesting. They're not out to get you, they're not out to get anyone. They're just searching for causal relationships in an environment which makes causal relationships very difficult to find.

If you're interested in reading more about how economics works, I'd suggest reading Mostly Harmless Econometrics. I find it a lot better than the more popular Freakonomics in explaining the basic (modern) tenents of applied economic research. Anyway, have a good day, I hope this discussion wasn't a complete waste of your time!

In my graduate econometrics course we used Mostly Harmless Econometrics. It's focused on the question of causal inference, and specifically how to do empirically rigorous studies when variables aren't exogenous. It covers a bunch of best practices in design experiments. It's not focused on networks, which is a rapidly emerging field of study in the social sciences. However, it does a very good job of explaining possible sources of error in statistical inference and research design.