Umps make calls based on race?

[pete the dog]

Table 3 shows standard errors for all of the point estimates, the most controversial of which is statistically significant. In particular, using the LPM model, the standard error of the .00341 estimate is .0017. It's the .00341 estimate that drives the 1% effect described, since 32% of called pitches are strikes. I didn't bother to calculate an appropriate t-statistic, but with N>1,000,000 and reverting to my grad school presumption that all t-statistics are 2, the effect appears to be statistically significant.

 

Note that the impact of pitch count is dramatically greater than any race effect. Again, the measured effect of race is extremely small, but statistically significant. I'm sure discrimination lawsuits have been won using less persuasive evidence.

 

Yes I've sent them comments. I guarantee I'm not the only one calling foul on these guys. I have a few problems with the paper:

 

There is a voluminous body of research on racial bias, a less emotionally charged word than discrimination, which these authors use in the title. There is no overt claim in the paper that the effect they study is the result of willful conduct, so why did they put racial discrimination front and center?

 

And yes, Table 3 is where statistical significance of their main argument is examined. But if you read the paper, time and time again they try to support their argument with utterly insignificant statistics, e.g. the 11 games I mentioned in my previous post.

 

And re Table 3 my question is, where are the results for White umpires?! Why did they only cite t-stats for less than 10% of the cases, the games called by Black and Hispanic umpires? They admit to leaving out White umpires without justification or any attempt to explain why. It's not hard to guess: their tenuous claim to significance probably evaporated quickly when they added the 90% of the games called by white umpires.

 

And they say this:

 

While none of the coefficients on the umpire race/ethnicity indicators is statistically significant, some suggestive patterns emerge from the first three panels. In Columns 1-3(a), both estimates of the 7

coefficients on the Black and Hispanic umpire indicators are negative, suggesting that pitches thrown by White pitchers are less likely to be called strikes by non-White umpires.

 

No, your results are either significant or they're not. You are not allowed to say things like "suggestive patterns emerge" and other weasel words in an academically honest paper.

 

Also, note that this is a discussion paper, not a refereed published work. I'm sure the authors would be interested in any criticism prior to submission for publication - see the first two words at the top of the paper.

 

Who said it was published? It would have been much more impressive had it been peer-reviewed and submitted for publication before calling up MSNBC (note the publication date and the date MSNBC reported the story), rather than throwing it out there with a "COMMENTS WELCOME" slapped at the top.

 

Another ridiculous quote:

"The results suggest that standard measures of salary discrimination that adjust for measured productivity may be flawed, and we derive the magnitude of the bias generally and apply it to several examples."

 

Really?? The results purport to prove MLB umpires employ racial discrimination in their ball/strike calls. Isn't a stretch to extrapolate "wage discrimination" metrics from pro athletes making $500k/yr and up to standard cases?

 

Well, I guess I should have actually read the paper before I posted. I didn't realize that this wasn't published, but was simply made available to the public and (more importantly) the media for inspection. In my opinion, alerting the media to un-peer-reviewed, unpublished data is a major no-no. Maybe the science is sound, maybe not. But this practice of science is pretty slimy.

 

Talking about "suggestive patterns" is also pretty weak, and would certainly need to be presented with serious caveats in any respectable journal.

 

Finally, as noted above, this paper may be about bias, but it most certainly is not about discrimination.

 

These guys look like media whores. Nevertheless, the data do show a significant bias, and I do think it matters.

[FergieJ31]

In my opinion, alerting the media to un-peer-reviewed, unpublished data is a major no-no. Maybe the science is sound, maybe not. But this practice of science is pretty slimy.

 

Talking about "suggestive patterns" is also pretty weak, and would certainly need to be presented with serious caveats in any respectable journal.

 

Exactly. The general public doesn't necessarily understand the purpose of a discussion paper. On the surface it looks like a legit and rigorous study.

 

Also please note: they have a VERY(*) tenuous claim of significance on cherry-picked data. Why were the games w/white umpires omitted in Table 3?

 

(*) t-stat of their main result is 2.0, which barely makes it significant at the 5% level, not the 1% level.

[UBlink]

In my opinion, alerting the media to un-peer-reviewed, unpublished data is a major no-no. Maybe the science is sound, maybe not. But this practice of science is pretty slimy.

 

Talking about "suggestive patterns" is also pretty weak, and would certainly need to be presented with serious caveats in any respectable journal.

 

Exactly. The general public doesn't necessarily understand the purpose of a discussion paper. On the surface it looks like a legit and rigorous study.

 

Also please note: they have a VERY(*) tenuous claim of significance on cherry-picked data. Why were the games w/white umpires omitted in Table 3?

(*) t-stat of their main result is 2.0, which barely makes it significant at the 5% level, not the 1% level.

 

I don't think the white-umped games were left out - what the model is measuring in the probit models is the difference between the black or hispanic umpires and the white umpires - in other words, the white umpires are the control group. So in the first cell, a black umpire is likely to call a ball thrown by a white pitcher a strike 0.13% less often than a white umpire (with a standard error of 0.22%).

[Balsa]

This just reminds me how much I hate real statistics. :D

[jyoung55]

This doesn't deserve the week or so worth of attention they got here. I'm disappointed every time I see it come back up to the top of the forum.

[FergieJ31]

I don't think the white-umped games were left out - what the model is measuring in the probit models is the difference between the black or hispanic umpires and the white umpires - in other words, the white umpires are the control group. So in the first cell, a black umpire is likely to call a ball thrown by a white pitcher a strike 0.13% less often than a white umpire (with a standard error of 0.22%).

 

The white umpires are the control group? Where do you see that? They state their control variables very clearly on page 7.

 

The numbers in the Table 3 Panels are not to be interpreted the way I think you're suggesting when you say: "So in the first cell, a black umpire is likely to call a ball thrown by a white pitcher a strike 0.13% less often than a white umpire". The numbers in these cells are the beta coefficients (regression slopes) of equation (1), the linear model they estimated.

 

Note that every single beta was statistically insignificant except one, and even that one is barely significant at the 5% level. Now at a 5% level of significance they should expect 1 in 20 of their measured parameters to show up as significant by pure chance. They estimated over a couple of dozen linear models, so it's not surprising they'd get one hit, especially by cherry-picking their data.

[UBlink]

The white umpires are the control group? Where do you see that? They state their control variables very clearly on page 7.

 

From page 8

 

Table 3 presents the results of estimating (1). The first three panels show the analysis for each possible pitcher-umpire race/ethnicity combination rather than using UPM, i.e., they estimate separate equations for each pitcher race/ethnicity and include individual indicators for umpire race/ethnicity. In the final panel we aggregate all pitchers, so that the indicator of interest becomes UPM as in (1). In each panel Columns (a) and (b) present estimates of the marginal effects from probit models, while the equations in Columns © are estimated as linear probability models (LPM) with pitcher fixed-effects and robust standard errors. In the first three panels the omitted category for umpire race/ethnicity is White.

 

It really has been decades but I believe in a probit model you can't use all the possibilities - one has to be omitted. Hence the bolded quote from page 8. My imprecise use of the term control group was unfortunate and I apologize.

 

The numbers in the Table 3 Panels are not to be interpreted the way I think you're suggesting when you say: "So in the first cell, a black umpire is likely to call a ball thrown by a white pitcher a strike 0.13% less often than a white umpire". The numbers in these cells are the beta coefficients (regression slopes) of equation (1), the linear model they estimated.

 

columns a nnd b are probit models with the interpretation I suggest, and c is a linear probability model so I think the interpretation is applicable there as well

 

Note that every single beta was statistically insignificant except one, and even that one is barely significant at the 5% level. Now at a 5% level of significance they should expect 1 in 20 of their measured parameters to show up as significant by pure chance. They estimated over a couple of dozen linear models, so it's not surprising they'd get one hit, especially by cherry-picking their data.

 

I don't see cherry-picking of data - I do see creating a single model that incorporates all the data, thus increasing the likelihood of getting a significant result. And unless things have changed a lot in empirical economic studies over the last 2 or 3 decades, this is a reasonably good example of how this kind of research is done. Most empirical research involves using whatever statistical tools are available in order to get a significant result.