This section of The Black Box Society, by Frank Pasquale, expresses why it’s so crucial that practitioners of digital social science cultivate reflexivity. This issue cuts across methodology, research ethics and politics: it’s not simply a narrowly technical issue. From pg 41:
Continuing unease about black box scoring reflects long-standing anxiety about misapplications of natural science methods to the social realm. A civil engineer might use data from a thousand bridges to estimate which one might next collapse; now financial engineers scrutinize millions of transactions to predict consumer defaults. But unlikely he engineer, whose studies do nothing to the bridges she examines, a credit scoring system increases the chances of a consumer defaulting once it labels him a risk and prices a loan accordingly.
Mark Carrigan 