The fear isn't that big data discriminates. We already know that it does. It's that you don't know if you've been discriminated against.
Interpretation
The quote highlights the concern about the discriminatory nature of big data and the lack of awareness individuals have regarding its effects on them.
Kate Crawford's quote emphasizes the troubling reality that big data systems often exhibit discrimination, yet the more alarming issue is the ignorance people have about how they might be affected by this discrimination. It stresses the importance of awareness in understanding the implications of data-driven decisions and biases embedded within algorithms, suggesting that the unseen influence of big data can lead to negative consequences for individuals without their knowledge.
In practice
In a discussion about the implications of AI in hiring practices, this quote can highlight ethical considerations.
If you're not thinking about the way systemic bias can be propagated through the criminal justice system or predictive policing, then it's very likely that, if you're designing a system based on historical data, you're going to be perpetuating those biases.
We need to be vigilant about how we design and train these machine-learning systems, or we will see ingrained forms of bias built into the artificial intelligence of the future.
As we move into an era in which personal devices are seen as proxies for public needs, we run the risk that already-existing inequities will be further entrenched. Thus, with every big data set, we need to ask which people are excluded. Which places are less visible? What happens if you live in the shadow of big data sets?
Only by developing a deeper understanding of AI systems as they act in the world can we ensure that this new infrastructure never turns toxic.
It is a failure of imagination and methodology to claim that it is necessary to experiment on millions of people without their consent in order to produce good data science.
If you have rooms that are very homogeneous, that have all had the same life experiences and educational backgrounds, and they're all relatively wealthy, their perspective on the world is going to mirror what they already know. That can be dangerous when we're making systems that will affect so many diverse populations.
If the Net becomes the center of the universe, which is what seems to be happening, then the dizzying array of machines that will be plugged into it will virtually guarantee that the specifics of which chip and which operating system you've got will be irrelevant.
Living as we do in the age of Facebook, we shouldn't be surprised that some countries are starting to imagine themselves more as social networks than as a physical place.
Everybody right now, they look at the current technology, and they think, 'OK, that's what artificial neural nets are.' And they don't realize how arbitrary it is. We just made it up! And there's no reason why we shouldn't make up something else.
It is poor civic hygiene to install technologies that could someday facilitate a police state.
We have spent so much time worrying about a 'cyber Pearl Harbor,'' the attack that takes out the power grid, that we have focused far too little on the subtle manipulation of data that can mean that no election, medical record, or self-driving car can be truly trusted.
None of us today know how to get computers to learn with the speed and flexibility of a child.
Subscribe for the occasional hand-picked quote. No noise.