Welcome to episode 146 of The Digital Life, a show about our adventures in the world of design and technology. I’m your host, Jon Follett, and with me is founder and co-host Dirk Knemeyer.
Today Dirk, we’re going to discuss the surveillance state in software. In particular, China is embarking on some interesting endeavors to use predictive analytics and big data to stop terrorism before it can happen. The key words there are before it can happen, which might sort of bring to mind the pre-crime unit of Minority Report fame which is the dystopian Philip K. Dick novel that was made into a movie starring Tom Cruise that everybody likes to cite for various reasons. Pre-crime is the concept that before even the criminal knows exactly what they’re doing, the surveillance state knows and clamps down on them. In this case terrorism is sort of the first and foremost element that the Chinese state wants to prevent. Of course, we have a similar mandate to protect United States citizens from terrorism and use all this data. Of course, the questions that all of that raises, especially using predictive analytics to do it, are interesting indeed. I’ll set up the news item which you shared with me yesterday.
China Electronics Technology Group, which is one of their state run defense contractors, is developing the software, and they’re basically going to assemble and analyze data on ordinary citizens. It’s everything from what they’re buying, what their employment is, where they go on holiday, things like that, to try to feed this predictive analytics data base. Secondly, at the beginning of the year there were some anti-terror laws in China that allowed the government … They went into effect at the beginning of the year and they allowed the government to access a whole slew of new types of data. Everything from bank accounts to even surveillance cameras. The name of their surveillance camera unit is unironically named Skynet for some reason. Thanks very much for that. Additionally you got your major Chinese tech companies which more or less comply with government requests for data, everything from their major search engine Baidu to their social messaging apps to their e-commerce sites. All of them supply information.
Then lastly the Chinese government revealed last year that it was building a nationwide database that would rank citizens and score them on their trustworthiness. Boy, I hope I get a good … Well, we all, if we were being ranked on trustworthiness, would want to get an A grade. Wouldn’t we? If the government was ranking us. All this is very creepy and it’s perhaps easier to access from this news item which is focused on China, of course, but we have similar surveillance state creepiness happening in the US. Pick your poison, Dirk. Where would you like to start?
Yeah, there’s lots of fodder there. I mean, look, as you’re sort of alluding, it’s easy to dismiss this as a China thing. Right? Totalitarian state regularly oppresses the liberties of the citizens. For me what’s compelling about this story is the fact that it’s a foreshadowing of things that we’re going to be dealing with in the United States, both in ways similar to what are being outlined here in a Chinese context, but also things far, far beyond that I believe. We are likely headed for a future where the government is using data to predict future behavior and takes preemptive action to control that behavior. Look, even thirty years ago we knew very little about the human animal. We would rely on things like characteristics that go back to classical times that we would bestow upon people. That’s a person with character. That’s a person with self-control. That’s a person with honor. That’s a person who is trustworthy.
There are notions that I think are going to appear increasingly quaint as we know more and more about the human animal. I did a lot of research into violent crime a few years ago. It’s something that I find troubling and I’ve put some time into some problem solving in those areas, but the number one predictor of a person who will commit a violent crime is their gender. There’s a specific characteristic, I believe it’s called major violent crimes. Ninety-something percent of the people who commit major violent crimes are men. Now, we could easily armchair theorize at what the reason is for that. One candidate would be high testosterone, for example. However, what’s going to happen is in the next decade people will have a much better idea. Science will tease out a much better idea of what is this strange brew that results in people killing, raping, doing horrible things. The fact is that …
I’m going to focus on rape and sexual crime. Those are epidemic. If you look at the statistics, I believe it’s one out of four women who attend a higher education institution are going to be either the victim of sexual violence, or the victim of attempted sexual violence. One in four. Now, that’s a stat that just kind of hovers in the background, but to me that’s an epidemic. With someone with a young daughter and the idea that flip two coins and get tails twice and she’s going to be vulnerable to that, that’s not acceptable. We are as a society finding it increasingly unacceptable to perpetrate horrible things on the disadvantaged. By disadvantaged I mean the non-white males, basically. We are becoming increasingly vigilant in protecting women, racial minorities, the disables, however you want to look at that.
I think we’re heading for this sort of tipping point, if you will, where we are going to have so much knowledge around human motivation and behavior, scientifically, not armchair theory, that comes together with our finally saying, “You know what? It is not acceptable for one in four women to have to deal with an attempted sexual violence.” These things are going to come together, and I think it’s going to result in at least the consideration, at the highest levels, and potentially in public forums, of prediction and control, and controlling people from doing those things that once upon a time were tolerated. Now they’re not tolerated, but they’re still not being forcefully addressed. I think we’ll forcefully address them, and I think something akin to pre-crime may be part of that.
Yeah. That’s a disturbing prospect I think, especially for …
Is it a disturbing prospect? Is it?
Sure. I mean, if you counter-balance what you were saying with the idea that people have free will to decide, of course, whether they’re going to do one thing or …
People don’t have free will.
Yeah. This is a great topic, then.
As a high school senior I was first exposed to the debate between hard determinism, soft determinism, and free will. The idea is to what degree do we ultimately have agency. I’m a hard determinist. I believe that everything we do is a predictable synthesis of the nature and nurturer that came before it. Even the weird, goofy things that we do, if we had … I didn’t use the computer metaphor in 1991, but if we had a brain that was big enough and fast enough and understood all of the context properly, it could predict every single thing. I believe that. I don’t believe that there is this ineffable, unpredictable thing that just spews out of us. I think that we’re ultimately … You know, it’s a synthesis. It’s A plus B equals C and we go. For me it’s not troubling at all. For me the troubling thing is that one in four women going into higher education are going to be victims of at least an attempted sexual assault. That’s nuts. That’s loony toons, man.
I think there are everything from cultural to educational to even levels of transparency and information that could change those circumstances. If the statistic is one in four now, is that because it’s tolerated by certain cultures, even celebrated? Is it because there isn’t widespread knowledge? In other words, maybe the facts are known but they’re not well appreciated or they’re dismissed by people, et cetera, et cetera. It may be that once you put all of these elements together in a witches brew of data, you can predict what folks will do, whether it’s terrorism or sexual assault. I also think that there are possible mitigating factors that go ahead of the concept of pre-crime, which is these events are inevitable so we’ve got to head them off at the pass. Who knows what particular technique will be used to head it off at the pass.
To tie this back to our discussion of the new Chinese software endeavor that we started with, I think that they’re going to be testing this software tool out specifically in territories where ethnic minorities are opposed to Chinese rule. Whether it is in Xinjiang or Tibet. Right? There is the opposite example, you know, the counter example, to your one of women in college. Here is a minority in Tibet, say that is resisting Chinese rule. Under the authoritarian regime, that’s seen as terrorism. Right? You have a minority where the so-called pre-crime software will be used to identify whether people are going to commit terrorists acts. Who knows what arrests them. It’s unclear what the output will be. I think it’s a double edge sword and I appreciate what you laid out so far, but I think that it’s a multi-faceted sort of aspect there. I don’t know what your reaction is.
Yeah. I think these things are going to be addressed. The question is just how, right? I mean, I think that “pre-crime” is part of it as one of the potential paths, but at the end of the day it’s social engineering. Right? Again, we don’t know well enough what the cause is. These things are going to sound like crazy sci-fi, but I think decades from now we’re going to be considering things like are we controlling the hormones of men so their testosterone can never go above a certain level? There may be pills or injections or something that’s happening to control the chemicals within us because those chemicals control how we act and control the things that we do. I’d like to focus on a sexual violence because it’s so common. It’s so common and the people who are committing it aren’t people who you can see walking down the street who are frothing at the mouth and look like they’re crazy.
I mean, they’re people who we know. They’re people who seem like normal people, but in different contexts and different ways and for different reasons, they manifest horribly malignant and destructive behavior on other individuals again, and again, and again, and again, and again. I do believe that there are aspects, particularly in male violence, in sort of traditional, aggressive male things, that are going to reach the point that they are no longer considered socially acceptable, and that we will legislate. I guess it would probably go through a legislative process, but I think we’ll essentially legislate them, maybe not out of existence, but try and legislate around them to control how part of our species behaves. Whether that’s pre-crime, whether it’s hormone control, whether it’s something even more, if you could believe it, if it’s even more seemingly dystopic in nature, I think those things are going to happen.
The fact is, if you go above humanity and you look at it in some objective way and you say, “You know, for one of these two genders, this super high percentage is being abused by the other one in ways that can fundamentally shatter their sense of self and the rest of their lives,” that ain’t acceptable. It’s not acceptable. Up to now we haven’t been able to see it. We haven’t been able to understand it. We haven’t been able to control it, but science is moving so fast that we’re going to understand the human animal very well. We’re going to grok what it is that results in that stuff happening, the combination of physiology, psychology, sociology, the whole nine yards. We’re going to be able to control it. That control is going to look invasive and intrusive and like something out of a science fiction movie, but is it better to put those controls on to protect the half of the people who are being imposed upon and violated? To me it’s a no brainer. It’s just a question of what does that look like when the time comes.
Yeah. I don’t know. That’s the same style of thinking that ended up with Alan Turing getting his hormonal therapy.
No, it’s not. Absolutely not.
No. I mean, Alan Turing, no way. What was done to him was done because people feared who he was as a person that wasn’t hurting or impacting others in any way, shape or form. What we’re talking about here is that one half of our population is pre-disposed to troubling degrees to harm and impose themselves upon both their gender, but also the other, in ways that are stone age behaviors. They’re not consistent with our rational, libertarian understanding of the human animal, of the self, and of how we as a species need to all live together.
Yeah. I just see some parallels with the idea that there is an authority that knows best. Right? An authority that has …
We already have that. We already have that.
… the mindset with … During the end of World War Two there was plenty of scientific thinking and the culture of the day said that Alan Turing was a deviant and that he needed to be controlled and that he was sort of altering society by his behavior and was detrimental to society. I do see that the things we think we know today or the things that get baked into these pre-crime algorithms or what have you, they may not seem so enlightened looking back on it forty years later. I think it’s dangerous to provide these tools that can say, “Oh, well you’re going to end up doing this certain thing, so we’re going to prescribe whatever treatment it is,” whether it’s for terrorists, violent behavior, sexual misconduct … If you haven’t done it, I think that’s dangerous.
It’s already happening. Right? The society already allows for those things, they’re just physical things, they’re not science things. I mean, if I stand up and come across the table and punch you in the face, and if there’s a video camera that is capturing that act, and if you call the police and if the tape is there and we go to court, yada, yada, yada, I’m in jail or I’m paying a fine. There’s something that the society is imposing upon me because I did that. The society has already decided that certain degrees of behavior need to be controlled. If I am likely to do certain types of behavior, I will be imprisoned to make sure that I don’t do those types of behavior. My thesis is as we understand the human animal better, we’ll take a much more sophisticated look at it. It won’t just be, “Oh, we’ve already let this person burgle eight people, so now we’re going to put him in jail longer because he’s more of a risk.” We don’t need to wait until crime after crime after crime after crime has been perpetrated.
I’m not talking about something that is universal and heavy handed. I’m talking about just being smart about it. Is it okay that all these women are being assaulted? I say absolutely not. That’s the starting point when you say, “What can we do? What’s the gap? How can we close this gap and protect people? What does that look like?” Right now you and I are still pretty ignorant about what is this alchemy that comes together that leads to young men, and it’s disproportionately young men, raping and beating and killing and whatever the different things are, other people? That’s going to be figured out at some point in a very sophisticated way. Once we have that data, what do we do with it? I think we would be irresponsible and primitive not to do something with it.
I really am troubled by how women are treated in the culture in a holistic way, but particularly when it comes to sexual violence. It has such a big impact on their lives. I mean, there’s women who the rest of their lives, for some women, can’t have a healthy sexual relationship, can’t trust other people. This is freaking devastating and we’re just going to let it happen despite having knowledge about these are the people who are likely to do it? These are the things, the factors, that are potentially going to lead them to do it? I mean, look, I’m a man. I’ve never committed an act of sexual violence. I’ve never been disposed to commit an act of sexual violence. I have been in situations where I’m so sexually charged up that I don’t have a lot of control over how I behave. Is it possible in different contexts that I may have found myself in a situation that was in reality or perceived by the other person as sexual violence? I would say yes, it’s possible, despite the fact I’ve never done it and never had a compulsion to do it.
I’ve seen my body and my degree of self-control over myself in such ways that I can see the bridge there. I don’t want that. I don’t want to be put in that position. I don’t want these chemicals that are firing in my body to result in me really hurting somebody in ways that are horribly detrimental to them and are possibly going to impact the rest of my life in negative ways as well. For me, I would proactively say if you can tell me what are the things that could get me over the red line, and how can I be brought within myself so that I don’t go over that red line, I would sign up in a second. I want to be part of a harmonious whole with other people where we’re all being actualized and being able to be our full and best selves. I believe that the things that lead all of these men to commit these horrible acts are things that are chemical, are things that are happening that they don’t have control over in the way traditionally people say, “Oh, just control it.” I think it’s different than that, and I think we really need to look at it and think about it.
Listeners, remember that while you’re listening to the show you can follow along with the things that we’re mentioning here in real time. Just head over to thedigitalife.com, that’s just one L in thedigitalife, and go to the page for this episode. We’ve included links to pretty much everything mentioned by everybody, so it’s a rich information resource to take advantage of while you’re listening, or afterward if you’re trying to remember something that you liked. If you want to follow us outside of the show, you can follow me on Twitter @jonfollett, that’s J-O-N F-O-L-L-E-T-T. Of course, the whole show is brought to you by Involution Studios which you can check out at goinvo.com, that’s G-O-I-N-V-O dot com. Dirk?
You can follow me on Twitter @dknemeyer, that’s at D-K-N-E-M-E-Y-E-R, or email me firstname.lastname@example.org.
That’s it for episode 146 of The Digital Life. For Dirk Knemeyer, I’m Jon Follett, and we’ll see you next time.