There are no computer systems that are “too important to fail.” Failure, as any competent engineer will tell you, is always an option. Yet modern societies increasingly depend on systems to be foolproof. Electrical grids, air traffic control, automobile control, and medical equipment are all life-critical systems, and none of us wants to depend on life-critical systems with a high failure rate. Nobody wants to trust a large portion of his life savings to a financial trading system that is subject to unpredictable failure either. The same is true of the Internet itself. What we need to do is take a step back and study the design, architecture, and feedback control of these systems. Without such a discussion, we will experience more of these trillion-dollar flash crashes.
It is clear that interfacing in native mode (i.e., IP) over the Internet provides great flexibility for any advanced computer-based system, whether that be a smart appliance hooked up to the Smart Grid, a location device on a connected automobile, or an HFT system around the world. But it is clear that today’s Internet is increasingly vulnerable to security attacks and anything attached to it will therefore increase its vulnerability.
Then there is the question of privacy. It’s becoming clear that almost anything anyone puts on the Internet can (and will) be found and integrated with all the other data that exists about people, what they do, where they are, what they read, what they have said, and so on.
What Should We Do?
When considering the attributes of today’s increasingly complex, large-scale systems, it’s time to make risk management a priority using these three recommendations:
- Push for a public group equivalent to the National Transportation Safety Board (NTSB).
- Push for uniform privacy and confidentiality rules worldwide.
- Require new initiatives to produce something equivalent to an “environmental impact statement” regarding the impact of system failures and breaches of privacy, confidentially, and security.