BACKGROUND OF THE INVENTION

Most information systems today are comprised of combinations of computers and communications systems. The daily lives of people today are ever more dependent on these information systems. Problems may occur in such information systems, either naturally occurring or resulting from attacks. There are two specific aspects of dealing with these problems that have a major impact on reliability, availability, robustness, affordability, and security of these information systems. These two aspects are commonly referred to as false positive filtering and root cause analysis.

False positive filtering and root cause analysis have been performed for quite some time. However, systems have greatly accelerated the increase in scale, complexity and volatility, which makes it difficult to perform false positive filtering and root cause analysis. This is true of a full range of information systems. This is also true at any level of granularity in information systems from inside semiconductors, inside memory systems, inside communications systems, inside computers, inside networks, inside record systems, inside Clouds, etc. All consist of layers of legacy subsystems, incompatible subsystems, vendor proprietary solutions, etc. These often-incompatible subsystems in layer upon layer are one cause of the complexity. The speed of adding new layers and making changes is what generates the volatility. The “digitization” of the world is what is creating the scale problem.

These problems may be illustrated through a couple of examples. In a first example, the Internet archive is taking snapshots of the entire Web and storing them in a single memory system. This is creating file sizes not seen outside of classified systems. These file sizes were never anticipated by the people who wrote the drivers for the memory devices holding these files. One of the results is that the Internet Archive is discovering error conditions caused by these drivers that have not been seen before.

In a second example, until the early 2000s, Intel Corporation had a policy that only three people in the corporation were allowed to know the full architecture of a processor chip. This was done to protect the corporation's intellectual property. Then, the scale, complexity and volatility became so great that it was impossible for anybody to understand the full architecture of a processor chip and the company discontinued all the controls it had previously enforced.

The scale, complexity, and volatility problems described above are also true of many other types of systems including: electrical grids, multimodal transportation systems, mechanical systems, and even manual processes and procedures. For example, electrical grids were designed to have a very few very large sources of power. Now, they are faced with a large and growing number of small sustainable sources. At the same time, climate change is forcing power companies to confront new operational requirements to prevent fires. This, while the world is trying to move from fossil fuels to electricity.

Airliners now have so many sensors that it is said that on a transatlantic flight, the cost of acquiring, communicating, storing, and processing all the data is greater than the cost of the fuel. Freight systems driven by globalization and on-line shopping are seeing a combination of dramatic increases in volume, from an increasing number and types of sources, with rapidly increasing demands for lowering costs while guaranteeing on time delivery.

The digitization of medical and health systems is producing ever increasing amounts of data. But, it is getting more and more difficult to move and combine it. The move to Smart Cities is confronting similar problems with incompatible data that needs to be combined.

Against this background, existing ways of doing false positive filtering and root cause analysis are struggling. These approaches can deal with scale, or complexity, or volatility but struggle to deal with the combination of all three. Generally speaking, well-trained staff can be very good at these tasks. They are particularly good at identifying dynamic problems and determining dynamic solutions to them. But, they can only handle a limited number in a unit of time. With today's scale the numbers are quite large while response time requirements make the units of time quite small. Thus, the staff is unable to identify and solve all of the dynamic problems within the unit of time. Furthermore, the complexity makes it from difficult to impossible to have staff trained in every aspect limiting the availability and functionality of staff. This limitation can lead to staff relying on hunches built on underlying prejudices, reliance on ‘playbooks’ that can lead to ‘fat finger failures’, and inaction based on fear of these two. So, manual systems can face potentially catastrophic outcomes.

On the other hand, today's centralized automated systems can handle large amounts of data. Unfortunately, the amounts of data being generated are so large that it can take hours, days, or longer to find a particular pattern in the sea of data. Artificial intelligence (AI) has helped with identifying the patterns to look for, but it can take weeks to train an AI system to identify new patterns and then it faces delays searching the sea of data. So, these systems face challenges in finding dynamic problems and responding with dynamic solutions.

These delays can be deadly. For example, on a flight from San Francisco to Amsterdam in 2019, an airplane lost cabin pressure and performed an emergency decent to 9,000 feet. The pilot flew around in circles for 1.5 hours talking to airline flight operations, and the airplane manufacturer's emergency services. Those discussions over that 1.5 hours resulted in a decision to fly at 9,000 feet altitude to Newark airport ˜2 hours away. Thirty minutes into that two hours, the oxygen masks deployed. This should not have happened at 9,000 feet. But the two operations staffs performing root cause analysis determined that a mechanical problem with the two air compressors was the root cause for the lost cabin pressure and a delayed reaction by the oxygen masks was a result of the mechanical problem. The plane made an emergency landing in Bangor, Me., which the pilot did not believe was necessary. After the plane landed, passengers were unable to get their checked luggage because the plane had lost hydraulics. Hydraulics are how the control surfaces on the plane are put into and maintained in the desired configuration. Without hydraulics, it is between extremely difficult and impossible to prevent a catastrophic crash.

The plane example is an example of a root cause analysis failure. The plane has two jet engines, each with a corresponding generator that delivers power to a centralized power management system. The air compressors and hydraulic systems are driven by electric motors. The hydraulics system, however, has a pressure tank to act as a buffer against large short duration demands for pressure. Changes in control surfaces, etc. draw on the pressure in the tank and the electrical pump replenishes the tank. It never occurred to airline flight operations nor manufacturer emergency support that there could be a problem in the electrical power management system. Their prejudices told them to look for mechanical trouble with the air compressors. Even when the oxygen masks deployed, they still looked for a mechanical fault in the air compressors and told themselves that the masks deploying was just a delayed reaction to the air pressure drop. In reality, what happened was that the electrical power management system was failing and as it failed it reduced electrical power to various parts of the plane. When the hydraulic pump stopped working, the pilot only had the pressure in the tank to fly and land the plane. By the time the plane was on the ground, all that pressure was gone. Another few minutes of flight time would have caused a catastrophic crash.

In this airplane example, time is an important parameter. The root cause analysis system must find the right answer in the right amount of time to keep the plane from crashing. The root cause analysis failure was a result of the fact that both airline flight operations staff and manufacturer emergency support staff acted based on their prejudices. The central site root cause analysis systems they were using were not able to find the true problem in time, because they had to search through the airplane's data lake and use the patterns their system had been trained to detect. The failure of the electrical power management system does not happen frequently enough that there was a predefined pattern that the AI system could use to match it against. Given the complexity of modern airplanes, and the combinations and permutations there are innumerable potential system failures that do not have patterns. The result was that the crew and passengers were in mortal danger because the combination of staff and AI tools could not find the root cause fast enough to save the plane. It was only luck that there was enough pressure in the hydraulic tank to land the plane.

Two recent lab tests on false positives and false negatives provide illustrative examples of other types of problems with scale, complexity, and volatility. Both false negatives and false positives are generally subsumed under the title of false positives. One lab test and audit of a particular organization's set of cyber security systems using extensive ex post facto analysis (that could not be completed in production for lack of time, human and computer resources, etc.) was performed. It found that those security systems had generated 10,000 alerts in one week of which all were considered false positives and on which no action was taken on any of the alerts. With the tools available to that lab, they found that there were 12 actual security events of which 7 were actionable.

In another lab test, all the input data that went to an industry leading cyber security product was collected along with all outputs of that system. Applying extensive ex post facto analysis, it was found that the security system had identified all the static attacks and applied all the static solutions. However, it was not able to find the dynamic attacks. That is, the type of attack that changes its mode of operation quickly so that there is no pattern that can be matched from one attack to another. The dynamic attacks that were found in the months of detailed analysis by cyber security and data analysis Ph.D.'s turned out to be 70% of the attacks.

In cyber security systems, time is also an important parameter. In 2019, the US Department of Defense published a report that said that if one small portion of an information system was successfully penetrated by a sophisticated attacker, the system had 19 minutes to find the attack and perform remediation or the entire information system would be compromised.

A common response to the problems described above has been the creation of specialized tools to solve each special type of problem. This can be called the 1,000 tools for the 1,000 problems approach. Examples in the cyber security space include special tools to protect against ransomware, or specialized tools to protect against certain types of email attacks, etc. Each specialized tool doing its own false positive filtering and root cause analysis. In the short term, the proliferation of these specialized tools may seem to provide benefits, but they actually increase the scale, complexity, and volatility problems.

Timeliness has been described above. But in addition to solving specific problems within specific tight time constraints, there is a bigger timeliness problem—keeping the whole system in sync. That is, not just taking the one action required to address the immediate action, but making sure that the whole system stays in sync by taking actions elsewhere necessary to maintain this synchronization.

This leads to the final and most comprehensive need. That is maintaining these large complex volatile systems that operate in a dynamic environment configured in such a fashion that they do the best possible job of meeting the overall objectives of the system as a whole. That is, the system does what it was intended to do in the way it was intended to do it and does not do what it was not intended to do. Root cause and false positive analysis systems in a wide range of types of applications are struggling with scale (e.g., number of components/subsystems and the volume of data), complexity (e.g., increasing number of incompatible layers and types of components/sub systems), and volatility (e.g., things changing rapidly making the use of patterns difficult (dynamic problems needing dynamic solutions) plus shrinking critical time frames.

