Magic’s just science that we don’t understand yet.
― Arthur C. Clarke
Scientific discovery and wonder can often be viewed as magic. Some things we can do with our knowledge of the universe can seem magical until you understand them. We commonly use technology that would seem magical to people only a few decades ago. Our ability to discover, innovate and build upon or knowledge creates opportunity for better, happier and longer healthy lives for humanity. In many ways technology is the most human of endeavors and sets us apart from the animal kingdom through its ability to harness, control and shape our World to our benefit. Scientific knowledge and discovery is the foundation of all technology, and from this foundation we can produce magical results. I’m increasingly aware of our tendency to shy away from doing the very work that yields magic.
The world is full of magic things, patiently waiting for our senses to grow sharper.
― W.B. Yeats
Today I’ll talk about a couple things: the magical power of models, methods, and algorithms, and what it takes to create the magic.
What do I mean by magic with abstract things like models, methods and algorithms in the first place? As I mentioned these things are all basically ideas and these ideas take shape through mathematics and power through computational simulation. Ultimately the combination of mathematical structure and computer code the ideas can produce almost magical capabilities in understanding and explaining the World around us allowing us to tame reality in new innovative ways. One little correction is immediately in order; models themselves can be useful without computers. Simple models can be solved via analytical means and these solutions provided classical physics with many breakthroughs in the era before computers. Computers offered the ability expand the scope of these solutions to far more difficult and general models of reality.
This then takes us to the magic from methods and algorithms, which are similar, but differing in character. The method is the means of taking a model and solving it. The method enables a model to be solved, the nature of that solution, and the basic efficiency of the solution. Ultimately the methods power what is possible to achieve with computers. All our modeling and simulation codes depend upon these methods for their core abilities
. Without the innovate methods to solve models, the computers would be far less powerful for science. Many great methods have been devised over the past few decades, and advances with methods open the door to new models or simply greater accuracy, or efficiency in their solution. Some methods are magical in their ability to open new models to solution and with those new perspectives on our reality.
Any sufficiently advanced technology is indistinguishable from magic.
― Arthur C. Clarke
Despite their centrality and essential nature in scientific computing, emphasis and focus on method creation is waning badly. Research into new or better methods has little priority today and the simple transfer (or porting) of existing methods onto new computers is the preferred choice. The blunt truth is that porting a method onto a new computer will produce progress, but no magic. The magic of methods can be more than simply enabling; the best methods bridge a divide between modeling and methods by containing elements of physical modeling. The key example of this character is shock capturing. Shock capturing magically created the ability to solve discontinuous problems in a general way, and paved the way for many if not most of our general application codes.
The magic isn’t limited to just making solutions possible, the means of making the solution possible also added important physical modeling to the equations. The core methodology used for shock capturing is the addition of subgrid dissipative physics (i.e., artificial viscosity). The foundation of shock capturing led directly to large eddy simulation and the ability to simulate turbulence. Improved shock capturing developed in the 1970’s and 1980’s created implicit large eddy simulation. To many this seemed completely magical; the modeling simply came for free. In reality this magic was predictable. The basic method of shock capturing was the same as the basic subgrid modeling in LES. Finding out that improved shock capturing gives automatic LES modeling is actually quite logical. In essence the connection is due to the model leaving key physics out of the equations. Nature doesn’t allow this to go unpunished.
One of the aspects of modern science is that it provides a proverbial two-edged sword by understanding the magic. In the understanding we lose the magic, but open the door for new more miraculous capabilities. For implicit LES we have begun to unveil the secrets of its seemingly magical success. The core of the success is simply the same as original shock capturing, producing viable solutions on finite grids occurs via getting physically relevant solutions, which by definition means a dissipative (vanishing viscosity) solution. The new improved shock capturing methods extended the basic ability to solve problems. If one were cognizant of the connection between LES and shock capturing, the magic of implicit LES should be been foreseen.
The real key is the movement to physical admissible second-order accurate methods. Before the advent of modern shock capturing methods guarantees of physical admissibility were limited to first order accuracy. The first-order accuracy brings with it large numerical errors that look just like physical viscosity, which renders all solutions effectively laminar in character. This intrinsic laminar character disappears with second-order accuracy. The trick is that the classical second-order results are oscillatory and prone to being unphysical. Modern shock capturing methods solve this issue and make solutions realizable. It turns out that the fundamental and leading truncation error in a second-order finite volume method produces the same form of dissipation as many models produce in the limit of vanishing viscosity. In other words, the second order solutions match the asymptotic structure of the solutions to the inviscid equations in a deep manner. This structural matching is the basis of the seemingly magic ability of second-order methods to produce convincingly turbulent calculations.
This magic is the tip of the iceberg, and science is about understanding the magic as a route to even greater wizardry. One of the great tragedies of the modern age is the disconnect between these magical results and what we are allowed to do.
We can also get magical results from algorithms. The algorithms are important mathematical tools that enable methods to work. In some cases algorithmic limitations produce significantly limiting efficiency for numerical methods. One of the clearest areas of algorithmic magic is numerical linear algebra. Breakthroughs in numerical linear algebra have produced immense and enabling capabilities for methods. If the linear algebra is inefficient it can limit the capacity for solving problems. Conversely a breakthrough in linear algebra scaling (like multigrid) can allow solutions with a speed, magnitude and efficiency that seems positively magical in nature.
Numerous algorithms have been developed that endow codes with seemingly magical abilities. A recent breakthrough where magical power is ascribable to is compressed sensing. This methodology has seeded a number of related algorithmic capabilities that defy normal rules. The biggest element of compressed sensing is its appetite for sparsity, and sparsity drives good scaling properties. We see magical ability to recover clear images from noisy signals. The key to all of this capability is the marriage of deep mathematical theory to applied mathematical practice, and algorithmic implementation. We should want as much of this sort of magical capabilities as possible. They do seemingly impossible things providing new unforeseen abilities.
In the republic of mediocrity, genius is dangerous.
― Robert G. Ingersoll
We don’t do much of this these days. Model, method and algorithm advancement is difficult and risky. Unfortunately our modern management programs don’t do difficult things well anymore. We do risky things even less. A risky failure prone research program is likely to not be funded. Our management is incapable of taking risks, and progress in all of these areas is very risky. We must be able to absorb many failures in attempting to achieve breakthroughs. Without accepting and managing through these failures, the breakthroughs will not occur. If the breakthroughs occur massive benefits will arise, but these benefit while doubtless are hard to estimate. We are living in the lunacy of the scheduled breakthrough. Our inability to seek success without the possibility of failure is nothing, but unbridled bullshit and the recipe for systematic failure.
There is always danger for those who are afraid.
― George Bernard Shaw
The truly unfortunate aspect of today’s world is the systematic lack of trust in people, expertise, institutions and facts in general. These trustworthiness crises are getting worse, not better, and may be approaching a critical fracture. The end result of the lack of trust is a lack of effective execution of work because people’s hands are tied. The level of control placed on how work is executed is incompatible with serendipitous breakthroughs and adaption of complex efforts. Instead we tend to have highly controlled and scripted work lacking any innovation and discovery. In other words the control and lack of trust conspire to remove magic as a potential result. Over the years this leads to a lessening of the wonderful things we can accomplish.
If we expect to continue discovering wonderful things we need to change how we manage our programs. We need to start trusting people, expertise, and institutions again. Trust is a wonderful thing. Trust is an empowering thing. Trust drives greater efficiency and allows people to learn and adapt. If we trust people they will discover serendipitous results. Most discoveries are not completely new ideas. A much more common occurrence is for old mature ideas to combine into entirely new ideas. This is a common source of magical and new capabilities. Currently the controls placed on work driven by lack of trust remove most of the potential for a marriage of new ideas. The new ideas simply never meet and never have a chance to become something new and amazing. We need to give trust and relinquish some control if we want great things to happen.
The problem with releasing control and giving trust is the acceptance of risk. Anything new, wonderful, even magical will also entail great risk of failure. If one desires the magic, one must also accept the possibility of failure. The two things are intrinsically linked and utterly dependent. Without risks the reward will not materialize. The ability to take large risks, highly prone to failure is necessary to expose discoveries. The magic is out there waiting to be uncovered by those with the courage to take the risks.