However, there are two problems that make the future of high frequency trading of unique global concern, irrespective of how popular it becomes.

The first is a scientific one: Financial markets represent the largest-ever sociotechnical system in existence, with a mix of state-of-the-art communications and computational power operating at speeds approaching the natural speed limit of light. Yet nobody, including Einstein, has ever produced a theory that predicts what might go wrong in an ultrafast global network of interconnected machines that carry out millions of operations in the blink of an eye -- or what can be done to prevent or manage it.

This leads to the second problem. How can regulators and governments possibly decide how to manage this emerging ultrafast financial jungle if nobody yet fully understands it?

My fellow researchers and I recently uncovered glimpses of what is already going wrong in the form of escalating patterns of "sub-second tsunamis." These tsunamis are huge spikes and dips in the price of an individual stock. Although the Flash Crash was fast, lasting only a few minutes, these sub-second tsunamis are over in the blink of an eye -- and there are thousands of them. A 10% daily change in a major stock would guarantee breaking news coverage, but these tsunamis typically send the price plummeting to almost zero. However they go unnoticed since the price quickly recovers as other algorithms jump in for the kill.

Their existence reveals a remarkable difference between the human trading world above the typical human response time of one second, and the all-machine ecology of trading algorithms below one second.

Just like cracks propagating in a structure prior to mechanical failure, these sub-second tsunamis escalated in the lead up to the 2008 financial meltdown. Most importantly, the stock showing highest proliferation are the banks that are now associated with the crisis. Yet nobody knew at the time. Indeed our research predicts a growing zoo of such tsunamis in the future, with each species having its own characteristic twist and turns.

Governments need the financial equivalent of an air traffic control system in order to know how to manage this brave new world, and hence what rules (if any) to impose. This in turn will require a joint research program between trading houses, regulators and academics.

However, instead of using conventional economics, the methodological approach should be built around complex adaptive systems and dynamical networks. Generative market models must be tested in real-time against high resolution data, to see if they can reproduce the observed price exchange dynamics down to the sub-second scale.

Our own research predicts that these sub-second market movements will be neither completely unpredictable nor predictable, but will instead have pockets of predictability that come and go in particular ways. Estimates of the market share of different trading algorithms will enable real-time system management, while ensuring that the secrecy of individual trading entities remains intact.

Without such a financial Manhattan Project, regulatory bodies will effectively be flying blind and may end up doing more harm than good.

David Fuller's view

Regulatory bodies will never be able to catch up with the latest in high frequency trading (HFT) because they are not developing it.  Also, I do not think a ban on HFT would be very effective.  Aside from having a Luddite quality, bans would be easily circumvented because HFT is already internationally pervasive and cuts across many markets, not just stocks and not just in the USA. 

However, HFT should be discouraged because it is highly predatory, often illegal and usually unethical.  News events that robots can profit from are infrequent relative the amount of HFT that now occurs.  HFT firms are often front running and they also engage in a form of sonar fishing, by triggering stops, affecting sentiment and shaking out other traders and investors with lightening (now-you-see-it-now-you-don’t) raids.   

Market makers and exchanges could negate a portion of HFT’s advantage by delaying order execution times by at least several seconds.  However, we are a long way from that in terms of global markets, although the commendable Brad Katsuyama of IEX has made a start.