This guy seems to think so, and his arguments seem pretty convincing to me, but I don't understand the financial system as well as I might.
yarchive.net/blog/computers/flash_crash.html
Gettys is namechecked in the piece.
Cheers,
-- jra
This guy seems to think so, and his arguments seem pretty convincing to me, but I don't understand the financial system as well as I might.
yarchive.net/blog/computers/flash_crash.html
Gettys is namechecked in the piece.
Cheers,
-- jra
Interesting Jay, thanks for forwarding that.
I'm not convinced, but I could be. Interesting hypothesis that, at
least for me, raises more questions that I'm not qualified to answer
either. Perhaps most fundamentally, what and where exactly is the
"queue" in the transmission system that Nanex refers to?
It would seem surprising that delays in general due to long queues
would not have been noticed before, since or would have caused other
more far reaching problems.
If there is serious debate about the cause of the problem, then it
may be necessary to invite an independent, neutral party to fully
investigate. One has to ask, wouldn't it be convenient for Nanex
if queueing delay from others into their systems were the cause?
John
bufferbloat is the boogieman... of late. I think that's foolish
I think this comment from jtk is really on point though! 'why only
then?' that sure seems convenient, eh?
Failures almost never have a single cause.
Transport networks are never perfect, i.e. delays, dropped packets, data
corruption, etc. They may be contributing factors, or a combination of
rare events. The hard question that SEC and the industry has been wrestling with is "Why?" not so much "How?"
The apparent condititions didn't change, but the system reacted differently during those seconds. Why? What was different?
The queuing like the RBC dudes were doing was in order transmission not
on the wire. given wires of various lengths having the request arrive on
different exchanges at different times based on distance was considered
unedesirable (by people loooking to reduce the opportunity for arbitrage
on latency).
I have have minimal experience with trading platforms but what switch
vendors were selling us as a latency sensitive customer (and HFT shops
at time) were broadcom or fulcrum asics which by virtue of being
cut-through are essentially minimally buffered.
Hi Jay,
My read is that the author got it upside down.
The intermediate cause of the problem was propagation delay (including
buffer bloat) which induced an oscillating set of states in the
trading software.
The root cause was a flipping jassack trying to out-time his
competitors by assuming a degree of instantaneity which proved untrue.
Don't do that. Don't make assumptions about network timing. You can
count on being wrong. If timing matters to your application, find a
way to continuously measure.
Regards,
Bill Herrin
P.S. Recruiters: No, I do NOT want to move to New York City and
engineer another half millisecond out of your network. I would,
however, welcome a law which bans both buying and selling instruments
of the same stock or commodity within 24 hours.
In 8/6/15 10:44 AM, William Herrin wrote:
The intermediate cause of the problem was propagation delay (including
buffer bloat) which induced an oscillating set of states in the
trading software.The root cause was a flipping jassack trying to out-time his
competitors by assuming a degree of instantaneity which proved untrue.
Don't do that. Don't make assumptions about network timing. You can
count on being wrong. If timing matters to your application, find a
way to continuously measure.
Similar things happen when folks decide they are going to twiddle the
knobs of NTP's behavior. NTP works locally, and gets/provides
information globally. More or less. When folks decide to make a change
in its core behavior, the usually don't consider how those changes will
affect anybody else.
I know enough about this to know I don't know anywhere near enough about
it, so I leave the knobs alone.
Various technical issues may have made things worse, but the central cause of the flash crash was due to lack of regulator and/or procedures for the now distributed markets (exchanges, ecns, dark-pools, etc...) on a market imbalance.
What started the whole thing was a selloff of a large quantity of e-mini futures, which caused a broad run on the market. This is a feature, not a bug. The thundering herd responded, as normal, by starting their own sell-off
Before the decentralized markets, when there was a market imbalance (all buyers or all sellers), the "specialist" or "market maker" would halt trading in a symbol, and work with the buyers/sellers to determine a new price and re-open with the new price. The problem is without coordination, when the NYSE/Nasdaq market makers halted trading, the distributed exchanges weren't halted. Since the market makers in those exchanges are required to always have quotes, they put out "stub" quotes of 0.01/$10000.00. Since there weren't valid quotes on the regular exchanges and because of RegNMS, those stub quotes got disseminated as BBO. This was a cosmetic issue and didn't effect trading.
Who really got screwed were people that had a stop order on their stocks and didn't realize there were no guarantees of trading through that price. For example, if you bought a stock at $100, and put a stop order to sell at $90, and there was a market imbalance, the price could trade discontinuously. For example the last valid quote could have been at $95.90, then halted, then re-opened at $82.50. The stop order would sell immediately at $82.50, not the $90 people thought. Then the stock could recover and be trading at $95.05 and you could really feel you were screwed. But that's how it is supposed to work.