"Tell this little sh*t to NEVER talk to me again!" I wasn't sure why the head trader of Morgan Stanley's Nasdaq desk was so mad with my suggestion that he honor his quotes, but when he said "We all KNOW that when you program trading FU?KERs start a program that all the prices are going to move" it became clearer…

It was 1985 and I was a 22 year old programmer in Morgan Stanley's IT department. For the first 18 months of my career I had worked only on back office systems, toiling in a quiet cubicle with an occasional meeting or phone call.

On a bright summer day, all of that changed. Tom (my boss) told me we were going to meet with "the traders", so we walked over to 1251 6th Ave, where the front office departments all worked. I still remember the first impression of chaos, action, and noise that hit me as we entered the trading floor. Yelling, sometimes cursing, sales traders were giving orders to traders and time stamping tickets while the traders yelled right back at them with fills or expletives about how they were getting screwed.

We were led to a conference room where Sam, a derivative trader who would soon become the head of index arbitrage, described their problem:

"We have this new advanced analytic system that we can use to make millions. It tells us within seconds when the pricing for stock index futures versus the underlying stock is wrong. We can then buy or sell the futures and do the opposite in the stocks before anyone else can react. Our problem is that we have trouble getting the orders placed and it is taking us 30 minutes or so to know what we have actually traded. In that time, we can't do more trades and often miss chances to either add to or close out our positions."

By the end of the meeting, Tom and I proposed a simple solution - we would build a system with pre- printed packets of tickets for the floor traders to execute (95% of the trades were on the NYSE) and build a screen on the floor for them to enter fills into a special basket trade entry system (with all the preset information loaded). The goal was to improve both speed and accuracy, and a few weeks later we put it into production. It was a major success, the entire process was reduced to under 5 minutes with over 99% accuracy. Millions of dollars of profits was the result, so we kept improving the system.

The first big leap forward was connecting it to "SuperDOT" a PC based innovation that the NYSE built to automate small orders. (Note - even though it was automated, for the next 20 years the specialist still had to "ok" each trade by hand) Since even relatively large baskets consisted of small individual orders, this system was used for the vast majority of stocks in the main index, which was the S&P 500. Since these orders paused at the specialists "booth", when programs started to get delivered to the floor, word spread quickly. It became floor lingo that "a wave" or "a program" was being executed, which, of course, triggered other traders on the floor to try and trade ahead of orders later in the program, or trade alongside orders being executed and also cancel their own orders on the opposite side.

At the same time, I remember asking the Nasdaq traders at Morgan Stanley if we could connect the basket system to either SOES (the Nasdaq small order entry system) or just automate the tickets and execution at THEIR own best bid or offer. I will never forget the expletive laced tirade that followed... (The head trader started by yelling at Sam - "Tell this little sh*t to NEVER talk to me again. We all KNOW that once you FU? KERS start a program that the prices are going to move. Why the H--L would we let YOU pick us off? You already act like you own this floor and ..." (It was a lot longer, but that was the gist...)

Over the next 5 plus years, this system became both more automated and global. Apart from a small setback after the 87 crash, where program trading was initially blamed, the business grew but gradually became much more competitive. Eventually, speed was no longer enough to trade profitably and firms that combined index arbitrage with more statistical techniques dominated the business. Of course, by that time, the spreads between futures and stocks and later ETFs and stocks had shrunk massively.

So, what can we learn from this anecdote:

First - HFT is NOT new- Speed advantages have been exploited before with the same result. The ability to generate profits from those advantages always decrease as competition increases, with the result of lower spreads between the securities being arbitraged.

Second - Traders will ALWAYS exploit legal information advantages (such as proximity to information like "a wave is being executed" or superior technology "our system tells us BEFORE others")

Third - market makers ALWAYS want to avoid trading with informed flow (and remember back then spreads were more than 10 TIMES the spreads of today)

Fourth - advantages in human dominated markets are more structural and long lasting. The floor traders and specialists that saw the "waves" kept that edge for over 20 years in one form or another...

Fifth - despite all of the advantages the index arbitrageurs possessed, those practices produced significant social good. The ability to build and trade ETFs (exchange traded funds) was a direct outcome of program trading efficiencies. Also index arbitrage has led directly to much lower implementation costs for investors that use ETFs to allocate assets. Those products are useable by investors precisely because their prices are kept in line with their real value by arbitrageurs.

All in all, there are many lessons, but for the purpose of this post, the key point is that all of us should be very careful when evaluating market structure commentary and proposals. The "good old days" were not necessarily that good and, despite the fear of the unknown, technology and competition generally lead to progress.

S&P Global provides industry-leading data, software and technology platforms and managed services to tackle some of the most difficult challenges in financial markets. We help our customers better understand complicated markets, reduce risk, operate more efficiently and comply with financial regulation.

This article was published by S&P Global Market Intelligence and not by S&P Global Ratings, which is a separately managed division of S&P Global.