From: John Conover <john@email.johncon.com>
Subject: Re: NtropiX+NNet idea
Date: 4 Jun 2001 19:05:40 -0000
Yea, that is exactly what the -d5 option to tsinvest does. It picks
its stocks based on "forecastability", i.e., H > 0.5, (among other
things,) and maintains a "history" of the patterns of ups/downs, then
bases its wager on 2P - 1, which is optimal, where P = H, (and "bets"
avg/rms amount of the portfolio on it, if the -a flag is used.)
If there is persistence, then a short term pattern will emerge in the
ups and downs, by definition. That's what H > 0.5 means.
For example, suppose H = 0.6, (and if you can find that, buy it-ASND
was the only stock that sustained that for more than a calendar
quarter in the last decade,) and, further, suppose the stock moved
down yesterday. Then there is a 60% chance that it will move down
today, too. So, one waits until it makes an up movement, and buys
since there is a 60% that it will move up the following day, too. Then
sell on the first down movement, and then start the whole scenario
again, (note that it is not a cyclic or periodic phenomena-its
stochastic-a probabilistic scenario, so one has to optimize wagering.)
On a day-to-day basis, H, (for a "typical" stock,) runs about 53%-56%
on the American exchanges, about 62% for currency exchanges, and is a
market inefficiency, (specifically, that not all holders of a stock
act instantaneously, as per EMH theoreticals, on market information.)
So, if something happens in the marketplace that moves a stock's
price, some stock holders will react virtually instantaneously. Others
will react at the end of the day, others the next day, others at the
end of the week, and so on. The H for inter-day trading is quite high,
(that is what the day traders attempt to exploit.) By about 3 days,
the stock's price has accommodated the information-all share holders
have reacted to the information, (meaning that the Lyapunov exponent,
defines the horizon of visibility, or predictability, at about 1.5
days, or so; the reaction to information is cut in about half, every
day.)
If information was accommodated by the market instantaneously, then,
as you say, the market dynamics would be strict Brownian motion, (H =
0.5,) and the market would be perfectly efficient, and fair, (i.e., no
one could have an advantage-and investing would be a short term
zero-sum game, too.) H != 0.5 is a market inefficiency, and is
exploitable, (see http://www.johncon.com/ntropix/usage.html, TABLE I
for an example on real data where the -d5 option is used.) In 1965
Paul Samuelson, using the approximation that information is
accommodated by the market instantaneously, showed that market
dynamics would be Brownian. In 1989, Brian Arthur showed that it is
the only long term stable solution to a market place, (aggregate
arbitrage system,) and all inefficiencies will, eventually, be
arbitraged away, (i.e., H != 0.5 is not sustainable, in the long run.)
See, also:
http://www.johncon.com/john/correspondence/990828002022.22436.html
http://www.johncon.com/john/correspondence/991109213225.19168.html
which are graphs of the tsinvest internal data structures for the -d5
option, and the series at:
http://www.johncon.com/john/correspondence/990205113415.1038.html
http://www.johncon.com/john/correspondence/980807151309.11811.html
http://www.johncon.com/john/correspondence/980807152940.11914.html
http://www.johncon.com/john/correspondence/980807154817.12009.html
http://www.johncon.com/john/correspondence/980807164313.12188.html
(look at the date, and what happened in the last graph.)
And, there is a very subtle caveat. A deterministic system does not
have to be a predictable system, (and is not, by definition, in
complex systems.) Although it is intellectually satisfying to discover
the mechanics of a deterministic system, one still has to address the
problem of how to use such knowledge to optimize the wagering
function. But that does not require knowledge of the underlying
mechanics-it can be determined directly, (by measuring, specifically,
the entropy of the system.)
I mean, if one does discover the mechanics of a deterministic system,
and the predictability of the system is discovered to be, say, 60%,
then one would wager a fraction of (2 * 0.6) - 1 on that
predictability. But one can measure the predictability by measuring
the entropy, without any knowledge of the underlying mechanics.
John
BTW, the numbers we see today for the Hurst exponent, H, are less than
they were a quarter of a century ago. Daily H > 0.6 was not uncommon.
However, with the advent of modern computer technology and networks,
it has been decreasing toward 0.5, (as it theoretically should.) It
used to be much easier to exploit market inefficiency, (until the mid
80's,) when information rate increased in the aggregate.
Jeff Haferman writes:
>
> I've been brainstorming a bit... the idea is to choose a
> pool of stocks with the Hurst coefficient "far" from 0.5,
> and then use these as candidates for neural net training.
>
> Underlying this is the notion that for H = 0.5, we have
> Brownian motion, and the time series is not predictable.
> H > 0.5 or H < 0.5 implies that we may be able to forecast
> the time-series (and, going further, I suppose we could compute
> the Lyapunov exponent to see how fast the time series decays,
> eg to get an idea of how far out we might be able to forecast).
>
> Any caveats before I start to undertake this exercise?
>
--
John Conover, john@email.johncon.com, http://www.johncon.com/