From: John Conover <john@email.johncon.com>
Subject: Re: NtropiX+NNet idea
Date: 5 Jun 2001 08:57:21 -0000
TABLE I of http://www.johncon.com/ntropix/usage.html is kind of
interesting, BTW.
If you look at the "-d5 -m0 -p -P" (row 2, colum 5,) characteristics,
the portfolio gain was 1.00329 per day, or 1.00329^253 = 2.2956X per
calendar year, for 253 trading days per year.
The best stocks today have Hurst exponents, H, (i.e., daily
persistence,) of about 0.54, when averaged over a year, or so. Based
on that, the tsinvest program should have wagered 2P - 1 = 8% per day,
(on average,) and it should have optimized the gain to be 0.08^2 =
0.0064, for a gain per day, G, of:
G = (1.08)^0.54 * (1.08)^(1 - 0.54) = 1.00320855854
which means the theoretical and emperical values are very close,
(about 1 part in a 10,000,) over the 2.5 year period, (it stumbled
into ASND, which ran about H = 0.55 for that time interval-but it had
to work its way through the '93 recession, too.)
Note that I do not suggest using tsinvest this way.
Those arguments are very dangerous and risky, (the -m0 allows it to
drop hedging the portfolio against individual stocks in the portfolio
under certain conditions-allowing it to invest everything in only one
stock.) Its probably more risk than any sane person would want to
take. (Its sole function is verification of pro forma against theory;
use the -r to dump the internal data structures, and that will print
the numbers its using, which can be compared against theory, and other
methodologies, like proposed by Hurst, etc.)
Its probably not very practical, either, since TABLE I does not
include broker's transaction costs, nor does it include broker
inefficiency, (which is an issue in day-trading.)
But its nice to know the theoreticals and empiricals are in close
agreement when exploiting persistence.
John
BTW, I don't use the -d5 -m0 options together. I don't encourage
anyone else do it, either. However, if one must, then by all means use
-c option to force the program to clamp down on its data set size
requirements, (which forces the program to compensate its wagering
functions for fractal run length dynamics, in addition to standard
statistical estimates.)
John Conover writes:
> Yea, that is exactly what the -d5 option to tsinvest does. It picks
> its stocks based on "forecastability", i.e., H > 0.5, (among other
> things,) and maintains a "history" of the patterns of ups/downs, then
> bases its wager on 2P - 1, which is optimal, where P = H, (and "bets"
> avg/rms amount of the portfolio on it, if the -a flag is used.)
>
> If there is persistence, then a short term pattern will emerge in the
> ups and downs, by definition. That's what H > 0.5 means.
>
> For example, suppose H = 0.6, (and if you can find that, buy it-ASND
> was the only stock that sustained that for more than a calendar
> quarter in the last decade,) and, further, suppose the stock moved
> down yesterday. Then there is a 60% chance that it will move down
> today, too. So, one waits until it makes an up movement, and buys
> since there is a 60% that it will move up the following day, too. Then
> sell on the first down movement, and then start the whole scenario
> again, (note that it is not a cyclic or periodic phenomena-its
> stochastic-a probabilistic scenario, so one has to optimize wagering.)
>
> On a day-to-day basis, H, (for a "typical" stock,) runs about 53%-56%
> on the American exchanges, about 62% for currency exchanges, and is a
> market inefficiency, (specifically, that not all holders of a stock
> act instantaneously, as per EMH theoreticals, on market information.)
> So, if something happens in the marketplace that moves a stock's
> price, some stock holders will react virtually instantaneously. Others
> will react at the end of the day, others the next day, others at the
> end of the week, and so on. The H for inter-day trading is quite high,
> (that is what the day traders attempt to exploit.) By about 3 days,
> the stock's price has accommodated the information-all share holders
> have reacted to the information, (meaning that the Lyapunov exponent,
> defines the horizon of visibility, or predictability, at about 1.5
> days, or so; the reaction to information is cut in about half, every
> day.)
>
> If information was accommodated by the market instantaneously, then,
> as you say, the market dynamics would be strict Brownian motion, (H =
> 0.5,) and the market would be perfectly efficient, and fair, (i.e., no
> one could have an advantage-and investing would be a short term
> zero-sum game, too.) H != 0.5 is a market inefficiency, and is
> exploitable, (see http://www.johncon.com/ntropix/usage.html, TABLE I
> for an example on real data where the -d5 option is used.) In 1965
> Paul Samuelson, using the approximation that information is
> accommodated by the market instantaneously, showed that market
> dynamics would be Brownian. In 1989, Brian Arthur showed that it is
> the only long term stable solution to a market place, (aggregate
> arbitrage system,) and all inefficiencies will, eventually, be
> arbitraged away, (i.e., H != 0.5 is not sustainable, in the long run.)
>
> See, also:
>
> http://www.johncon.com/john/correspondence/990828002022.22436.html
> http://www.johncon.com/john/correspondence/991109213225.19168.html
>
> which are graphs of the tsinvest internal data structures for the -d5
> option, and the series at:
>
> http://www.johncon.com/john/correspondence/990205113415.1038.html
> http://www.johncon.com/john/correspondence/980807151309.11811.html
> http://www.johncon.com/john/correspondence/980807152940.11914.html
> http://www.johncon.com/john/correspondence/980807154817.12009.html
> http://www.johncon.com/john/correspondence/980807164313.12188.html
>
> (look at the date, and what happened in the last graph.)
>
> And, there is a very subtle caveat. A deterministic system does not
> have to be a predictable system, (and is not, by definition, in
> complex systems.) Although it is intellectually satisfying to discover
> the mechanics of a deterministic system, one still has to address the
> problem of how to use such knowledge to optimize the wagering
> function. But that does not require knowledge of the underlying
> mechanics-it can be determined directly, (by measuring, specifically,
> the entropy of the system.)
>
> I mean, if one does discover the mechanics of a deterministic system,
> and the predictability of the system is discovered to be, say, 60%,
> then one would wager a fraction of (2 * 0.6) - 1 on that
> predictability. But one can measure the predictability by measuring
> the entropy, without any knowledge of the underlying mechanics.
>
> John
>
> BTW, the numbers we see today for the Hurst exponent, H, are less than
> they were a quarter of a century ago. Daily H > 0.6 was not uncommon.
> However, with the advent of modern computer technology and networks,
> it has been decreasing toward 0.5, (as it theoretically should.) It
> used to be much easier to exploit market inefficiency, (until the mid
> 80's,) when information rate increased in the aggregate.
>
> Jeff Haferman writes:
> >
> > I've been brainstorming a bit... the idea is to choose a
> > pool of stocks with the Hurst coefficient "far" from 0.5,
> > and then use these as candidates for neural net training.
> >
> > Underlying this is the notion that for H = 0.5, we have
> > Brownian motion, and the time series is not predictable.
> > H > 0.5 or H < 0.5 implies that we may be able to forecast
> > the time-series (and, going further, I suppose we could compute
> > the Lyapunov exponent to see how fast the time series decays,
> > eg to get an idea of how far out we might be able to forecast).
> >
> > Any caveats before I start to undertake this exercise?
> >
--
John Conover, john@email.johncon.com, http://www.johncon.com/