Software For Algorithmic Trading Of Equities:

While most prevailing equity market investment or trading methodologies rely on static analysis-for example, the concept of P/E ratios-entropic analysis provides solutions that concurrently maximizes portfolio value growth, while minimizing risk exposure. The analysis uses stochastic models of equity prices that exploit market dynamics. The concepts are useful for programmed trading-ie., day trading-and/or exploiting market "bubbles".

This is the home page for the tsinvest program. The tsinvest.tar.gz tape archive contains the C sources for three programs used for entropic analysis of equity prices. The archive consists of the programs, tsinvest, tsinvestsim, and tsshannoneffective. The programs use non-linear extensions to the random walk fractal model of equity prices. The extensions are similar to the discreet time "logistic" (parabolic) function. There is a mathematical and numerical methods development, for those so inclined, and a derivation of the model is contained in the tsinvest documentation. The programs are useful for quantitative financial analysis of equities and portfolio optimization. The tsinvest program scans all stocks in a market deciding which should be invested in at any time by calculating the fractal statistics of all equities. The program uses statistical estimation techniques to estimate the accuracy of the fractal statistics. A 2.5 year "fragment" of the US exchange's daily "ticker" from 1993 to 1996, consisting of 454 stocks, is included as a demonstration on the Usage page. Different decision criteria are available as command line options which alter the statistical methodology and portfolio optimization. The paradigm of the non-linear model is validated on the quality assurance, (QA) page.

Additionally, two other utility programs are distributed with the tsinvest sources. The program csv2tsinvest translates the Yahoo! historical stock price database spreadsheet format, csv, (available from http://chart.yahoo.com/d, which is an online historical database of the daily closes of all stocks in the US equity markets,) to the tsinvest time series database format. The tsinvest database format is a standard Unix sequential, tab delimited, file format for extensibility and flexibility. The program tsinvestdb is a template for expediting the development of programs that manipulate the tsinvest time series database(s), such as data blades.

There is additional information, regarding the concepts of financial economics, in the FAQs, which also contains a suitable bibliography, and a Links page to related sites for a general survey of the subject. Collateral programs can be found on the Utilities page.

If very large time series databases of financial data are to be manipulated, there is a suite of command line utilities for fast binary searching sorted flat file constant databases. Partial key searches and files with tab delimited fields are supported, (as used in the tsinvest program suite,) in a multiuser environment, and ISAM, (Indexed Sequential Access Method,) databases can be constructed across a network-a useful topology for constructing "data blade" mechanisms. The ndex programs have their own site at NdeX. The NdeX archive is at http://www.johncon.com/ndex/archive/.

Historical Perspective

The theoretical foundations of the entropic methods used in modern finance were formalized by the mathematicians Jakob Bernoulli, (Ars Conjectandi, 1713,) and Abraham de Moivre a few years later, (The Doctrine of Chances.) The dynamics of simple fractal systems, (for example, Brownian motion and the fixed increment fractals,) appear in the formalization of repeated trial convergence, (sample average,) in the Law of Large Numbers, and the formal development of the normal, (or Gaussian,) distribution/bell curve. Specific applications to fractal dynamics were further developed by A. A. Markov (1903-1922,) and Norbert Wiener, (1874-1964.)

The concept of entropic analysis of equity prices is not new. It was first proposed by Louis Bachelier (1870-1946) and published in his 1900 doctoral thesis. His "theory of speculation," (Theorie de la Speculation,) was discounted by none other than Henri Poincare, observing that "M. Bachelier has evidenced an original and precise mind [but] the subject is somewhat remote from those our other candidates are in the habit of treating." Nevertheless, the thesis anticipated many of the mathematical discoveries made later by Wiener and Markov, and outlined the importance of such ideas in today's financial markets, stating that "it is evident that the present theory solves the majority of problems in the study of speculation by the calculus of probability."

J. L. Kelly, Jr. established the isomorphism between the information-theoretic concept of information rate in a binary symmetric channel and speculation under uncertainty. The concept was presented in the Bell System Technical Journal, (A New Interpretation of Information Rate,) in 1956. The importance of Kelly's contribution can not be underestimated since it made the large mathematical infrastructure of information theory, developed by Claude Shannon in the mid 1940's, (The Mathematical Theory of Communication,) applicable to the analysis and optimization of speculative endeavors.

In 1963, Benoit Mandelbrot, (The Variation of Certain Speculative Prices,) citing Wesley Claire Mitchell's work of 1915, (Wieser's Theory of Social Economics,) formulated a theory of price fluctuations in speculative markets based on the probability distributions discovered by the French mathematician Paul Levy. As pointed out by Mandelbrot, the so-called log-normal distribution is of interest in finance since wealth in a multi-agent system evolves into a log-normal distribution. The Gaussian/Normal distribution is a special case of the more general Levy distributions, and is often used as an approximation to log-normal distributions for mathematical expediency.

In 1965, Paul Samuelson formally proved Bachelier's empirical theories on the characteristics of speculative markets, (Proof that Properly Anticipated Prices Fluctuate Randomly.) In the early 1970's, Fisher Black, Myron S. Scholes, and Robert C. Merton, extended the theory into a methodology for virtually zero risk option and derivative pricing, and established the isomorphism between the standard deviation of the fluctuations in price of a financial instrument, and investment risk.

By the mid 1990's, largely through the efforts of W. Brian Arthur of the Santa Fe Institute, bounded rationality and self-referential indeterminism were generally accepted as the cause of the stochastic characteristics of speculative markets. Today, stochastic analysis is a firmly entrenched branch of information theory, and economic applications of entropic principles are found in such disciplines as finance, strategic marketing, and the optimization of business operations and process-see the NdustriX site for particulars.

For a more detailed historical perspective, "Choice under Risk and Uncertainty" is highly recommended as a general survey.

Mathematical Analysis & Numerical Methods

For a brief mathematical development and description of the associated numerical methods:

• Quantitative Analysis of Non-Linear High Entropy Economic Systems I contains a derivation of the non-linear high entropy model and its optimization. As empirical examples, GE's equity price is analyzed, the equity price growth of the component companies in the DJIA demonstrated to be nearly optimally maximal, and an intuitive development of the leptokurtosis in the increments of the DJIA offered.

• Quantitative Analysis of Non-Linear High Entropy Economic Systems II establishes the relationship between the log-normal characteristics of the non-linear high entropy model, and its Brownian motion/random walk equivalent. The errors created by ignoring the log-normal evolution of the non-linear high entropy model are analyzed. The distribution of wealth in the US, (including a general analysis of the effect of taxes and governmental inefficiencies,) and, the chances of Linux on the desktop are offered as empirical examples.

• Quantitative Analysis of Non-Linear High Entropy Economic Systems III develops a methodology for analyzing the dynamics of the log-normal characteristics of the non-linear high entropy model, and for empirical examples, analyzes the dynamics of the DJIA-and the US stock market crash of 1929-the US GDP, (which contains an Addendum on the duration of economies and civilizations,) the North American integrated circuit market, GE's equity price, why IT projects have such a high failure rate, and concludes with an analysis of the dynamics of the Yen/DollarUS Exchange Rate, gold prices, and, the US CPIAI-including an analysis of deflation in the Great Depression.

• Quantitative Analysis of Non-Linear High Entropy Economic Systems IV demonstrates a methodology for analyzing the dynamics of non-linear high entropy systems using GE's equity price as a detailed empirical example, including the annual fluctuation of equity prices on the US exchanges, and concludes with a simulation of the durability of US public companies-how long they are listed on an exchange-including the durability of hedge funds.

• Quantitative Analysis of Non-Linear High Entropy Economic Systems V is an application of the principles of non-linear high entropy systems to financial engineering and demonstrates a simple portfolio management strategy, including a quarter century simulation of the strategy on the historical data of ten equities from the DJIA, and an analytical simulation of the dynamics of the venture capital industry.

• Quantitative Analysis of Non-Linear High Entropy Economic Systems VI discusses the effect of leptokurtosis on financial time series, using the DJIA as an example of determining the risk of an investment. GE's equity price is also analyzed, and an empirical method offered for analyzing long term rare catastrophic economic events.

• Quantitative Analysis of Non-Linear High Entropy Economic Systems VII offers a concluding example of a detailed analysis of the DJIA, an example of the ubiquity of non-linear high entropy systems by analyzing Internet web server hits, and, The DJIA, revisited, concerning the 2008 financial crisis, followed by regression studies in non-linear high entropy economic systems regarding the frailty of analysis in macroeconomics, and concludes with a comparison of the Laplacian and Normal/Gaussian probability distributions of the marginal increments in financial time series.

• Quantitative Analysis of Non-Linear High Entropy Economic Systems Addendum analyzes the 2002 US GDP and the NASDAQ index and draws optimistic conclusions about the near-to-mid term future of the high-tech industry, and includes an analysis of the 2000 decline in the DJIA, NASDAQ and S&P500 indices, an analysis of significant declines in the US equity market indices over the last hundred years, (and offers a simple methodology for calculating the severity of the declines,) a 2003 mathematical analysis of the poll numbers in the California Governor Recall, the probabilities of the outcomes of war, a compendum on the projections of the 2004 Presidential election, an analysis of the assumptions about the US GDP used in Social Security solvency arguments, log-normal distributed poll data in the multi-candidate 2008 US Presidential Election, an analysis of 2008 asset values/housing prices, an analysis of light sweet crude futures, a short analysis of 2008 US FDIC Bank failures, and concludes with forensics of the "Great Recession."

• Quantitative Analysis of Non-Linear High Entropy Economic Systems Notes and Asides offers implementation details and suggestions for using the concepts and software available on this site.

All are from the NtropiX Mailing List, and/or, NdustriX Mailing List distributions.

Historical Economics

For applications of quantitative analysis of non-Linear high entropy economic systems (e.g., econophysics,) to historical data, see the Historical Economics page on the author's site. The analysis includes the world GDP since the late stone age, (about -5000 CE,) and the US since 1792.

Add Ons

• Ronald McEwan has written a Microsoft Excel® spreadsheet, entropy.xls, that implements the functionality of tsinvest, (the spreadsheet also works with Sun Microsystems® StarOffice® which is available as Open Source Software, OpenOffice). It includes a utility for downloading daily, weekly and monthly data from Yahoo!. You will have to manually re-scale the y axis of the chart depending on the price range desired. The spreadsheet looks at 60 days worth of data, but can be easily modified for specific needs.

• Ron has also compiled the tsinvest program on Microsoft XP® the executable is available as tsinvest.exe.

• Ron, with the help of Don Gullet, from the Xltraders group (on Yahoo!,) wrote Intraday_Entropy.xls, Microsoft Excel® spreadsheet, to get Intraday Data from the Business Week site. There is an adjustable look back range that can be changed with the spinner. New symbols can be added to the Symbols page. The drop down list in the main page lets you select the symbol. The chart will not automatically adjust for the amount of data that gets retrieved, so the user will have to do that themselves.

• The tsticker Perl 5 script is a demonstration/template for a way to obtain near real time ticker data over the Internet from the Yahoo Quote server for the tsinvest program suite. This script requires Dj Padzensky's excellent Finance::YahooQuote package.

• The QuoteMonster Perl script, available from http://xmlworks.com/quotemonster/, can be used to retrieve historical data for many different stocks at one time, over a specified time interval, from the http://chart.yahoo.com/d stock price database in csv spreadsheet format. The program can be configured to be compatible with the csv2tsinvest program that is distributed with the tsinvest sources.

Availability

The tsinvest programs are freely available for Download and distributed as source code, at no charge, and provided under License.

Mailing list

A mailing list, maintained by the author, is available for users of the tsinvest program suite. To join, see the Mailing List page.

Archive

The NtropiX archive is at http://www.johncon.com/ntropix/archive/.