Software For Algorithmic Trading Of Equities:

The tests directory in the source tree contains data files for the tsinvestsim(1) program for regression testing of the tsinvest(1) and tsinvestsim(1) programs.

Note: some/most of the simulations are for 100,000 days, and the complete regression suite takes over 300 hours, (ie., about two weeks,) of execution time on a Pentium 90. Additionally, the output files of the regression suite require over half of a gigabyte of disk space. A makefile is supplied to construct the regression suite. The results are documented in each data file.

Inventory:

• non-volatile.data, a test file for tsinvestsim(1), of a market with 300 equities, with too little volatility, ie., rms < 2P - 1, with Shannon probabilities, P, ranging, in a linear fashion, from 0.51 to 0.51299. (Real markets go from about 0.505 to 0.560, or so, and are typically, non-volatile.) The volatility is 50% too low.

  • The daily gain in value of the index, i, should be 1.000266, and the gain in value of a portfolio of the top ten equities, g, should be 1.000327.

  • This file is intended to test whether the tsinvest(1) program can exploit markets where the difference in the growth rates of equities is not large. Ideally, what should happen, after many days, (say, 100,000,) is that the equities invested in are 299, 298, 297, ..., and the value of the capital should be greater than the value of the average index.

  • To exercise this file:

    • tsinvestsim non-volatile.data 100000 | tsinvest -i -s -t

    where the -d 1|2|3|4|5|6 argument of tsinvest(1) is used to alter the wagering strategy. The measured results are:

    Arg	-d1	-d2	-d3	-d4	-d5
    g	1.000265	1.000265	1.000265	1.000265	1.000265
    i	1.000288	1.000317	1.000295	1.000275	1.000270

    
    

• non-volatile.equal.antipersistent.data, a test file for tsinvestsim(1), of a market with 300 equities, with too little volatility, ie., rms < 2P - 1, with Shannon probabilities, P, identical, and equal to 0.51, and an antipersistence, H, ranging, in a linear fashion, from 0.4 to 0.5. (Real markets have Shannon probabilities that go from about 0.505 to 0.560, or so, and antipersistences running from about 0.400 to 0.500, or so.) The volatility is 50% too low. This is a good "bear" market simulation.

  • The daily gain in value of the index, i, should be 1.000200, and the gain in value of a portfolio of the top ten equities, g, should be 1.000195. The gain in value of a portfolio of the top ten equites, g, based on the selection criteria of antipersistence, (ie., the -d5 option,) should be about 1.001997, (assuming a probability of an up movement of 1 - H, or about 0.6.)

  • This file is intended to test how well the tsinvest(1) program does in a market where there is nothing to exploit. Ideally, what should happen, after many days, (say, 100,000,) is that value of the capital should be less than, but nearly equal to, the value of the average index. There is no strategic advantage in investing in any stock over any other stock-in point of fact, the optimal strategy is to invest equally in all 300 equities. Anything less than this will result in a loss, in comparison to the average index of all equities.

  • To exercise this file:

    • tsinvestsim non-volatile.equal.antipersistent.data 100000 | tsinvest -i -s -t

    where the -d 1|2|3|4|5|6 argument of tsinvest(1) is used to alter the wagering strategy. The measured results are:

    Arg	-d1	-d2	-d3	-d4	-d5
    g	1.000176	1.000176	1.000176	1.000176	1.000176
    i	1.000166	1.000180	1.000166	1.000177	1.001925

    
    

• non-volatile.equal.data, a test file for tsinvestsim(1), of a market with 300 equities, with too little volatility, ie., rms < 2P - 1, with Shannon probabilities, P, identical, and equal to 0.51. (Real markets go from about 0.505 to 0.560, or so.) The volatility is 50% too low. This is a good "bear" market simulation.

  • The daily gain in value of the index, i, should be 1.000200, and the gain in value of a portfolio of the top ten equities, g, should be 1.000195.

  • This file is intended to test how well the tsinvest(1) program does in a market where there is nothing to exploit. Ideally, what should happen, after many days, (say, 100,000,) is that value of the capital should be less than, but nearly equal to, the value of the average index.

  • To exercise this file:

    • tsinvestsim non-volatile.equal.data 100000 | tsinvest -i -s -t

    where the -d 1|2|3|4|5|6 argument of tsinvest(1) is used to alter the wagering strategy. The measured results are:

    Arg	-d1	-d2	-d3	-d4	-d5
    g	1.000199	1.000199	1.000199	1.000199	1.000199
    i	1.000193	1.000200	1.000192	1.000196	1.000178

    
    

• non-volatile.equal.persistent.data, a test file for tsinvestsim(1), of a market with 300 equities, with too little volatility, ie., rms < 2P - 1, with Shannon probabilities, P, identical, and equal to 0.51, and a persistence, H, ranging, in a linear fashion, from 0.5 to 0.6. (Real markets have Shannon probabilities that go from about 0.505 to 0.560, or so, and persistences running from about 0.500 to 0.600, or so.) The volatility is 50% too low. This is a good "bear" market simulation.

  • The daily gain in value of the index, i, should be 1.000200, and the gain in value of a portfolio of the top ten equities, g, should be 1.000195. The gain in value of a portfolio of the top ten equites, g, based on the selection criteria of antipersistence, (ie., the -d5 option,) should be about 1.001997, (assuming a probability of an up movement of H, or about 0.6.)

  • This file is intended to test how well the tsinvest(1) program does in a market where there is nothing to exploit. Ideally, what should happen, after many days, (say, 100,000,) is that value of the capital should be less than, but nearly equal to, the value of the average index. There is no strategic advantage in investing in any stock over any other stock-in point of fact, the optimal strategy is to invest equally in all 300 equities. Anything less than this will result in a loss, in comparison to the average index of all equities.

  • To exercise this file:

    • tsinvestsim non-volatile.equal.persistent.data 100000 | tsinvest -i -s -t

    where the -d 1|2|3|4|5|6 argument of tsinvest(1) is used to alter the wagering strategy. The measured results are:

    Arg	-d1	-d2	-d3	-d4	-d5
    g	1.000226	1.000226	1.000226	1.000226	1.000226
    i	1.000253	1.000231	1.000255	1.000226	1.001915

    
    

• optimal.data, a test file for tsinvestsim(1), of a market with 300 equities, all optimal, ie., rms = 2P - 1, with Shannon probabilities, P, ranging, in a linear fashion, from 0.51 to 0.51299. (Real markets go from about 0.505 to 0.560, or so.)

  • The daily gain in value of the index, i, should be 1.000531, and the gain in value of a portfolio of the top ten equities, g, should be 1.000637.

  • This file is intended to test whether the tsinvest(1) program can exploit markets where the difference in the growth rates of equities is not large. Ideally, what should happen, after many days, (say, 100,000,) is that the equities invested in are 299, 298, 297, ..., and the value of the capital should be greater than the value of the average index.

  • To exercise this file:

    • tsinvestsim optimal.data 100000 | tsinvest -i -s -t

    where the -d 1|2|3|4|5|6 argument of tsinvest(1) is used to alter the wagering strategy. The measured results are:

    Arg	-d1	-d2	-d3	-d4	-d5
    g	1.000530	1.000530	1.000530	1.000530	1.000530
    i	1.000553	1.000616	1.000575	1.000579	1.000523

    
    

• optimal.equal.antipersistent.data, a test file for tsinvestsim(1), of a market with 300 equities, all optimal, ie., rms = 2P - 1, with Shannon probabilities, P, identical, and equal to 0.51, and a antipersistence, H, ranging, in a linear fashion, from 0.4 to 0.5. (Real markets have Shannon probabilities that go from about 0.505 to 0.560, or so, and antipersistences running from about 0.400 to 0.500.)

  • The daily gain in value of the index, i, should be 1.000399, and the gain in value of a portfolio of the top ten equities, g, should be 1.000380. The gain in value of a portfolio of the top ten equites, g, based on the selection criteria of antipersistence, (ie., the -d5 option,) should be about 1.003988, (assuming a probability of an up movement of 1 - H, or about 0.6.)

  • This file is intended to test how well the tsinvest(1) program does in a market where there is nothing to exploit. Ideally, what should happen, after many days, (say, 100,000,) is that value of the capital should be less than, but nearly equal to, the value of the average index. There is no strategic advantage in investing in any stock over any other stock-in point of fact, the optimal strategy is to invest equally in all 300 equities. Anything less than this will result in a loss, in comparison to the average index of all equities.

  • To exercise this file:

    • tsinvestsim optimal.equal.antipersistent.data 100000 | tsinvest -i -s -t

    where the -d 1|2|3|4|5|6 argument of tsinvest(1) is used to alter the wagering strategy. The measured results are:

    Arg	-d1	-d2	-d3	-d4	-d5
    g	1.000352	1.000352	1.000352	1.000352	1.000352
    i	1.000322	1.000351	1.000320	1.000325	1.003843

    
    

• optimal.equal.data, a test file for tsinvestsim(1), of a market with 300 equities, all optimal, ie., rms = 2P - 1, with Shannon probabilities, P, identical, and equal to 0.51. (Real markets go from about 0.505 to 0.560, or so.)

  • The daily gain in value of the index, i, should be 1.000399, and the gain in value of a portfolio of the top ten equities, g, should be 1.000380.

  • This file is intended to test how well the tsinvest(1) program does in a market where there is nothing to exploit. Ideally, what should happen, after many days, (say, 100,000,) is that value of the capital should be less than, but nearly equal to, the value of the average index.

  • To exercise this file:

    • tsinvestsim optimal.equal.data 100000 | tsinvest -i -s -t

    where the -d 1|2|3|4|5|6 argument of tsinvest(1) is used to alter the wagering strategy. The measured results are:

    Arg	-d1	-d2	-d3	-d4	-d5
    g	1.000399	1.000399	1.000399	1.000399	1.000399
    i	1.000377	1.000390	1.000378	1.000379	1.000346

    
    

• optimal.equal.persistent.data, a test file for tsinvestsim(1), of a market with 300 equities, all optimal, ie., rms = 2P - 1, with Shannon probabilities, P, identical, and equal to 0.51, and a persistence, H, ranging, in a linear fashion, from 0.5 to 0.6. (Real markets have Shannon probabilities that go from about 0.505 to 0.560, or so, and persistences running from about 0.500 to 0.600.)

  • The daily gain in value of the index, i, should be 1.000399, and the gain in value of a portfolio of the top ten equities, g, should be 1.000380. The gain in value of a portfolio of the top ten equites, g, based on the selection criteria of antipersistence, (ie., the -d5 option,) should be about 1.003988, (assuming a probability of an up movement of H, or about 0.6.)

  • This file is intended to test how well the tsinvest(1) program does in a market where there is nothing to exploit. Ideally, what should happen, after many days, (say, 100,000,) is that value of the capital should be less than, but nearly equal to, the value of the average index. There is no strategic advantage in investing in any stock over any other stock-in point of fact, the optimal strategy is to invest equally in all 300 equities. Anything less than this will result in a loss, in comparison to the average index of all equities.

  • To exercise this file:

    • tsinvestsim optimal.equal.persistent.data 100000 | tsinvest -i -s -t

    where the -d 1|2|3|4|5|6 argument of tsinvest(1) is used to alter the wagering strategy. The measured results are:

    Arg	-d1	-d2	-d3	-d4	-d5
    g	1.000453	1.000453	1.000453	1.000453	1.000453
    i	1.000499	1.000451	1.000496	1.000452	1.003821

    
    

• volatile.data, a test file for tsinvestsim(1), of a market with 300 equities, all too volatile, ie., rms > 2P - 1, with Shannon probabilities, P, ranging, in a linear fashion, from 0.51 to 0.51299. (Real markets go from about 0.505 to 0.560, or so, and are typically, non-volatile, but some equities exhibit volatility.) The volatility is 50% too high.

  • The daily gain in value of the index, i, should be 1.000796, and the gain in value of a portfolio of the top ten equities, g, should be 1.000931.

  • This file is intended to test whether the tsinvest(1) program can exploit markets where the difference in the growth rates of equities is not large. Ideally, what should happen, after many days, (say, 100,000,) is that the equities invested in are 299, 298, 297, ..., and the value of the capital should be greater than the value of the average index.

  • To exercise this file:

    • tsinvestsim volatile.data 100000 | tsinvest -i -s -t

    where the -d 1|2|3|4|5|6 argument of tsinvest(1) is used to alter the wagering strategy. The measured results are:

    Arg	-d1	-d2	-d3	-d4	-d5
    g	1.000800	1.000800	1.000800	1.000800	1.000800
    i	1.000780	1.001055	1.000848	1.000877	1.000622

    
    

• volatile.equal.antipersistent.data, a test file for tsinvestsim(1), of a market with 300 equities, all too volatile, ie., rms > 2P - 1, with Shannon probabilities, P, identical, and equal to 0.51, and a antipersistence, H, ranging, in a linear fashion, from 0.4 to 0.5. (Real markets have Shannon probabilities that go from about 0.505 to 0.560, or so, and antipersistences running from about 0.400 to 0.500, or so.) The volatility is 50% too high.

  • The daily gain in value of the index, i, should be 1.000599, and the gain in value of a portfolio of the top ten equities, g, should be 1.000555. The gain in value of a portfolio of the top ten equites, g, based on the selection criteria of antipersistence, (ie., the -d5 option,) should be about 1.005973, (assuming a probability of an up movement of 1 - H, or about 0.6.)

  • This file is intended to test how well the tsinvest(1) program does in a market where there is nothing to exploit. Ideally, what should happen, after many days, (say, 100,000,) is that value of the capital should be less than, but nearly equal to, the value of the average index. There is no strategic advantage in investing in any stock over any other stock-in point of fact, the optimal strategy is to invest equally in all 300 equities. Anything less than this will result in a loss, in comparison to the average index of all equities.

  • To exercise this file:

    • tsinvestsim volatile.equal.antipersistent.data 100000 | tsinvest -i -s -t

    where the -d 1|2|3|4|5|6 argument of tsinvest(1) is used to alter the wagering strategy. The measured results are:

    Arg	-d1	-d2	-d3	-d4	-d5
    g	1.000536	1.000536	1.000536	1.000536	1.000536
    i	1.000400	1.000730	1.000451	1.000517	1.005375

    
    

• volatile.equal.data, a test file for tsinvestsim(1), of a market with 300 equities, all too volatile, ie., rms > 2P - 1, with Shannon probabilities, P, identical, and equal to 0.51. (Real markets go from about 0.505 to 0.560, or so.) The volatility is 50% too high.

  • The daily gain in value of the index, i, should be 1.000599, and the gain in value of a portfolio of the top ten equities, g, should be 1.000555.

  • This file is intended to test how well the tsinvest(1) program does in a market where there is nothing to exploit. Ideally, what should happen, after many days, (say, 100,000,) is that value of the capital should be less than, but nearly equal to, the value of the average index.

  • To exercise this file:

    • tsinvestsim volatile.equal.data 100000 | tsinvest -i -s -t

    where the -d 1|2|3|4|5|6 argument of tsinvest(1) is used to alter the wagering strategy. The measured results are:

    Arg	-d1	-d2	-d3	-d4	-d5
    g	1.000600	1.000600	1.000600	1.000600	1.000600
    i	1.000555	1.000647	1.000556	1.000558	1.000336

    
    

• volatile.equal.persistent.data, a test file for tsinvestsim(1), of a market with 300 equities, all too volatile, ie., rms > 2P - 1, with Shannon probabilities, P, identical, and equal to 0.51, and a persistence, H, ranging, in a linear fashion, from 0.5 to 0.6. (Real markets have Shannon probabilities that go from about 0.505 to 0.560, or so, and persistences running from about 0.500 to 0.600, or so.) The volatility is 50% too high.

  • The daily gain in value of the index, i, should be 1.000599, and the gain in value of a portfolio of the top ten equities, g, should be 1.000555. The gain in value of a portfolio of the top ten equites, g, based on the selection criteria of antipersistence, (ie., the -d5 option,) should be about 1.005973, (assuming a probability of an up movement of H, or about 0.6.)

  • This file is intended to test how well the tsinvest(1) program does in a market where there is nothing to exploit. Ideally, what should happen, after many days, (say, 100,000,) is that value of the capital should be less than, but nearly equal to, the value of the average index. There is no strategic advantage in investing in any stock over any other stock-in point of fact, the optimal strategy is to invest equally in all 300 equities. Anything less than this will result in a loss, in comparison to the average index of all equities.

  • To exercise this file:

    • tsinvestsim volatile.equal.persistent.data 100000 | tsinvest -i -s -t

    where the -d 1|2|3|4|5|6 argument of tsinvest(1) is used to alter the wagering strategy. The measured results are:

    Arg	-d1	-d2	-d3	-d4	-d5
    g	1.000679	1.000679	1.000679	1.000679	1.000679
    i	1.000736	1.000696	1.000728	1.000670	1.005578

    
    

• crash-up.data, a test file for tsinvestsim(1), of a deteriorating market with 300 equities, simulating the US equity markets for 3,254 trading days between 15 August, 1921, and 6 June, 1932, inclusive. During the 2,401 trading day period between 15 August, 1921 and 7 September, 1929, the US equity markets had a substantial gain of about 5.7X in value, (DJIA values of 66.02 to 375.44.) During the 853 trading day period between 7 September, 1929, and 6 June, 1932, the markets had a significant reversal, loosing about 90% of their 7 September, 1929 value, (DJIA values of 375.44 to 42.68,) for about a 30% loss on the decade 1921-1931, and did not regain their 7 September, 1929 values until mid 1956.

  • To make the tsinvest(1) data file, tsinvestsim(1) is executed twice-the first time on this file, which is when the market was increasing:

    • tsinvestsim crash-up.data 2401 | sed 's/^/UP/' > crash.data

    with the terminal values for each stock at the end of the simulation used to make another tsinvestsim(1) input file with the same initial values, which is when the market was decreasing:

    • egrep '^UP2400' crash.data | cut -f2,3 | sed 's/ /, p = 0.44, f = 0.02, h = 0.55, i = /' > crash-down.data

    (where sed(1) is replacing a tab character,) which is then executed by tsinvestsim(1):

    • tsinvestsim crash-down.data 853 | sed 's/^/DOWN/' >> crash.data

  • and, to exercise this file:

    • tsinvest -i -t -s -c crash.data

    where the -d 1|2|3|4|5|6 argument of tsinvest(1) is used to alter the wagering strategy. The measured results are:

    Arg	-d1 -c	-d2 -c	-d3 -c	-d4 -c	-d5 -c
    g	0.999866	0.999866	0.999866	0.999866	0.999866
    i	1.000134	1.000000	1.000000	0.999950	1.000450

    
    

• crash-down.data, a test file for tsinvestsim(1), machine generated from the crash-up.data file. The file crash-up.data represents the escalation in equity values, from 1921 on, and the file crash-down.data represents the deterioration in equity values, from 1929 on.

• stocks.data, a test file for tsinvestsim(1), of a market with 498 equities. This file was generated by dumping the internal data structures of the tsinvest(1) program after it had completed execution of the file "stocks", (a daily fragment of the American stock exchanges, consisting of 498 equities, from January 1, 1993, to June 6, 1996, as supplied by http://www.ai.mit.edu/stocks.html,) using the -r option, to make a new file for tsinvestsim(1).

  • This file is intended to test how well the tsinvestsim(1) and tsinvest(1) programs model real markets. The data output from the tsinvest(1) program should be similar with the real, and dumped data.

  • To exercise this file:

    • tsinvestsim stocks.data 671 | tsinvest -i -s -t

    where the -d 1|2|3|4|5|6 argument of tsinvest(1) is used to alter the wagering strategy.

• losers.data, a test file for tsinvest(1), of a market with 49 equities, all decreasing in value. This file was generated by dumping the internal data structures of the tsinvest(1) program after it had completed execution of the file "stocks", (a daily fragment of the American stock exchanges, consisting of 454 equities, from January 1, 1993, to June 6, 1996, as supplied by http://www.ai.mit.edu/stocks.html,) using the -r option, (the -p -P options were used, also,) to make a new file for tsinvest(1).

  • To make the tsinvest(1) data file:

    • tsinvest -p -P -r stocks > stocks.data

    The file stocks.data was then edited in a text editor, and stocks picked such that:

    1. The gain, G, of the value of the stock was less than unity, ie., it decreased in value.

    2. The root mean square, rms, of the marginal returns, squared, was greater than the average of the marginal returns, ie., the stock's rms^2 > avg, it was too volatile.

    3. The likelihood of an up movement in the stock's value, P, was greater than 50%.

    The file "stocks" was then filtered, using egrep(1), to make a new tsinvest(1) database of the stocks that lost value do to being too volatile, between January 1, 1993, to June 6, 1996.

    This file is intended to test how well the tsinvest(1) program does in a market that is deteriorating, and test how well it does assembling the portfolio, taking advantage of moving money around in the portfolio, (ie., asset allocation.)

    Note that the -D0 and -j options were used; normally, the tsinvest(1) program will not invest in stocks that are declining in value-the -D0 option over rides this default behavior, and forces the program to commit to managing investments in stocks that are declining in value; and the -j option prints the average of the stocks, as opposed to the average balanced growth.

  • To exercise this file:

    • tsinvest -i -s -t losers.data

    where the -d 1|2|3|4|5|6 argument of tsinvest(1) is used to alter the wagering strategy. The measured results are:

    Arg	-d1	-d2	-d3	-d4	-d5
    g	0.999987	0.999987	0.999987	0.999987	0.999987
    i	0.999364	1.001251	0.999413	1.001209	0.999845

    
    

    • tsinvest -i -s -t -m0 losers.data

    where the -d 1|2|3|4|5|6 argument of tsinvest(1) is used to alter the wagering strategy. The measured results are:

    Arg	-d1 -m0	-d2 -m0	-d3 -m0	-d4 -m0	-d5 -m0
    g	0.999987	0.999987	0.999987	0.999987	0.999987
    i	0.999291	1.001649	0.999388	1.000336	1.000943

    
    

A license is hereby granted to reproduce this software source code and to create executable versions from this source code for personal, non-commercial use. The copyright notice included with the software must be maintained in all copies produced.

THIS PROGRAM IS PROVIDED "AS IS". THE AUTHOR PROVIDES NO WARRANTIES WHATSOEVER, EXPRESSED OR IMPLIED, INCLUDING WARRANTIES OF MERCHANTABILITY, TITLE, OR FITNESS FOR ANY PARTICULAR PURPOSE. THE AUTHOR DOES NOT WARRANT THAT USE OF THIS PROGRAM DOES NOT INFRINGE THE INTELLECTUAL PROPERTY RIGHTS OF ANY THIRD PARTY IN ANY COUNTRY.

So there.

Copyright © 1994-2011, John Conover, All Rights Reserved.

Comments and/or bug reports should be addressed to:

john@email.johncon.com

http://www.johncon.com/

http://www.johncon.com/ntropix/

http://www.johncon.com/ndustrix/

http://www.johncon.com/nformatix/

http://www.johncon.com/ndex/

John Conover

john@email.johncon.com

January 6, 2006