Common Mistakes when Applying Computational Intelligence and Machine Learning to Stock Market modelling

Abstract

For a number of reasons, computational intelligence and machine learning methods have been largely dismissed by the professional community. The reasons for this are numerous and varied, but inevitably amongst the reasons given is that the systems designed often do not perform as expected by their designers. The reasons for this lack of performance is a direct result of mistakes that are commonly seen in market-prediction systems. This paper examines some of the more common mistakes, namely dataset insufficiency; inappropriate scaling; time-series tracking; inappropriate target quantification and inappropriate measures of performance. The rationale that leads to each of these mistakes is examined, as well as the nature of the errors they introduce to the analysis / design. Alternative ways of performing each task are also recommended in order to avoid perpetuating these mistakes, and hopefully to aid in clearing the way for the use of these powerful techniques in industry.