Creating a portfolio based on hisotrical stock market prices
We download stock prices on some European indexes (CAC 40, DAX 100...) which can represent a country, a sector or a company size (small/mid caps). The goal is to have a diversity in the data. So we'll run the algorithm on those indexes and only stick with the top N companies of each index. We're then only looking at the best companies in each group (the term "best" is only based on the outcome of our algorithm). Finally, we run the algorithm on our list of stock filtered to get the final portfolio.
We use this process for three main reasons :
Firstly, Our algorithm is O(n²*T) with n the number of stocks and T the number of timestamps. So using the algorithm separately on different index enables to reduce the cost of computational cost.
Secondly, if we computed the algorithm on all the stocks at once, we will have a better theorical solution. However, it's not feasable in reality. The maximum weight for a stock will be less than 5%, having a lot of 0.001% etc.. With the fees of brokers, we will be pay a lot as the fees tend to be lighter as the money increased. It will result in a lost in operational cost of the fund.
Thirdly, fragmenting in subset of stocks enable our algorithm to work on smaller data without losing much diversity. For example, if I was US based, the data companies would have take at least 50% weight of my porfolio with the GAFAM and Nvidia. But by taking indexes, I can target one ESG index with one small caps index, one french index and one energy index. If the indexes selection is well done (10 to 20 indexes), the data will be diversified.