After revisiting my Rydah Springboot server, React Client, and Puzzle Gen about 2 years since I stopped developing it, I was completely lost about how some of these processes worked.
Here is an explanation for future me.
PuzzleGen
1) Download 5m and daily stock data from Stooq
2) Arrange it for parsing by placing it in the right directory format under TradingSim.
TradingSim/data_final/5m_us_txt/data should contain all the folders of various exchanges for 5m, and daily_us_txt (in place of 5m_us_txt) is the directory for daily data.
3) Run generateDataScript with i = 0 and DailyData dh uncommented, to generate the Daily_data.txt file that is used for parsing to get the daily candles for our DB.
4) Run generateDataScript with i on 1 and DailyData, to update the MySQL table with all daily data for all useful tickers.
5) Run generateDataScript with i on 0 and 5m data, to generate the puzzles. This will run much slower, parsing through all the 5m data directories, and checking which puzzles are valid enough to keep, batching them in 500’s, and then writing them to a file “5m_data.txt.” When this is done, we will have a massive file.
It will take many hours to finish generating puzzles, however, so plan accordingly. To accommodate this, it is possible to stop and start whenever necessary by editing the strings at the top of FiveMinuteData.java. See below:
private final String abs_path = “TradingSim\data_final\5_us_txt\data\”;
private final String starting_file = “aadr”;
private final String starting_date = “20241217”;
By changing the starting_file and starting_date, we can pick up at any point we want to. But try to only do this after it either finishes a batch of puzzles, or you will have to start it from the ticker that came after the last finished batch.
6) Run generateDataScript with i on 1 and 5m data, which will now look at “5m_data.txt”, and store all the 5m candles that pertain to each puzzle in the database. In other words, we are first generating the valid puzzles, and then only storing the data needed to render these puzzles.
7)
How Puzzles Work
The puzzles are generated by calculating the distance between an initial candle “i” and every candle “j through ZZ” during the same day.
We then assess if the gap between any pair of candles provides a valuable trading opportunity, and if so, we store this data as a ‘puzzle’ in a list. There may be 20 different puzzles generated on a single ticker on a single day, because candle A to B, A to C, A to D, A to E, etc. may all be high profit trades. In addition B to D, B to E, etc. may also provide valid puzzles.
Thus, we will have a lot of puzzles. This is a good thing for randomly presenting one to a user and not risking repeating too often. A check will almost certainly be present to prevent repetition anyway.
The idea is that the react client would request a puzzle from the server, and upon selecting one (based on user exp, and other factors), the server would fetch the 5m data for the relevant stock chart, up to that point in time of that ticker, and display it on the chart. The user would then make an estimate of where they feel the market should go based on the technicals present. After they enter a price/order values, they can press play or run straight to the end of that day’s chart, and see their results.
This may no longer be of use with the new approach using websockets and realtime data, but it is worth keeping just in case. It can provide a worthy minigame if nothing else.