Cyclical Pre-Training for Cryptocurrency Price Predictions

img
Is it possible to use recent news headlines to predict the price of extremely volatile cryptocurrency prices in the future? The answer was found to be yes with the best results being 5% above chance. How was this done? Initial baseline approaches with BERT failed to have an accuracy more than flipping a coin. Instead, using cyclical pre-training with GPT and simplified questions answering proved to be the best strategy. The idea was that the model would hold onto pertinent price information between pre-training on a day's worth of news information and fine-tuning on past price questions. What does this do for the field of NLP? This work is related to sentiment analysis with a temporal aspect, which has applications beyond the financial sector such as election prediction and potentially early disease outbreak detection. This strategy may be helpful for certain models to improve their ability with numeracy and cause and effect as well.