Using News to Predict Stock Movements

Using news to predict stock movements. Stock movement prediction is an area of interest for a wide range of industries nowadays. In a world with a growing amount of information almost any trader is interested in knowing about a future situation on the market and informational space in order to make a successful investment and make a profit. Precise and effective prediction algorithms indirectly help investors by providing additional supportive information such as a future price of a stock or its direction. The objectives of the study are: 1) to predict movement directions of S&P 500 companies’ stocks using market and news data 2) identify the importance of news features in the final prediction; 3) exploration and creation of new approaches for sentiment analysis. Tasks: to identify the importance of different types of news on stock price movements; built an algorithm which predicts stock movements in the following day by applying modern machine learning approaches and refine state of the art prediction algorithms for the problem; make comparisons of different algorithms and features. Market data for S&P 500 was obtained from Yahoo! Finance; news headlines were scrapped from Reuters and Stockwatch. Preprocessing for both types of data was applied and technical analysis features were generated. The models we used were for stock movements prediction: logistic regression, KNN, Gradient boosting, Random Forest. Then we implemented Bidirectional Encoder Representations from Transformers model to perform both sentiment analysis and classification of stock price movement only using news. Results: 1) Visualization of attention layers in the network is implemented, which enabled to highlight the most important parts of the sentence, which have the highest importance in text classification. 2) The most well-performing model was Gradient boosting. 3) News features appeared to make a significant contribution to the final score.