Financial Sentiment Analysis with Transformers

Problem Statement

Financial markets are highly sensitive to news. Manual analysis of the vast amount of financial news generated daily is impossible. Automated sentiment analysis can provide real-time insights for trading strategies.

The objective is to classify financial news headlines into three categories: Positive, Negative, and Neutral.

Dataset

We used the FinancialPhraseBank dataset, which consists of 4,840 sentences from English language financial news categorized by sentiment.

Class distribution:

- Neutral: 59%

- Positive: 28%

- Negative: 13%

Methodology

We fine-tuned a pre-trained BERT (Bidirectional Encoder Representations from Transformers) model.

The architecture consists of:

1. BERT Base Uncased encoder

2. Dropout layer (p=0.1)

3. Linear classification head

1
2import torch.nn as nn
3from transformers import BertModel
4
5class SentimentClassifier(nn.Module):
6    def __init__(self, n_classes):
7        super(SentimentClassifier, self).__init__()
8        self.bert = BertModel.from_pretrained('bert-base-uncased')
9        self.drop = nn.Dropout(p=0.3)
10        self.out = nn.Linear(self.bert.config.hidden_size, n_classes)
11    
12    def forward(self, input_ids, attention_mask):
13        _, pooled_output = self.bert(
14          input_ids=input_ids,
15          attention_mask=attention_mask
16        )
17        output = self.drop(pooled_output)
18        return self.out(output)
19      

Results

The model achieved an accuracy of 94% on the test set.

Confusion Matrix analysis showed that the model rarely confuses Positive and Negative sentiments, with most errors occurring between Neutral and other classes.

Limitations

The model struggles with sarcasm and complex financial jargon that requires deep domain knowledge.