ningkko commited on
Commit
7e34597
1 Parent(s): 6533da6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -7
README.md CHANGED
@@ -11,19 +11,29 @@ should probably proofread and complete it, then remove this comment. -->
11
 
12
  # drug-stance-bert
13
 
14
- This model is a fine-tuned version of [cardiffnlp/twitter-roberta-base-sentiment](https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment) on [COVID-CQ](https://github.com/eceveco/COVID-CQ).
15
 
16
  ## Model description
17
 
18
- More information needed
19
 
20
- ## Intended uses & limitations
21
 
22
- More information needed
 
 
 
 
 
 
23
 
24
- ## Training and evaluation data
25
 
26
- More information needed
 
 
 
 
 
27
 
28
  ## Training procedure
29
 
@@ -36,7 +46,7 @@ The following hyperparameters were used during training:
36
  - seed: 42
37
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
38
  - lr_scheduler_type: linear
39
- - num_epochs: 4.0
40
 
41
  ### Framework versions
42
 
 
11
 
12
  # drug-stance-bert
13
 
14
+ This model is a fine-tuned version of [cardiffnlp/twitter-roberta-base-sentiment](https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment) on [COVID-CQ](https://github.com/eceveco/COVID-CQ), a dataset that contains 3-label annotated opinions (negative, neutral, and positive) of the tweet initiators regarding the use of Chloroquine or Hydroxychloroquine for the treatment or prevention of the coronavirus.
15
 
16
  ## Model description
17
 
18
+ "We developed two COVID-drug-stance RoBERTa-base models by fine-tuning a pre-trained Twitter-specific stance detection model on a stance data set called COVID-CQ. The data were divided into training-dev-test validation datasets with a 70:10:20 ratio. Model I (COVID-drug-stance-BERT) was trained on the original tweet data, and Model II (COVID-drug-stance-BERT-masked) was trained on tweets with drug names masked as “[mask]” for model generalizability on different drugs. The two models had similar performance on the COVID-19 validation set: COVID-drug-stance-BERT had an accuracy of 86.88%, and the masked model had an accuracy of 86.67%. The two models were then evaluated by predicting tweet initiators’ attitudes towards the drug mentioned in each tweet using randomly selected test sets (100 tweets) of each drug. As suggested by the evaluation in Table 2, Model I had better performance and was therefore used in this study".
19
 
 
20
 
21
+ | **Drug** | **Model I: Original Tweet** | | | **Model II: Drug Names Masked** | | |
22
+ |------------------------|:---------------------------:|:-----------:|:------------:|:-------------------------------:|:-----------:|:------------:|
23
+ | | **Precision** | **Recall** | **F1-Score** | **Precision** | **Recall** | **F1-Score** |
24
+ | **Hydroxychloroquine** | 0.93 | 0.92 | **0.92** | 0.84 | 0.83 | 0.83 |
25
+ | **Ivermectin** | 0.92 | 0.91 | **0.91** | 0.72 | 0.68 | 0.68 |
26
+ | **Molnupiravir** | 0.89 | 0.89 | **0.89** | 0.78 | 0.77 | 0.77 |
27
+ | **Remdesivir** | 0.82 | 0.79 | **0.79** | 0.70 | 0.66 | 0.66 |
28
 
29
+ The model uploaded here is Model I.
30
 
31
+ ## Intended uses & limitations
32
+ Predict opinions (negative, neutral, and positive) of tweet initiators regarding the use of a drug for the treatment or prevention of the coronavirus.
33
+
34
+ The model works better when there's only one drug name in a single tweet.
35
+ ## Training and evaluation data
36
+ COVID-CQ
37
 
38
  ## Training procedure
39
 
 
46
  - seed: 42
47
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
48
  - lr_scheduler_type: linear
49
+ - num_epochs: 3.0
50
 
51
  ### Framework versions
52