mtyrrell commited on
Commit
eadaa42
·
1 Parent(s): 1555bc6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -3
README.md CHANGED
@@ -34,15 +34,22 @@ It achieves the following results on the evaluation set:
34
 
35
  ## Model description
36
 
37
- More information needed
38
 
39
  ## Intended uses & limitations
40
 
41
- More information needed
 
 
 
42
 
43
  ## Training and evaluation data
44
 
45
- More information needed
 
 
 
 
46
 
47
  ## Training procedure
48
 
 
34
 
35
  ## Model description
36
 
37
+ The model is a binary text classifier using 'sentence-transformers/all-mpnet-base-v2' and fine-tuned on text sourced from national climate policy documents.
38
 
39
  ## Intended uses & limitations
40
 
41
+ The classifier assigns a class of 'Unconditional' or 'Conditional' to denote the strength of commitments as portrayed in extracted passages from the documents. The intended use is for climate policy researchers and analysts seeking to automate the process of reviewing lengthy, non-standardized PDF documents to produce summaries and reports.
42
+
43
+ Due to inconsistencies in the training data, the classifier performance leaves room for improvement. The classifier exhibits reasonably good training metrics (F1 ~ 0.85), balanced between precise identification of true positive classifications (precision ~ 0.85) and a wide net to capture as many true positives as possible (recall ~ 0.85). When tested on real world unseen test data, the performance was subptimal for a binary classifier (F1 ~ 0.5). However, testing was based on a small out-of-sample dataset containing it's own inconsistencies. Therefore classification may prove more robust in practice.
44
+
45
 
46
  ## Training and evaluation data
47
 
48
+ The dataset is comprised of data from 2 sources:
49
+ - [ClimateWatch NDC Sector data](https://www.climatewatchdata.org/data-explorer/historical-emissions?historical-emissions-data-sources=climate-watch&historical-emissions-gases=all-ghg&historical-emissions-regions=All%20Selected&historical-emissions-sectors=total-including-lucf%2Ctotal-including-lucf&page=1)
50
+ - [IKI TraCS Climate Strategies for Transport Tracker](https://changing-transport.org/wp-content/uploads/20220722_Tracker_Database.xlsx) implemented by GIZ and funded by theInternational Climate Initiative (IKI) of the German Federal Ministry for Economic Affairs and Climate Action (BMWK).
51
+
52
+ From the first source, we take
53
 
54
  ## Training procedure
55