CE fine-tuned epoch 2/3 best_val=0.6000

README.md

@@ -28,25 +28,25 @@ model-index:
       type: ce-val
     metrics:
     - type: accuracy
-      value: 0.5636363636363636
+      value: 0.6
       name: Accuracy
     - type: accuracy_threshold
-      value: 0.512444794178009
+      value: 0.5077031850814819
       name: Accuracy Threshold
     - type: f1
-      value: 0.6711297071129707
+      value: 0.6791489361702127
       name: F1
     - type: f1_threshold
-      value: 0.4493926167488098
+      value: 0.4203318953514099
       name: F1 Threshold
     - type: precision
-      value: 0.5127877237851662
+      value: 0.5236220472440944
       name: Precision
     - type: recall
-      value: 0.9709443099273608
+      value: 0.9661016949152542
       name: Recall
     - type: average_precision
-      value: 0.565654586279988
+      value: 0.6083361348816916
       name: Average Precision
 ---
 
@@ -99,25 +99,25 @@ from sentence_transformers import CrossEncoder
 model = CrossEncoder("cross_encoder_model_id")
 # Get scores for pairs of inputs
 pairs = [
-    ['An independent inquiry found CRU is a small research unit with limited resources and their rigour and honesty are not in doubt.', 'The media and other scientific organisations were criticised for having "sometimes neglected" to reflect the uncertainties, doubts and assumptions of the work done by the CRU.'],
-    ['As president, Obama will immediately close the Mississippi River Gulf Outlet, which experts say funneled floodwater into New Orleans.', 'Levees along the MRGO and the Intracoastal Waterway were breached in approximately 20 places, directly flooding most of St. Bernard Parish and New Orleans East.'],
-    ['If we double atmospheric carbon dioxide[…] we’d only raise global surface temperatures by about a degree Celsius.', 'Not only do increasing carbon dioxide concentrations lead to increases in global surface temperature, but increasing global temperatures also cause increasing concentrations of carbon dioxide.'],
-    ['But as that upper layer warms up, the oxygen-rich waters are less likely to mix down into cooler layers of the ocean because the warm waters are less dense and do not sink as readily.', 'Water that is saltier or cooler will be denser, and will sink in relation to the surrounding water.'],
-    ['Less than half of published scientists endorse global warming.', 'Scientists Reach 100% Consensus on Anthropogenic Global Warming.'],
+    ['The last time the planet was even four degrees warmer, Peter Brannen points out in The Ends of the World, his new history of the planet’s major extinction events, the oceans were hundreds of feet higher.', 'Almost all scientists acknowledge that the rate of species loss is greater now than at any time in human history, with extinctions occurring at rates hundreds of times higher than background extinction rates.'],
+    ['[S]unspot activity on the surface of our star has dropped to a new low.', 'At solar-cycle minimum, the toroidal field is, correspondingly, at minimum strength, sunspots are relatively rare, and the poloidal field is at its maximum strength.'],
+    ['More money is dedicated within the Department of Homeland Security to climate change than what\'s spent combating "Islamist terrorists radicalizing over the Internet in the United States of America."', 'Homeland security is officially defined by the National Strategy for Homeland Security as "a concerted national effort to prevent terrorist attacks within the United States, reduce America\'s vulnerability to terrorism, and minimize the damage and recover from attacks that do occur".'],
+    ['Worst-case global heating scenarios may need to be revised upwards in light of a better understanding of the role of clouds, scientists have said.', 'With this information, scientists can produce scenarios of how greenhouse gas emissions may vary in the future.'],
+    ['Prof Adam Scaife, a climate modelling expert at the UK’s Met Office, said the evidence for a link to shrinking Arctic ice was now good: ‘The consensus points towards that being a real effect.’”', 'Some models of modern climate exhibit Arctic amplification without changes in snow and ice cover.'],
 ]
 scores = model.predict(pairs)
 print(scores)
-# [0.5286 0.4566 0.49   0.5111 0.6522]
+# [0.6799 0.5912 0.5    0.4181 0.4313]
 
 # Or rank different texts based on similarity to a single text
 ranks = model.rank(
-    'An independent inquiry found CRU is a small research unit with limited resources and their rigour and honesty are not in doubt.',
+    'The last time the planet was even four degrees warmer, Peter Brannen points out in The Ends of the World, his new history of the planet’s major extinction events, the oceans were hundreds of feet higher.',
     [
-        'The media and other scientific organisations were criticised for having "sometimes neglected" to reflect the uncertainties, doubts and assumptions of the work done by the CRU.',
-        'Levees along the MRGO and the Intracoastal Waterway were breached in approximately 20 places, directly flooding most of St. Bernard Parish and New Orleans East.',
-        'Not only do increasing carbon dioxide concentrations lead to increases in global surface temperature, but increasing global temperatures also cause increasing concentrations of carbon dioxide.',
-        'Water that is saltier or cooler will be denser, and will sink in relation to the surrounding water.',
-        'Scientists Reach 100% Consensus on Anthropogenic Global Warming.',
+        'Almost all scientists acknowledge that the rate of species loss is greater now than at any time in human history, with extinctions occurring at rates hundreds of times higher than background extinction rates.',
+        'At solar-cycle minimum, the toroidal field is, correspondingly, at minimum strength, sunspots are relatively rare, and the poloidal field is at its maximum strength.',
+        'Homeland security is officially defined by the National Strategy for Homeland Security as "a concerted national effort to prevent terrorist attacks within the United States, reduce America\'s vulnerability to terrorism, and minimize the damage and recover from attacks that do occur".',
+        'With this information, scientists can produce scenarios of how greenhouse gas emissions may vary in the future.',
+        'Some models of modern climate exhibit Arctic amplification without changes in snow and ice cover.',
     ]
 )
 # [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
@@ -158,13 +158,13 @@ You can finetune this model on your own dataset.
 
 | Metric                | Value      |
 |:----------------------|:-----------|
-| accuracy              | 0.5636     |
-| accuracy_threshold    | 0.5124     |
-| f1                    | 0.6711     |
-| f1_threshold          | 0.4494     |
-| precision             | 0.5128     |
-| recall                | 0.9709     |
-| **average_precision** | **0.5657** |
+| accuracy              | 0.6        |
+| accuracy_threshold    | 0.5077     |
+| f1                    | 0.6791     |
+| f1_threshold          | 0.4203     |
+| precision             | 0.5236     |
+| recall                | 0.9661     |
+| **average_precision** | **0.6083** |
 
 <!--
 ## Bias, Risks and Limitations
@@ -190,13 +190,13 @@ You can finetune this model on your own dataset.
   |         | sentence_0                                                                        | sentence_1                                                                         | label                                                          |
   |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------|
   | type    | string                                                                            | string                                                                             | float                                                          |
-  | details | <ul><li>min: 7 tokens</li><li>mean: 26.73 tokens</li><li>max: 66 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 31.55 tokens</li><li>max: 133 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.53</li><li>max: 1.0</li></ul> |
+  | details | <ul><li>min: 7 tokens</li><li>mean: 26.66 tokens</li><li>max: 80 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 31.89 tokens</li><li>max: 256 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.52</li><li>max: 1.0</li></ul> |
 * Samples:
-  | sentence_0                                                                                                                                         | sentence_1                                                                                                                                                                                                    | label            |
-  |:---------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------|
-  | <code>An independent inquiry found CRU is a small research unit with limited resources and their rigour and honesty are not in doubt.</code>       | <code>The media and other scientific organisations were criticised for having "sometimes neglected" to reflect the uncertainties, doubts and assumptions of the work done by the CRU.</code>                  | <code>0.0</code> |
-  | <code>As president, Obama will immediately close the Mississippi River Gulf Outlet, which experts say funneled floodwater into New Orleans.</code> | <code>Levees along the MRGO and the Intracoastal Waterway were breached in approximately 20 places, directly flooding most of St. Bernard Parish and New Orleans East.</code>                                 | <code>1.0</code> |
-  | <code>If we double atmospheric carbon dioxide[…] we’d only raise global surface temperatures by about a degree Celsius.</code>                     | <code>Not only do increasing carbon dioxide concentrations lead to increases in global surface temperature, but increasing global temperatures also cause increasing concentrations of carbon dioxide.</code> | <code>0.0</code> |
+  | sentence_0                                                                                                                                                                                                               | sentence_1                                                                                                                                                                                                                                                                                               | label            |
+  |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------|
+  | <code>The last time the planet was even four degrees warmer, Peter Brannen points out in The Ends of the World, his new history of the planet’s major extinction events, the oceans were hundreds of feet higher.</code> | <code>Almost all scientists acknowledge that the rate of species loss is greater now than at any time in human history, with extinctions occurring at rates hundreds of times higher than background extinction rates.</code>                                                                            | <code>0.0</code> |
+  | <code>[S]unspot activity on the surface of our star has dropped to a new low.</code>                                                                                                                                     | <code>At solar-cycle minimum, the toroidal field is, correspondingly, at minimum strength, sunspots are relatively rare, and the poloidal field is at its maximum strength.</code>                                                                                                                       | <code>1.0</code> |
+  | <code>More money is dedicated within the Department of Homeland Security to climate change than what's spent combating "Islamist terrorists radicalizing over the Internet in the United States of America."</code>      | <code>Homeland security is officially defined by the National Strategy for Homeland Security as "a concerted national effort to prevent terrorist attacks within the United States, reduce America's vulnerability to terrorism, and minimize the damage and recover from attacks that do occur".</code> | <code>1.0</code> |
 * Loss: [<code>BinaryCrossEntropyLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#binarycrossentropyloss) with these parameters:
   ```json
   {
@@ -322,10 +322,14 @@ You can finetune this model on your own dataset.
 | 0.5391 | 1000 | 0.6983        | -                        |
 | 0.8086 | 1500 | 0.6982        | -                        |
 | -1     | -1   | -             | 0.5657                   |
+| 0.2695 | 500  | 0.6835        | -                        |
+| 0.5391 | 1000 | 0.6840        | -                        |
+| 0.8086 | 1500 | 0.6848        | -                        |
+| -1     | -1   | -             | 0.6083                   |
 
 
 ### Training Time
-- **Training**: 7.1 minutes
+- **Training**: 7.2 minutes
 
 ### Framework Versions
 - Python: 3.12.13

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:922a849772df9d09c87547738e9d2e009866a84a654b460b49743b396fc81a74
+oid sha256:97b936db2695b03518b21df64ef8888ab2b08529a82fbaff59162c06c34daf7f
 size 737716172