CE fine-tuned epoch 2/3 best_val=0.5455

README.md

@@ -28,16 +28,16 @@ model-index:
       type: ce-val
     metrics:
     - type: accuracy
-      value: 0.5284848484848484
+      value: 0.5454545454545454
       name: Accuracy
     - type: accuracy_threshold
-      value: 3.9093470573425293
+      value: 1.165013074874878
       name: Accuracy Threshold
     - type: f1
       value: 0.6677471636952999
       name: F1
     - type: f1_threshold
-      value: -10.441707611083984
+      value: -9.491117477416992
       name: F1 Threshold
     - type: precision
       value: 0.5018270401948843
@@ -46,7 +46,7 @@ model-index:
       value: 0.9975786924939467
       name: Recall
     - type: average_precision
-      value: 0.5295137880648401
+      value: 0.5451929668211959
       name: Average Precision
 ---
 
@@ -99,25 +99,25 @@ from sentence_transformers import CrossEncoder
 model = CrossEncoder("cross_encoder_model_id")
 # Get scores for pairs of inputs
 pairs = [
-    ['They (Clinton and Obama) have never to my knowledge been involved in legislation nor hearings nor engagement on this issue (climate change).', 'Gore has been involved with environmental issues since 1976, when as a freshman congressman, he held the "first congressional hearings on the climate change, and co-sponsor[ed] hearings on toxic waste and global warming."'],
-    ['This increase is the result of humans emitting more carbon dioxide into the atmosphere and hence more being absorbed into the oceans.', 'Humans have a substantial influence on the rise of sea level because we emit increasing levels of carbon dioxide into the atmosphere through automobile use and industry.'],
-    ["Venus doesn't have a runaway greenhouse effect", "More recent studies have suggested that several billion years ago, Venus's atmosphere was much more like Earth's than it is now and that there were probably substantial quantities of liquid water on the surface, but a runaway greenhouse effect was caused by the evaporation of that original water, which generated a critical level of greenhouse gases in its atmosphere."],
-    ['At four degrees, the deadly European heat wave of 2003, which killed as many as 2,000 people a day, will be a normal summer.', 'For comparison, the 2003 European heat wave killed an estimated 35,000–70,000 people, with temperatures slightly less than in India and Pakistan.'],
-    ['Under the most ambitious scenarios, they found a strong likelihood that Antarctica would remain fairly stable.”', 'Its remains have been found in Africa, Antarctica, Europe, and North America.'],
+    ['The last time the planet was even four degrees warmer, Peter Brannen points out in The Ends of the World, his new history of the planet’s major extinction events, the oceans were hundreds of feet higher.', 'Almost all scientists acknowledge that the rate of species loss is greater now than at any time in human history, with extinctions occurring at rates hundreds of times higher than background extinction rates.'],
+    ['[S]unspot activity on the surface of our star has dropped to a new low.', "Patches of the star's surface with a lower temperature and luminosity than average are known as starspots."],
+    ['More money is dedicated within the Department of Homeland Security to climate change than what\'s spent combating "Islamist terrorists radicalizing over the Internet in the United States of America."', "The center works on the Internet's routing infrastructure (the SPRI program) and Domain Name System (DNSSEC), identity theft and other online criminal activity (ITTC), Internet traffic and networks research (PREDICT datasets and the DETER testbed), Department of Defense and HSARPA exercises (Livewire and Determined Promise), and wireless security in cooperation with Canada."],
+    ['Worst-case global heating scenarios may need to be revised upwards in light of a better understanding of the role of clouds, scientists have said.', 'With this information, scientists can produce scenarios of how greenhouse gas emissions may vary in the future.'],
+    ['Prof Adam Scaife, a climate modelling expert at the UK’s Met Office, said the evidence for a link to shrinking Arctic ice was now good: ‘The consensus points towards that being a real effect.’”', 'Some models of modern climate exhibit Arctic amplification without changes in snow and ice cover.'],
 ]
 scores = model.predict(pairs)
 print(scores)
-# [ 2.0512  5.9885  7.7773  5.7437 -3.7705]
+# [-0.2177  0.5965 -3.8169 -1.2369  0.4599]
 
 # Or rank different texts based on similarity to a single text
 ranks = model.rank(
-    'They (Clinton and Obama) have never to my knowledge been involved in legislation nor hearings nor engagement on this issue (climate change).',
+    'The last time the planet was even four degrees warmer, Peter Brannen points out in The Ends of the World, his new history of the planet’s major extinction events, the oceans were hundreds of feet higher.',
     [
-        'Gore has been involved with environmental issues since 1976, when as a freshman congressman, he held the "first congressional hearings on the climate change, and co-sponsor[ed] hearings on toxic waste and global warming."',
-        'Humans have a substantial influence on the rise of sea level because we emit increasing levels of carbon dioxide into the atmosphere through automobile use and industry.',
-        "More recent studies have suggested that several billion years ago, Venus's atmosphere was much more like Earth's than it is now and that there were probably substantial quantities of liquid water on the surface, but a runaway greenhouse effect was caused by the evaporation of that original water, which generated a critical level of greenhouse gases in its atmosphere.",
-        'For comparison, the 2003 European heat wave killed an estimated 35,000–70,000 people, with temperatures slightly less than in India and Pakistan.',
-        'Its remains have been found in Africa, Antarctica, Europe, and North America.',
+        'Almost all scientists acknowledge that the rate of species loss is greater now than at any time in human history, with extinctions occurring at rates hundreds of times higher than background extinction rates.',
+        "Patches of the star's surface with a lower temperature and luminosity than average are known as starspots.",
+        "The center works on the Internet's routing infrastructure (the SPRI program) and Domain Name System (DNSSEC), identity theft and other online criminal activity (ITTC), Internet traffic and networks research (PREDICT datasets and the DETER testbed), Department of Defense and HSARPA exercises (Livewire and Determined Promise), and wireless security in cooperation with Canada.",
+        'With this information, scientists can produce scenarios of how greenhouse gas emissions may vary in the future.',
+        'Some models of modern climate exhibit Arctic amplification without changes in snow and ice cover.',
     ]
 )
 # [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
@@ -158,13 +158,13 @@ You can finetune this model on your own dataset.
 
 | Metric                | Value      |
 |:----------------------|:-----------|
-| accuracy              | 0.5285     |
-| accuracy_threshold    | 3.9093     |
+| accuracy              | 0.5455     |
+| accuracy_threshold    | 1.165      |
 | f1                    | 0.6677     |
-| f1_threshold          | -10.4417   |
+| f1_threshold          | -9.4911    |
 | precision             | 0.5018     |
 | recall                | 0.9976     |
-| **average_precision** | **0.5295** |
+| **average_precision** | **0.5452** |
 
 <!--
 ## Bias, Risks and Limitations
@@ -190,13 +190,13 @@ You can finetune this model on your own dataset.
   |         | sentence_0                                                                        | sentence_1                                                                         | label                                                          |
   |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------|
   | type    | string                                                                            | string                                                                             | float                                                          |
-  | details | <ul><li>min: 7 tokens</li><li>mean: 27.81 tokens</li><li>max: 82 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 32.86 tokens</li><li>max: 247 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.49</li><li>max: 1.0</li></ul> |
+  | details | <ul><li>min: 7 tokens</li><li>mean: 27.57 tokens</li><li>max: 82 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 33.03 tokens</li><li>max: 333 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.52</li><li>max: 1.0</li></ul> |
 * Samples:
-  | sentence_0                                                                                                                                                | sentence_1                                                                                                                                                                                                                                                                                                                                                                                     | label            |
-  |:----------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------|
-  | <code>They (Clinton and Obama) have never to my knowledge been involved in legislation nor hearings nor engagement on this issue (climate change).</code> | <code>Gore has been involved with environmental issues since 1976, when as a freshman congressman, he held the "first congressional hearings on the climate change, and co-sponsor[ed] hearings on toxic waste and global warming."</code>                                                                                                                                                     | <code>1.0</code> |
-  | <code>This increase is the result of humans emitting more carbon dioxide into the atmosphere and hence more being absorbed into the oceans.</code>        | <code>Humans have a substantial influence on the rise of sea level because we emit increasing levels of carbon dioxide into the atmosphere through automobile use and industry.</code>                                                                                                                                                                                                         | <code>0.0</code> |
-  | <code>Venus doesn't have a runaway greenhouse effect</code>                                                                                               | <code>More recent studies have suggested that several billion years ago, Venus's atmosphere was much more like Earth's than it is now and that there were probably substantial quantities of liquid water on the surface, but a runaway greenhouse effect was caused by the evaporation of that original water, which generated a critical level of greenhouse gases in its atmosphere.</code> | <code>0.0</code> |
+  | sentence_0                                                                                                                                                                                                               | sentence_1                                                                                                                                                                                                                                                                                                                                                                                            | label            |
+  |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------|
+  | <code>The last time the planet was even four degrees warmer, Peter Brannen points out in The Ends of the World, his new history of the planet’s major extinction events, the oceans were hundreds of feet higher.</code> | <code>Almost all scientists acknowledge that the rate of species loss is greater now than at any time in human history, with extinctions occurring at rates hundreds of times higher than background extinction rates.</code>                                                                                                                                                                         | <code>0.0</code> |
+  | <code>[S]unspot activity on the surface of our star has dropped to a new low.</code>                                                                                                                                     | <code>Patches of the star's surface with a lower temperature and luminosity than average are known as starspots.</code>                                                                                                                                                                                                                                                                               | <code>1.0</code> |
+  | <code>More money is dedicated within the Department of Homeland Security to climate change than what's spent combating "Islamist terrorists radicalizing over the Internet in the United States of America."</code>      | <code>The center works on the Internet's routing infrastructure (the SPRI program) and Domain Name System (DNSSEC), identity theft and other online criminal activity (ITTC), Internet traffic and networks research (PREDICT datasets and the DETER testbed), Department of Defense and HSARPA exercises (Livewire and Determined Promise), and wireless security in cooperation with Canada.</code> | <code>1.0</code> |
 * Loss: [<code>BinaryCrossEntropyLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#binarycrossentropyloss) with these parameters:
   ```json
   {
@@ -317,11 +317,11 @@ You can finetune this model on your own dataset.
 ### Training Logs
 | Epoch | Step | ce-val_average_precision |
 |:-----:|:----:|:------------------------:|
-| -1    | -1   | 0.5295                   |
+| -1    | -1   | 0.5452                   |
 
 
 ### Training Time
-- **Training**: 34.4 seconds
+- **Training**: 34.1 seconds
 
 ### Framework Versions
 - Python: 3.12.13

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c50a517d1275313dcf63127bb3c7e7282772be50cbbe9f885894b98627ea0d19
+oid sha256:9f5527a4edf90324016218b6ec6a80d61152677cf3287945a911f9a26de6a6bb
 size 90866404