CE fine-tuned epoch 2/3

README.md

@@ -28,25 +28,25 @@ model-index:
       type: ce-val
     metrics:
     - type: accuracy
-      value: 0.8130841121495327
+      value: 0.8299065420560747
       name: Accuracy
     - type: accuracy_threshold
-      value: -0.2579485774040222
+      value: -0.3896324336528778
       name: Accuracy Threshold
     - type: f1
-      value: 0.8845265588914549
+      value: 0.8914285714285715
       name: F1
     - type: f1_threshold
-      value: -0.43757033348083496
+      value: -1.0047539472579956
       name: F1 Threshold
     - type: precision
-      value: 0.8362445414847162
+      value: 0.8351177730192719
       name: Precision
     - type: recall
-      value: 0.9387254901960784
+      value: 0.9558823529411765
       name: Recall
     - type: average_precision
-      value: 0.9335939639352934
+      value: 0.9535681876950206
       name: Average Precision
 ---
 
@@ -99,25 +99,25 @@ from sentence_transformers import CrossEncoder
 model = CrossEncoder("cross_encoder_model_id")
 # Get scores for pairs of inputs
 pairs = [
-    ['Yet a study published just this week, by the Bjerknes Centre for Climate Research in Bergen, Norway, found that the natural climate system can change abruptly, without the need for any external forces.', "Analysis of the layering and chemical composition of the cores has provided a revolutionary new record of climate change in the Northern Hemisphere going back about 100,000\xa0years and illustrated that the world's weather and temperature have often shifted rapidly from one seemingly stable state to another, with worldwide consequences."],
-    ['Schmittner finds low climate sensitivity.', 'Herbaceous perennials are also able to tolerate the extremes of cold in temperate and Arctic winters, with less sensitivity than trees or shrubs.'],
-    ['Meeting the 2025 emissions reduction target alone could subtract $250 billion from our GDP and eliminate 2.7 million jobs.', 'Judging by the continued growth in the Renewable Fuel Standard and the extension of the biodiesel tax incentive, the number of jobs can increase to 50,725, $2.7 billion in income, and reaching $5 billion in GDP by 2012 and 2013.'],
-    ['Preventing global warming is relatively cheap; business-as-usual will cause accelerating climate damage costs that economists struggle to even estimate.', 'Researchers have warned that current economic modeling may seriously underestimate the impact of potentially catastrophic climate change and point to the need for new models that give a more accurate picture of potential damages.'],
-    ['Domino-effect of climate events could move Earth into a ‘hothouse’ state', 'This one change creates a domino effect which alters many events and characters in the DC Universe.'],
+    ['Climate change is a hoax invented by the Chinese.', '"The concept of global warming was created by and for the Chinese in order to make U.S. manufacturing non-competitive" (Tweet).'],
+    ['“The oceans, which absorb more than 90% of the extra CO2 pumped into the atmosphere“', 'Most of the CO 2 taken up by the ocean, which is about 30% of the total released into the atmosphere, forms carbonic acid in equilibrium with bicarbonate.'],
+    ['“The jet stream forms a boundary between the cold north and the warmer south, but the lower temperature difference means the winds are now weaker.', 'Therefore, the strong eastward moving jet streams are in part a simple consequence of the fact that the Equator is warmer than the North and South poles.'],
+    ['climate models predict too much warming in the troposphere', 'While the satellite data now show global warming, there is still some difference between what climate models predict and what the satellite data show for warming of the lower troposphere, with the climate models predicting slightly more warming than what the satellites measure.'],
+    ['Nine years into that 11-year hurricane drought, a NASA scientist computed it as a 1-in-177-year event.', 'It is approximately 177 light years from the Earth.'],
 ]
 scores = model.predict(pairs)
 print(scores)
-# [ 3.0222  1.1202  2.4127  1.3557 -0.496 ]
+# [ 3.6842  2.0192  1.9189 -0.9618  0.3676]
 
 # Or rank different texts based on similarity to a single text
 ranks = model.rank(
-    'Yet a study published just this week, by the Bjerknes Centre for Climate Research in Bergen, Norway, found that the natural climate system can change abruptly, without the need for any external forces.',
+    'Climate change is a hoax invented by the Chinese.',
     [
-        "Analysis of the layering and chemical composition of the cores has provided a revolutionary new record of climate change in the Northern Hemisphere going back about 100,000\xa0years and illustrated that the world's weather and temperature have often shifted rapidly from one seemingly stable state to another, with worldwide consequences.",
-        'Herbaceous perennials are also able to tolerate the extremes of cold in temperate and Arctic winters, with less sensitivity than trees or shrubs.',
-        'Judging by the continued growth in the Renewable Fuel Standard and the extension of the biodiesel tax incentive, the number of jobs can increase to 50,725, $2.7 billion in income, and reaching $5 billion in GDP by 2012 and 2013.',
-        'Researchers have warned that current economic modeling may seriously underestimate the impact of potentially catastrophic climate change and point to the need for new models that give a more accurate picture of potential damages.',
-        'This one change creates a domino effect which alters many events and characters in the DC Universe.',
+        '"The concept of global warming was created by and for the Chinese in order to make U.S. manufacturing non-competitive" (Tweet).',
+        'Most of the CO 2 taken up by the ocean, which is about 30% of the total released into the atmosphere, forms carbonic acid in equilibrium with bicarbonate.',
+        'Therefore, the strong eastward moving jet streams are in part a simple consequence of the fact that the Equator is warmer than the North and South poles.',
+        'While the satellite data now show global warming, there is still some difference between what climate models predict and what the satellite data show for warming of the lower troposphere, with the climate models predicting slightly more warming than what the satellites measure.',
+        'It is approximately 177 light years from the Earth.',
     ]
 )
 # [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
@@ -158,13 +158,13 @@ You can finetune this model on your own dataset.
 
 | Metric                | Value      |
 |:----------------------|:-----------|
-| accuracy              | 0.8131     |
-| accuracy_threshold    | -0.2579    |
-| f1                    | 0.8845     |
-| f1_threshold          | -0.4376    |
-| precision             | 0.8362     |
-| recall                | 0.9387     |
-| **average_precision** | **0.9336** |
+| accuracy              | 0.8299     |
+| accuracy_threshold    | -0.3896    |
+| f1                    | 0.8914     |
+| f1_threshold          | -1.0048    |
+| precision             | 0.8351     |
+| recall                | 0.9559     |
+| **average_precision** | **0.9536** |
 
 <!--
 ## Bias, Risks and Limitations
@@ -187,16 +187,16 @@ You can finetune this model on your own dataset.
 * Size: 4,815 training samples
 * Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>label</code>
 * Approximate statistics based on the first 1000 samples:
-  |         | sentence_0                                                                       | sentence_1                                                                         | label                                                          |
-  |:--------|:---------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------|
-  | type    | string                                                                           | string                                                                             | float                                                          |
-  | details | <ul><li>min: 7 tokens</li><li>mean: 26.5 tokens</li><li>max: 66 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 34.21 tokens</li><li>max: 333 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.77</li><li>max: 1.0</li></ul> |
+  |         | sentence_0                                                                        | sentence_1                                                                        | label                                                          |
+  |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------|
+  | type    | string                                                                            | string                                                                            | float                                                          |
+  | details | <ul><li>min: 7 tokens</li><li>mean: 27.21 tokens</li><li>max: 73 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 35.1 tokens</li><li>max: 333 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.78</li><li>max: 1.0</li></ul> |
 * Samples:
-  | sentence_0                                                                                                                                                                                                             | sentence_1                                                                                                                                                                                                                                                                                                                                                   | label            |
-  |:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------|
-  | <code>Yet a study published just this week, by the Bjerknes Centre for Climate Research in Bergen, Norway, found that the natural climate system can change abruptly, without the need for any external forces.</code> | <code>Analysis of the layering and chemical composition of the cores has provided a revolutionary new record of climate change in the Northern Hemisphere going back about 100,000 years and illustrated that the world's weather and temperature have often shifted rapidly from one seemingly stable state to another, with worldwide consequences.</code> | <code>1.0</code> |
-  | <code>Schmittner finds low climate sensitivity.</code>                                                                                                                                                                 | <code>Herbaceous perennials are also able to tolerate the extremes of cold in temperate and Arctic winters, with less sensitivity than trees or shrubs.</code>                                                                                                                                                                                               | <code>1.0</code> |
-  | <code>Meeting the 2025 emissions reduction target alone could subtract $250 billion from our GDP and eliminate 2.7 million jobs.</code>                                                                                | <code>Judging by the continued growth in the Renewable Fuel Standard and the extension of the biodiesel tax incentive, the number of jobs can increase to 50,725, $2.7 billion in income, and reaching $5 billion in GDP by 2012 and 2013.</code>                                                                                                            | <code>1.0</code> |
+  | sentence_0                                                                                                                                                      | sentence_1                                                                                                                                                              | label            |
+  |:----------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------|
+  | <code>Climate change is a hoax invented by the Chinese.</code>                                                                                                  | <code>"The concept of global warming was created by and for the Chinese in order to make U.S. manufacturing non-competitive" (Tweet).</code>                            | <code>1.0</code> |
+  | <code>“The oceans, which absorb more than 90% of the extra CO2 pumped into the atmosphere“</code>                                                               | <code>Most of the CO 2 taken up by the ocean, which is about 30% of the total released into the atmosphere, forms carbonic acid in equilibrium with bicarbonate.</code> | <code>1.0</code> |
+  | <code>“The jet stream forms a boundary between the cold north and the warmer south, but the lower temperature difference means the winds are now weaker.</code> | <code>Therefore, the strong eastward moving jet streams are in part a simple consequence of the fact that the Equator is warmer than the North and South poles.</code>  | <code>1.0</code> |
 * Loss: [<code>BinaryCrossEntropyLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#binarycrossentropyloss) with these parameters:
   ```json
   {
@@ -317,11 +317,11 @@ You can finetune this model on your own dataset.
 ### Training Logs
 | Epoch | Step | ce-val_average_precision |
 |:-----:|:----:|:------------------------:|
-| 1.0   | 301  | 0.9336                   |
+| 1.0   | 301  | 0.9536                   |
 
 
 ### Training Time
-- **Training**: 24.8 seconds
+- **Training**: 27.9 seconds
 
 ### Framework Versions
 - Python: 3.12.13

eval/CrossEncoderClassificationEvaluator_ce-val_results.csv

@@ -1,2 +1,3 @@
 epoch,steps,Accuracy,Accuracy_Threshold,F1,F1_Threshold,Precision,Recall,Average_Precision
 1.0,301,0.8130841121495327,-0.25794858,0.8845265588914549,-0.43757033,0.8362445414847162,0.9387254901960784,0.9335939639352934
+1.0,301,0.8299065420560747,-0.38963243,0.8914285714285715,-1.004754,0.8351177730192719,0.9558823529411765,0.9535681876950206

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:160442534ea4f7d804bce531ea9d64d7765f92e6df6c366ebfb0002c2043e133
+oid sha256:9e0ca103c6d4a5cde8fa579d1f6f9007149e9cede51d84a9a59e524ea135df68
 size 90866404