geolip-hypersphere-experiments / hypersphere_convergence.txt
AbstractPhil's picture
Create hypersphere_convergence.txt
48e9436 verified
======================================================================
GEOLIP HYPERSPHERE STRUCTURAL ANALYSIS
Checkpoint: checkpoints/dual_stream_best.pt
Device: cuda
======================================================================
Loading checkpoint...
mAP=0.838 epoch=17
D_ANCHOR=256 N_ANCHORS=512 N_EXPERTS=3
Anchors: torch.Size([512, 256])
Expert rotations: N/A (dual-stream — perspectives in projectors)
Loading expert features...
Generating embeddings...
Embeddings: torch.Size([5000, 256])
======================================================================
SCAN 1: ANCHOR GEOMETRY
======================================================================
Pairwise cosine:
mean=0.0001 std=0.0442
max=0.5800 min=-0.5539
Max neighbor cosine per anchor:
mean=0.2330 std=0.0823
max=0.5800 min=0.1204
pairs with cos > 0.9: 0 (0.00%)
pairs with cos > 0.8: 0 (0.00%)
pairs with cos > 0.7: 0 (0.00%)
pairs with cos > 0.5: 3 (0.00%)
pairs with cos > 0.3: 47 (0.04%)
pairs with cos > 0.0: 64430 (49.25%)
Anchor spectral:
effective rank: 256.0/256
sv_max=1.4151 sv_10=1.4146 sv_50=1.4145 sv_min=1.413725
top-10 SVs explain 3.9%
top-25 SVs explain 9.8%
top-50 SVs explain 19.5%
top-100 SVs explain 39.1%
top-128 SVs explain 50.0%
top-200 SVs explain 78.1%
Anchor pentachoron CV: 0.0286
mean_vol=0.092596 std_vol=0.002649
======================================================================
SCAN 2: ANCHOR UTILIZATION
======================================================================
Active anchors: 201/512 (39.3%)
Visit counts: mean=9.8 std=25.6
max=264 min=1 (among active)
top 10: [np.float32(264.0), np.float32(194.0), np.float32(161.0), np.float32(140.0), np.float32(129.0), np.float32(114.0), np.float32(111.0), np.float32(103.0), np.float32(96.0), np.float32(90.0)]
Entropy: 4.5776 / 6.2383 (73.4%)
Gini coefficient: 0.8568 (0=equal, 1=one anchor gets all)
anchors with 1-5 visits: 66
anchors with 5-20 visits: 64
anchors with 20-50 visits: 39
anchors with 50-100 visits: 24
anchors with 100-500 visits: 8
anchors with 500-5000 visits: 0
======================================================================
SCAN 3: EMBEDDING MANIFOLD GEOMETRY
======================================================================
Effective dimensionality: 71.6/256
top-5 SVs explain 33.5%
top-10 SVs explain 51.1%
top-20 SVs explain 72.6%
top-50 SVs explain 94.2%
top-100 SVs explain 99.4%
top-128 SVs explain 99.8%
top-200 SVs explain 100.0%
Self-similarity (off-diagonal):
mean=0.0025 std=0.1920
max=0.9931 min=-0.5248
Norms: mean=1.000000 std=0.000000
Global pentachoron CV: 0.2199
mean_vol=0.080570 std_vol=0.017715
======================================================================
SCAN 4: EXPERT PERSPECTIVE DIVERGENCE
======================================================================
Per-image expert agreement:
clip_l14_openai × dinov2_b14 : mean=1.0000 std=0.0000 min=1.0000
clip_l14_openai × siglip_b16_384 : mean=1.0000 std=0.0000 min=1.0000
dinov2_b14 × siglip_b16_384 : mean=1.0000 std=0.0000 min=1.0000
Per-anchor expert divergence:
mean divergence: 0.0000 std: 0.0000
max divergence: 0.0000 (anchor 0)
min divergence: 0.0000 (anchor 0)
Top 10 most contentious anchors:
#1 anchor 39: div=0.0000 visits=0
#2 anchor 38: div=0.0000 visits=0
#3 anchor 37: div=0.0000 visits=0
#4 anchor 36: div=0.0000 visits=0
#5 anchor 35: div=0.0000 visits=0
#6 anchor 34: div=0.0000 visits=0
#7 anchor 33: div=0.0000 visits=0
#8 anchor 32: div=0.0000 visits=2
#9 anchor 47: div=0.0000 visits=0
#10 anchor 46: div=0.0000 visits=1
Top 10 most unanimous anchors:
#1 anchor 504: div=0.0000 visits=0
#2 anchor 505: div=0.0000 visits=2
#3 anchor 506: div=0.0000 visits=27
#4 anchor 507: div=0.0000 visits=17
#5 anchor 508: div=0.0000 visits=4
#6 anchor 509: div=0.0000 visits=0
#7 anchor 510: div=0.0000 visits=0
#8 anchor 511: div=0.0000 visits=0
#9 anchor 496: div=0.0000 visits=0
#10 anchor 497: div=0.0000 visits=0
Expert rotation eigenspectra:
clip_l14_openai : ortho_err=0.000000 eval_min=1.0000 eval_max=1.0000
dinov2_b14 : ortho_err=0.000000 eval_min=1.0000 eval_max=1.0000
siglip_b16_384 : ortho_err=0.000000 eval_min=1.0000 eval_max=1.0000
Expert whitener condition:
clip_l14_openai : cond=1.00 sv_max=1.0000 sv_min=1.000000
dinov2_b14 : cond=1.00 sv_max=1.0000 sv_min=1.000000
siglip_b16_384 : cond=1.00 sv_max=1.0000 sv_min=1.000000
======================================================================
SCAN 5: NEAREST ANCHOR DISTANCES
======================================================================
k= 0: mean_dist=0.5721 std=0.0978 max=0.8074 min=0.2505
k= 1: mean_dist=0.6521 std=0.0692 max=0.8282 min=0.4311
k= 2: mean_dist=0.6931 std=0.0559 max=0.8344 min=0.5052
k= 4: mean_dist=0.7466 std=0.0436 max=0.8548 min=0.5865
k= 9: mean_dist=0.8172 std=0.0243 max=0.8985 min=0.7391
k= 19: mean_dist=0.8814 std=0.0183 max=0.9495 min=0.8327
k= 49: mean_dist=0.9374 std=0.0132 max=0.9738 min=0.9056
k= 99: mean_dist=0.9671 std=0.0084 max=0.9896 min=0.9445
anchors with cos > 0.9: mean=0.0 max=0 min=0
anchors with cos > 0.8: mean=0.0 max=0 min=0
anchors with cos > 0.7: mean=0.0 max=1 min=0
anchors with cos > 0.5: mean=0.3 max=2 min=0
anchors with cos > 0.3: mean=2.7 max=8 min=0
anchors with cos > 0.0: mean=227.2 max=284 min=148
======================================================================
SCAN 6: PER-CLASS ANCHOR AFFINITY
======================================================================
Anchor specialization:
classes per active anchor: mean=12.8 std=9.4
max=40 min=1
Class spread (anchors per class):
mean=32.2 std=28.8
max=170 (person)
min=2 (bear)
Top 10 by anchor spread:
person : 170 anchors, 2693 images
chair : 105 anchors, 580 images
car : 98 anchors, 535 images
bottle : 85 anchors, 379 images
dining table : 84 anchors, 501 images
handbag : 81 anchors, 292 images
cup : 80 anchors, 390 images
backpack : 76 anchors, 228 images
truck : 66 anchors, 250 images
book : 65 anchors, 230 images
Bottom 10 by anchor spread:
sheep : 6 anchors, 65 images
tennis racket : 5 anchors, 167 images
giraffe : 5 anchors, 101 images
hair drier : 4 anchors, 9 images
toaster : 4 anchors, 8 images
baseball glove : 4 anchors, 100 images
elephant : 4 anchors, 89 images
skis : 4 anchors, 120 images
zebra : 3 anchors, 85 images
bear : 2 anchors, 49 images
======================================================================
SCAN 7: INTER-CLASS GEOMETRIC DISTANCES
======================================================================
Inter-class cosine (78 classes with >10 images):
mean=0.0531
max=0.9947 min=-0.4158
Most similar class pairs:
#1: mouse × keyboard cos=0.9947
#2: baseball bat × baseball glove cos=0.9905
#3: microwave × refrigerator cos=0.9783
#4: spoon × bowl cos=0.9757
#5: skis × snowboard cos=0.9707
#6: microwave × oven cos=0.9615
#7: knife × dining table cos=0.9582
#8: cup × dining table cos=0.9580
#9: laptop × keyboard cos=0.9577
#10: fork × knife cos=0.9565
#11: laptop × mouse cos=0.9531
#12: oven × refrigerator cos=0.9510
#13: knife × spoon cos=0.9489
#14: apple × orange cos=0.9461
#15: broccoli × carrot cos=0.9433
Most distant class pairs:
#1: person × zebra cos=-0.4158
#2: person × giraffe cos=-0.3890
#3: airplane × bottle cos=-0.3845
#4: snowboard × potted plant cos=-0.3742
#5: snowboard × chair cos=-0.3692
#6: snowboard × vase cos=-0.3677
#7: bench × toothbrush cos=-0.3641
#8: skis × potted plant cos=-0.3635
#9: truck × toothbrush cos=-0.3629
#10: bird × bottle cos=-0.3616
#11: airplane × cup cos=-0.3575
#12: car × toothbrush cos=-0.3572
#13: skis × vase cos=-0.3550
#14: skis × chair cos=-0.3484
#15: horse × bottle cos=-0.3445
Intra-class spread:
Tightest 10:
giraffe : spread=0.0566 (n=101)
zebra : spread=0.0626 (n=85)
tennis racket : spread=0.0785 (n=167)
baseball glove : spread=0.0871 (n=100)
skis : spread=0.0880 (n=120)
elephant : spread=0.1058 (n=89)
snowboard : spread=0.1145 (n=49)
skateboard : spread=0.1188 (n=127)
surfboard : spread=0.1377 (n=149)
baseball bat : spread=0.1512 (n=97)
Loosest 10:
potted plant : spread=0.5553 (n=172)
umbrella : spread=0.5566 (n=174)
cup : spread=0.5587 (n=390)
car : spread=0.5765 (n=535)
bottle : spread=0.6014 (n=379)
handbag : spread=0.6207 (n=292)
bench : spread=0.6600 (n=235)
chair : spread=0.6621 (n=580)
backpack : spread=0.6764 (n=228)
person : spread=0.8186 (n=2693)
======================================================================
SCAN 8: LOCAL PENTACHORON CV
======================================================================
Clusters with 10+ members: 102
Local CV: mean=0.4302 std=0.1654
max=1.0834 min=0.1965
Global CV: 0.2199
Ratio (local/global): 1.9564
Highest local CV (most diverse clusters):
anchor 86: CV=1.0834 n= 103 mean_vol=0.002920
anchor 469: CV=0.8698 n= 89 mean_vol=0.000587
anchor 364: CV=0.8659 n= 194 mean_vol=0.007823
anchor 175: CV=0.8403 n= 88 mean_vol=0.001932
anchor 68: CV=0.8222 n= 161 mean_vol=0.000757
anchor 220: CV=0.7987 n= 114 mean_vol=0.001308
anchor 437: CV=0.7678 n= 129 mean_vol=0.001834
anchor 498: CV=0.7410 n= 78 mean_vol=0.001262
anchor 205: CV=0.6792 n= 48 mean_vol=0.001457
anchor 339: CV=0.6555 n= 11 mean_vol=0.003612
Lowest local CV (most uniform clusters):
anchor 361: CV=0.2631 n= 41 mean_vol=0.012367
anchor 254: CV=0.2615 n= 32 mean_vol=0.019875
anchor 155: CV=0.2565 n= 34 mean_vol=0.017756
anchor 53: CV=0.2527 n= 12 mean_vol=0.025066
anchor 479: CV=0.2503 n= 43 mean_vol=0.021270
anchor 157: CV=0.2354 n= 11 mean_vol=0.031356
anchor 289: CV=0.2137 n= 18 mean_vol=0.018227
anchor 430: CV=0.2126 n= 15 mean_vol=0.041118
anchor 486: CV=0.2002 n= 11 mean_vol=0.016117
anchor 131: CV=0.1965 n= 14 mean_vol=0.030370
======================================================================
SCAN 9: PROJECTOR ANALYSIS
======================================================================
clip_l14_openai:
self-sim: mean=0.0538 std=0.1757
eff_dim: 76.3/256
cos→fused: mean=0.9160 std=0.0323
dinov2_b14:
self-sim: mean=0.1254 std=0.1798
eff_dim: 74.5/256
cos→fused: mean=0.8665 std=0.0459
siglip_b16_384:
self-sim: mean=0.0332 std=0.1800
eff_dim: 76.3/256
cos→fused: mean=0.9303 std=0.0284
Cross-expert agreement (projected):
clip_l14_openai × dinov2_b14 : cos=0.6581 std=0.1092
clip_l14_openai × siglip_b16_384 : cos=0.8297 std=0.0749
dinov2_b14 × siglip_b16_384 : cos=0.6966 std=0.1019
Expert uniqueness (leave-one-out):
Without clip_l14_openai : cos_to_full=0.9756 (uniqueness=0.0244)
Without dinov2_b14 : cos_to_full=0.9652 (uniqueness=0.0348)
Without siglip_b16_384 : cos_to_full=0.9791 (uniqueness=0.0209)
======================================================================
SCAN 9.5: DUAL-STREAM ANALYSIS
======================================================================
Shared × Native cosine per expert:
clip_l14_openai : mean=0.3337 std=0.0146 min=0.2898 max=0.4650
dinov2_b14 : mean=0.3576 std=0.0256 min=0.2771 max=0.5046
siglip_b16_384 : mean=0.3503 std=0.0191 min=0.2433 max=0.4444
Displacement (shared - native):
clip_l14_openai : L2_mean=1.1543 std=0.0127
dinov2_b14 : L2_mean=1.1332 std=0.0228
siglip_b16_384 : L2_mean=1.1398 std=0.0168
Native effective dimensionality:
clip_l14_openai : eff_dim=71.3/256
dinov2_b14 : eff_dim=62.4/256
siglip_b16_384 : eff_dim=62.8/256
Cross-expert native agreement:
clip_l14_openai × dinov2_b14 : mean=0.2059 std=0.0037
clip_l14_openai × siglip_b16_384 : mean=0.2078 std=0.0041
dinov2_b14 × siglip_b16_384 : mean=0.7933 std=0.0048
Shared × Other's Native (cross-stream):
clip_l14_ope_shared × dinov2_b14 _native: mean=0.1277
clip_l14_ope_shared × siglip_b16_3_native: mean=0.2346
dinov2_b14_shared × clip_l14_ope_native: mean=0.0136
dinov2_b14_shared × siglip_b16_3_native: mean=0.2824
siglip_b16_3_shared × clip_l14_ope_native: mean=0.2211
siglip_b16_3_shared × dinov2_b14 _native: mean=0.1836
Native triangulation divergence from shared:
clip_l14_openai : tri_cos=0.2093 tri_diff=0.0572
dinov2_b14 : tri_cos=0.2464 tri_diff=0.0573
siglip_b16_384 : tri_cos=0.3196 tri_diff=0.0553
Native pairwise triangulation correlation:
clip_l14_openai × dinov2_b14 : mean=0.2059 std=0.0037
clip_l14_openai × siglip_b16_384 : mean=0.2078 std=0.0041
dinov2_b14 × siglip_b16_384 : mean=0.7933 std=0.0048
Information content:
Shared tri eff_dim: 71.6
Combined eff_dim: 126.2
Info gain from native: +54.5 dims
======================================================================
SCAN 10: TRIANGULATION STRUCTURE
======================================================================
clip_l14_openai triangulation:
mean=0.9993 std=0.0625
min=0.2505 max=1.3152
nearest: mean=0.5721 std=0.0978
dinov2_b14 triangulation:
mean=0.9993 std=0.0625
min=0.2505 max=1.3152
nearest: mean=0.5721 std=0.0978
siglip_b16_384 triangulation:
mean=0.9993 std=0.0625
min=0.2505 max=1.3152
nearest: mean=0.5721 std=0.0978
Expert triangulation correlation:
clip_l14_openai × dinov2_b14 : per_img_cos mean=1.0000 std=0.0000
clip_l14_openai × siglip_b16_384 : per_img_cos mean=1.0000 std=0.0000
dinov2_b14 × siglip_b16_384 : per_img_cos mean=1.0000 std=0.0000
======================================================================
SUMMARY
======================================================================
Checkpoint: checkpoints/dual_stream_best.pt
mAP: 0.838
Anchors: 512 × 256-d, 201 active (39%)
Embedding eff_dim: 71.6/256
Anchor eff_rank: 256.0/256
Global CV: 0.2199
Anchor CV: 0.0286
Local CV (mean): 0.4302
Utilization entropy: 73.4%
Utilization Gini: 0.8568
======================================================================
ANALYSIS COMPLETE
======================================================================