🧠
multilingual-e5-large model by intfloat
⭐ 18.3
🧠
multilingual-e5-large
by intfloat Model
ID: hf-model--intfloat--multilingual-e5-large
FNI 18.3
Top 74%
Audited 18.3 FNI Score
Tiny 0.56B Params
4k Context
Hot 3.3M Downloads
8G GPU ~2GB Est. VRAM
⚡ Quick Commands
🦙 Ollama Run
ollama run multilingual-e5-large 🤗 HF Download
huggingface-cli download intfloat/multilingual-e5-large 📊
Engineering Specs
⚡ Hardware
Parameters
0.56B
Architecture
XLMRobertaModel
Context Length
4K
Model Size
9.4GB
🧠 Lifecycle
Library
-
Precision
float16
Tokenizer
-
🌐 Identity
Source
HuggingFace
License
Open Access
💾
Est. VRAM Benchmark
~1.7GB
Analyze Hardware
* Technical estimation for FP16/Q4 weights. Does not include OS overhead or long-context batching. For Technical Reference Only.
🕸️ Neural Mesh Hub
Interconnecting Research, Data & Ecosystem
📈 Interest Trend
--
📡 Knowledge Mesh Synchronizing...
Real-time Trend Indexing In-Progress
Real-time Trend Indexing In-Progress
* Real-time activity index across HuggingFace, GitHub and Research citations.
🔍 Semantic Keywords
🏷️ sentence-transformers 🏷️ pytorch 🏷️ onnx 🏷️ safetensors 🏷️ openvino 🏷️ xlm-roberta 🏷️ mteb 🏷️ sentence transformers 🏷️ sentence-similarity 🏷️ feature-extraction 🏷️ multilingual 🏷️ af 🏷️ am 🏷️ ar 🏷️ as 🏷️ az 🏷️ be 🏷️ bg 🏷️ bn 🏷️ br 🏷️ bs 🏷️ ca 🏷️ cs 🏷️ cy 🏷️ da 🏷️ de 🏷️ el 🏷️ en 🏷️ eo 🏷️ es 🏷️ et 🏷️ eu 🏷️ fa 🏷️ fi 🏷️ fr 🏷️ fy 🏷️ ga 🏷️ gd 🏷️ gl 🏷️ gu 🏷️ ha 🏷️ he 🏷️ hi 🏷️ hr 🏷️ hu 🏷️ hy 🏷️ id 🏷️ is 🏷️ it 🏷️ ja 🏷️ jv 🏷️ ka 🏷️ kk 🏷️ km 🏷️ kn 🏷️ ko 🏷️ ku 🏷️ ky 🏷️ la 🏷️ lo 🏷️ lt 🏷️ lv 🏷️ mg 🏷️ mk 🏷️ ml 🏷️ mn 🏷️ mr 🏷️ ms 🏷️ my 🏷️ ne 🏷️ nl 🏷️ no 🏷️ om 🏷️ or 🏷️ pa 🏷️ pl 🏷️ ps 🏷️ pt 🏷️ ro 🏷️ ru 🏷️ sa 🏷️ sd 🏷️ si 🏷️ sk 🏷️ sl 🏷️ so 🏷️ sq 🏷️ sr 🏷️ su 🏷️ sv 🏷️ sw 🏷️ ta 🏷️ te 🏷️ th 🏷️ tl 🏷️ tr 🏷️ ug 🏷️ uk 🏷️ ur 🏷️ uz 🏷️ vi 🏷️ xh 🏷️ yi 🏷️ zh 🏷️ arxiv:2402.05672 🏷️ arxiv:2108.08787 🏷️ arxiv:2104.08663 🏷️ arxiv:2210.07316 🏷️ license:mit 🏷️ model-index 🏷️ text-embeddings-inference 🏷️ endpoints_compatible 🏷️ deploy:azure 🏷️ region:us
No similar models found.
Social Proof
FNI RankTop 74%
HuggingFace Hub
1.1KLikes
3.3MDownloads
🔬Technical Deep Dive
Full Specifications [+]▾
🚀 What's Next?
⚡ Quick Commands
🦙 Ollama Run
ollama run multilingual-e5-large 🤗 HF Download
huggingface-cli download intfloat/multilingual-e5-large 🖥️
Hardware Compatibility
Multi-Tier Validation Matrix
Live Sync
🎮 Compatible
RTX 3060 / 4060 Ti
Entry 8GB VRAM
🎮 Compatible
RTX 4070 Super
Mid 12GB VRAM
💻 Compatible
RTX 4080 / Mac M3
High 16GB VRAM
🚀 Compatible
RTX 3090 / 4090
Pro 24GB VRAM
🏗️ Compatible
RTX 6000 Ada
Workstation 48GB VRAM
🏭 Compatible
A100 / H100
Datacenter 80GB VRAM
ℹ️
Pro Tip: Compatibility is estimated for 4-bit quantization (Q4). High-precision (FP16) or ultra-long context windows will significantly increase VRAM requirements.
README
tags:
- mteb
- Sentence Transformers
- sentence-similarity
- feature-extraction
- sentence-transformers model-index:
- name: multilingual-e5-large
results:
- task:
type: Classification
dataset:
type: mteb/amazon_counterfactual
name: MTEB AmazonCounterfactualClassification (en)
config: en
split: test
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
metrics:
- type: accuracy value: 79.05970149253731
- type: ap value: 43.486574390835635
- type: f1 value: 73.32700092140148
- task:
type: Classification
dataset:
type: mteb/amazon_counterfactual
name: MTEB AmazonCounterfactualClassification (de)
config: de
split: test
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
metrics:
- type: accuracy value: 71.22055674518201
- type: ap value: 81.55756710830498
- type: f1 value: 69.28271787752661
- task:
type: Classification
dataset:
type: mteb/amazon_counterfactual
name: MTEB AmazonCounterfactualClassification (en-ext)
config: en-ext
split: test
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
metrics:
- type: accuracy value: 80.41979010494754
- type: ap value: 29.34879922376344
- type: f1 value: 67.62475449011278
- task:
type: Classification
dataset:
type: mteb/amazon_counterfactual
name: MTEB AmazonCounterfactualClassification (ja)
config: ja
split: test
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
metrics:
- type: accuracy value: 77.8372591006424
- type: ap value: 26.557560591210738
- type: f1 value: 64.96619417368707
- task:
type: Classification
dataset:
type: mteb/amazon_polarity
name: MTEB AmazonPolarityClassification
config: default
split: test
revision: e2d317d38cd51312af73b3d32a06d1a08b442046
metrics:
- type: accuracy value: 93.489875
- type: ap value: 90.98758636917603
- type: f1 value: 93.48554819717332
- task:
type: Classification
dataset:
type: mteb/amazon_reviews_multi
name: MTEB AmazonReviewsClassification (en)
config: en
split: test
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
metrics:
- type: accuracy value: 47.564
- type: f1 value: 46.75122173518047
- task:
type: Classification
dataset:
type: mteb/amazon_reviews_multi
name: MTEB AmazonReviewsClassification (de)
config: de
split: test
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
metrics:
- type: accuracy value: 45.400000000000006
- type: f1 value: 44.17195682400632
- task:
type: Classification
dataset:
type: mteb/amazon_reviews_multi
name: MTEB AmazonReviewsClassification (es)
config: es
split: test
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
metrics:
- type: accuracy value: 43.068
- type: f1 value: 42.38155696855596
- task:
type: Classification
dataset:
type: mteb/amazon_reviews_multi
name: MTEB AmazonReviewsClassification (fr)
config: fr
split: test
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
metrics:
- type: accuracy value: 41.89
- type: f1 value: 40.84407321682663
- task:
type: Classification
dataset:
type: mteb/amazon_reviews_multi
name: MTEB AmazonReviewsClassification (ja)
config: ja
split: test
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
metrics:
- type: accuracy value: 40.120000000000005
- type: f1 value: 39.522976223819114
- task:
type: Classification
dataset:
type: mteb/amazon_reviews_multi
name: MTEB AmazonReviewsClassification (zh)
config: zh
split: test
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
metrics:
- type: accuracy value: 38.832
- type: f1 value: 38.0392533394713
- task:
type: Retrieval
dataset:
type: arguana
name: MTEB ArguAna
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 30.725
- type: map_at_10 value: 46.055
- type: map_at_100 value: 46.900999999999996
- type: map_at_1000 value: 46.911
- type: map_at_3 value: 41.548
- type: map_at_5 value: 44.297
- type: mrr_at_1 value: 31.152
- type: mrr_at_10 value: 46.231
- type: mrr_at_100 value: 47.07
- type: mrr_at_1000 value: 47.08
- type: mrr_at_3 value: 41.738
- type: mrr_at_5 value: 44.468999999999994
- type: ndcg_at_1 value: 30.725
- type: ndcg_at_10 value: 54.379999999999995
- type: ndcg_at_100 value: 58.138
- type: ndcg_at_1000 value: 58.389
- type: ndcg_at_3 value: 45.156
- type: ndcg_at_5 value: 50.123
- type: precision_at_1 value: 30.725
- type: precision_at_10 value: 8.087
- type: precision_at_100 value: 0.9769999999999999
- type: precision_at_1000 value: 0.1
- type: precision_at_3 value: 18.54
- type: precision_at_5 value: 13.542000000000002
- type: recall_at_1 value: 30.725
- type: recall_at_10 value: 80.868
- type: recall_at_100 value: 97.653
- type: recall_at_1000 value: 99.57300000000001
- type: recall_at_3 value: 55.619
- type: recall_at_5 value: 67.71000000000001
- task:
type: Clustering
dataset:
type: mteb/arxiv-clustering-p2p
name: MTEB ArxivClusteringP2P
config: default
split: test
revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
metrics:
- type: v_measure value: 44.30960650674069
- task:
type: Clustering
dataset:
type: mteb/arxiv-clustering-s2s
name: MTEB ArxivClusteringS2S
config: default
split: test
revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53
metrics:
- type: v_measure value: 38.427074197498996
- task:
type: Reranking
dataset:
type: mteb/askubuntudupquestions-reranking
name: MTEB AskUbuntuDupQuestions
config: default
split: test
revision: 2000358ca161889fa9c082cb41daa8dcfb161a54
metrics:
- type: map value: 60.28270056031872
- type: mrr value: 74.38332673789738
- task:
type: STS
dataset:
type: mteb/biosses-sts
name: MTEB BIOSSES
config: default
split: test
revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
metrics:
- type: cos_sim_pearson value: 84.05942144105269
- type: cos_sim_spearman value: 82.51212105850809
- type: euclidean_pearson value: 81.95639829909122
- type: euclidean_spearman value: 82.3717564144213
- type: manhattan_pearson value: 81.79273425468256
- type: manhattan_spearman value: 82.20066817871039
- task:
type: BitextMining
dataset:
type: mteb/bucc-bitext-mining
name: MTEB BUCC (de-en)
config: de-en
split: test
revision: d51519689f32196a32af33b075a01d0e7c51e252
metrics:
- type: accuracy value: 99.46764091858039
- type: f1 value: 99.37717466945023
- type: precision value: 99.33194154488518
- type: recall value: 99.46764091858039
- task:
type: BitextMining
dataset:
type: mteb/bucc-bitext-mining
name: MTEB BUCC (fr-en)
config: fr-en
split: test
revision: d51519689f32196a32af33b075a01d0e7c51e252
metrics:
- type: accuracy value: 98.29407880255337
- type: f1 value: 98.11248073959938
- type: precision value: 98.02443319392472
- type: recall value: 98.29407880255337
- task:
type: BitextMining
dataset:
type: mteb/bucc-bitext-mining
name: MTEB BUCC (ru-en)
config: ru-en
split: test
revision: d51519689f32196a32af33b075a01d0e7c51e252
metrics:
- type: accuracy value: 97.79009352268791
- type: f1 value: 97.5176076665512
- type: precision value: 97.38136473848286
- type: recall value: 97.79009352268791
- task:
type: BitextMining
dataset:
type: mteb/bucc-bitext-mining
name: MTEB BUCC (zh-en)
config: zh-en
split: test
revision: d51519689f32196a32af33b075a01d0e7c51e252
metrics:
- type: accuracy value: 99.26276987888363
- type: f1 value: 99.20133403545726
- type: precision value: 99.17500438827453
- type: recall value: 99.26276987888363
- task:
type: Classification
dataset:
type: mteb/banking77
name: MTEB Banking77Classification
config: default
split: test
revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
metrics:
- type: accuracy value: 84.72727272727273
- type: f1 value: 84.67672206031433
- task:
type: Clustering
dataset:
type: mteb/biorxiv-clustering-p2p
name: MTEB BiorxivClusteringP2P
config: default
split: test
revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40
metrics:
- type: v_measure value: 35.34220182511161
- task:
type: Clustering
dataset:
type: mteb/biorxiv-clustering-s2s
name: MTEB BiorxivClusteringS2S
config: default
split: test
revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908
metrics:
- type: v_measure value: 33.4987096128766
- task:
type: Retrieval
dataset:
type: BeIR/cqadupstack
name: MTEB CQADupstackRetrieval
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 25.558249999999997
- type: map_at_10 value: 34.44425000000001
- type: map_at_100 value: 35.59833333333333
- type: map_at_1000 value: 35.706916666666665
- type: map_at_3 value: 31.691749999999995
- type: map_at_5 value: 33.252916666666664
- type: mrr_at_1 value: 30.252666666666666
- type: mrr_at_10 value: 38.60675
- type: mrr_at_100 value: 39.42666666666666
- type: mrr_at_1000 value: 39.48408333333334
- type: mrr_at_3 value: 36.17441666666665
- type: mrr_at_5 value: 37.56275
- type: ndcg_at_1 value: 30.252666666666666
- type: ndcg_at_10 value: 39.683
- type: ndcg_at_100 value: 44.68541666666667
- type: ndcg_at_1000 value: 46.94316666666668
- type: ndcg_at_3 value: 34.961749999999995
- type: ndcg_at_5 value: 37.215666666666664
- type: precision_at_1 value: 30.252666666666666
- type: precision_at_10 value: 6.904166666666667
- type: precision_at_100 value: 1.0989999999999995
- type: precision_at_1000 value: 0.14733333333333334
- type: precision_at_3 value: 16.037666666666667
- type: precision_at_5 value: 11.413583333333333
- type: recall_at_1 value: 25.558249999999997
- type: recall_at_10 value: 51.13341666666666
- type: recall_at_100 value: 73.08366666666667
- type: recall_at_1000 value: 88.79483333333334
- type: recall_at_3 value: 37.989083333333326
- type: recall_at_5 value: 43.787833333333325
- task:
type: Retrieval
dataset:
type: climate-fever
name: MTEB ClimateFEVER
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 10.338
- type: map_at_10 value: 18.360000000000003
- type: map_at_100 value: 19.942
- type: map_at_1000 value: 20.134
- type: map_at_3 value: 15.174000000000001
- type: map_at_5 value: 16.830000000000002
- type: mrr_at_1 value: 23.257
- type: mrr_at_10 value: 33.768
- type: mrr_at_100 value: 34.707
- type: mrr_at_1000 value: 34.766000000000005
- type: mrr_at_3 value: 30.977
- type: mrr_at_5 value: 32.528
- type: ndcg_at_1 value: 23.257
- type: ndcg_at_10 value: 25.733
- type: ndcg_at_100 value: 32.288
- type: ndcg_at_1000 value: 35.992000000000004
- type: ndcg_at_3 value: 20.866
- type: ndcg_at_5 value: 22.612
- type: precision_at_1 value: 23.257
- type: precision_at_10 value: 8.124
- type: precision_at_100 value: 1.518
- type: precision_at_1000 value: 0.219
- type: precision_at_3 value: 15.679000000000002
- type: precision_at_5 value: 12.117
- type: recall_at_1 value: 10.338
- type: recall_at_10 value: 31.154
- type: recall_at_100 value: 54.161
- type: recall_at_1000 value: 75.21900000000001
- type: recall_at_3 value: 19.427
- type: recall_at_5 value: 24.214
- task:
type: Retrieval
dataset:
type: dbpedia-entity
name: MTEB DBPedia
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 8.498
- type: map_at_10 value: 19.103
- type: map_at_100 value: 27.375
- type: map_at_1000 value: 28.981
- type: map_at_3 value: 13.764999999999999
- type: map_at_5 value: 15.950000000000001
- type: mrr_at_1 value: 65.5
- type: mrr_at_10 value: 74.53800000000001
- type: mrr_at_100 value: 74.71799999999999
- type: mrr_at_1000 value: 74.725
- type: mrr_at_3 value: 72.792
- type: mrr_at_5 value: 73.554
- type: ndcg_at_1 value: 53.37499999999999
- type: ndcg_at_10 value: 41.286
- type: ndcg_at_100 value: 45.972
- type: ndcg_at_1000 value: 53.123
- type: ndcg_at_3 value: 46.172999999999995
- type: ndcg_at_5 value: 43.033
- type: precision_at_1 value: 65.5
- type: precision_at_10 value: 32.725
- type: precision_at_100 value: 10.683
- type: precision_at_1000 value: 1.978
- type: precision_at_3 value: 50
- type: precision_at_5 value: 41.349999999999994
- type: recall_at_1 value: 8.498
- type: recall_at_10 value: 25.070999999999998
- type: recall_at_100 value: 52.383
- type: recall_at_1000 value: 74.91499999999999
- type: recall_at_3 value: 15.207999999999998
- type: recall_at_5 value: 18.563
- task:
type: Classification
dataset:
type: mteb/emotion
name: MTEB EmotionClassification
config: default
split: test
revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
metrics:
- type: accuracy value: 46.5
- type: f1 value: 41.93833713984145
- task:
type: Retrieval
dataset:
type: fever
name: MTEB FEVER
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 67.914
- type: map_at_10 value: 78.10000000000001
- type: map_at_100 value: 78.333
- type: map_at_1000 value: 78.346
- type: map_at_3 value: 76.626
- type: map_at_5 value: 77.627
- type: mrr_at_1 value: 72.74199999999999
- type: mrr_at_10 value: 82.414
- type: mrr_at_100 value: 82.511
- type: mrr_at_1000 value: 82.513
- type: mrr_at_3 value: 81.231
- type: mrr_at_5 value: 82.065
- type: ndcg_at_1 value: 72.74199999999999
- type: ndcg_at_10 value: 82.806
- type: ndcg_at_100 value: 83.677
- type: ndcg_at_1000 value: 83.917
- type: ndcg_at_3 value: 80.305
- type: ndcg_at_5 value: 81.843
- type: precision_at_1 value: 72.74199999999999
- type: precision_at_10 value: 10.24
- type: precision_at_100 value: 1.089
- type: precision_at_1000 value: 0.11299999999999999
- type: precision_at_3 value: 31.268
- type: precision_at_5 value: 19.706000000000003
- type: recall_at_1 value: 67.914
- type: recall_at_10 value: 92.889
- type: recall_at_100 value: 96.42699999999999
- type: recall_at_1000 value: 97.92
- type: recall_at_3 value: 86.21
- type: recall_at_5 value: 90.036
- task:
type: Retrieval
dataset:
type: fiqa
name: MTEB FiQA2018
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 22.166
- type: map_at_10 value: 35.57
- type: map_at_100 value: 37.405
- type: map_at_1000 value: 37.564
- type: map_at_3 value: 30.379
- type: map_at_5 value: 33.324
- type: mrr_at_1 value: 43.519000000000005
- type: mrr_at_10 value: 51.556000000000004
- type: mrr_at_100 value: 52.344
- type: mrr_at_1000 value: 52.373999999999995
- type: mrr_at_3 value: 48.868
- type: mrr_at_5 value: 50.319
- type: ndcg_at_1 value: 43.519000000000005
- type: ndcg_at_10 value: 43.803
- type: ndcg_at_100 value: 50.468999999999994
- type: ndcg_at_1000 value: 53.111
- type: ndcg_at_3 value: 38.893
- type: ndcg_at_5 value: 40.653
- type: precision_at_1 value: 43.519000000000005
- type: precision_at_10 value: 12.253
- type: precision_at_100 value: 1.931
- type: precision_at_1000 value: 0.242
- type: precision_at_3 value: 25.617
- type: precision_at_5 value: 19.383
- type: recall_at_1 value: 22.166
- type: recall_at_10 value: 51.6
- type: recall_at_100 value: 76.574
- type: recall_at_1000 value: 92.192
- type: recall_at_3 value: 34.477999999999994
- type: recall_at_5 value: 41.835
- task:
type: Retrieval
dataset:
type: hotpotqa
name: MTEB HotpotQA
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 39.041
- type: map_at_10 value: 62.961999999999996
- type: map_at_100 value: 63.79899999999999
- type: map_at_1000 value: 63.854
- type: map_at_3 value: 59.399
- type: map_at_5 value: 61.669
- type: mrr_at_1 value: 78.082
- type: mrr_at_10 value: 84.321
- type: mrr_at_100 value: 84.49600000000001
- type: mrr_at_1000 value: 84.502
- type: mrr_at_3 value: 83.421
- type: mrr_at_5 value: 83.977
- type: ndcg_at_1 value: 78.082
- type: ndcg_at_10 value: 71.229
- type: ndcg_at_100 value: 74.10900000000001
- type: ndcg_at_1000 value: 75.169
- type: ndcg_at_3 value: 66.28699999999999
- type: ndcg_at_5 value: 69.084
- type: precision_at_1 value: 78.082
- type: precision_at_10 value: 14.993
- type: precision_at_100 value: 1.7239999999999998
- type: precision_at_1000 value: 0.186
- type: precision_at_3 value: 42.737
- type: precision_at_5 value: 27.843
- type: recall_at_1 value: 39.041
- type: recall_at_10 value: 74.96300000000001
- type: recall_at_100 value: 86.199
- type: recall_at_1000 value: 93.228
- type: recall_at_3 value: 64.105
- type: recall_at_5 value: 69.608
- task:
type: Classification
dataset:
type: mteb/imdb
name: MTEB ImdbClassification
config: default
split: test
revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
metrics:
- type: accuracy value: 90.23160000000001
- type: ap value: 85.5674856808308
- type: f1 value: 90.18033354786317
- task:
type: Retrieval
dataset:
type: msmarco
name: MTEB MSMARCO
config: default
split: dev
revision: None
metrics:
- type: map_at_1 value: 24.091
- type: map_at_10 value: 36.753
- type: map_at_100 value: 37.913000000000004
- type: map_at_1000 value: 37.958999999999996
- type: map_at_3 value: 32.818999999999996
- type: map_at_5 value: 35.171
- type: mrr_at_1 value: 24.742
- type: mrr_at_10 value: 37.285000000000004
- type: mrr_at_100 value: 38.391999999999996
- type: mrr_at_1000 value: 38.431
- type: mrr_at_3 value: 33.440999999999995
- type: mrr_at_5 value: 35.75
- type: ndcg_at_1 value: 24.742
- type: ndcg_at_10 value: 43.698
- type: ndcg_at_100 value: 49.145
- type: ndcg_at_1000 value: 50.23800000000001
- type: ndcg_at_3 value: 35.769
- type: ndcg_at_5 value: 39.961999999999996
- type: precision_at_1 value: 24.742
- type: precision_at_10 value: 6.7989999999999995
- type: precision_at_100 value: 0.95
- type: precision_at_1000 value: 0.104
- type: precision_at_3 value: 15.096000000000002
- type: precision_at_5 value: 11.183
- type: recall_at_1 value: 24.091
- type: recall_at_10 value: 65.068
- type: recall_at_100 value: 89.899
- type: recall_at_1000 value: 98.16
- type: recall_at_3 value: 43.68
- type: recall_at_5 value: 53.754999999999995
- task:
type: Classification
dataset:
type: mteb/mtop_domain
name: MTEB MTOPDomainClassification (en)
config: en
split: test
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
metrics:
- type: accuracy value: 93.66621067031465
- type: f1 value: 93.49622853272142
- task:
type: Classification
dataset:
type: mteb/mtop_domain
name: MTEB MTOPDomainClassification (de)
config: de
split: test
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
metrics:
- type: accuracy value: 91.94702733164272
- type: f1 value: 91.17043441745282
- task:
type: Classification
dataset:
type: mteb/mtop_domain
name: MTEB MTOPDomainClassification (es)
config: es
split: test
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
metrics:
- type: accuracy value: 92.20146764509674
- type: f1 value: 91.98359080555608
- task:
type: Classification
dataset:
type: mteb/mtop_domain
name: MTEB MTOPDomainClassification (fr)
config: fr
split: test
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
metrics:
- type: accuracy value: 88.99780770435328
- type: f1 value: 89.19746342724068
- task:
type: Classification
dataset:
type: mteb/mtop_domain
name: MTEB MTOPDomainClassification (hi)
config: hi
split: test
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
metrics:
- type: accuracy value: 89.78486912871998
- type: f1 value: 89.24578823628642
- task:
type: Classification
dataset:
type: mteb/mtop_domain
name: MTEB MTOPDomainClassification (th)
config: th
split: test
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
metrics:
- type: accuracy value: 88.74502712477394
- type: f1 value: 89.00297573881542
- task:
type: Classification
dataset:
type: mteb/mtop_intent
name: MTEB MTOPIntentClassification (en)
config: en
split: test
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
metrics:
- type: accuracy value: 77.9046967624259
- type: f1 value: 59.36787125785957
- task:
type: Classification
dataset:
type: mteb/mtop_intent
name: MTEB MTOPIntentClassification (de)
config: de
split: test
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
metrics:
- type: accuracy value: 74.5280360664976
- type: f1 value: 57.17723440888718
- task:
type: Classification
dataset:
type: mteb/mtop_intent
name: MTEB MTOPIntentClassification (es)
config: es
split: test
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
metrics:
- type: accuracy value: 75.44029352901934
- type: f1 value: 54.052855531072964
- task:
type: Classification
dataset:
type: mteb/mtop_intent
name: MTEB MTOPIntentClassification (fr)
config: fr
split: test
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
metrics:
- type: accuracy value: 70.5606013153774
- type: f1 value: 52.62215934386531
- task:
type: Classification
dataset:
type: mteb/mtop_intent
name: MTEB MTOPIntentClassification (hi)
config: hi
split: test
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
metrics:
- type: accuracy value: 73.11581211903908
- type: f1 value: 52.341291845645465
- task:
type: Classification
dataset:
type: mteb/mtop_intent
name: MTEB MTOPIntentClassification (th)
config: th
split: test
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
metrics:
- type: accuracy value: 74.28933092224233
- type: f1 value: 57.07918745504911
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (af)
config: af
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 62.38063214525892
- type: f1 value: 59.46463723443009
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (am)
config: am
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 56.06926698049766
- type: f1 value: 52.49084283283562
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ar)
config: ar
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 60.74983187626093
- type: f1 value: 56.960640620165904
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (az)
config: az
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 64.86550100874243
- type: f1 value: 62.47370548140688
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (bn)
config: bn
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 63.971082716879636
- type: f1 value: 61.03812421957381
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (cy)
config: cy
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 54.98318762609282
- type: f1 value: 51.51207916008392
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (da)
config: da
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.45527908540686
- type: f1 value: 66.16631905400318
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (de)
config: de
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.32750504371216
- type: f1 value: 66.16755288646591
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (el)
config: el
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.09213180901143
- type: f1 value: 66.95654394661507
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (en)
config: en
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 73.75588433086752
- type: f1 value: 71.79973779656923
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (es)
config: es
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 70.49428379287154
- type: f1 value: 68.37494379215734
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (fa)
config: fa
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.90921318090115
- type: f1 value: 66.79517376481645
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (fi)
config: fi
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 70.12104909213181
- type: f1 value: 67.29448842879584
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (fr)
config: fr
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.34095494283793
- type: f1 value: 67.01134288992947
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (he)
config: he
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 67.61264290517822
- type: f1 value: 64.68730512660757
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (hi)
config: hi
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 67.79757901815738
- type: f1 value: 65.24938539425598
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (hu)
config: hu
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.68728984532616
- type: f1 value: 67.0487169762553
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (hy)
config: hy
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 62.07464694014795
- type: f1 value: 59.183532276789286
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (id)
config: id
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 70.04707464694015
- type: f1 value: 67.66829629003848
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (is)
config: is
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 62.42434431741762
- type: f1 value: 59.01617226544757
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (it)
config: it
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 70.53127101546738
- type: f1 value: 68.10033760906255
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ja)
config: ja
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 72.50504371217215
- type: f1 value: 69.74931103158923
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (jv)
config: jv
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 57.91190316072628
- type: f1 value: 54.05551136648796
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ka)
config: ka
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 51.78211163416275
- type: f1 value: 49.874888544058535
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (km)
config: km
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 47.017484868863484
- type: f1 value: 44.53364263352014
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (kn)
config: kn
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 62.16207128446537
- type: f1 value: 59.01185692320829
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ko)
config: ko
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.42501681237391
- type: f1 value: 67.13169450166086
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (lv)
config: lv
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 67.0780094149294
- type: f1 value: 64.41720167850707
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ml)
config: ml
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 65.57162071284466
- type: f1 value: 62.414138683804424
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (mn)
config: mn
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 61.71149966375252
- type: f1 value: 58.594805125087234
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ms)
config: ms
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 66.03900470746471
- type: f1 value: 63.87937257883887
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (my)
config: my
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 60.8776059179556
- type: f1 value: 57.48587618059131
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (nb)
config: nb
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.87895090786819
- type: f1 value: 66.8141299430347
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (nl)
config: nl
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 70.45057162071285
- type: f1 value: 67.46444039673516
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (pl)
config: pl
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 71.546738399462
- type: f1 value: 68.63640876702655
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (pt)
config: pt
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 70.72965702757229
- type: f1 value: 68.54119560379115
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ro)
config: ro
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 68.35574983187625
- type: f1 value: 65.88844917691927
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ru)
config: ru
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 71.70477471418964
- type: f1 value: 69.19665697061978
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (sl)
config: sl
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 67.0880968392737
- type: f1 value: 64.76962317666086
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (sq)
config: sq
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 65.18493611297916
- type: f1 value: 62.49984559035371
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (sv)
config: sv
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 71.75857431069265
- type: f1 value: 69.20053687623418
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (sw)
config: sw
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 58.500336247478145
- type: f1 value: 55.2972398687929
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ta)
config: ta
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 62.68997982515132
- type: f1 value: 59.36848202755348
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (te)
config: te
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 63.01950235373235
- type: f1 value: 60.09351954625423
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (th)
config: th
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 68.29186281102892
- type: f1 value: 67.57860496703447
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (tl)
config: tl
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 64.77471418964357
- type: f1 value: 61.913983147713836
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (tr)
config: tr
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.87222595830532
- type: f1 value: 66.03679033708141
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ur)
config: ur
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 64.04505716207127
- type: f1 value: 61.28569169817908
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (vi)
config: vi
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.38466711499663
- type: f1 value: 67.20532357036844
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (zh-CN)
config: zh-CN
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 71.12306657700067
- type: f1 value: 68.91251226588182
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (zh-TW)
config: zh-TW
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 66.20040349697378
- type: f1 value: 66.02657347714175
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (af)
config: af
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 68.73907195696032
- type: f1 value: 66.98484521791418
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (am)
config: am
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 60.58843308675185
- type: f1 value: 58.95591723092005
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ar)
config: ar
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 66.22730329522528
- type: f1 value: 66.0894499712115
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (az)
config: az
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 66.48285137861465
- type: f1 value: 65.21963176785157
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (bn)
config: bn
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 67.74714189643578
- type: f1 value: 66.8212192745412
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (cy)
config: cy
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 59.09213180901143
- type: f1 value: 56.70735546356339
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (da)
config: da
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 75.05716207128448
- type: f1 value: 74.8413712365364
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (de)
config: de
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 74.69737726967047
- type: f1 value: 74.7664341963
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (el)
config: el
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 73.90383322125084
- type: f1 value: 73.59201554448323
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (en)
config: en
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 77.51176866173503
- type: f1 value: 77.46104434577758
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (es)
config: es
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 74.31069266980496
- type: f1 value: 74.61048660675635
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (fa)
config: fa
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 72.95225285810356
- type: f1 value: 72.33160006574627
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (fi)
config: fi
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 73.12373907195696
- type: f1 value: 73.20921012557481
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (fr)
config: fr
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 73.86684599865501
- type: f1 value: 73.82348774610831
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (he)
config: he
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 71.40215198386012
- type: f1 value: 71.11945183971858
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (hi)
config: hi
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 72.12844653665098
- type: f1 value: 71.34450495911766
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (hu)
config: hu
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 74.52252858103566
- type: f1 value: 73.98878711342999
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (hy)
config: hy
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 64.93611297915265
- type: f1 value: 63.723200467653385
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (id)
config: id
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 74.11903160726295
- type: f1 value: 73.82138439467096
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (is)
config: is
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 67.15198386012105
- type: f1 value: 66.02172193802167
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (it)
config: it
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 74.32414256893072
- type: f1 value: 74.30943421170574
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ja)
config: ja
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 77.46805648957633
- type: f1 value: 77.62808409298209
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (jv)
config: jv
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 63.318762609280434
- type: f1 value: 62.094284066075076
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ka)
config: ka
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 58.34902488231338
- type: f1 value: 57.12893860987984
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (km)
config: km
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 50.88433086751849
- type: f1 value: 48.2272350802058
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (kn)
config: kn
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 66.4425016812374
- type: f1 value: 64.61463095996173
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ko)
config: ko
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 75.04707464694015
- type: f1 value: 75.05099199098998
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (lv)
config: lv
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 70.50437121721586
- type: f1 value: 69.83397721096314
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ml)
config: ml
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 69.94283792871553
- type: f1 value: 68.8704663703913
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (mn)
config: mn
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 64.79488903833222
- type: f1 value: 63.615424063345436
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ms)
config: ms
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 69.88231338264963
- type: f1 value: 68.57892302593237
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (my)
config: my
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 63.248150638870214
- type: f1 value: 61.06680605338809
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (nb)
config: nb
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 74.84196368527236
- type: f1 value: 74.52566464968763
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (nl)
config: nl
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 74.8285137861466
- type: f1 value: 74.8853197608802
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (pl)
config: pl
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 74.13248150638869
- type: f1 value: 74.3982040999179
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (pt)
config: pt
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 73.49024882313383
- type: f1 value: 73.82153848368573
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ro)
config: ro
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 71.72158708809684
- type: f1 value: 71.85049433180541
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ru)
config: ru
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 75.137861466039
- type: f1 value: 75.37628348188467
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (sl)
config: sl
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 71.86953597848016
- type: f1 value: 71.87537624521661
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (sq)
config: sq
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 70.27572293207801
- type: f1 value: 68.80017302344231
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (sv)
config: sv
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 76.09952925353059
- type: f1 value: 76.07992707688408
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (sw)
config: sw
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 63.140551445864155
- type: f1 value: 61.73855010331415
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ta)
config: ta
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 66.27774041694687
- type: f1 value: 64.83664868894539
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (te)
config: te
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 66.69468728984533
- type: f1 value: 64.76239666920868
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (th)
config: th
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 73.44653665097512
- type: f1 value: 73.14646052013873
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (tl)
config: tl
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 67.71351714862139
- type: f1 value: 66.67212180163382
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (tr)
config: tr
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 73.9946200403497
- type: f1 value: 73.87348793725525
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ur)
config: ur
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 68.15400134498992
- type: f1 value: 67.09433241421094
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (vi)
config: vi
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 73.11365164761264
- type: f1 value: 73.59502539433753
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (zh-CN)
config: zh-CN
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 76.82582380632145
- type: f1 value: 76.89992945316313
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (zh-TW)
config: zh-TW
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 71.81237390719569
- type: f1 value: 72.36499770986265
- task:
type: Clustering
dataset:
type: mteb/medrxiv-clustering-p2p
name: MTEB MedrxivClusteringP2P
config: default
split: test
revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73
metrics:
- type: v_measure value: 31.480506569594695
- task:
type: Clustering
dataset:
type: mteb/medrxiv-clustering-s2s
name: MTEB MedrxivClusteringS2S
config: default
split: test
revision: 35191c8c0dca72d8ff3efcd72aa802307d469663
metrics:
- type: v_measure value: 29.71252128004552
- task:
type: Reranking
dataset:
type: mteb/mind_small
name: MTEB MindSmallReranking
config: default
split: test
revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69
metrics:
- type: map value: 31.421396787056548
- type: mrr value: 32.48155274872267
- task:
type: Retrieval
dataset:
type: nfcorpus
name: MTEB NFCorpus
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 5.595
- type: map_at_10 value: 12.642000000000001
- type: map_at_100 value: 15.726
- type: map_at_1000 value: 17.061999999999998
- type: map_at_3 value: 9.125
- type: map_at_5 value: 10.866000000000001
- type: mrr_at_1 value: 43.344
- type: mrr_at_10 value: 52.227999999999994
- type: mrr_at_100 value: 52.898999999999994
- type: mrr_at_1000 value: 52.944
- type: mrr_at_3 value: 49.845
- type: mrr_at_5 value: 51.115
- type: ndcg_at_1 value: 41.949999999999996
- type: ndcg_at_10 value: 33.995
- type: ndcg_at_100 value: 30.869999999999997
- type: ndcg_at_1000 value: 39.487
- type: ndcg_at_3 value: 38.903999999999996
- type: ndcg_at_5 value: 37.236999999999995
- type: precision_at_1 value: 43.344
- type: precision_at_10 value: 25.480000000000004
- type: precision_at_100 value: 7.672
- type: precision_at_1000 value: 2.028
- type: precision_at_3 value: 36.636
- type: precision_at_5 value: 32.632
- type: recall_at_1 value: 5.595
- type: recall_at_10 value: 16.466
- type: recall_at_100 value: 31.226
- type: recall_at_1000 value: 62.778999999999996
- type: recall_at_3 value: 9.931
- type: recall_at_5 value: 12.884
- task:
type: Retrieval
dataset:
type: nq
name: MTEB NQ
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 40.414
- type: map_at_10 value: 56.754000000000005
- type: map_at_100 value: 57.457
- type: map_at_1000 value: 57.477999999999994
- type: map_at_3 value: 52.873999999999995
- type: map_at_5 value: 55.175
- type: mrr_at_1 value: 45.278
- type: mrr_at_10 value: 59.192
- type: mrr_at_100 value: 59.650000000000006
- type: mrr_at_1000 value: 59.665
- type: mrr_at_3 value: 56.141
- type: mrr_at_5 value: 57.998000000000005
- type: ndcg_at_1 value: 45.278
- type: ndcg_at_10 value: 64.056
- type: ndcg_at_100 value: 66.89
- type: ndcg_at_1000 value: 67.364
- type: ndcg_at_3 value: 56.97
- type: ndcg_at_5 value: 60.719
- type: precision_at_1 value: 45.278
- type: precision_at_10 value: 9.994
- type: precision_at_100 value: 1.165
- type: precision_at_1000 value: 0.121
- type: precision_at_3 value: 25.512
- type: precision_at_5 value: 17.509
- type: recall_at_1 value: 40.414
- type: recall_at_10 value: 83.596
- type: recall_at_100 value: 95.72
- type: recall_at_1000 value: 99.24
- type: recall_at_3 value: 65.472
- type: recall_at_5 value: 74.039
- task:
type: Retrieval
dataset:
type: quora
name: MTEB QuoraRetrieval
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 70.352
- type: map_at_10 value: 84.369
- type: map_at_100 value: 85.02499999999999
- type: map_at_1000 value: 85.04
- type: map_at_3 value: 81.42399999999999
- type: map_at_5 value: 83.279
- type: mrr_at_1 value: 81.05
- type: mrr_at_10 value: 87.401
- type: mrr_at_100 value: 87.504
- type: mrr_at_1000 value: 87.505
- type: mrr_at_3 value: 86.443
- type: mrr_at_5 value: 87.10799999999999
- type: ndcg_at_1 value: 81.04
- type: ndcg_at_10 value: 88.181
- type: ndcg_at_100 value: 89.411
- type: ndcg_at_1000 value: 89.507
- type: ndcg_at_3 value: 85.28099999999999
- type: ndcg_at_5 value: 86.888
- type: precision_at_1 value: 81.04
- type: precision_at_10 value: 13.406
- type: precision_at_100 value: 1.5350000000000001
- type: precision_at_1000 value: 0.157
- type: precision_at_3 value: 37.31
- type: precision_at_5 value: 24.54
- type: recall_at_1 value: 70.352
- type: recall_at_10 value: 95.358
- type: recall_at_100 value: 99.541
- type: recall_at_1000 value: 99.984
- type: recall_at_3 value: 87.111
- type: recall_at_5 value: 91.643
- task:
type: Clustering
dataset:
type: mteb/reddit-clustering
name: MTEB RedditClustering
config: default
split: test
revision: 24640382cdbf8abc73003fb0fa6d111a705499eb
metrics:
- type: v_measure value: 46.54068723291946
- task:
type: Clustering
dataset:
type: mteb/reddit-clustering-p2p
name: MTEB RedditClusteringP2P
config: default
split: test
revision: 282350215ef01743dc01b456c7f5241fa8937f16
metrics:
- type: v_measure value: 63.216287629895994
- task:
type: Retrieval
dataset:
type: scidocs
name: MTEB SCIDOCS
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 4.023000000000001
- type: map_at_10 value: 10.071
- type: map_at_100 value: 11.892
- type: map_at_1000 value: 12.196
- type: map_at_3 value: 7.234
- type: map_at_5 value: 8.613999999999999
- type: mrr_at_1 value: 19.900000000000002
- type: mrr_at_10 value: 30.516
- type: mrr_at_100 value: 31.656000000000002
- type: mrr_at_1000 value: 31.723000000000003
- type: mrr_at_3 value: 27.400000000000002
- type: mrr_at_5 value: 29.270000000000003
- type: ndcg_at_1 value: 19.900000000000002
- type: ndcg_at_10 value: 17.474
- type: ndcg_at_100 value: 25.020999999999997
- type: ndcg_at_1000 value: 30.728
- type: ndcg_at_3 value: 16.588
- type: ndcg_at_5 value: 14.498
- type: precision_at_1 value: 19.900000000000002
- type: precision_at_10 value: 9.139999999999999
- type: precision_at_100 value: 2.011
- type: precision_at_1000 value: 0.33899999999999997
- type: precision_at_3 value: 15.667
- type: precision_at_5 value: 12.839999999999998
- type: recall_at_1 value: 4.023000000000001
- type: recall_at_10 value: 18.497
- type: recall_at_100 value: 40.8
- type: recall_at_1000 value: 68.812
- type: recall_at_3 value: 9.508
- type: recall_at_5 value: 12.983
- task:
type: STS
dataset:
type: mteb/sickr-sts
name: MTEB SICK-R
config: default
split: test
revision: a6ea5a8cab320b040a23452cc28066d9beae2cee
metrics:
- type: cos_sim_pearson value: 83.967008785134
- type: cos_sim_spearman value: 80.23142141101837
- type: euclidean_pearson value: 81.20166064704539
- type: euclidean_spearman value: 80.18961335654585
- type: manhattan_pearson value: 81.13925443187625
- type: manhattan_spearman value: 80.07948723044424
- task:
type: STS
dataset:
type: mteb/sts12-sts
name: MTEB STS12
config: default
split: test
revision: a0d554a64d88156834ff5ae9920b964011b16384
metrics:
- type: cos_sim_pearson value: 86.94262461316023
- type: cos_sim_spearman value: 80.01596278563865
- type: euclidean_pearson value: 83.80799622922581
- type: euclidean_spearman value: 79.94984954947103
- type: manhattan_pearson value: 83.68473841756281
- type: manhattan_spearman value: 79.84990707951822
- task:
type: STS
dataset:
type: mteb/sts13-sts
name: MTEB STS13
config: default
split: test
revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca
metrics:
- type: cos_sim_pearson value: 80.57346443146068
- type: cos_sim_spearman value: 81.54689837570866
- type: euclidean_pearson value: 81.10909881516007
- type: euclidean_spearman value: 81.56746243261762
- type: manhattan_pearson value: 80.87076036186582
- type: manhattan_spearman value: 81.33074987964402
- task:
type: STS
dataset:
type: mteb/sts14-sts
name: MTEB STS14
config: default
split: test
revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375
metrics:
- type: cos_sim_pearson value: 79.54733787179849
- type: cos_sim_spearman value: 77.72202105610411
- type: euclidean_pearson value: 78.9043595478849
- type: euclidean_spearman value: 77.93422804309435
- type: manhattan_pearson value: 78.58115121621368
- type: manhattan_spearman value: 77.62508135122033
- task:
type: STS
dataset:
type: mteb/sts15-sts
name: MTEB STS15
config: default
split: test
revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3
metrics:
- type: cos_sim_pearson value: 88.59880017237558
- type: cos_sim_spearman value: 89.31088630824758
- type: euclidean_pearson value: 88.47069261564656
- type: euclidean_spearman value: 89.33581971465233
- type: manhattan_pearson value: 88.40774264100956
- type: manhattan_spearman value: 89.28657485627835
- task:
type: STS
dataset:
type: mteb/sts16-sts
name: MTEB STS16
config: default
split: test
revision: 4d8694f8f0e0100860b497b999b3dbed754a0513
metrics:
- type: cos_sim_pearson value: 84.08055117917084
- type: cos_sim_spearman value: 85.78491813080304
- type: euclidean_pearson value: 84.99329155500392
- type: euclidean_spearman value: 85.76728064677287
- type: manhattan_pearson value: 84.87947428989587
- type: manhattan_spearman value: 85.62429454917464
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (ko-ko)
config: ko-ko
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 82.14190939287384
- type: cos_sim_spearman value: 82.27331573306041
- type: euclidean_pearson value: 81.891896953716
- type: euclidean_spearman value: 82.37695542955998
- type: manhattan_pearson value: 81.73123869460504
- type: manhattan_spearman value: 82.19989168441421
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (ar-ar)
config: ar-ar
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 76.84695301843362
- type: cos_sim_spearman value: 77.87790986014461
- type: euclidean_pearson value: 76.91981583106315
- type: euclidean_spearman value: 77.88154772749589
- type: manhattan_pearson value: 76.94953277451093
- type: manhattan_spearman value: 77.80499230728604
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (en-ar)
config: en-ar
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 75.44657840482016
- type: cos_sim_spearman value: 75.05531095119674
- type: euclidean_pearson value: 75.88161755829299
- type: euclidean_spearman value: 74.73176238219332
- type: manhattan_pearson value: 75.63984765635362
- type: manhattan_spearman value: 74.86476440770737
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (en-de)
config: en-de
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 85.64700140524133
- type: cos_sim_spearman value: 86.16014210425672
- type: euclidean_pearson value: 86.49086860843221
- type: euclidean_spearman value: 86.09729326815614
- type: manhattan_pearson value: 86.43406265125513
- type: manhattan_spearman value: 86.17740150939994
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (en-en)
config: en-en
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 87.91170098764921
- type: cos_sim_spearman value: 88.12437004058931
- type: euclidean_pearson value: 88.81828254494437
- type: euclidean_spearman value: 88.14831794572122
- type: manhattan_pearson value: 88.93442183448961
- type: manhattan_spearman value: 88.15254630778304
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (en-tr)
config: en-tr
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 72.91390577997292
- type: cos_sim_spearman value: 71.22979457536074
- type: euclidean_pearson value: 74.40314008106749
- type: euclidean_spearman value: 72.54972136083246
- type: manhattan_pearson value: 73.85687539530218
- type: manhattan_spearman value: 72.09500771742637
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (es-en)
config: es-en
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 80.9301067983089
- type: cos_sim_spearman value: 80.74989828346473
- type: euclidean_pearson value: 81.36781301814257
- type: euclidean_spearman value: 80.9448819964426
- type: manhattan_pearson value: 81.0351322685609
- type: manhattan_spearman value: 80.70192121844177
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (es-es)
config: es-es
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 87.13820465980005
- type: cos_sim_spearman value: 86.73532498758757
- type: euclidean_pearson value: 87.21329451846637
- type: euclidean_spearman value: 86.57863198601002
- type: manhattan_pearson value: 87.06973713818554
- type: manhattan_spearman value: 86.47534918791499
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (fr-en)
config: fr-en
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 85.48720108904415
- type: cos_sim_spearman value: 85.62221757068387
- type: euclidean_pearson value: 86.1010129512749
- type: euclidean_spearman value: 85.86580966509942
- type: manhattan_pearson value: 86.26800938808971
- type: manhattan_spearman value: 85.88902721678429
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (it-en)
config: it-en
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 83.98021347333516
- type: cos_sim_spearman value: 84.53806553803501
- type: euclidean_pearson value: 84.61483347248364
- type: euclidean_spearman value: 85.14191408011702
- type: manhattan_pearson value: 84.75297588825967
- type: manhattan_spearman value: 85.33176753669242
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (nl-en)
config: nl-en
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 84.51856644893233
- type: cos_sim_spearman value: 85.27510748506413
- type: euclidean_pearson value: 85.09886861540977
- type: euclidean_spearman value: 85.62579245860887
- type: manhattan_pearson value: 84.93017860464607
- type: manhattan_spearman value: 85.5063988898453
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (en)
config: en
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 62.581573200584195
- type: cos_sim_spearman value: 63.05503590247928
- type: euclidean_pearson value: 63.652564812602094
- type: euclidean_spearman value: 62.64811520876156
- type: manhattan_pearson value: 63.506842893061076
- type: manhattan_spearman value: 62.51289573046917
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (de)
config: de
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 48.2248801729127
- type: cos_sim_spearman value: 56.5936604678561
- type: euclidean_pearson value: 43.98149464089
- type: euclidean_spearman value: 56.108561882423615
- type: manhattan_pearson value: 43.86880305903564
- type: manhattan_spearman value: 56.04671150510166
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (es)
config: es
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 55.17564527009831
- type: cos_sim_spearman value: 64.57978560979488
- type: euclidean_pearson value: 58.8818330154583
- type: euclidean_spearman value: 64.99214839071281
- type: manhattan_pearson value: 58.72671436121381
- type: manhattan_spearman value: 65.10713416616109
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (pl)
config: pl
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 26.772131864023297
- type: cos_sim_spearman value: 34.68200792408681
- type: euclidean_pearson value: 16.68082419005441
- type: euclidean_spearman value: 34.83099932652166
- type: manhattan_pearson value: 16.52605949659529
- type: manhattan_spearman value: 34.82075801399475
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (tr)
config: tr
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 54.42415189043831
- type: cos_sim_spearman value: 63.54594264576758
- type: euclidean_pearson value: 57.36577498297745
- type: euclidean_spearman value: 63.111466379158074
- type: manhattan_pearson value: 57.584543715873885
- type: manhattan_spearman value: 63.22361054139183
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (ar)
config: ar
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 47.55216762405518
- type: cos_sim_spearman value: 56.98670142896412
- type: euclidean_pearson value: 50.15318757562699
- type: euclidean_spearman value: 56.524941926541906
- type: manhattan_pearson value: 49.955618528674904
- type: manhattan_spearman value: 56.37102209240117
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (ru)
config: ru
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 49.20540980338571
- type: cos_sim_spearman value: 59.9009453504406
- type: euclidean_pearson value: 49.557749853620535
- type: euclidean_spearman value: 59.76631621172456
- type: manhattan_pearson value: 49.62340591181147
- type: manhattan_spearman value: 59.94224880322436
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (zh)
config: zh
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 51.508169956576985
- type: cos_sim_spearman value: 66.82461565306046
- type: euclidean_pearson value: 56.2274426480083
- type: euclidean_spearman value: 66.6775323848333
- type: manhattan_pearson value: 55.98277796300661
- type: manhattan_spearman value: 66.63669848497175
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (fr)
config: fr
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 72.86478788045507
- type: cos_sim_spearman value: 76.7946552053193
- type: euclidean_pearson value: 75.01598530490269
- type: euclidean_spearman value: 76.83618917858281
- type: manhattan_pearson value: 74.68337628304332
- type: manhattan_spearman value: 76.57480204017773
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (de-en)
config: de-en
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 55.922619099401984
- type: cos_sim_spearman value: 56.599362477240774
- type: euclidean_pearson value: 56.68307052369783
- type: euclidean_spearman value: 54.28760436777401
- type: manhattan_pearson value: 56.67763566500681
- type: manhattan_spearman value: 53.94619541711359
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (es-en)
config: es-en
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 66.74357206710913
- type: cos_sim_spearman value: 72.5208244925311
- type: euclidean_pearson value: 67.49254562186032
- type: euclidean_spearman value: 72.02469076238683
- type: manhattan_pearson value: 67.45251772238085
- type: manhattan_spearman value: 72.05538819984538
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (it)
config: it
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 71.25734330033191
- type: cos_sim_spearman value: 76.98349083946823
- type: euclidean_pearson value: 73.71642838667736
- type: euclidean_spearman value: 77.01715504651384
- type: manhattan_pearson value: 73.61712711868105
- type: manhattan_spearman value: 77.01392571153896
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (pl-en)
config: pl-en
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 63.18215462781212
- type: cos_sim_spearman value: 65.54373266117607
- type: euclidean_pearson value: 64.54126095439005
- type: euclidean_spearman value: 65.30410369102711
- type: manhattan_pearson value: 63.50332221148234
- type: manhattan_spearman value: 64.3455878104313
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (zh-en)
config: zh-en
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 62.30509221440029
- type: cos_sim_spearman value: 65.99582704642478
- type: euclidean_pearson value: 63.43818859884195
- type: euclidean_spearman value: 66.83172582815764
- type: manhattan_pearson value: 63.055779168508764
- type: manhattan_spearman value: 65.49585020501449
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (es-it)
config: es-it
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 59.587830825340404
- type: cos_sim_spearman value: 68.93467614588089
- type: euclidean_pearson value: 62.3073527367404
- type: euclidean_spearman value: 69.69758171553175
- type: manhattan_pearson value: 61.9074580815789
- type: manhattan_spearman value: 69.57696375597865
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (de-fr)
config: de-fr
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 57.143220125577066
- type: cos_sim_spearman value: 67.78857859159226
- type: euclidean_pearson value: 55.58225107923733
- type: euclidean_spearman value: 67.80662907184563
- type: manhattan_pearson value: 56.24953502726514
- type: manhattan_spearman value: 67.98262125431616
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (de-pl)
config: de-pl
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 21.826928900322066
- type: cos_sim_spearman value: 49.578506634400405
- type: euclidean_pearson value: 27.939890138843214
- type: euclidean_spearman value: 52.71950519136242
- type: manhattan_pearson value: 26.39878683847546
- type: manhattan_spearman value: 47.54609580342499
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (fr-pl)
config: fr-pl
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 57.27603854632001
- type: cos_sim_spearman value: 50.709255283710995
- type: euclidean_pearson value: 59.5419024445929
- type: euclidean_spearman value: 50.709255283710995
- type: manhattan_pearson value: 59.03256832438492
- type: manhattan_spearman value: 61.97797868009122
- task:
type: STS
dataset:
type: mteb/stsbenchmark-sts
name: MTEB STSBenchmark
config: default
split: test
revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
metrics:
- type: cos_sim_pearson value: 85.00757054859712
- type: cos_sim_spearman value: 87.29283629622222
- type: euclidean_pearson value: 86.54824171775536
- type: euclidean_spearman value: 87.24364730491402
- type: manhattan_pearson value: 86.5062156915074
- type: manhattan_spearman value: 87.15052170378574
- task:
type: Reranking
dataset:
type: mteb/scidocs-reranking
name: MTEB SciDocsRR
config: default
split: test
revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab
metrics:
- type: map value: 82.03549357197389
- type: mrr value: 95.05437645143527
- task:
type: Retrieval
dataset:
type: scifact
name: MTEB SciFact
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 57.260999999999996
- type: map_at_10 value: 66.259
- type: map_at_100 value: 66.884
- type: map_at_1000 value: 66.912
- type: map_at_3 value: 63.685
- type: map_at_5 value: 65.35499999999999
- type: mrr_at_1 value: 60.333000000000006
- type: mrr_at_10 value: 67.5
- type: mrr_at_100 value: 68.013
- type: mrr_at_1000 value: 68.038
- type: mrr_at_3 value: 65.61099999999999
- type: mrr_at_5 value: 66.861
- type: ndcg_at_1 value: 60.333000000000006
- type: ndcg_at_10 value: 70.41
- type: ndcg_at_100 value: 73.10600000000001
- type: ndcg_at_1000 value: 73.846
- type: ndcg_at_3 value: 66.133
- type: ndcg_at_5 value: 68.499
- type: precision_at_1 value: 60.333000000000006
- type: precision_at_10 value: 9.232999999999999
- type: precision_at_100 value: 1.0630000000000002
- type: precision_at_1000 value: 0.11299999999999999
- type: precision_at_3 value: 25.667
- type: precision_at_5 value: 17.067
- type: recall_at_1 value: 57.260999999999996
- type: recall_at_10 value: 81.94399999999999
- type: recall_at_100 value: 93.867
- type: recall_at_1000 value: 99.667
- type: recall_at_3 value: 70.339
- type: recall_at_5 value: 76.25
- task:
type: PairClassification
dataset:
type: mteb/sprintduplicatequestions-pairclassification
name: MTEB SprintDuplicateQuestions
config: default
split: test
revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46
metrics:
- type: cos_sim_accuracy value: 99.74356435643564
- type: cos_sim_ap value: 93.13411948212683
- type: cos_sim_f1 value: 86.80521991300147
- type: cos_sim_precision value: 84.00374181478017
- type: cos_sim_recall value: 89.8
- type: dot_accuracy value: 99.67920792079208
- type: dot_ap value: 89.27277565444479
- type: dot_f1 value: 83.9276990718124
- type: dot_precision value: 82.04393505253104
- type: dot_recall value: 85.9
- type: euclidean_accuracy value: 99.74257425742574
- type: euclidean_ap value: 93.17993008259062
- type: euclidean_f1 value: 86.69396110542476
- type: euclidean_precision value: 88.78406708595388
- type: euclidean_recall value: 84.7
- type: manhattan_accuracy value: 99.74257425742574
- type: manhattan_ap value: 93.14413755550099
- type: manhattan_f1 value: 86.82483594144371
- type: manhattan_precision value: 87.66564729867483
- type: manhattan_recall value: 86
- type: max_accuracy value: 99.74356435643564
- type: max_ap value: 93.17993008259062
- type: max_f1 value: 86.82483594144371
- task:
type: Clustering
dataset:
type: mteb/stackexchange-clustering
name: MTEB StackExchangeClustering
config: default
split: test
revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259
metrics:
- type: v_measure value: 57.525863806168566
- task:
type: Clustering
dataset:
type: mteb/stackexchange-clustering-p2p
name: MTEB StackExchangeClusteringP2P
config: default
split: test
revision: 815ca46b2622cec33ccafc3735d572c266efdb44
metrics:
- type: v_measure value: 32.68850574423839
- task:
type: Reranking
dataset:
type: mteb/stackoverflowdupquestions-reranking
name: MTEB StackOverflowDupQuestions
config: default
split: test
revision: e185fbe320c72810689fc5848eb6114e1ef5ec69
metrics:
- type: map value: 49.71580650644033
- type: mrr value: 50.50971903913081
- task:
type: Summarization
dataset:
type: mteb/summeval
name: MTEB SummEval
config: default
split: test
revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c
metrics:
- type: cos_sim_pearson value: 29.152190498799484
- type: cos_sim_spearman value: 29.686180371952727
- type: dot_pearson value: 27.248664793816342
- type: dot_spearman value: 28.37748983721745
- task:
type: Retrieval
dataset:
type: trec-covid
name: MTEB TRECCOVID
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 0.20400000000000001
- type: map_at_10 value: 1.6209999999999998
- type: map_at_100 value: 9.690999999999999
- type: map_at_1000 value: 23.733
- type: map_at_3 value: 0.575
- type: map_at_5 value: 0.885
- type: mrr_at_1 value: 78
- type: mrr_at_10 value: 86.56700000000001
- type: mrr_at_100 value: 86.56700000000001
- type: mrr_at_1000 value: 86.56700000000001
- type: mrr_at_3 value: 85.667
- type: mrr_at_5 value: 86.56700000000001
- type: ndcg_at_1 value: 76
- type: ndcg_at_10 value: 71.326
- type: ndcg_at_100 value: 54.208999999999996
- type: ndcg_at_1000 value: 49.252
- type: ndcg_at_3 value: 74.235
- type: ndcg_at_5 value: 73.833
- type: precision_at_1 value: 78
- type: precision_at_10 value: 74.8
- type: precision_at_100 value: 55.50000000000001
- type: precision_at_1000 value: 21.836
- type: precision_at_3 value: 78
- type: precision_at_5 value: 78
- type: recall_at_1 value: 0.20400000000000001
- type: recall_at_10 value: 1.894
- type: recall_at_100 value: 13.245999999999999
- type: recall_at_1000 value: 46.373
- type: recall_at_3 value: 0.613
- type: recall_at_5 value: 0.991
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (sqi-eng)
config: sqi-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 95.89999999999999
- type: f1 value: 94.69999999999999
- type: precision value: 94.11666666666667
- type: recall value: 95.89999999999999
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (fry-eng)
config: fry-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 68.20809248554913
- type: f1 value: 63.431048720066066
- type: precision value: 61.69143958161298
- type: recall value: 68.20809248554913
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (kur-eng)
config: kur-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 71.21951219512195
- type: f1 value: 66.82926829268293
- type: precision value: 65.1260162601626
- type: recall value: 71.21951219512195
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (tur-eng)
config: tur-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 97.2
- type: f1 value: 96.26666666666667
- type: precision value: 95.8
- type: recall value: 97.2
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (deu-eng)
config: deu-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 99.3
- type: f1 value: 99.06666666666666
- type: precision value: 98.95
- type: recall value: 99.3
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (nld-eng)
config: nld-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 97.39999999999999
- type: f1 value: 96.63333333333333
- type: precision value: 96.26666666666668
- type: recall value: 97.39999999999999
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (ron-eng)
config: ron-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 96
- type: f1 value: 94.86666666666666
- type: precision value: 94.31666666666668
- type: recall value: 96
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (ang-eng)
config: ang-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 47.01492537313433
- type: f1 value: 40.178867566927266
- type: precision value: 38.179295828549556
- type: recall value: 47.01492537313433
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (ido-eng)
config: ido-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 86.5
- type: f1 value: 83.62537480063796
- type: precision value: 82.44555555555554
- type: recall value: 86.5
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (jav-eng)
config: jav-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 80.48780487804879
- type: f1 value: 75.45644599303138
- type: precision value: 73.37398373983739
- type: recall value: 80.48780487804879
- task: type: BitextMining dataset:
- task:
type: Classification
dataset:
type: mteb/amazon_counterfactual
name: MTEB AmazonCounterfactualClassification (en)
config: en
split: test
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
metrics:
[Content truncated...]
100,024 chars • Full Disclosure Protocol Active
ZEN MODE • README
tags:
- mteb
- Sentence Transformers
- sentence-similarity
- feature-extraction
- sentence-transformers model-index:
- name: multilingual-e5-large
results:
- task:
type: Classification
dataset:
type: mteb/amazon_counterfactual
name: MTEB AmazonCounterfactualClassification (en)
config: en
split: test
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
metrics:
- type: accuracy value: 79.05970149253731
- type: ap value: 43.486574390835635
- type: f1 value: 73.32700092140148
- task:
type: Classification
dataset:
type: mteb/amazon_counterfactual
name: MTEB AmazonCounterfactualClassification (de)
config: de
split: test
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
metrics:
- type: accuracy value: 71.22055674518201
- type: ap value: 81.55756710830498
- type: f1 value: 69.28271787752661
- task:
type: Classification
dataset:
type: mteb/amazon_counterfactual
name: MTEB AmazonCounterfactualClassification (en-ext)
config: en-ext
split: test
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
metrics:
- type: accuracy value: 80.41979010494754
- type: ap value: 29.34879922376344
- type: f1 value: 67.62475449011278
- task:
type: Classification
dataset:
type: mteb/amazon_counterfactual
name: MTEB AmazonCounterfactualClassification (ja)
config: ja
split: test
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
metrics:
- type: accuracy value: 77.8372591006424
- type: ap value: 26.557560591210738
- type: f1 value: 64.96619417368707
- task:
type: Classification
dataset:
type: mteb/amazon_polarity
name: MTEB AmazonPolarityClassification
config: default
split: test
revision: e2d317d38cd51312af73b3d32a06d1a08b442046
metrics:
- type: accuracy value: 93.489875
- type: ap value: 90.98758636917603
- type: f1 value: 93.48554819717332
- task:
type: Classification
dataset:
type: mteb/amazon_reviews_multi
name: MTEB AmazonReviewsClassification (en)
config: en
split: test
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
metrics:
- type: accuracy value: 47.564
- type: f1 value: 46.75122173518047
- task:
type: Classification
dataset:
type: mteb/amazon_reviews_multi
name: MTEB AmazonReviewsClassification (de)
config: de
split: test
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
metrics:
- type: accuracy value: 45.400000000000006
- type: f1 value: 44.17195682400632
- task:
type: Classification
dataset:
type: mteb/amazon_reviews_multi
name: MTEB AmazonReviewsClassification (es)
config: es
split: test
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
metrics:
- type: accuracy value: 43.068
- type: f1 value: 42.38155696855596
- task:
type: Classification
dataset:
type: mteb/amazon_reviews_multi
name: MTEB AmazonReviewsClassification (fr)
config: fr
split: test
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
metrics:
- type: accuracy value: 41.89
- type: f1 value: 40.84407321682663
- task:
type: Classification
dataset:
type: mteb/amazon_reviews_multi
name: MTEB AmazonReviewsClassification (ja)
config: ja
split: test
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
metrics:
- type: accuracy value: 40.120000000000005
- type: f1 value: 39.522976223819114
- task:
type: Classification
dataset:
type: mteb/amazon_reviews_multi
name: MTEB AmazonReviewsClassification (zh)
config: zh
split: test
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
metrics:
- type: accuracy value: 38.832
- type: f1 value: 38.0392533394713
- task:
type: Retrieval
dataset:
type: arguana
name: MTEB ArguAna
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 30.725
- type: map_at_10 value: 46.055
- type: map_at_100 value: 46.900999999999996
- type: map_at_1000 value: 46.911
- type: map_at_3 value: 41.548
- type: map_at_5 value: 44.297
- type: mrr_at_1 value: 31.152
- type: mrr_at_10 value: 46.231
- type: mrr_at_100 value: 47.07
- type: mrr_at_1000 value: 47.08
- type: mrr_at_3 value: 41.738
- type: mrr_at_5 value: 44.468999999999994
- type: ndcg_at_1 value: 30.725
- type: ndcg_at_10 value: 54.379999999999995
- type: ndcg_at_100 value: 58.138
- type: ndcg_at_1000 value: 58.389
- type: ndcg_at_3 value: 45.156
- type: ndcg_at_5 value: 50.123
- type: precision_at_1 value: 30.725
- type: precision_at_10 value: 8.087
- type: precision_at_100 value: 0.9769999999999999
- type: precision_at_1000 value: 0.1
- type: precision_at_3 value: 18.54
- type: precision_at_5 value: 13.542000000000002
- type: recall_at_1 value: 30.725
- type: recall_at_10 value: 80.868
- type: recall_at_100 value: 97.653
- type: recall_at_1000 value: 99.57300000000001
- type: recall_at_3 value: 55.619
- type: recall_at_5 value: 67.71000000000001
- task:
type: Clustering
dataset:
type: mteb/arxiv-clustering-p2p
name: MTEB ArxivClusteringP2P
config: default
split: test
revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
metrics:
- type: v_measure value: 44.30960650674069
- task:
type: Clustering
dataset:
type: mteb/arxiv-clustering-s2s
name: MTEB ArxivClusteringS2S
config: default
split: test
revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53
metrics:
- type: v_measure value: 38.427074197498996
- task:
type: Reranking
dataset:
type: mteb/askubuntudupquestions-reranking
name: MTEB AskUbuntuDupQuestions
config: default
split: test
revision: 2000358ca161889fa9c082cb41daa8dcfb161a54
metrics:
- type: map value: 60.28270056031872
- type: mrr value: 74.38332673789738
- task:
type: STS
dataset:
type: mteb/biosses-sts
name: MTEB BIOSSES
config: default
split: test
revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
metrics:
- type: cos_sim_pearson value: 84.05942144105269
- type: cos_sim_spearman value: 82.51212105850809
- type: euclidean_pearson value: 81.95639829909122
- type: euclidean_spearman value: 82.3717564144213
- type: manhattan_pearson value: 81.79273425468256
- type: manhattan_spearman value: 82.20066817871039
- task:
type: BitextMining
dataset:
type: mteb/bucc-bitext-mining
name: MTEB BUCC (de-en)
config: de-en
split: test
revision: d51519689f32196a32af33b075a01d0e7c51e252
metrics:
- type: accuracy value: 99.46764091858039
- type: f1 value: 99.37717466945023
- type: precision value: 99.33194154488518
- type: recall value: 99.46764091858039
- task:
type: BitextMining
dataset:
type: mteb/bucc-bitext-mining
name: MTEB BUCC (fr-en)
config: fr-en
split: test
revision: d51519689f32196a32af33b075a01d0e7c51e252
metrics:
- type: accuracy value: 98.29407880255337
- type: f1 value: 98.11248073959938
- type: precision value: 98.02443319392472
- type: recall value: 98.29407880255337
- task:
type: BitextMining
dataset:
type: mteb/bucc-bitext-mining
name: MTEB BUCC (ru-en)
config: ru-en
split: test
revision: d51519689f32196a32af33b075a01d0e7c51e252
metrics:
- type: accuracy value: 97.79009352268791
- type: f1 value: 97.5176076665512
- type: precision value: 97.38136473848286
- type: recall value: 97.79009352268791
- task:
type: BitextMining
dataset:
type: mteb/bucc-bitext-mining
name: MTEB BUCC (zh-en)
config: zh-en
split: test
revision: d51519689f32196a32af33b075a01d0e7c51e252
metrics:
- type: accuracy value: 99.26276987888363
- type: f1 value: 99.20133403545726
- type: precision value: 99.17500438827453
- type: recall value: 99.26276987888363
- task:
type: Classification
dataset:
type: mteb/banking77
name: MTEB Banking77Classification
config: default
split: test
revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
metrics:
- type: accuracy value: 84.72727272727273
- type: f1 value: 84.67672206031433
- task:
type: Clustering
dataset:
type: mteb/biorxiv-clustering-p2p
name: MTEB BiorxivClusteringP2P
config: default
split: test
revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40
metrics:
- type: v_measure value: 35.34220182511161
- task:
type: Clustering
dataset:
type: mteb/biorxiv-clustering-s2s
name: MTEB BiorxivClusteringS2S
config: default
split: test
revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908
metrics:
- type: v_measure value: 33.4987096128766
- task:
type: Retrieval
dataset:
type: BeIR/cqadupstack
name: MTEB CQADupstackRetrieval
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 25.558249999999997
- type: map_at_10 value: 34.44425000000001
- type: map_at_100 value: 35.59833333333333
- type: map_at_1000 value: 35.706916666666665
- type: map_at_3 value: 31.691749999999995
- type: map_at_5 value: 33.252916666666664
- type: mrr_at_1 value: 30.252666666666666
- type: mrr_at_10 value: 38.60675
- type: mrr_at_100 value: 39.42666666666666
- type: mrr_at_1000 value: 39.48408333333334
- type: mrr_at_3 value: 36.17441666666665
- type: mrr_at_5 value: 37.56275
- type: ndcg_at_1 value: 30.252666666666666
- type: ndcg_at_10 value: 39.683
- type: ndcg_at_100 value: 44.68541666666667
- type: ndcg_at_1000 value: 46.94316666666668
- type: ndcg_at_3 value: 34.961749999999995
- type: ndcg_at_5 value: 37.215666666666664
- type: precision_at_1 value: 30.252666666666666
- type: precision_at_10 value: 6.904166666666667
- type: precision_at_100 value: 1.0989999999999995
- type: precision_at_1000 value: 0.14733333333333334
- type: precision_at_3 value: 16.037666666666667
- type: precision_at_5 value: 11.413583333333333
- type: recall_at_1 value: 25.558249999999997
- type: recall_at_10 value: 51.13341666666666
- type: recall_at_100 value: 73.08366666666667
- type: recall_at_1000 value: 88.79483333333334
- type: recall_at_3 value: 37.989083333333326
- type: recall_at_5 value: 43.787833333333325
- task:
type: Retrieval
dataset:
type: climate-fever
name: MTEB ClimateFEVER
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 10.338
- type: map_at_10 value: 18.360000000000003
- type: map_at_100 value: 19.942
- type: map_at_1000 value: 20.134
- type: map_at_3 value: 15.174000000000001
- type: map_at_5 value: 16.830000000000002
- type: mrr_at_1 value: 23.257
- type: mrr_at_10 value: 33.768
- type: mrr_at_100 value: 34.707
- type: mrr_at_1000 value: 34.766000000000005
- type: mrr_at_3 value: 30.977
- type: mrr_at_5 value: 32.528
- type: ndcg_at_1 value: 23.257
- type: ndcg_at_10 value: 25.733
- type: ndcg_at_100 value: 32.288
- type: ndcg_at_1000 value: 35.992000000000004
- type: ndcg_at_3 value: 20.866
- type: ndcg_at_5 value: 22.612
- type: precision_at_1 value: 23.257
- type: precision_at_10 value: 8.124
- type: precision_at_100 value: 1.518
- type: precision_at_1000 value: 0.219
- type: precision_at_3 value: 15.679000000000002
- type: precision_at_5 value: 12.117
- type: recall_at_1 value: 10.338
- type: recall_at_10 value: 31.154
- type: recall_at_100 value: 54.161
- type: recall_at_1000 value: 75.21900000000001
- type: recall_at_3 value: 19.427
- type: recall_at_5 value: 24.214
- task:
type: Retrieval
dataset:
type: dbpedia-entity
name: MTEB DBPedia
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 8.498
- type: map_at_10 value: 19.103
- type: map_at_100 value: 27.375
- type: map_at_1000 value: 28.981
- type: map_at_3 value: 13.764999999999999
- type: map_at_5 value: 15.950000000000001
- type: mrr_at_1 value: 65.5
- type: mrr_at_10 value: 74.53800000000001
- type: mrr_at_100 value: 74.71799999999999
- type: mrr_at_1000 value: 74.725
- type: mrr_at_3 value: 72.792
- type: mrr_at_5 value: 73.554
- type: ndcg_at_1 value: 53.37499999999999
- type: ndcg_at_10 value: 41.286
- type: ndcg_at_100 value: 45.972
- type: ndcg_at_1000 value: 53.123
- type: ndcg_at_3 value: 46.172999999999995
- type: ndcg_at_5 value: 43.033
- type: precision_at_1 value: 65.5
- type: precision_at_10 value: 32.725
- type: precision_at_100 value: 10.683
- type: precision_at_1000 value: 1.978
- type: precision_at_3 value: 50
- type: precision_at_5 value: 41.349999999999994
- type: recall_at_1 value: 8.498
- type: recall_at_10 value: 25.070999999999998
- type: recall_at_100 value: 52.383
- type: recall_at_1000 value: 74.91499999999999
- type: recall_at_3 value: 15.207999999999998
- type: recall_at_5 value: 18.563
- task:
type: Classification
dataset:
type: mteb/emotion
name: MTEB EmotionClassification
config: default
split: test
revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
metrics:
- type: accuracy value: 46.5
- type: f1 value: 41.93833713984145
- task:
type: Retrieval
dataset:
type: fever
name: MTEB FEVER
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 67.914
- type: map_at_10 value: 78.10000000000001
- type: map_at_100 value: 78.333
- type: map_at_1000 value: 78.346
- type: map_at_3 value: 76.626
- type: map_at_5 value: 77.627
- type: mrr_at_1 value: 72.74199999999999
- type: mrr_at_10 value: 82.414
- type: mrr_at_100 value: 82.511
- type: mrr_at_1000 value: 82.513
- type: mrr_at_3 value: 81.231
- type: mrr_at_5 value: 82.065
- type: ndcg_at_1 value: 72.74199999999999
- type: ndcg_at_10 value: 82.806
- type: ndcg_at_100 value: 83.677
- type: ndcg_at_1000 value: 83.917
- type: ndcg_at_3 value: 80.305
- type: ndcg_at_5 value: 81.843
- type: precision_at_1 value: 72.74199999999999
- type: precision_at_10 value: 10.24
- type: precision_at_100 value: 1.089
- type: precision_at_1000 value: 0.11299999999999999
- type: precision_at_3 value: 31.268
- type: precision_at_5 value: 19.706000000000003
- type: recall_at_1 value: 67.914
- type: recall_at_10 value: 92.889
- type: recall_at_100 value: 96.42699999999999
- type: recall_at_1000 value: 97.92
- type: recall_at_3 value: 86.21
- type: recall_at_5 value: 90.036
- task:
type: Retrieval
dataset:
type: fiqa
name: MTEB FiQA2018
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 22.166
- type: map_at_10 value: 35.57
- type: map_at_100 value: 37.405
- type: map_at_1000 value: 37.564
- type: map_at_3 value: 30.379
- type: map_at_5 value: 33.324
- type: mrr_at_1 value: 43.519000000000005
- type: mrr_at_10 value: 51.556000000000004
- type: mrr_at_100 value: 52.344
- type: mrr_at_1000 value: 52.373999999999995
- type: mrr_at_3 value: 48.868
- type: mrr_at_5 value: 50.319
- type: ndcg_at_1 value: 43.519000000000005
- type: ndcg_at_10 value: 43.803
- type: ndcg_at_100 value: 50.468999999999994
- type: ndcg_at_1000 value: 53.111
- type: ndcg_at_3 value: 38.893
- type: ndcg_at_5 value: 40.653
- type: precision_at_1 value: 43.519000000000005
- type: precision_at_10 value: 12.253
- type: precision_at_100 value: 1.931
- type: precision_at_1000 value: 0.242
- type: precision_at_3 value: 25.617
- type: precision_at_5 value: 19.383
- type: recall_at_1 value: 22.166
- type: recall_at_10 value: 51.6
- type: recall_at_100 value: 76.574
- type: recall_at_1000 value: 92.192
- type: recall_at_3 value: 34.477999999999994
- type: recall_at_5 value: 41.835
- task:
type: Retrieval
dataset:
type: hotpotqa
name: MTEB HotpotQA
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 39.041
- type: map_at_10 value: 62.961999999999996
- type: map_at_100 value: 63.79899999999999
- type: map_at_1000 value: 63.854
- type: map_at_3 value: 59.399
- type: map_at_5 value: 61.669
- type: mrr_at_1 value: 78.082
- type: mrr_at_10 value: 84.321
- type: mrr_at_100 value: 84.49600000000001
- type: mrr_at_1000 value: 84.502
- type: mrr_at_3 value: 83.421
- type: mrr_at_5 value: 83.977
- type: ndcg_at_1 value: 78.082
- type: ndcg_at_10 value: 71.229
- type: ndcg_at_100 value: 74.10900000000001
- type: ndcg_at_1000 value: 75.169
- type: ndcg_at_3 value: 66.28699999999999
- type: ndcg_at_5 value: 69.084
- type: precision_at_1 value: 78.082
- type: precision_at_10 value: 14.993
- type: precision_at_100 value: 1.7239999999999998
- type: precision_at_1000 value: 0.186
- type: precision_at_3 value: 42.737
- type: precision_at_5 value: 27.843
- type: recall_at_1 value: 39.041
- type: recall_at_10 value: 74.96300000000001
- type: recall_at_100 value: 86.199
- type: recall_at_1000 value: 93.228
- type: recall_at_3 value: 64.105
- type: recall_at_5 value: 69.608
- task:
type: Classification
dataset:
type: mteb/imdb
name: MTEB ImdbClassification
config: default
split: test
revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
metrics:
- type: accuracy value: 90.23160000000001
- type: ap value: 85.5674856808308
- type: f1 value: 90.18033354786317
- task:
type: Retrieval
dataset:
type: msmarco
name: MTEB MSMARCO
config: default
split: dev
revision: None
metrics:
- type: map_at_1 value: 24.091
- type: map_at_10 value: 36.753
- type: map_at_100 value: 37.913000000000004
- type: map_at_1000 value: 37.958999999999996
- type: map_at_3 value: 32.818999999999996
- type: map_at_5 value: 35.171
- type: mrr_at_1 value: 24.742
- type: mrr_at_10 value: 37.285000000000004
- type: mrr_at_100 value: 38.391999999999996
- type: mrr_at_1000 value: 38.431
- type: mrr_at_3 value: 33.440999999999995
- type: mrr_at_5 value: 35.75
- type: ndcg_at_1 value: 24.742
- type: ndcg_at_10 value: 43.698
- type: ndcg_at_100 value: 49.145
- type: ndcg_at_1000 value: 50.23800000000001
- type: ndcg_at_3 value: 35.769
- type: ndcg_at_5 value: 39.961999999999996
- type: precision_at_1 value: 24.742
- type: precision_at_10 value: 6.7989999999999995
- type: precision_at_100 value: 0.95
- type: precision_at_1000 value: 0.104
- type: precision_at_3 value: 15.096000000000002
- type: precision_at_5 value: 11.183
- type: recall_at_1 value: 24.091
- type: recall_at_10 value: 65.068
- type: recall_at_100 value: 89.899
- type: recall_at_1000 value: 98.16
- type: recall_at_3 value: 43.68
- type: recall_at_5 value: 53.754999999999995
- task:
type: Classification
dataset:
type: mteb/mtop_domain
name: MTEB MTOPDomainClassification (en)
config: en
split: test
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
metrics:
- type: accuracy value: 93.66621067031465
- type: f1 value: 93.49622853272142
- task:
type: Classification
dataset:
type: mteb/mtop_domain
name: MTEB MTOPDomainClassification (de)
config: de
split: test
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
metrics:
- type: accuracy value: 91.94702733164272
- type: f1 value: 91.17043441745282
- task:
type: Classification
dataset:
type: mteb/mtop_domain
name: MTEB MTOPDomainClassification (es)
config: es
split: test
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
metrics:
- type: accuracy value: 92.20146764509674
- type: f1 value: 91.98359080555608
- task:
type: Classification
dataset:
type: mteb/mtop_domain
name: MTEB MTOPDomainClassification (fr)
config: fr
split: test
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
metrics:
- type: accuracy value: 88.99780770435328
- type: f1 value: 89.19746342724068
- task:
type: Classification
dataset:
type: mteb/mtop_domain
name: MTEB MTOPDomainClassification (hi)
config: hi
split: test
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
metrics:
- type: accuracy value: 89.78486912871998
- type: f1 value: 89.24578823628642
- task:
type: Classification
dataset:
type: mteb/mtop_domain
name: MTEB MTOPDomainClassification (th)
config: th
split: test
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
metrics:
- type: accuracy value: 88.74502712477394
- type: f1 value: 89.00297573881542
- task:
type: Classification
dataset:
type: mteb/mtop_intent
name: MTEB MTOPIntentClassification (en)
config: en
split: test
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
metrics:
- type: accuracy value: 77.9046967624259
- type: f1 value: 59.36787125785957
- task:
type: Classification
dataset:
type: mteb/mtop_intent
name: MTEB MTOPIntentClassification (de)
config: de
split: test
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
metrics:
- type: accuracy value: 74.5280360664976
- type: f1 value: 57.17723440888718
- task:
type: Classification
dataset:
type: mteb/mtop_intent
name: MTEB MTOPIntentClassification (es)
config: es
split: test
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
metrics:
- type: accuracy value: 75.44029352901934
- type: f1 value: 54.052855531072964
- task:
type: Classification
dataset:
type: mteb/mtop_intent
name: MTEB MTOPIntentClassification (fr)
config: fr
split: test
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
metrics:
- type: accuracy value: 70.5606013153774
- type: f1 value: 52.62215934386531
- task:
type: Classification
dataset:
type: mteb/mtop_intent
name: MTEB MTOPIntentClassification (hi)
config: hi
split: test
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
metrics:
- type: accuracy value: 73.11581211903908
- type: f1 value: 52.341291845645465
- task:
type: Classification
dataset:
type: mteb/mtop_intent
name: MTEB MTOPIntentClassification (th)
config: th
split: test
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
metrics:
- type: accuracy value: 74.28933092224233
- type: f1 value: 57.07918745504911
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (af)
config: af
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 62.38063214525892
- type: f1 value: 59.46463723443009
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (am)
config: am
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 56.06926698049766
- type: f1 value: 52.49084283283562
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ar)
config: ar
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 60.74983187626093
- type: f1 value: 56.960640620165904
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (az)
config: az
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 64.86550100874243
- type: f1 value: 62.47370548140688
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (bn)
config: bn
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 63.971082716879636
- type: f1 value: 61.03812421957381
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (cy)
config: cy
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 54.98318762609282
- type: f1 value: 51.51207916008392
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (da)
config: da
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.45527908540686
- type: f1 value: 66.16631905400318
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (de)
config: de
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.32750504371216
- type: f1 value: 66.16755288646591
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (el)
config: el
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.09213180901143
- type: f1 value: 66.95654394661507
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (en)
config: en
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 73.75588433086752
- type: f1 value: 71.79973779656923
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (es)
config: es
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 70.49428379287154
- type: f1 value: 68.37494379215734
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (fa)
config: fa
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.90921318090115
- type: f1 value: 66.79517376481645
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (fi)
config: fi
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 70.12104909213181
- type: f1 value: 67.29448842879584
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (fr)
config: fr
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.34095494283793
- type: f1 value: 67.01134288992947
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (he)
config: he
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 67.61264290517822
- type: f1 value: 64.68730512660757
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (hi)
config: hi
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 67.79757901815738
- type: f1 value: 65.24938539425598
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (hu)
config: hu
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.68728984532616
- type: f1 value: 67.0487169762553
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (hy)
config: hy
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 62.07464694014795
- type: f1 value: 59.183532276789286
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (id)
config: id
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 70.04707464694015
- type: f1 value: 67.66829629003848
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (is)
config: is
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 62.42434431741762
- type: f1 value: 59.01617226544757
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (it)
config: it
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 70.53127101546738
- type: f1 value: 68.10033760906255
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ja)
config: ja
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 72.50504371217215
- type: f1 value: 69.74931103158923
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (jv)
config: jv
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 57.91190316072628
- type: f1 value: 54.05551136648796
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ka)
config: ka
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 51.78211163416275
- type: f1 value: 49.874888544058535
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (km)
config: km
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 47.017484868863484
- type: f1 value: 44.53364263352014
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (kn)
config: kn
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 62.16207128446537
- type: f1 value: 59.01185692320829
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ko)
config: ko
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.42501681237391
- type: f1 value: 67.13169450166086
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (lv)
config: lv
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 67.0780094149294
- type: f1 value: 64.41720167850707
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ml)
config: ml
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 65.57162071284466
- type: f1 value: 62.414138683804424
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (mn)
config: mn
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 61.71149966375252
- type: f1 value: 58.594805125087234
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ms)
config: ms
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 66.03900470746471
- type: f1 value: 63.87937257883887
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (my)
config: my
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 60.8776059179556
- type: f1 value: 57.48587618059131
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (nb)
config: nb
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.87895090786819
- type: f1 value: 66.8141299430347
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (nl)
config: nl
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 70.45057162071285
- type: f1 value: 67.46444039673516
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (pl)
config: pl
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 71.546738399462
- type: f1 value: 68.63640876702655
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (pt)
config: pt
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 70.72965702757229
- type: f1 value: 68.54119560379115
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ro)
config: ro
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 68.35574983187625
- type: f1 value: 65.88844917691927
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ru)
config: ru
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 71.70477471418964
- type: f1 value: 69.19665697061978
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (sl)
config: sl
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 67.0880968392737
- type: f1 value: 64.76962317666086
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (sq)
config: sq
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 65.18493611297916
- type: f1 value: 62.49984559035371
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (sv)
config: sv
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 71.75857431069265
- type: f1 value: 69.20053687623418
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (sw)
config: sw
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 58.500336247478145
- type: f1 value: 55.2972398687929
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ta)
config: ta
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 62.68997982515132
- type: f1 value: 59.36848202755348
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (te)
config: te
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 63.01950235373235
- type: f1 value: 60.09351954625423
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (th)
config: th
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 68.29186281102892
- type: f1 value: 67.57860496703447
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (tl)
config: tl
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 64.77471418964357
- type: f1 value: 61.913983147713836
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (tr)
config: tr
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.87222595830532
- type: f1 value: 66.03679033708141
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (ur)
config: ur
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 64.04505716207127
- type: f1 value: 61.28569169817908
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (vi)
config: vi
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 69.38466711499663
- type: f1 value: 67.20532357036844
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (zh-CN)
config: zh-CN
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 71.12306657700067
- type: f1 value: 68.91251226588182
- task:
type: Classification
dataset:
type: mteb/amazon_massive_intent
name: MTEB MassiveIntentClassification (zh-TW)
config: zh-TW
split: test
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
metrics:
- type: accuracy value: 66.20040349697378
- type: f1 value: 66.02657347714175
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (af)
config: af
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 68.73907195696032
- type: f1 value: 66.98484521791418
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (am)
config: am
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 60.58843308675185
- type: f1 value: 58.95591723092005
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ar)
config: ar
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 66.22730329522528
- type: f1 value: 66.0894499712115
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (az)
config: az
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 66.48285137861465
- type: f1 value: 65.21963176785157
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (bn)
config: bn
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 67.74714189643578
- type: f1 value: 66.8212192745412
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (cy)
config: cy
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 59.09213180901143
- type: f1 value: 56.70735546356339
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (da)
config: da
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 75.05716207128448
- type: f1 value: 74.8413712365364
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (de)
config: de
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 74.69737726967047
- type: f1 value: 74.7664341963
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (el)
config: el
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 73.90383322125084
- type: f1 value: 73.59201554448323
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (en)
config: en
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 77.51176866173503
- type: f1 value: 77.46104434577758
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (es)
config: es
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 74.31069266980496
- type: f1 value: 74.61048660675635
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (fa)
config: fa
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 72.95225285810356
- type: f1 value: 72.33160006574627
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (fi)
config: fi
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 73.12373907195696
- type: f1 value: 73.20921012557481
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (fr)
config: fr
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 73.86684599865501
- type: f1 value: 73.82348774610831
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (he)
config: he
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 71.40215198386012
- type: f1 value: 71.11945183971858
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (hi)
config: hi
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 72.12844653665098
- type: f1 value: 71.34450495911766
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (hu)
config: hu
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 74.52252858103566
- type: f1 value: 73.98878711342999
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (hy)
config: hy
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 64.93611297915265
- type: f1 value: 63.723200467653385
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (id)
config: id
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 74.11903160726295
- type: f1 value: 73.82138439467096
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (is)
config: is
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 67.15198386012105
- type: f1 value: 66.02172193802167
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (it)
config: it
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 74.32414256893072
- type: f1 value: 74.30943421170574
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ja)
config: ja
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 77.46805648957633
- type: f1 value: 77.62808409298209
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (jv)
config: jv
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 63.318762609280434
- type: f1 value: 62.094284066075076
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ka)
config: ka
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 58.34902488231338
- type: f1 value: 57.12893860987984
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (km)
config: km
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 50.88433086751849
- type: f1 value: 48.2272350802058
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (kn)
config: kn
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 66.4425016812374
- type: f1 value: 64.61463095996173
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ko)
config: ko
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 75.04707464694015
- type: f1 value: 75.05099199098998
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (lv)
config: lv
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 70.50437121721586
- type: f1 value: 69.83397721096314
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ml)
config: ml
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 69.94283792871553
- type: f1 value: 68.8704663703913
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (mn)
config: mn
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 64.79488903833222
- type: f1 value: 63.615424063345436
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ms)
config: ms
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 69.88231338264963
- type: f1 value: 68.57892302593237
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (my)
config: my
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 63.248150638870214
- type: f1 value: 61.06680605338809
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (nb)
config: nb
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 74.84196368527236
- type: f1 value: 74.52566464968763
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (nl)
config: nl
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 74.8285137861466
- type: f1 value: 74.8853197608802
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (pl)
config: pl
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 74.13248150638869
- type: f1 value: 74.3982040999179
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (pt)
config: pt
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 73.49024882313383
- type: f1 value: 73.82153848368573
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ro)
config: ro
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 71.72158708809684
- type: f1 value: 71.85049433180541
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ru)
config: ru
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 75.137861466039
- type: f1 value: 75.37628348188467
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (sl)
config: sl
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 71.86953597848016
- type: f1 value: 71.87537624521661
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (sq)
config: sq
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 70.27572293207801
- type: f1 value: 68.80017302344231
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (sv)
config: sv
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 76.09952925353059
- type: f1 value: 76.07992707688408
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (sw)
config: sw
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 63.140551445864155
- type: f1 value: 61.73855010331415
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ta)
config: ta
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 66.27774041694687
- type: f1 value: 64.83664868894539
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (te)
config: te
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 66.69468728984533
- type: f1 value: 64.76239666920868
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (th)
config: th
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 73.44653665097512
- type: f1 value: 73.14646052013873
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (tl)
config: tl
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 67.71351714862139
- type: f1 value: 66.67212180163382
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (tr)
config: tr
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 73.9946200403497
- type: f1 value: 73.87348793725525
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (ur)
config: ur
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 68.15400134498992
- type: f1 value: 67.09433241421094
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (vi)
config: vi
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 73.11365164761264
- type: f1 value: 73.59502539433753
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (zh-CN)
config: zh-CN
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 76.82582380632145
- type: f1 value: 76.89992945316313
- task:
type: Classification
dataset:
type: mteb/amazon_massive_scenario
name: MTEB MassiveScenarioClassification (zh-TW)
config: zh-TW
split: test
revision: 7d571f92784cd94a019292a1f45445077d0ef634
metrics:
- type: accuracy value: 71.81237390719569
- type: f1 value: 72.36499770986265
- task:
type: Clustering
dataset:
type: mteb/medrxiv-clustering-p2p
name: MTEB MedrxivClusteringP2P
config: default
split: test
revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73
metrics:
- type: v_measure value: 31.480506569594695
- task:
type: Clustering
dataset:
type: mteb/medrxiv-clustering-s2s
name: MTEB MedrxivClusteringS2S
config: default
split: test
revision: 35191c8c0dca72d8ff3efcd72aa802307d469663
metrics:
- type: v_measure value: 29.71252128004552
- task:
type: Reranking
dataset:
type: mteb/mind_small
name: MTEB MindSmallReranking
config: default
split: test
revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69
metrics:
- type: map value: 31.421396787056548
- type: mrr value: 32.48155274872267
- task:
type: Retrieval
dataset:
type: nfcorpus
name: MTEB NFCorpus
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 5.595
- type: map_at_10 value: 12.642000000000001
- type: map_at_100 value: 15.726
- type: map_at_1000 value: 17.061999999999998
- type: map_at_3 value: 9.125
- type: map_at_5 value: 10.866000000000001
- type: mrr_at_1 value: 43.344
- type: mrr_at_10 value: 52.227999999999994
- type: mrr_at_100 value: 52.898999999999994
- type: mrr_at_1000 value: 52.944
- type: mrr_at_3 value: 49.845
- type: mrr_at_5 value: 51.115
- type: ndcg_at_1 value: 41.949999999999996
- type: ndcg_at_10 value: 33.995
- type: ndcg_at_100 value: 30.869999999999997
- type: ndcg_at_1000 value: 39.487
- type: ndcg_at_3 value: 38.903999999999996
- type: ndcg_at_5 value: 37.236999999999995
- type: precision_at_1 value: 43.344
- type: precision_at_10 value: 25.480000000000004
- type: precision_at_100 value: 7.672
- type: precision_at_1000 value: 2.028
- type: precision_at_3 value: 36.636
- type: precision_at_5 value: 32.632
- type: recall_at_1 value: 5.595
- type: recall_at_10 value: 16.466
- type: recall_at_100 value: 31.226
- type: recall_at_1000 value: 62.778999999999996
- type: recall_at_3 value: 9.931
- type: recall_at_5 value: 12.884
- task:
type: Retrieval
dataset:
type: nq
name: MTEB NQ
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 40.414
- type: map_at_10 value: 56.754000000000005
- type: map_at_100 value: 57.457
- type: map_at_1000 value: 57.477999999999994
- type: map_at_3 value: 52.873999999999995
- type: map_at_5 value: 55.175
- type: mrr_at_1 value: 45.278
- type: mrr_at_10 value: 59.192
- type: mrr_at_100 value: 59.650000000000006
- type: mrr_at_1000 value: 59.665
- type: mrr_at_3 value: 56.141
- type: mrr_at_5 value: 57.998000000000005
- type: ndcg_at_1 value: 45.278
- type: ndcg_at_10 value: 64.056
- type: ndcg_at_100 value: 66.89
- type: ndcg_at_1000 value: 67.364
- type: ndcg_at_3 value: 56.97
- type: ndcg_at_5 value: 60.719
- type: precision_at_1 value: 45.278
- type: precision_at_10 value: 9.994
- type: precision_at_100 value: 1.165
- type: precision_at_1000 value: 0.121
- type: precision_at_3 value: 25.512
- type: precision_at_5 value: 17.509
- type: recall_at_1 value: 40.414
- type: recall_at_10 value: 83.596
- type: recall_at_100 value: 95.72
- type: recall_at_1000 value: 99.24
- type: recall_at_3 value: 65.472
- type: recall_at_5 value: 74.039
- task:
type: Retrieval
dataset:
type: quora
name: MTEB QuoraRetrieval
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 70.352
- type: map_at_10 value: 84.369
- type: map_at_100 value: 85.02499999999999
- type: map_at_1000 value: 85.04
- type: map_at_3 value: 81.42399999999999
- type: map_at_5 value: 83.279
- type: mrr_at_1 value: 81.05
- type: mrr_at_10 value: 87.401
- type: mrr_at_100 value: 87.504
- type: mrr_at_1000 value: 87.505
- type: mrr_at_3 value: 86.443
- type: mrr_at_5 value: 87.10799999999999
- type: ndcg_at_1 value: 81.04
- type: ndcg_at_10 value: 88.181
- type: ndcg_at_100 value: 89.411
- type: ndcg_at_1000 value: 89.507
- type: ndcg_at_3 value: 85.28099999999999
- type: ndcg_at_5 value: 86.888
- type: precision_at_1 value: 81.04
- type: precision_at_10 value: 13.406
- type: precision_at_100 value: 1.5350000000000001
- type: precision_at_1000 value: 0.157
- type: precision_at_3 value: 37.31
- type: precision_at_5 value: 24.54
- type: recall_at_1 value: 70.352
- type: recall_at_10 value: 95.358
- type: recall_at_100 value: 99.541
- type: recall_at_1000 value: 99.984
- type: recall_at_3 value: 87.111
- type: recall_at_5 value: 91.643
- task:
type: Clustering
dataset:
type: mteb/reddit-clustering
name: MTEB RedditClustering
config: default
split: test
revision: 24640382cdbf8abc73003fb0fa6d111a705499eb
metrics:
- type: v_measure value: 46.54068723291946
- task:
type: Clustering
dataset:
type: mteb/reddit-clustering-p2p
name: MTEB RedditClusteringP2P
config: default
split: test
revision: 282350215ef01743dc01b456c7f5241fa8937f16
metrics:
- type: v_measure value: 63.216287629895994
- task:
type: Retrieval
dataset:
type: scidocs
name: MTEB SCIDOCS
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 4.023000000000001
- type: map_at_10 value: 10.071
- type: map_at_100 value: 11.892
- type: map_at_1000 value: 12.196
- type: map_at_3 value: 7.234
- type: map_at_5 value: 8.613999999999999
- type: mrr_at_1 value: 19.900000000000002
- type: mrr_at_10 value: 30.516
- type: mrr_at_100 value: 31.656000000000002
- type: mrr_at_1000 value: 31.723000000000003
- type: mrr_at_3 value: 27.400000000000002
- type: mrr_at_5 value: 29.270000000000003
- type: ndcg_at_1 value: 19.900000000000002
- type: ndcg_at_10 value: 17.474
- type: ndcg_at_100 value: 25.020999999999997
- type: ndcg_at_1000 value: 30.728
- type: ndcg_at_3 value: 16.588
- type: ndcg_at_5 value: 14.498
- type: precision_at_1 value: 19.900000000000002
- type: precision_at_10 value: 9.139999999999999
- type: precision_at_100 value: 2.011
- type: precision_at_1000 value: 0.33899999999999997
- type: precision_at_3 value: 15.667
- type: precision_at_5 value: 12.839999999999998
- type: recall_at_1 value: 4.023000000000001
- type: recall_at_10 value: 18.497
- type: recall_at_100 value: 40.8
- type: recall_at_1000 value: 68.812
- type: recall_at_3 value: 9.508
- type: recall_at_5 value: 12.983
- task:
type: STS
dataset:
type: mteb/sickr-sts
name: MTEB SICK-R
config: default
split: test
revision: a6ea5a8cab320b040a23452cc28066d9beae2cee
metrics:
- type: cos_sim_pearson value: 83.967008785134
- type: cos_sim_spearman value: 80.23142141101837
- type: euclidean_pearson value: 81.20166064704539
- type: euclidean_spearman value: 80.18961335654585
- type: manhattan_pearson value: 81.13925443187625
- type: manhattan_spearman value: 80.07948723044424
- task:
type: STS
dataset:
type: mteb/sts12-sts
name: MTEB STS12
config: default
split: test
revision: a0d554a64d88156834ff5ae9920b964011b16384
metrics:
- type: cos_sim_pearson value: 86.94262461316023
- type: cos_sim_spearman value: 80.01596278563865
- type: euclidean_pearson value: 83.80799622922581
- type: euclidean_spearman value: 79.94984954947103
- type: manhattan_pearson value: 83.68473841756281
- type: manhattan_spearman value: 79.84990707951822
- task:
type: STS
dataset:
type: mteb/sts13-sts
name: MTEB STS13
config: default
split: test
revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca
metrics:
- type: cos_sim_pearson value: 80.57346443146068
- type: cos_sim_spearman value: 81.54689837570866
- type: euclidean_pearson value: 81.10909881516007
- type: euclidean_spearman value: 81.56746243261762
- type: manhattan_pearson value: 80.87076036186582
- type: manhattan_spearman value: 81.33074987964402
- task:
type: STS
dataset:
type: mteb/sts14-sts
name: MTEB STS14
config: default
split: test
revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375
metrics:
- type: cos_sim_pearson value: 79.54733787179849
- type: cos_sim_spearman value: 77.72202105610411
- type: euclidean_pearson value: 78.9043595478849
- type: euclidean_spearman value: 77.93422804309435
- type: manhattan_pearson value: 78.58115121621368
- type: manhattan_spearman value: 77.62508135122033
- task:
type: STS
dataset:
type: mteb/sts15-sts
name: MTEB STS15
config: default
split: test
revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3
metrics:
- type: cos_sim_pearson value: 88.59880017237558
- type: cos_sim_spearman value: 89.31088630824758
- type: euclidean_pearson value: 88.47069261564656
- type: euclidean_spearman value: 89.33581971465233
- type: manhattan_pearson value: 88.40774264100956
- type: manhattan_spearman value: 89.28657485627835
- task:
type: STS
dataset:
type: mteb/sts16-sts
name: MTEB STS16
config: default
split: test
revision: 4d8694f8f0e0100860b497b999b3dbed754a0513
metrics:
- type: cos_sim_pearson value: 84.08055117917084
- type: cos_sim_spearman value: 85.78491813080304
- type: euclidean_pearson value: 84.99329155500392
- type: euclidean_spearman value: 85.76728064677287
- type: manhattan_pearson value: 84.87947428989587
- type: manhattan_spearman value: 85.62429454917464
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (ko-ko)
config: ko-ko
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 82.14190939287384
- type: cos_sim_spearman value: 82.27331573306041
- type: euclidean_pearson value: 81.891896953716
- type: euclidean_spearman value: 82.37695542955998
- type: manhattan_pearson value: 81.73123869460504
- type: manhattan_spearman value: 82.19989168441421
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (ar-ar)
config: ar-ar
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 76.84695301843362
- type: cos_sim_spearman value: 77.87790986014461
- type: euclidean_pearson value: 76.91981583106315
- type: euclidean_spearman value: 77.88154772749589
- type: manhattan_pearson value: 76.94953277451093
- type: manhattan_spearman value: 77.80499230728604
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (en-ar)
config: en-ar
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 75.44657840482016
- type: cos_sim_spearman value: 75.05531095119674
- type: euclidean_pearson value: 75.88161755829299
- type: euclidean_spearman value: 74.73176238219332
- type: manhattan_pearson value: 75.63984765635362
- type: manhattan_spearman value: 74.86476440770737
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (en-de)
config: en-de
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 85.64700140524133
- type: cos_sim_spearman value: 86.16014210425672
- type: euclidean_pearson value: 86.49086860843221
- type: euclidean_spearman value: 86.09729326815614
- type: manhattan_pearson value: 86.43406265125513
- type: manhattan_spearman value: 86.17740150939994
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (en-en)
config: en-en
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 87.91170098764921
- type: cos_sim_spearman value: 88.12437004058931
- type: euclidean_pearson value: 88.81828254494437
- type: euclidean_spearman value: 88.14831794572122
- type: manhattan_pearson value: 88.93442183448961
- type: manhattan_spearman value: 88.15254630778304
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (en-tr)
config: en-tr
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 72.91390577997292
- type: cos_sim_spearman value: 71.22979457536074
- type: euclidean_pearson value: 74.40314008106749
- type: euclidean_spearman value: 72.54972136083246
- type: manhattan_pearson value: 73.85687539530218
- type: manhattan_spearman value: 72.09500771742637
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (es-en)
config: es-en
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 80.9301067983089
- type: cos_sim_spearman value: 80.74989828346473
- type: euclidean_pearson value: 81.36781301814257
- type: euclidean_spearman value: 80.9448819964426
- type: manhattan_pearson value: 81.0351322685609
- type: manhattan_spearman value: 80.70192121844177
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (es-es)
config: es-es
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 87.13820465980005
- type: cos_sim_spearman value: 86.73532498758757
- type: euclidean_pearson value: 87.21329451846637
- type: euclidean_spearman value: 86.57863198601002
- type: manhattan_pearson value: 87.06973713818554
- type: manhattan_spearman value: 86.47534918791499
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (fr-en)
config: fr-en
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 85.48720108904415
- type: cos_sim_spearman value: 85.62221757068387
- type: euclidean_pearson value: 86.1010129512749
- type: euclidean_spearman value: 85.86580966509942
- type: manhattan_pearson value: 86.26800938808971
- type: manhattan_spearman value: 85.88902721678429
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (it-en)
config: it-en
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 83.98021347333516
- type: cos_sim_spearman value: 84.53806553803501
- type: euclidean_pearson value: 84.61483347248364
- type: euclidean_spearman value: 85.14191408011702
- type: manhattan_pearson value: 84.75297588825967
- type: manhattan_spearman value: 85.33176753669242
- task:
type: STS
dataset:
type: mteb/sts17-crosslingual-sts
name: MTEB STS17 (nl-en)
config: nl-en
split: test
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
metrics:
- type: cos_sim_pearson value: 84.51856644893233
- type: cos_sim_spearman value: 85.27510748506413
- type: euclidean_pearson value: 85.09886861540977
- type: euclidean_spearman value: 85.62579245860887
- type: manhattan_pearson value: 84.93017860464607
- type: manhattan_spearman value: 85.5063988898453
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (en)
config: en
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 62.581573200584195
- type: cos_sim_spearman value: 63.05503590247928
- type: euclidean_pearson value: 63.652564812602094
- type: euclidean_spearman value: 62.64811520876156
- type: manhattan_pearson value: 63.506842893061076
- type: manhattan_spearman value: 62.51289573046917
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (de)
config: de
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 48.2248801729127
- type: cos_sim_spearman value: 56.5936604678561
- type: euclidean_pearson value: 43.98149464089
- type: euclidean_spearman value: 56.108561882423615
- type: manhattan_pearson value: 43.86880305903564
- type: manhattan_spearman value: 56.04671150510166
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (es)
config: es
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 55.17564527009831
- type: cos_sim_spearman value: 64.57978560979488
- type: euclidean_pearson value: 58.8818330154583
- type: euclidean_spearman value: 64.99214839071281
- type: manhattan_pearson value: 58.72671436121381
- type: manhattan_spearman value: 65.10713416616109
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (pl)
config: pl
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 26.772131864023297
- type: cos_sim_spearman value: 34.68200792408681
- type: euclidean_pearson value: 16.68082419005441
- type: euclidean_spearman value: 34.83099932652166
- type: manhattan_pearson value: 16.52605949659529
- type: manhattan_spearman value: 34.82075801399475
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (tr)
config: tr
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 54.42415189043831
- type: cos_sim_spearman value: 63.54594264576758
- type: euclidean_pearson value: 57.36577498297745
- type: euclidean_spearman value: 63.111466379158074
- type: manhattan_pearson value: 57.584543715873885
- type: manhattan_spearman value: 63.22361054139183
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (ar)
config: ar
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 47.55216762405518
- type: cos_sim_spearman value: 56.98670142896412
- type: euclidean_pearson value: 50.15318757562699
- type: euclidean_spearman value: 56.524941926541906
- type: manhattan_pearson value: 49.955618528674904
- type: manhattan_spearman value: 56.37102209240117
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (ru)
config: ru
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 49.20540980338571
- type: cos_sim_spearman value: 59.9009453504406
- type: euclidean_pearson value: 49.557749853620535
- type: euclidean_spearman value: 59.76631621172456
- type: manhattan_pearson value: 49.62340591181147
- type: manhattan_spearman value: 59.94224880322436
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (zh)
config: zh
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 51.508169956576985
- type: cos_sim_spearman value: 66.82461565306046
- type: euclidean_pearson value: 56.2274426480083
- type: euclidean_spearman value: 66.6775323848333
- type: manhattan_pearson value: 55.98277796300661
- type: manhattan_spearman value: 66.63669848497175
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (fr)
config: fr
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 72.86478788045507
- type: cos_sim_spearman value: 76.7946552053193
- type: euclidean_pearson value: 75.01598530490269
- type: euclidean_spearman value: 76.83618917858281
- type: manhattan_pearson value: 74.68337628304332
- type: manhattan_spearman value: 76.57480204017773
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (de-en)
config: de-en
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 55.922619099401984
- type: cos_sim_spearman value: 56.599362477240774
- type: euclidean_pearson value: 56.68307052369783
- type: euclidean_spearman value: 54.28760436777401
- type: manhattan_pearson value: 56.67763566500681
- type: manhattan_spearman value: 53.94619541711359
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (es-en)
config: es-en
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 66.74357206710913
- type: cos_sim_spearman value: 72.5208244925311
- type: euclidean_pearson value: 67.49254562186032
- type: euclidean_spearman value: 72.02469076238683
- type: manhattan_pearson value: 67.45251772238085
- type: manhattan_spearman value: 72.05538819984538
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (it)
config: it
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 71.25734330033191
- type: cos_sim_spearman value: 76.98349083946823
- type: euclidean_pearson value: 73.71642838667736
- type: euclidean_spearman value: 77.01715504651384
- type: manhattan_pearson value: 73.61712711868105
- type: manhattan_spearman value: 77.01392571153896
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (pl-en)
config: pl-en
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 63.18215462781212
- type: cos_sim_spearman value: 65.54373266117607
- type: euclidean_pearson value: 64.54126095439005
- type: euclidean_spearman value: 65.30410369102711
- type: manhattan_pearson value: 63.50332221148234
- type: manhattan_spearman value: 64.3455878104313
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (zh-en)
config: zh-en
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 62.30509221440029
- type: cos_sim_spearman value: 65.99582704642478
- type: euclidean_pearson value: 63.43818859884195
- type: euclidean_spearman value: 66.83172582815764
- type: manhattan_pearson value: 63.055779168508764
- type: manhattan_spearman value: 65.49585020501449
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (es-it)
config: es-it
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 59.587830825340404
- type: cos_sim_spearman value: 68.93467614588089
- type: euclidean_pearson value: 62.3073527367404
- type: euclidean_spearman value: 69.69758171553175
- type: manhattan_pearson value: 61.9074580815789
- type: manhattan_spearman value: 69.57696375597865
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (de-fr)
config: de-fr
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 57.143220125577066
- type: cos_sim_spearman value: 67.78857859159226
- type: euclidean_pearson value: 55.58225107923733
- type: euclidean_spearman value: 67.80662907184563
- type: manhattan_pearson value: 56.24953502726514
- type: manhattan_spearman value: 67.98262125431616
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (de-pl)
config: de-pl
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 21.826928900322066
- type: cos_sim_spearman value: 49.578506634400405
- type: euclidean_pearson value: 27.939890138843214
- type: euclidean_spearman value: 52.71950519136242
- type: manhattan_pearson value: 26.39878683847546
- type: manhattan_spearman value: 47.54609580342499
- task:
type: STS
dataset:
type: mteb/sts22-crosslingual-sts
name: MTEB STS22 (fr-pl)
config: fr-pl
split: test
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
metrics:
- type: cos_sim_pearson value: 57.27603854632001
- type: cos_sim_spearman value: 50.709255283710995
- type: euclidean_pearson value: 59.5419024445929
- type: euclidean_spearman value: 50.709255283710995
- type: manhattan_pearson value: 59.03256832438492
- type: manhattan_spearman value: 61.97797868009122
- task:
type: STS
dataset:
type: mteb/stsbenchmark-sts
name: MTEB STSBenchmark
config: default
split: test
revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
metrics:
- type: cos_sim_pearson value: 85.00757054859712
- type: cos_sim_spearman value: 87.29283629622222
- type: euclidean_pearson value: 86.54824171775536
- type: euclidean_spearman value: 87.24364730491402
- type: manhattan_pearson value: 86.5062156915074
- type: manhattan_spearman value: 87.15052170378574
- task:
type: Reranking
dataset:
type: mteb/scidocs-reranking
name: MTEB SciDocsRR
config: default
split: test
revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab
metrics:
- type: map value: 82.03549357197389
- type: mrr value: 95.05437645143527
- task:
type: Retrieval
dataset:
type: scifact
name: MTEB SciFact
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 57.260999999999996
- type: map_at_10 value: 66.259
- type: map_at_100 value: 66.884
- type: map_at_1000 value: 66.912
- type: map_at_3 value: 63.685
- type: map_at_5 value: 65.35499999999999
- type: mrr_at_1 value: 60.333000000000006
- type: mrr_at_10 value: 67.5
- type: mrr_at_100 value: 68.013
- type: mrr_at_1000 value: 68.038
- type: mrr_at_3 value: 65.61099999999999
- type: mrr_at_5 value: 66.861
- type: ndcg_at_1 value: 60.333000000000006
- type: ndcg_at_10 value: 70.41
- type: ndcg_at_100 value: 73.10600000000001
- type: ndcg_at_1000 value: 73.846
- type: ndcg_at_3 value: 66.133
- type: ndcg_at_5 value: 68.499
- type: precision_at_1 value: 60.333000000000006
- type: precision_at_10 value: 9.232999999999999
- type: precision_at_100 value: 1.0630000000000002
- type: precision_at_1000 value: 0.11299999999999999
- type: precision_at_3 value: 25.667
- type: precision_at_5 value: 17.067
- type: recall_at_1 value: 57.260999999999996
- type: recall_at_10 value: 81.94399999999999
- type: recall_at_100 value: 93.867
- type: recall_at_1000 value: 99.667
- type: recall_at_3 value: 70.339
- type: recall_at_5 value: 76.25
- task:
type: PairClassification
dataset:
type: mteb/sprintduplicatequestions-pairclassification
name: MTEB SprintDuplicateQuestions
config: default
split: test
revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46
metrics:
- type: cos_sim_accuracy value: 99.74356435643564
- type: cos_sim_ap value: 93.13411948212683
- type: cos_sim_f1 value: 86.80521991300147
- type: cos_sim_precision value: 84.00374181478017
- type: cos_sim_recall value: 89.8
- type: dot_accuracy value: 99.67920792079208
- type: dot_ap value: 89.27277565444479
- type: dot_f1 value: 83.9276990718124
- type: dot_precision value: 82.04393505253104
- type: dot_recall value: 85.9
- type: euclidean_accuracy value: 99.74257425742574
- type: euclidean_ap value: 93.17993008259062
- type: euclidean_f1 value: 86.69396110542476
- type: euclidean_precision value: 88.78406708595388
- type: euclidean_recall value: 84.7
- type: manhattan_accuracy value: 99.74257425742574
- type: manhattan_ap value: 93.14413755550099
- type: manhattan_f1 value: 86.82483594144371
- type: manhattan_precision value: 87.66564729867483
- type: manhattan_recall value: 86
- type: max_accuracy value: 99.74356435643564
- type: max_ap value: 93.17993008259062
- type: max_f1 value: 86.82483594144371
- task:
type: Clustering
dataset:
type: mteb/stackexchange-clustering
name: MTEB StackExchangeClustering
config: default
split: test
revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259
metrics:
- type: v_measure value: 57.525863806168566
- task:
type: Clustering
dataset:
type: mteb/stackexchange-clustering-p2p
name: MTEB StackExchangeClusteringP2P
config: default
split: test
revision: 815ca46b2622cec33ccafc3735d572c266efdb44
metrics:
- type: v_measure value: 32.68850574423839
- task:
type: Reranking
dataset:
type: mteb/stackoverflowdupquestions-reranking
name: MTEB StackOverflowDupQuestions
config: default
split: test
revision: e185fbe320c72810689fc5848eb6114e1ef5ec69
metrics:
- type: map value: 49.71580650644033
- type: mrr value: 50.50971903913081
- task:
type: Summarization
dataset:
type: mteb/summeval
name: MTEB SummEval
config: default
split: test
revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c
metrics:
- type: cos_sim_pearson value: 29.152190498799484
- type: cos_sim_spearman value: 29.686180371952727
- type: dot_pearson value: 27.248664793816342
- type: dot_spearman value: 28.37748983721745
- task:
type: Retrieval
dataset:
type: trec-covid
name: MTEB TRECCOVID
config: default
split: test
revision: None
metrics:
- type: map_at_1 value: 0.20400000000000001
- type: map_at_10 value: 1.6209999999999998
- type: map_at_100 value: 9.690999999999999
- type: map_at_1000 value: 23.733
- type: map_at_3 value: 0.575
- type: map_at_5 value: 0.885
- type: mrr_at_1 value: 78
- type: mrr_at_10 value: 86.56700000000001
- type: mrr_at_100 value: 86.56700000000001
- type: mrr_at_1000 value: 86.56700000000001
- type: mrr_at_3 value: 85.667
- type: mrr_at_5 value: 86.56700000000001
- type: ndcg_at_1 value: 76
- type: ndcg_at_10 value: 71.326
- type: ndcg_at_100 value: 54.208999999999996
- type: ndcg_at_1000 value: 49.252
- type: ndcg_at_3 value: 74.235
- type: ndcg_at_5 value: 73.833
- type: precision_at_1 value: 78
- type: precision_at_10 value: 74.8
- type: precision_at_100 value: 55.50000000000001
- type: precision_at_1000 value: 21.836
- type: precision_at_3 value: 78
- type: precision_at_5 value: 78
- type: recall_at_1 value: 0.20400000000000001
- type: recall_at_10 value: 1.894
- type: recall_at_100 value: 13.245999999999999
- type: recall_at_1000 value: 46.373
- type: recall_at_3 value: 0.613
- type: recall_at_5 value: 0.991
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (sqi-eng)
config: sqi-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 95.89999999999999
- type: f1 value: 94.69999999999999
- type: precision value: 94.11666666666667
- type: recall value: 95.89999999999999
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (fry-eng)
config: fry-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 68.20809248554913
- type: f1 value: 63.431048720066066
- type: precision value: 61.69143958161298
- type: recall value: 68.20809248554913
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (kur-eng)
config: kur-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 71.21951219512195
- type: f1 value: 66.82926829268293
- type: precision value: 65.1260162601626
- type: recall value: 71.21951219512195
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (tur-eng)
config: tur-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 97.2
- type: f1 value: 96.26666666666667
- type: precision value: 95.8
- type: recall value: 97.2
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (deu-eng)
config: deu-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 99.3
- type: f1 value: 99.06666666666666
- type: precision value: 98.95
- type: recall value: 99.3
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (nld-eng)
config: nld-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 97.39999999999999
- type: f1 value: 96.63333333333333
- type: precision value: 96.26666666666668
- type: recall value: 97.39999999999999
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (ron-eng)
config: ron-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 96
- type: f1 value: 94.86666666666666
- type: precision value: 94.31666666666668
- type: recall value: 96
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (ang-eng)
config: ang-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 47.01492537313433
- type: f1 value: 40.178867566927266
- type: precision value: 38.179295828549556
- type: recall value: 47.01492537313433
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (ido-eng)
config: ido-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 86.5
- type: f1 value: 83.62537480063796
- type: precision value: 82.44555555555554
- type: recall value: 86.5
- task:
type: BitextMining
dataset:
type: mteb/tatoeba-bitext-mining
name: MTEB Tatoeba (jav-eng)
config: jav-eng
split: test
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
metrics:
- type: accuracy value: 80.48780487804879
- type: f1 value: 75.45644599303138
- type: precision value: 73.37398373983739
- type: recall value: 80.48780487804879
- task: type: BitextMining dataset:
- task:
type: Classification
dataset:
type: mteb/amazon_counterfactual
name: MTEB AmazonCounterfactualClassification (en)
config: en
split: test
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
metrics:
[Content truncated...]
📝 Limitations & Considerations
- • Benchmark scores may vary based on evaluation methodology and hardware configuration.
- • VRAM requirements are estimates; actual usage depends on quantization and batch size.
- • FNI scores are relative rankings and may change as new models are added.
- ⚠ License Unknown: Verify licensing terms before commercial use.
- • Source: Unknown
📜
Cite this model
Academic & Research Attribution
BibTeX
@misc{hf_model__intfloat__multilingual_e5_large,
author = {intfloat},
title = {undefined Model},
year = {2026},
howpublished = {\url{https://huggingface.co/intfloat/multilingual-e5-large}},
note = {Accessed via Free2AITools Knowledge Fortress}
} APA Style
intfloat. (2026). undefined [Model]. Free2AITools. https://huggingface.co/intfloat/multilingual-e5-large
🔄 Daily sync (03:00 UTC)
AI Summary: Based on Hugging Face metadata. Not a recommendation.
🛡️ Model Transparency Report
Verified data manifest for traceability and transparency.
100% Data Disclosure Active
🆔 Identity & Source
- id
- hf-model--intfloat--multilingual-e5-large
- author
- intfloat
- tags
- sentence-transformerspytorchonnxsafetensorsopenvinoxlm-robertamtebsentence transformerssentence-similarityfeature-extractionmultilingualafamarasazbebgbnbrbscacscydadeeleneoeseteufafifrfygagdglguhahehihrhuhyidisitjajvkakkkmknkokukylaloltlvmgmkmlmnmrmsmynenlnoomorpaplpsptrorusasdsiskslsosqsrsusvswtatethtltrugukuruzvixhyizharxiv:2402.05672arxiv:2108.08787arxiv:2104.08663arxiv:2210.07316license:mitmodel-indextext-embeddings-inferenceendpoints_compatibledeploy:azureregion:us
⚙️ Technical Specs
- architecture
- XLMRobertaModel
- params billions
- 0.56
- context length
- 4,096
- vram gb
- 1.7
- vram is estimated
- true
- vram formula
- VRAM ≈ (params * 0.75) + 0.8GB (KV) + 0.5GB (OS)
📊 Engagement & Metrics
- likes
- 1,095
- downloads
- 3,338,259
Free2AITools Constitutional Data Pipeline: Curated disclosure mode active. (V15.x Standard)