CASMI 2022 - Results

Thank you for your participation in the CASMI 2022! This CASMI trial was aimed mostly to gauge which methods are used by the community, with which method or parameter variants, and how close we are with respect to compound ID of unknowns.

An excerpt of methods used by the participants, and uploaded submitted results, can be found here: https://drive.google.com/drive/u/3/folders/12cCAdUjV_mXZ7tavdz9A_r18B6rDk437

A big round of applause and a "Thank you" shoutout also to the CASMI panelists and those of you who sent in recordings to share your thoughts on the CASMI 2022 trial. 

List of participants:

  1. Adamo Young - U. of Toronto
  2. Adriano Rutz - U. of Geneva
  3. Armando Alcazar - U. of British Columbia
  4. Carl Namini & Jonathan Mosley - the United States Environmental Protection Agency
  5. David Elser - U. of Strasbourg
  6. Dejan Nikolic - U. of Illinois Chicago
  7. Fei Wang - U. of Alberta
  8. Gianfranco Frigerio & Begona Talavera Andujar - U. of Luxembourg
  9. Hiroshi Tsugawa - Tokyo U. of Agriculture & Technology
  10. Kai Dührkop - U. of Jena
  11. Louis-Félix Nothias - U. of Geneva
  12. Sajjan Mehta & Paolo Bonini - oloBion
  13. Shipei Xing - U. of British Columbia
  14. Si Wei - Nanjing U.
  15. Habtom Ressom - OmicsCraft LLC
  16. Yuhui Hong - Indiana U.

Results will be discussed based on four different categories:

1. Correct annotation of adducts

2. Correct elemental formulas

3. Correct compound structure classes

4. Correct 2D chemical structures

View workshop presentation here: https://drive.google.com/drive/folders/1h3Hf8vxDQUhDN9aIRKIOBKGR91qgg3cL?usp=sharing

Correct annotation of adducts

Group Priority/Bonus Total Attempts for Adducts Correct number of Adducts Correct % of Adducts
Hong_pub priority 249 32 13
Hong_pred and pub priority 249 45 18
Wang priority 249 135 54
Ressom priority 234 129 55
Young priority 130 74 57
Alcazar priority 223 149 67
Frigerio/Andujar_msdial priority 196 131 67
Namini/Mosley priority 239 166 69
Rutz priority 228 162 71
Xing priority 246 187 76
Frigerio/Andujar_patroon priority 22 17 77
Wei_MSKU_1 priority 249 208 84
Wei_MSKU_2 priority 249 209 84
Wei_MSKU_3 priority 249 209 84
Mehta priority 156 133 85
Tsugawa priority 249 222 89
Nothias priority 228 209 92
Duhrkop priority 222 207 93
Nikolic_POSO priority 79 74 94
Nikolic_KUCA priority 102 96 94
Elser priority Did not submit Did not submit Did not submit

 

Group Priority/Bonus Total Attempts for Adducts Correct number of Adducts Correct % of Adducts
Hong_pub bonus 247 40 16
Hong_pred and pub bonus 247 51 21
Wang bonus 247 147 60
Mehta bonus 33 24 73
Rutz bonus 219 167 76
Xing bonus 247 214 87
Nothias bonus 219 208 95
Duhrkop bonus 219 210 96
Elser bonus Did not submit Did not submit Did not submit

Correct elemental formulas

Group Priority/Bonus Total Attempts for Formulas Correct number of Formulas Correct % of Formulas
Young priority 130 0 0
Hong_pub priority 249 8 3
Hong_pred and pub priority 249 20 8
Frigerio/Andujar_patroon priority 22 5 23
Ressom priority 234 74 32
Wang priority 249 91 37
Namini/Mosley priority 238 136 57
Frigerio/Andujar_msdial priority 196 113 58
Alcazar priority 222 145 65
Wei_MSKU_3 priority 248 172 69
Wei_MSKU_1 priority 248 181 73
Wei_MSKU_2 priority 248 184 74
Tsugawa priority 249 186 75
Xing priority 246 184 75
Mehta priority 156 117 75
Nikolic_KUCA priority 102 86 84
Nikolic_POSO priority 79 69 87
Rutz priority 228 208 91
Nothias priority 228 211 93
Duhrkop priority 222 208 94
Elser priority Did not submit Did not submit Did not submit

 

Group Priority/Bonus Total Attempts for Formulas Correct number of Formulas Correct % of Formulas
Hong_pub bonus 247 12 5
Hong_pred and pub bonus 247 21 9
Wang bonus 247 86 35
Mehta bonus 33 20 61
Xing bonus 246 211 86
Rutz bonus 219 206 94
Nothias bonus 219 206 94
Duhrkop bonus 219 207 95
Elser bonus Did not submit Did not submit Did not submit

Correct compound structure classes

*For several challenges the class could not be determined by ClassyFire, therefore a key could not be found. These challenges were removed and only challenges with a key were kept.

Group Priority/Bonus Total Attempts of Classes Correct number of Classes Correct % of Classes
Nikolic_POSO priority 36 1 3
Nikolic_KUCA priority 52 5 10
Hong_pub priority 183 52 28
Hong_pred and pub priority 179 56 31
Frigerio/Andujar_msdial priority 135 47 35
Namini/Mosley priority 185 83 45
Frigerio/Andujar_patroon priority 13 6 46
Tsugawa priority 187 106 57
Alcazar priority 135 77 57
Nothias priority 178 107 60
Rutz priority 176 106 60
Mehta priority 122 77 63
Wang priority 195 131 67
Duhrkop priority 168 116 69
Young priority Did not submit Did not submit Did not submit
Ressom priority Did not submit Did not submit Did not submit
Wei_MSKU_3 priority Did not submit Did not submit Did not submit
Wei_MSKU_1 priority Did not submit Did not submit Did not submit
Wei_MSKU_2 priority Did not submit Did not submit Did not submit
Xing priority Did not submit Did not submit Did not submit
Elser priority Did not submit Did not submit Did not submit

 

Group Priority/Bonus Total Attempts of Classes Correct number of Classes Correct % of Classes
Hong_pub bonus 175 53 30
Hong_pred and pub bonus 176 58 33
Mehta bonus 28 16 57
Rutz bonus 172 120 70
Nothias bonus 172 122 71
Duhrkop bonus 165 123 75
Wang bonus 194 149 77
Xing bonus Did not submit Did not submit Did not submit
Elser bonus Did not submit Did not submit Did not submit

Correct 2D chemical structures

Group Priority/Bonus Total Attempts of 2D Structure Correct number of 2D Structure Correct % of 2D Structure
Young priority 130 0 0
Elser priority 97 0 0
Hong_pub priority 249 3 1
Hong_pred and pub priority 249 3 1
Ressom priority 234 10 4
Frigerio/Andujar_patroon priority 22 1 5
Wei_MSKU_3 priority 247 12 5
Alcazar priority 215 12 6
Frigerio/Andujar_msdial priority 196 11 6
Wei_MSKU_2 priority 247 14 6
Wei_MSKU_1 priority 247 18 7
Wang priority 249 23 9
Namini/Mosley priority 235 25 11
Tsugawa priority 241 30 12
Xing priority 246 37 15
Nikolic_POSO priority 37 7 19
Mehta priority 156 32 21
Nothias priority 228 58 25
Rutz priority 228 58 25
Duhrkop priority 222 57 26
Nikolic_KUCA priority 55 21 38

 

Group Priority/Bonus Total Attempts of 2D Structure Correct number of 2D Structure Correct % of 2D Structure
Elser bonus 122 0 0
Hong_pub bonus 247 5 2
Wang bonus 247 10 4
Hong_pred and pub bonus 247 15 6
Xing bonus 246 31 13
Mehta bonus 33 7 21
Rutz bonus 219 62 28
Nothias bonus 219 62 28
Duhrkop bonus 219 64 29