max-lakera commited on
Commit
71acd6e
·
1 Parent(s): bbebd05

Update index.html

Browse files
Files changed (1) hide show
  1. index.html +31 -18
index.html CHANGED
@@ -13,35 +13,48 @@
13
  <p>
14
  This is a demo of MLTest on the dataset
15
  <a href="https://huggingface.co/datasets/marmal88/skin_cancer"><code>marmal88/skin_cancer</code></a>.
 
16
  </p>
17
  <p>
18
  The model has been trained on five models: two variants of Swin Transformers, ViT, ResNet, and BEiT. The test results for each model can
19
  be inspected in the dashboard below.
20
  </p>
21
  <p>
22
- <b>Performance tests</b>: TODO
 
 
23
  </p>
24
  <p>
25
- <b>Failure clusters</b>: TODO
 
 
 
 
26
  </p>
27
  <p>
28
- <b>Robustness</b>: TODO
29
- The following robustness tests were enabled:
30
- - kind: Brightness
31
- - kind: CompressImage
32
- - kind: Contrast
33
- - kind: DarkSpots
34
- - kind: GaussianBlur
35
- - kind: GaussianNoise
36
- - kind: Glare
37
- - kind: GlassBlur
38
- - kind: HorizontalFlip
39
- - kind: MedianBlur
40
- - kind: MotionBlur
41
- - kind: OilSpots
42
- - kind: Perspective
43
- - kind: VerticalFlip
 
 
 
 
 
44
 
 
45
  The full list of transforms supported by MLTest can be found in the <a href="https://docs.lakera.ai/configuration/robustness">documentation</a>.
46
  </p>
47
  <p>
 
13
  <p>
14
  This is a demo of MLTest on the dataset
15
  <a href="https://huggingface.co/datasets/marmal88/skin_cancer"><code>marmal88/skin_cancer</code></a>.
16
+ Each image is labeled with one of three different kinds of cancers.
17
  </p>
18
  <p>
19
  The model has been trained on five models: two variants of Swin Transformers, ViT, ResNet, and BEiT. The test results for each model can
20
  be inspected in the dashboard below.
21
  </p>
22
  <p>
23
+ <b>Performance tests</b>:
24
+ in order to measure how well the model performs, we compute common performance metrics like accuracy,
25
+ precision, recall, F1 score, and more.
26
  </p>
27
  <p>
28
+ <b>Failure clusters</b>:
29
+ these clusters give meaningful insights when the model is failing and can be inspected in the "Failure Clusters" tab.
30
+ These failure clusters are automatically detected
31
+ for different combinations of metadata.
32
+ For example, the BEiT transformer has a significantly lower accuracy on images taken of cancers of the back with class label <code>0</code>.
33
  </p>
34
  <p>
35
+ <b>Robustness</b>: these tests help ML developers evaluate how well their model performs under different conditions.
36
+ These conditions could include different levels of brightness, compression, and many other types of interference.
37
+ </p><p>
38
+ The following robustness tests were enabled for this test case:
39
+ </p>
40
+ <ul>
41
+ <li>kind: Brightness</li>
42
+ <li>kind: CompressImage</li>
43
+ <li>kind: Contrast</li>
44
+ <li>kind: DarkSpots</li>
45
+ <li>kind: GaussianBlur</li>
46
+ <li>kind: GaussianNoise</li>
47
+ <li>kind: Glare</li>
48
+ <li>kind: GlassBlur</li>
49
+ <li>kind: HorizontalFlip</li>
50
+ <li>kind: MedianBlur</li>
51
+ <li>kind: MotionBlur</li>
52
+ <li>kind: OilSpots</li>
53
+ <li>kind: Perspective</li>
54
+ <li>kind: VerticalFlip</li>
55
+ </ul>
56
 
57
+ <p>
58
  The full list of transforms supported by MLTest can be found in the <a href="https://docs.lakera.ai/configuration/robustness">documentation</a>.
59
  </p>
60
  <p>