Auto saved by Logseq

pages/ReadingNotes/Reproducibility crisis of ML experiments.md

@@ -0,0 +1,18 @@
+---
+tags:
+ - '#readingnotes'
+ - '#projects/proposals'
+ - '#machinelearning'
+---
+
+The factors that compromise the reproducibility of ML experiments are manifold including the following ones:
+- Unavailability or outdated source code
+- Unavailability of the datasets used for training and evaluation
+- Unavailability of the reference implementation
+- Unclear or missing description of the parameters that need to be set for obtaining the presented results
+- Missing guidelines on the selection of the training, test and evaluation data
+- Missing information of the required libraries/packages and their corresponding version
+- Possible tweaks performed in the source code and that are not mentioned in the paper
+- Missing information about the underpinning techniques, which have been used
+- Lack of documentation about the preprocessing phases, like data preparation and cleaning
+- Hardware requirements that could be not satisfied to train the large neural network