reproducibility and seed in dmn()

[handibles]

Thanks for the implementation, and the great reference page. I appreciate that this issue relates to an older package, but hoping a description will be of help to others.

In the reference, the best grouping for the test data is always k=4, and is always at a Laplace value of 39477 (run 6 times):

For my own microbiome count data, there is a huge variability in optimal $k$ (and group membership), and the Laplace value changes across a range of values.

I've tried to counter this by setting seed before each call to mclapply, and I've tried setting the mc.set.seed param in mclapply to FALSE as per the manpage for mclapply - this does seem to temper the variability, but I still see more varaibility than in the tutorial.

Is this simply a reflection of the complexity in my own data? How deterministic is the dmn() implementation?

As an aside, I'm also seeing an additional, second level of variability in outcome if the order of rows is shuffled. I might end up permuting the rows and polling for solution from this randomised start.

[mtmorgan]

The help page ?dmn (help page) suggests setting seed= a number between 0 and .Machine$integer.max. Reproducibility for dmngroup might additionally involve setting .lapply=lapply. I might be able to provide something additional if you provide the specific commands you are using, especially if you're able to share your data.

[jmitchell81]

I have also encountered the same issue of result reproducibility as @handibles. I had run dmn() with k=1 to k=8 and the same seed=123 for each value of k using the mclapply approach demonstrated in the package's vignette. k=7 appeared to be the optimal fit based on laplace distance and AIC. I then reran my script with k=1 to k=12 and the same seed=123. In the results from the second run, the group membership of samples in the k=7 result has changed drastically, despite using the same seed. My script used 'set.seed()' before mclapply as well as the seed argument to dmn() .

[mtmorgan]

@jmitchell81 a fully reproducible example, with real or simulated / obscured data and the explicit (minimal) code necessary to reproduce would be very helpful. Feel free to share an example dataset off line if you don't want to post it here.