tag:blogger.com,1999:blog-7890764972166411105.post4538785506206991395..comments2024-04-13T15:46:56.356+02:00Comments on Nick Brown's blog: An interesting lack of randomness in a published dataset: Scott and Dixson (2016)Nick Brownhttp://www.blogger.com/profile/00172030184497186082noreply@blogger.comBlogger4125tag:blogger.com,1999:blog-7890764972166411105.post-87123950808745702192023-07-02T14:25:37.762+02:002023-07-02T14:25:37.762+02:00I didn't approach this very scientifically, be...I didn't approach this very scientifically, because I had no intention of calculating an exact overall probability, and I knew that there were limitations with, for example, some lack of independence in the two runs of each fish (corresponding to the paired columns of data). At first I chose all the "chunks" that appeared to have a mean of around 12; although I calculated their SD I didn't calculate their mean, but that seemed OK as I hadn't specified a threshold (e.g., would 13.3 count?). I then thought that I could give the authors the benefit of the doubt by including any chunk where even one fish scored 12 (i.e., 50-50 for each flume), which of course is a bit silly since 12 isn't all that different from 13. So I'm not sure that my method is especially valid, but when seven of the results give less than 1 in a million chance, and five of those are totally independent, you don't need to do the full formal analysis, I think.Nick Brownhttps://www.blogger.com/profile/07481236547943428014noreply@blogger.comtag:blogger.com,1999:blog-7890764972166411105.post-83561277506816006662022-08-10T21:40:47.659+02:002022-08-10T21:40:47.659+02:00(This reply is coming from Nick - Google won't...(This reply is coming from Nick - Google won't let me sign in, for some reason)<br /><br />I didn't approach this very scientifically, because I had no intention of calculating an exact overall probability, and I knew that there were limitations with, for example, some lack of independence in the two runs of each fish (corresponding to the paired columns of data). At first I chose all the "chunks" that appeared to have a mean of around 12; although I calculated their SD I didn't calculate their mean, but that seemed OK as I hadn't specified a threshold (e.g., would 13.3 count?). I then thought that I could give the authors the benefit of the doubt by including any chunk where even one fish scored 12 (i.e., 50-50 for each flume), which of course is a bit silly since 12 isn't all that different from 13. So I'm not sure that my method is especially valid, but when seven of the results give less than 1 in a million chance, and five of those are totally independent, you don't need to do the full formal analysis, I think.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-7890764972166411105.post-73737863847135669212022-08-10T08:48:36.192+02:002022-08-10T08:48:36.192+02:00One question about this: did you select rows that ...One question about this: did you select rows that had a mean of about 12, or did you select fish species where the authors said there was no treatment effect? In the former case we clearly shouldn’t expect much variation in the number of times the fish picks each option—we’ve selected to some extent on the outcome variable not varying much. In the latter case, it’s less clear: we wouldn’t necessarily expect totally random variation, since “there’s no treatment effect” implicitly throws out fish with p<0.05. <br /><br />This doesn’t affect the spirit of your critique but it does what we would expect the variation in the outcome to look like.Jason Kerwinhttps://www.blogger.com/profile/11446743337803791862noreply@blogger.comtag:blogger.com,1999:blog-7890764972166411105.post-4525765660815053702022-08-10T04:09:15.477+02:002022-08-10T04:09:15.477+02:00I cannot wait to reuse 'homeopathic' in my...I cannot wait to reuse 'homeopathic' in my next stats-based argument. A thousand thanks.Anonymousnoreply@blogger.com