What's the feature?
More and more of our CRO sponsors require it either for the primary or at least sensitivity analysis in case of severe conditional non-normality of the response (under the assumption that such analysis still makes sense for the population).
Unlike ranked (testing for stochastic superiority, which doesn't map to anything concrete, like test of locations, dispersions, shapes - unless strong conditions hold) or quantile-based tests, this analysis does not change the planned null and alternative hypotheses, keeping the estimand in its original form.
It returns both resampling-based (bootstrap, permutation - depending on function; both are implemented) and classic (normality-based) output, so one can instantly compare them and assess the impact of the distributional problems on the inference.
In R there are two packages for that, nptest and MKinfer. I've never tested nptest, but have been working on daily basis with MKinfer (several times I examined the code while going through the Jannsen's papers, so I trust it).
> (tmp <- MKinfer::perm.t.test(rnorm(10), rnorm(10)))
Permutation Welch Two Sample t-test
data: rnorm(10) and rnorm(10)
number of permutations: 9999
(Monte-Carlo) permutation p-value = 0.3622
permutation difference of means (SE) = -0.4273352 (0.4676681)
95 percent (Monte-Carlo) permutation percentile confidence interval:
-1.3329959 0.4889624
Results without permutation:
t = -0.92699, df = 15.88, p-value = 0.3678
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-1.4321321 0.5610827
sample estimates:
mean of x mean of y
-0.5369586 -0.1014340
> str(tmp)
List of 18
$ statistic : Named num -0.927
..- attr(*, "names")= chr "t"
$ parameter : Named num 15.9
..- attr(*, "names")= chr "df"
$ p.value : num 0.368
$ perm.p.value : num 0.362
$ R : num 9999
$ R.true : int 9999
$ p.min : logi FALSE
$ conf.int : num [1:2] -1.432 0.561
..- attr(*, "conf.level")= num 0.95
$ perm.conf.int : Named num [1:2] -1.333 0.489
..- attr(*, "names")= chr [1:2] "2.5%" "97.5%"
..- attr(*, "conf.level")= num 0.95
$ estimate : Named num [1:2] -0.537 -0.101
..- attr(*, "names")= chr [1:2] "mean of x" "mean of y"
$ perm.estimate : Named num -0.427
..- attr(*, "names")= chr "permutation difference of means"
$ null.value : Named num 0
..- attr(*, "names")= chr "difference in means"
$ stderr : num 0.47
$ perm.stderr : num 0.468
$ alternative : chr "two.sided"
$ method : chr "Permutation Welch Two Sample t-test"
$ data.name : chr "rnorm(10) and rnorm(10)"
$ perm.statistic: NULL
- attr(*, "class")= chr [1:2] "perm.htest" "htest"
It implements two classes: perm.htest and htest, so broom::tidy() handles it, but returns only the classic htest part.
What's the feature?
More and more of our CRO sponsors require it either for the primary or at least sensitivity analysis in case of severe conditional non-normality of the response (under the assumption that such analysis still makes sense for the population).
Unlike ranked (testing for stochastic superiority, which doesn't map to anything concrete, like test of locations, dispersions, shapes - unless strong conditions hold) or quantile-based tests, this analysis does not change the planned null and alternative hypotheses, keeping the estimand in its original form.
It returns both resampling-based (bootstrap, permutation - depending on function; both are implemented) and classic (normality-based) output, so one can instantly compare them and assess the impact of the distributional problems on the inference.
In R there are two packages for that, nptest and MKinfer. I've never tested nptest, but have been working on daily basis with MKinfer (several times I examined the code while going through the Jannsen's papers, so I trust it).
It implements two classes:
perm.htestandhtest, so broom::tidy() handles it, but returns only the classichtestpart.