Gordon does not support Suite in testFilter

[kostya29]

When I have testFilter with value testFilter = "my.package.TestSuite" and run job :app:gordonBrand
I got
Execution failed for task ':app:gordonBrand'.

No test cases found

[abergfeld]

@kostya29

Are you referring to this style of tests? https://developer.android.com/training/testing/unit-testing/instrumented-unit-tests#test-suites

If so that would make sense because we are collecting tests based on annotated methods in classes. It could be possible to try and parse the annotation on the class and determine what other tests to run. Gordon was built with the idea of running tests based on packages where each package is a feature / domain.

Alternatively Gordon's filtering supports comma separated classes as well to create a similar test run.

If that isn't what you're referring to could you elaborate and maybe add example test classes / functions and the command you're running?

[kostya29]

I have suite in package com.brand.subdomain.suite

@RunWith(Suite::class) @SuiteClasses( LoginTest::class, RespondentTest::class, AlarmTest::class, CorridorTest::class, RoomTest::class, CameraCareTest::class, CommandButtonsTest::class, BrandTest::class, UtilsTest::class ) class Regression

These test classes
LoginTest::class,
RespondentTest::class,
AlarmTest::class,
CorridorTest::class,
RoomTest::class,
CameraCareTest::class,
CommandButtonsTest::class
are situated in package com.brand.subdomain.instrumentation.ui
and BrandTest::class, UtilsTest::class are situated in package com.brand.subdomain.instrumentation.unit

My gradle command is :app:gordonBrand as I wrote in first message, in gradle config
I have
' gordon {
retryQuota = 3
testTimeoutMillis = 600_000L
testFilter = "com.brand.subdomain.suite.Regression"
}'

Also at this time usual gradle command (not gordon)
:app:connectedBrandDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.class=com.brand.subdomain.suite.Regression
works successfully

[joshschriever]

@kostya29 this might be something we can add support for at some point. In the meantime, you could always just set your test filter to
"com.brand.subdomain.instrumentation.ui,com.brand.subdomain.instrumentation.unit"
or
"LoginTest,RespondentTest,AlarmTest,CorridorTest,RoomTest,CameraCareTest,CommandButtonsTest,BrandTest,UtilsTest"

[kostya29]

@joshschriever When I use "LoginTest,RespondentTest,AlarmTest,CorridorTest,RoomTest,CameraCareTest,CommandButtonsTest,BrandTest,UtilsTest" gordon runs these tests in another order then in this string, there is not way to point out order of execution tests for gordon

[joshschriever]

Running tests in a particular order is something I don't think we'll support in any case. If your tests only work when run in a particular order, I would recommend finding a way to decouple them.

For example, my company has a suite of UI tests where clearing shared preferences in-between tests decouples them, so we use a simple custom test application, in which we override onCreate like so:

override fun onCreate() {
    PreferenceManager.getDefaultSharedPreferences(this).edit().clear().commit()
    super.onCreate()
}