Support for factors in input data

[grst]

Description of feature

Right now, the only input format supported are text files, these are inherently limited in that they don't support:

• data types (numeric vs. character)
• factor levels (i.e. categorical variables with a predefined ordering).

Data types are guessed on reading in the data and factor levels are lost and replaced with alphabetical order of the categories.

For the "simple" contrasts (variable, baseline, target) this is not so much of an issue, as every comparison is defined explicitly. Numerical variables can be converted to categoricals for this purpose (I'm actually not sure what happens if a numeric variable is supplied to blocking_factors).

For complex contrasts with formula and make_contrast_str, the order of factors is often very important as the contrast string depends on which category is considered the baseline. For complex model designs, it is also convenient and common practice to explicitly set the baseline (e.g. pre-treatment), and then specify contrasts relative to the baseline.

There should be some way to specify factor levels (and possibly variable types) in differentialabundance. Two options I could imagine:

• support some sort of binary format as input, e.g. parquet as a language-agnostic tabular format, or SummarizedExperiment/AnnData as discussed in Support SummarizedExperiment as input #365.
• allow to explicitly specify the factor levels and possibly variable types, e.g.

  variables:
     VISIT: 
        type: categorical
        categories: ["SCREENING 01", "C02_V02"]
     age:
       type: numeric

@pinin4fjords I'm happy to hear your thoughts on this!

[pinin4fjords]

Yep, sounds good to me, optional variables as a peer of contrasts.

[grst]

Nice.
I was thinking that the content of variables could be a standard JSON schema. Then the fields that are specified there would be validated against that schema. Additional fields not specified are ignored by JSON schema validation. Factor levels can be represented as enum.

Above example would look like

variables: 
  VISIT:
    type: string
    description: sampling timepoint of the clinical trial 
    enum: ["SCREENING 01", "C02_V02"]
  age:
    type: integer
    minimum: 0

Alternatively, we can also keep it simple, but custom, with just type and categories.

[pinin4fjords]

We should probably work up some prototypes to assess the implications, but I'm open.