KCIDB Protocol Fundamentals

[tales-aparecida]

Feedback thread about https://github.com/kernelci/kcidb/wiki/KCIDB-Protocol-Fundamentals

[tales-aparecida]

5. All results can be submitted in one go, one object per submission, and everything in between.

I tripped a bit while reading that. How about:

Results can be submitted in groups of any size, from one at a time to all at once.

[spbnick]

Thanks! Used that with a small amendment.

[tales-aparecida]

7. No field can have null value, except under misc objects.

I would love some short rationale about this, but at very least I'd like to append some sort of guidance, like "don't include the field in the payload to keep it empty"

But my phrasing might cause some confusion with "null" vs "empty strings".

[spbnick]

Yeah, I'd like to add rationales too. The null is a literal JavaScript value, though, is appropriately quoted, and is distinct from empty strings. A clarification could be useful still.

There are a few rationales for this, but the ones I remember right now is reducing the useless (duplicate) variability of the inputs so we don't have to check for extra conditions (like checking for both a missing field and a null in lots of places), and leaving the NULL in database to signify "missing data/field" (as opposed to using it to store the input null).

[tales-aparecida]

I believe "null" is forbidden because

8. Order of submissions doesn't matter.

and

9. Between two submissions of an object (having the same ID) specified fields will be picked over unspecified, but if a field is specified in both with different values, it's value will be picked non-deterministically. NOTE: misc objects are picked as a whole.

Meaning that sending null can non-deterministically overwrite data. Is that correct?

[spbnick]

Meaning that sending null can non-deterministically overwrite data. Is that correct?

If we treat null as equivalent to "missing field", then not. But then we have all these checks there, and many ways to express the same data by padding with arbitrary sets of extra fields set to null.

[tales-aparecida]

11. Received data is acted upon only once it is good enough for a particular purpose. Purposes may differ in requirements for the data.

Could you give me an example of "acted upon"?

[spbnick]

Examples of acted upon:

• Generate a notification on a particular condition, which only requires this particular set of data (e.g. doesn't care about a compiler).
• Show a particular object in a dashboard (not all the fields need to be shown there to provide value, but some might be required).
• Involve this particular result in bisection, or other historical analysis with a particular profile
• And so on.

And in general it's good to keep the data requirements to the minimum needed for a particular task. This way more people/systems can contribute.

[spbnick]

I made some additions and refinements to the wiki page.

[tales-aparecida]

14. The schema requires only the absolute minimum set of fields: IDs, origin, and references to other objects. This is required to allow gradual submissions without having to repeat non-essential fields each time.

A discussion during today's dashboard meeting touched the "non-essential fields" part, when talking about Checkout.git_repository_url. Could we walk towards making more fields essential?

From the top of my head I'd start with

• Checkout: git_repository_url and git_repository_branch; essential to identify source
• Build: architecture is also pretty essential in the workflows we know about

[spbnick]

Can you tell me whether the work he's doing in his CI system is based on a published tree and branch? And whether you would think using that repo URL and based branch would fit the requirements of KCIDB?

I don't really know (but we can ask Mark). I suspect that he makes a series of local commits on top of ones previously published in his repos (just as we all do) and then tests them. The patchset_hash is intended for something which might not necessarily has been even made with git.

We could, of course require the reported commit to be published, as I said above, no matter how exactly we actually enforce that (in JSON or in DB).

However, we need to think if we would disable some actually useful cases by that, and recall that there are also borderline cases, like branches being force-pushed to (a part of workflow for some maintainers): now the commits are there, next day they're not and are nowhere to be found.

[tales-aparecida]

which might not necessarily has been even made with git... We could, of course require the reported commit to be published...

That's not my goal here. Like I've said before, I'm really more interested in the base repository url and branch.

The story I'm interested in telling is:

"Your test failed, but we know that it had already been failing in that repository+branch (+config, TBH, but that's more complex) "

[tales-aparecida]

I'm not, in any way, suggesting "KCIDB payloads should point to live code", I'm more interested in "KCIDB payloads should have identifiers that allow crossing data more easily, both from the same origin or different ones"

[spbnick]

I'm not, in any way, suggesting "KCIDB payloads should point to live code", I'm more interested in "KCIDB payloads should have identifiers that allow crossing data more easily, both from the same origin or different ones"

Sure. However, what's the point in analyzing a test result for a commit which was never published in any repo, and likely was a WIP? And how does it even fit into that branch's history? I mean it physically never belonged there, so you cannot say if it was before or after anything. And note most submissions refer to published commits, and do have the repository and the branch.

[spbnick]

If e.g. Mark happened to be testing a commit locally and submitted it without a repo and a branch, and then published it, then we would need to submit another checkout getting it out of there, and then we could aggregate and correlate the earlier results without repo/branch with the rest of the history via the commit hash.