[uss_qualifier/resources] Update naming and documentation for ResourceModifyingResource

[BenjaminPelletier]

This PR attempts to clarify naming, documentation, and usage of ResourceModifyingResource following #1465 leveraging the context of #1470

[the-glu]

Shouldn't it be the opposite ? A different key must produce different variants; the same may produce equivalents result?

I would assume that the spirit is to ensure different resources when needed because that more important that having the same values no?

(We can however have a 'must' for the same key -> same result)

[BenjaminPelletier]

I'm not sure I understand the question; the change says "different key values -> different variants" + "same key -> equivalent results" -- the same thing as the original, just using "key" instead of "index".

One substantive difference is the removal of "(unique)" because that will probably often be the case, but there's no reason it necessarily needs to be in any case where we want a resource that spawns resources based on a template resource.

[the-glu]

That indeed about the "(unique)" removed and the addition of 'generally': since the final goal is to be able to have resources that can be used in parallel without conflicts, I would expect that those resources generator, by 'contract' return different variants in every case, not in most of the cases.

[BenjaminPelletier]

The use case prompting the creation of this resource needs unique variants in all cases and should always produce equivalent results with equivalent keys. But the way this resource is defined allows it to be used in more general ways. Just because one use of a tool has certain requirements doesn't mean those requirements need to be imposed at the tool level rather than the tool-usage level. For instance, suppose we wanted to iterate over all ordered {uss1, uss2} pairs of participants, but a test needed a token that corresponded with the unordered pair of {uss1, uss2} (so {uss1, uss2} has a different token resource than {uss1, uss3}, but the same token resource as {uss2, uss1}). The suite may still be iterating over ordered pairs, but the resource producing modified versions of the underlying token resource may produce the same modified resource for multiple keys.

[the-glu]

Just because one use of a tool has certain requirements doesn't mean those requirements need to be imposed at the tool level rather than the tool-usage level.

Ok, but should we then define an sub-type of the base type than enforce this? I would expect instances used in 'parallel generator' to follow stricter rules (This probably answer the type question bellow as well)

but the resource producing modified versions of the underlying token resource may produce the same modified resource for multiple keys

(True, but in that case since it's unordered, it would be an equal key ({uss1,uss2}=={uss2,uss1}) even if it's has different values. But that don't mean other cases wouldn't exists.)

[BenjaminPelletier]

Ok, but should we then define an sub-type of the base type than enforce this? I would expect instances used in 'parallel generator' to follow stricter rules (This probably answer the type question bellow as well)

I wouldn't expect we'd need to. A good test design will produce test scenarios that don't interfere with each other. Not interfering sometimes means "unique" modified resources (a necessary but not sufficient condition to deconflict geospatial resources), but not always (like in the case of iterating over ordered pairs but only needing unique unordered pairs for the token resource). But if we did need "unique" modified resources, I'd expect that to be introduced along with the thing that actually needs that constraint. The content in this PR (and #1465) doesn't need uniqueness -- the concept of stamping out modified copies of a template resource based on a key is a good standalone concept that doesn't need the additional constraint to work.

(True, but in that case since it's unordered, it would be an equal key ({uss1,uss2}=={uss2,uss1}) even if it's has different values. But that don't mean other cases wouldn't exists.)

No, the key would be ordered as the test scenarios (or whatever is being primarily iterated) are distinguishable based on order in the example, but the particular resource being generated by modifying a base resource doesn't depend on order. To elaborate on the example: suppose we have a test scenario that has two roles and uses a token resource that needs to be "unique" according to participants in the scenario, but not according to what roles they're playing. The action generator would iterate over every (role1, role2) assignment of the {uss1, uss2, uss3, ...} participants when instantiating test scenarios. And, each test scenario would need a token resource based on some template token resource. But, the token resource needed would only depend on the participants rather than role assignments, so the modified token resources spawned would not be unique. At the same time, geospatial deconfliction is needed, so every set of flight intents would need to be "unique".

Scenario	Role 1	Role 2	Flight intents resource	Token resource
1	uss1	uss1	A	X
2	uss1	uss2	B	Y
3	uss2	uss1	C	Y
4	uss2	uss2	D	Z

[the-glu]

A good test design will produce test scenarios that don't interfere with each other.

Ok - that move the "responsibility" up to test definitions to only combine things that works together without an explicit 'contract', but that valid.

[the-glu]

Is it needed to allow for a dynamic type there?

I would assume thoses ResourceModifyingResource will only be used by generators, and we don't want generators to need to know various types, just to use 'modify' in a generic way.

Using an int mean we could use an simple index, or use hash() (that return an int) on more generic objects.

[BenjaminPelletier]

If this object is a generic resource-modifying resource, then it should support all the reasonable use cases when a resource produces other resources from a template resource. We could narrow its scope to IndexBasedResourceModifyingResource and thereby limit the key to an int, but the generic key allows more use cases and doesn't require any additional development to do so. I would expect modifiers intended specifically for iteration parallelization via index would definitely use int. But, I'm not sure why we would need to limit to a specific key type here rather than using the appropriate key type when this tool is used.

Basically, this is a map that maps values (probably from an iterator) to corresponding resources. I'm not sure why we would want to intentionally limit it to int -> resource rather than T -> resource when the latter supports the former with no additional code, but also other Ts as well.

[the-glu]

Yes, ours comments crossed and indeed it make sense to have it very generic at this level, but it would make sense to have a specific subtype for iteration parallelization then?

Either the subtype enforce the type (to be an int or something else), or parallelizer need to handle multiple cases (like ignoring ResourceModifyingResource it cannot handle, handle multiple type (like str() is easy), and that seems complex)*

(*That in future PR, but right now passing and '.modifing()' resources to have them parallelizable is quite simple and clean: b7d58e8#diff-d44fbbd1adab54bf42eb54d25ec991f1e3f2a22f9cfad7aa517ec7a374e0c09eR106 )

[BenjaminPelletier]

I would expect subtype or not to be independent of parallelization. Action generators generate a sequence (perhaps set, in the future) of actions and we're just looking to generate resources to go along with those actions that will reduce interference between actions. It would be valuable to do this even without parallelization because then failing iteration N has a lower chance of causing iterations after N to fail also.

The only thing parallelization would do differently is enumerate all those actions up front and then dispatch them to workers with some degree of simultaneity rather than sequentially.

[the-glu]

I would expect subtype or not to be independent of parallelization.

Yes I do agree, but I think the question is still valid:

Since ResourceModifyingResource are now generic, how do we make "users/callers" of those sending the correct value in modify() ?

I thought about a subtype, and "users" can just do something like that:

if isinstance(ResourceModifyingResourceUsingInt, rmr):
  new_resource = rmr.modify(index)
elif isinstance(ResourceModifyingResourceUsingUSSPair, rmr):
  new_resource = rmr.modify({base_uss, target_uss})

There are also other solutions:

• "if tree" directly on 'implementation types', e.g. if isinstance(GeospatialModifyingResource): ...
  • (probably hard to maintain)
• "users/callers" can introspect to detect the generic type and try to pass data accordingly
  • But we may have the same type for different data (e.g. date).
    • But we can always create new types
• Instead of the generic type, we pass a **context to modify, and "users/callers" send a relevant context as they work, and ResourceModifyingResource grab what they need from it (moving responsibility of handling various cases from "users/callers" to resources.
  • Example: Action generator send current values of their loop (FlightPlannerCombinations send participants) and index.

What did you have in mind in practice?

[BenjaminPelletier]

Action generators will presumably be the main users/callers as they are generally the ones that need to produce a large number of variants of a resource. We want to tell the action generator that, instead of passing resource-modifying resource X through to each of its actions, it should instead invoke X's modify function per action and pass the resulting resource X' into the action. Action generators today iterate over different things: DSS instances, combinations and permutations of flight planners, and a plain counter. When the action generator is preparing an action, it will have the iterated element, and that element will always be of a particular type for that particular action generator (e.g., dict of role to flight planner resource). It could also presumably have the iteration index (like for i, element in enumerate(elements):). It seems like the only gap is letting the action generator know it needs to call modify.

One way to implement could be to allow the test designer to specify a list of resources that should be treated as ResourceModifyingResources by the action generator (this could take the form of a mapping of "resource name in the pool available to the action generator" to "resource name in the pool available to the action"). The resource generator would prepare each action in the same way it does now, except any resource listed as a ResourceModifyingResource it would not move to the action's pool, but instead call modify on that resource and put the resulting resource in the action's pool instead. The key type can be determined by inspecting the ResourceModifyingResource, and the action generator only knows how to provide certain kinds of keys. Or, if the action generator only provides one type of key, it can just provide that key without checking since any mismatch would be a test design error. If the key type isn't one the action generator knows how to provide, it raises an error. Note that hypothetically the action generator could just inspect every resource in the available pool to see if it's a ResourceModifyingResource and treat it accordingly. But, I don't think we want to do that because we may want to pass through some ResourceModifyingResources to a lower level.

Having written the above, it occurs to me that ResourceModifyingResource is perhaps even more specific than we need -- it seems like what we really want is a ResourceGeneratingResource (a resource that generates resources based on a key), and a ResourceModifyingResource is a special case of ResourceGeneratingResource that performs its generation by modifying a base resource.

[the-glu]

Or, if the action generator only provides one type of key, it can just provide that key without checking since any mismatch would be a test design error.

I don't think we want to do that: That would generate typing errors and tracktraces no?

But, I don't think we want to do that because we may want to pass through some ResourceModifyingResources to a lower level.

Okay, but how should we indicate when to pass or modify? (Maybe check next PRs in the original tree since it's a practical implementation, even with wrong naming)

ResourceModifyingResource / ResourceGeneratingResource

Is there really a difference and/or distinction needed? 'How' it's generated is not really important for 'callers' no?

[BenjaminPelletier]

Having written the above, it occurs to me that...

Added a commit to add a ResourceProvidingResource base class.