Let's discuss how to handle validation

[patrick91]

Let's collect ideas and requirements around validation in Strawberry

[lijok]

Example of a problem:

@strawberry.mutation
def create_mapping(
    self, info: Info, input: CreateMappingInput
) -> strawberry.union(
    name="CreateMappingResult",  # noqa: F821
    types=(
        Mapping,
        error.IntegrationDoesNotExistError,
        error.FieldDoesNotExistError,
        error.EntityDoesNotExistError,
        error.DestinationFieldAlreadyMappedError,
        error.MissingRequiredFieldMappingsError,
        error.EntityAlreadyMappedError,
        error.FilterValueInvalidForFilterTypeError,
        error.FieldTypeIncorrectForFilterTypeError,
        error.FilterNotFoundError,
        error.FieldMappingDataTypeMismatchError,
    ),
):
    mapping = Mapping.from_input(input=input)
    is_valid, reason = mapping.validate()
    if not is_valid:
        return reason
    ...


@strawberry.type
class Mapping(Node):
    def validate(
        self,
    ) -> Tuple[
        bool,
        Optional[
            Union[
                error.FieldDoesNotExistError,
                error.EntityDoesNotExistError,
                error.MissingRequiredFieldMappingsError,
                error.EntityAlreadyMappedError,
                error.DestinationFieldAlreadyMappedError,
                error.FilterValueInvalidForFilterTypeError,
                error.FieldTypeIncorrectForFilterTypeError,
                error.FilterNotFoundError,
                error.FieldMappingDataTypeMismatchError,
            ]
        ],
    ]:
        validators = [
            validation.validate_entities,
            validation.validate_match_on_fields,
            validation.validate_field_mapping,
            validation.validate_filters,
        ]

        for validator in validators:
            error = validator(mapping=self)
            if error:
                return False, error

        return True, None


def validate_entities(
    mapping: "mapping.Mapping",
) -> Union[error.EntityDoesNotExistError, error.EntityAlreadyMappedError]:
    validators = [
        _validate_source_entity_exists,
        _validate_destination_entity_exists,
        _validate_destination_entity_is_unique,
    ]

    for validator in validators:
        error = validator(mapping=mapping)
        if error:
            return error

...

[jokull]

I kind of like not having any higher level abstractions for validation. Just use strawberry.union for the possible error/success payloads.

[codepyth]

how can i use field proper validations like its accepting email as abcd@gmail instead of abcd@gmail.com? how i can do that in graphql strawberry

[jkimbo]

I've been thinking about how the API would look for this. One possibility:

import strawberry

def validate_name(name: str) -> str:
    if ' ' not in name:
        raise ValueError('must contain a space')
    return name.title()

def validate_password(password: str) -> str:
	if len(password) < 4:
		raise ValueError("Password is too short")

def root_validator(input):
	# This is run after all the field validators have been run
	if self.password_confirm != self.password:
		raise ValueError("passwords do not match")

	# Also field values can be modified in this function

@strawberry.input(root_validator=root_validator)
class NewUserInput:
	name: str = strawberry.field(validators=[validate_name])
    email: str = strawberry.field(validators=[validate_email])
    password: str = strawberry.field(validators=[validate_password])
    password_confirm: str

@strawberry.mutation
def create_user(info, user_data: NewUserInput) -> Union[User, InputError]:
	try:
		user_data.validate(info)
    except strawberry.ValidationError as error:
        return InputError(errors=error.errors)

This API is kind of a mix between Pydantic and Django forms.

Some notes and open questions:

1. Validator functions are just plain functions that take a value, raise a ValueError/AssertionError and return a value (so they can be used to "clean" data as well).
2. Validators only make sense on input types so we should raise an error if they are used on a normal type
3. Is it ok to pass validators to strawberry.field when it's only applicable for input types? Should we have a strawberry.input_field?
4. Unsure how best to do full object validation. In the example above I've added the root_validator decorator but we could also rely on field ordering and allow individual validators to access other field values (like pydantic). We would probably need a root_validator equivalent though...
   • root_validator function passed to the input decorator
5. Validators should be allowed to be async (only if running in an async context)
6. Need to make sure that nested input types work
7. Should we be passing the info object to the validate function so that it can be accessed by the validator functions?
   • Yes
8. Should we be building this into Strawberry or should we be uses an alternative library (like Pydantic) to provide this for us? My current thinking is that there is value having this be built into Strawberry because it's quite a common use case.
   • Yes

---

Thoughts @strawberry-graphql/core ?

[DoctorJohn]

I removed a question from my first response because I was usure it was valid. Just correct me if I'm wrong:

2. Validators only make sense on input types so we should raise an error if they are used on a normal type

Consider the following query which I just sent to Githubs GraphQL API:

query { 
  viewer { 
    followers(after: "asdf", first: 10) {
      nodes {
        bio
      }
    }
  }
}

Unsurprisingly asdf isn't the id of any of my followers and a response containing errors was returned. Wouldn't validating field arguments similar to after (filters, cursors, ordering, etc.) be a usecase for field validation of non-input fields?

[BryceBeagle]

A couple comments here:

1. We should make sure we don't continue to add too many args/kwargs to strawberry.field. There are already a lot and this would be one more. If we could find another solution, potentially with a separate decorator, I think that would be best

2. Do we want to allow field/value modification in a validator? Is that commonplace? To me, "validation" would be a read-only check operation, and some other term like "sanitization" or "normalization" might be better for such a task.

   Edit: Didn't realize there were hidden comments and @DoctorJohn already touched on this.

3. Should the actual mutation or field be doing its error handling internally? Potentially it would be better to separate the expected behavior logic from the error handling to avoid having to add the boilerplate block of

   try:
       field.validate()
   except ValidationError as error:
       return InputError(errors=error.errors)

   to every field.

[jkimbo]

I'm unsure about the cleaning part. On the one hand I would expect a validation function purely conceptually to leave the data untouched, on the other hand I can see how cleaning data could be useful (Django forms). Would every validation function be required to return a value even when no data cleaning was performed? Or can we detect whether a function returned nothing vs. None to make returning untouched values optional?

@DoctorJohn @BryceBeagle I understand your concern with this but I was just copying how pydantic does it (https://pydantic-docs.helpmanual.io/usage/validators/). I'd also like to avoid having to have another method to so cleaning/normalisation. I do think returning values should be optional though.

Unsurprisingly asdf isn't the id of any of my followers and a response containing errors was returned. Wouldn't validating field arguments similar to after (filters, cursors, ordering, etc.) be a usecase for field validation of non-input fields?

@DoctorJohn I'm not sure that validation of arguments is in the scope of this. I would expect that for arguments you would just check the values in your resolver. I'm not sure how you could call .validate on those values anyway...

We should make sure we don't continue to add too many args/kwargs to strawberry.field. There are already a lot and this would be one more. If we could find another solution, potentially with a separate decorator, I think that would be best

@BryceBeagle I agree with this. Maybe we can leverage custom fields and create a validated_field? If we're going to be passing the root validator to the @strawberry.input decorator should we also be creating a custom validation_input decorator?

Should the actual mutation or field be doing its error handling internally? Potentially it would be better to separate the expected behavior logic from the error handling to avoid having to add the boilerplate block of

@BryceBeagle I was thinking that this would be another use case for custom fields. We could use something like validated_mutation to automatically call the .validate method and change the return type as well. For example:

@strawberry.validated_mutation(error_type=InputError)
def create_user(info, user_data: NewUserInput) -> User:
	# user_data has been validated

[patrick91]

I do think returning values should be optional though.

Couldn't this cause issues when you want to return None after sanitising a field?

We should make sure we don't continue to add too many args/kwargs to strawberry.field

Maybe validators is worth keeping, at the moment we have:

• name: Optional[str]
• is_subscription: bool <- this should go away, or at least made non public
• description: Optional[str]
• permission_classes: Optional[List[Type[BasePermission]]]
• federation: Optional[FederationFieldParams] <- this should be replaced by directives (it is also non public)
• deprecation_reason: Optional[str]
• default: Any
• default_factory: Union[Callable, object]

one more should be ok I think 😊 but I'm not opposed to having validated_field :)

def create_user(info, user_data: NewUserInput) -> User:

this is interesting, but a bit odd as an API, the validation is defined on the input data, but the error type is defined on the mutation itself.

[jkimbo]

Couldn't this cause issues when you want to return None after sanitising a field?

Hmm that is a good point. Ok I guess returning values can't be optional then. I don't think this is a big problem since it seems to be fine in pydantic.

Maybe validators is worth keeping,

I don't have a strong opinion either way. I would lean slightly towards adding validators to strawberry.field because it's simpler. I'll go with that for now.

this is interesting, but a bit odd as an API, the validation is defined on the input data, but the error type is defined on the mutation itself.

I don't think you can define the error type on the input data because you don't know where that input is going to be used. It could also be nested inside another input.

[nffdiogosilva]

What is the current status of this topic?

[0xxfu]

So, there should be no validation function, there is no relevant introduction in the documentation

[XChikuX]

Ideal usecase would be to not re-invent the wheel and have first class validation support built-in with Pydantic.

This issue #2181 shows us some examples of what I'm talking about.

[XChikuX]

It would be wonderful if we could handle first class Pydantic support #2181

[DevJake]

Any movement on this by chance? I'm trying to enforce strict validation on a password field for creation of new member accounts so that the password is automatically and consistently hashed. Without model-level or field-level validation, I have to trust that other developers never forget to hash incoming passwords... 😰