Smanor/sentinel support: add support for sentinel tokens

[ShayManor]

feat: add support for sentinel tokens

Description

Add support for more complicated masking in the form:
BOS {prompt} SENTINEL {completion} EOS
Where the sentinel token is passed with the sentinel flag.
If there is no sentinel ("" or nothing is passed), the functionality is the same as before.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Other (please describe):

Checklist

• I have performed a self-review of my own code.
• I have commented my code, particularly in hard-to-understand areas.
• I have made corresponding changes to the documentation (if applicable).
• My changes generate no new warnings.
• I have added tests that prove my fix is effective or my feature works.
• New and existing unit tests pass locally with my changes.
• Any dependent changes have been merged and published in downstream modules.

Additional Notes