The description of the "Tag set fandom only?" option says:
If you don't want all the wrangled characters available, then check this option, and ONLY the characters you've associated in your tag set will be allowed.
However, the setting in question is only used for the autocomplete, and is not checked anywhere in the Prompt or ChallengeSignup validations (that I can see). If the tag set includes a canonical fandom and both "Fandom only?" and "Tag set fandom only?" are selected, then you can sign up with any canonical characters or relationships from that fandom, even those not included in the tag set, as long as you ignore the autocomplete and enter the name manually.
Create a tag set.
Pick a canonical fandom X that has at least two canonical characters, A and B, wrangled into it. (If you have wrangling permissions and you want to make sure that you're not experiencing instead, you might want to create a brand new canonical fandom and two brand new canonical character tags associated with it.)
Add fandom X and character A to the tag set, but not character B.
Create a gift exchange that uses the tag set, and allows the user to enter at least one fandom and at least one character.
Make sure that you check off both "Fandom only?" and "Tag set fandom only?" under "Character Settings."
Sign up for your gift exchange with fandom X and character B. (Don't rely on the autocomplete, just type it in manually.)
Notice that your prompt is considered valid, even though "Tag set fandom only?" is checked, and you're using a character that isn't in the tag set.