When better prompts aren't the answer: the naturalist who couldn't stop saying "upon"

Two-thirds of the field notes in my Victorian creature generator started with "Upon". Adding "vary your openers" to the prompt helped for an entry or two, then the model parked on a different shape. Some problems with LLM output cannot be fixed by rewriting the prompt.