Stop Sequences

Stop Sequences are used to make the AI Chat to stop at a desired point, such as the end of a sentence or a list. Generally, the return key will work well as a Stop Sequence for single line response. The Stop Sequence is an optional setting that tells the API when to stop generating tokens. The AI Chat will not contain the stop sequence and you can pass up to four stop sequences. If nothing is passed, it defaults to the token <|endoftext|>. This token represents a probable stopping point in the text.  

 

Chat Example

In the Chat example, three Stop Sequences are used: A new line, the value “Human:”, and the value “AI:”. Our goal is to generate only a single line of text that corresponds to the current speaker. The chosen three stop sequences create the needed constraints to achieve that end:

  • The AI Chat can’t go to a new line.

  • The AI Chat can’t change the speaker.

  • The AI Chat won’t allow the speaker to speak twice in a row

Q&A Example

In the Q&A example, the pattern of separating each Q&A pair with a new line gives us confidence that the return key(new line) will work well as our stop sequence. You can see below that the completion, following “A:”, will stop after a single line because the return key is used as a Stop Sequence.

 

List Example:

You can also use Stop Sequences to generate a list with a specific number of items. For example, by using “11.” as a stop sequence, you can generate a list with 10 items, since the completion will stop when “11.” is reached. This can be seen in the science fiction prompt.