Flashcard
Token
A token is a small piece of text that an AI reads or writes. One word can be one token or several, and punctuation can count too. AI products use tokens to set limits and measure use, so tokens are not the same as words.
Flashcard
A token is a small piece of text that an AI reads or writes. One word can be one token or several, and punctuation can count too. AI products use tokens to set limits and measure use, so tokens are not the same as words.