tiktoken

tiktoken is a fast BPE tokeniser for use with OpenAI's models.

⚠️ Further information - our curation - is coming soon