Search results

Tik Tok follow updates

Status: Open.

#571 In openai/tiktoken;

· 79892s4tvv-code opened on Jun 17, 2026

[Security] Reachable Rust panic (denial of service) when tokenizing with an untrusted encoder missing single-byte tokens

Status: Open.

#568 In openai/tiktoken;

· geo-chen opened on Jun 10, 2026

CoreBPE::new has an unconstrained NSE type parameter, making normal downstream calls fail type inference

Status: Open.

#557 In openai/tiktoken;

· fallintoplace opened on May 24, 2026

Use Trusted Publishing to upload to PyPI

Status: Open.

#547 In openai/tiktoken;

· nsoranzo opened on May 19, 2026

encode_with_unstable does not handle surrogate pairs like encode

Status: Open.

#541 In openai/tiktoken;

· shreejaykurhade opened on May 13, 2026

encode_ordinary_batch — reproducible multi-second tail stalls on 32-core box (o200k_base, num_threads=8)

Status: Open.

#530 In openai/tiktoken;

· daechoi opened on Apr 21, 2026

tiktoken 0.12 cannot handle qwen3.5 in huggingface

Status: Open.

#521 In openai/tiktoken;

· jslhcl opened on Apr 8, 2026

[Bug] tiktoken 0.9.0 ignores TIKTOKEN_OFFLINE and TIKTOKEN_CACHE_DIR, still attempts network download

Status: Open.

#513 In openai/tiktoken;

· gavin9508 opened on Mar 24, 2026

[RFC] Korean Tokenization: Jamo Decomposition as Pre-tokenizer (A Blueprint for Compositional Scripts)

Status: Open.

#511 In openai/tiktoken;

· nicezic opened on Mar 22, 2026

Add riscv64 (linux_riscv64) wheel to PyPI releases

Status: Open.

#502 In openai/tiktoken;

· gounthar opened on Mar 11, 2026

Security: Cache Poisoning via Shared Temp Directory with Optional Hash Verification

Status: Open.

#500 In openai/tiktoken;

· spartan8806 opened on Mar 8, 2026

Token count discrepancy between tiktoken and API response when messages contain tool calls

Status: Open.

#474 In openai/tiktoken;

· yangchou19 opened on Jan 6, 2026