All pull requests

Pull requests

• fix(unigram): drop long lattice paths iteratively

  #2658·founderinaction opened Nov 6, 2026·

  ·
  1

• fix(parallelism): release the pool lock around fork()

  #2656·McPatate opened Nov 6, 2026Member·1/9·

  ·
  2

• docs: fix typo in README

  #2654·dheiver2 opened Nov 6, 2026·

  ·

• perf(bitcannon): add RVV literal matching

  #2650·xiazhuozhao opened Nov 4, 2026·

  ·
  5

• perf(parallel): only cut sequences that could hold up the batch

  #2649·McPatate opened Nov 4, 2026Member·

  ·
  1

• perf(parallelism): read the cached pool without a lock

  #2648·McPatate opened Nov 4, 2026Member·9/9·

  ·
  1

• refactor(parallelism)!: move the MaybeParallel* iterators to tk-train

  #2647·McPatate opened Nov 4, 2026Member·8/9·

  ·
  1

• feat(python): add encode_iter to stream a batch's encodings

  #2644·McPatate opened Nov 3, 2026Member·

  ·
  1

• feat(python): add set_max_threads / get_max_threads and a per-call max_threads

  #2643·McPatate opened Nov 3, 2026Member·7/9·

  ·
  1

• feat(parallelism)!: stop reading TOKENIZERS_PARALLELISM

  #2642·McPatate opened Nov 3, 2026Member·4/9·

  ·
  1

• perf(parallel): cut around normalized added tokens

  #2641·McPatate opened Nov 3, 2026Member·3/3·

  ·
  1

• feat(parallelism): default forked children to one thread

  #2640·McPatate opened Nov 3, 2026Member·6/9·

  ·
  1

• feat(parallel): add per-call EncodeOptions::max_threads

  #2639·McPatate opened Nov 3, 2026Member·3/9·

  ·
  1

• perf(parallel): reassemble chunked sequences in one copy

  #2638·McPatate opened Nov 3, 2026Member·

  ·
  1

• perf(parallel): split large inputs at safe pre-token boundaries

  #2637·McPatate opened Nov 3, 2026Member·2/3·

  ·
  1

• feat(bitcannon): add safe cut points after letter runs

  #2636·McPatate opened Nov 3, 2026Member·1/3·

  ·
  1

• fix(parallel): error instead of hanging when a handle crosses a fork

  #2635·McPatate opened Nov 3, 2026Member·5/9·

  ·
  1

• feat(parallelism)!: replace set_num_threads with set_max_threads

  #2634·McPatate opened Nov 3, 2026Member·2/9·

  ·
  1

• Fix ByteFallback decoding of mixed valid and invalid UTF-8

  #2627·DQNeoLab opened Nov 3, 2026·

  ·
  1

  1

• chore(ci): refresh workflow references (#2384)

  #2624·hf-security-analysis[bot] opened Nov 2, 2026Bot·

  ·

• chore(ci): refresh workflow references (#2622)

  #2623·hf-security-analysis[bot] opened Nov 2, 2026Bot·

  ·

• feat(java): experimental Java bindings over the C ABI

  #2622·ArthurZucker opened Nov 2, 2026Collaborator·

  ·

• docs(python): correct GPT-2 example outputs

  #2620·LeON-Nie-code opened Nov 2, 2026·

  ·
  1

• chore(deps): bump the actions group with 9 updates

  dependenciesgithub_actions
  #2619·dependabot[bot] opened Nov 1, 2026Bot·

  ·
  1

• Fix WordPiece trainer default continuation prefix

  #2618·dlowzzxx opened Oct 30, 2026Contributor·

  ·