Paketnamen Beschreibungen Quellcodepaketnamen Paket-Inhalte Alle Optionen

[  stonking  ]

Links für tokenizers

Ubuntu-Ressourcen:

• Fehlerberichte
• Ubuntu-Changelog
• Copyright-Datei
• Debian-Quellcode-Depot (Git)

Betreuer:

• Ubuntu MOTU Developers (E-Mail-Archiv)

Please consider filing a bug or asking a question via Launchpad before contacting the maintainer directly.

Original Maintainers (usually from Debian):

• Debian Deep Learning Team (E-Mail-Archiv)
• Kohei Sendai
• Mo Zhou

It should generally not be necessary for users to contact the original maintainer.

Die folgenden Binärpakete werden aus diesem Quellcode-Paket gebaut:

python3-tokenizers

Fast State-of-the-Art Tokenizers for Research and Production

Andere Pakete mit Bezug zu tokenizers

build-depends

build-depends-indep

• adep: debhelper-compat (= 13)

  Paket nicht verfügbar

• adep: dh-sequence-python3

  virtuelles Paket, bereitgestellt durch dh-python

• adep: cargo

  Rust package manager

• adep: librust-compact-str-dev

  Memory efficient string type - Rust source code

• adep: librust-daachorse-dev

  Daachorse: Double-Array Aho-Corasick - Rust source code

• adep: librust-dary-heap-dev

  D-ary heap - Rust source code

• adep: librust-derive-builder-dev

  Rust macro to automatically implement the builder pattern for arbitrary structs - Rust source code

• adep: librust-env-logger-dev

  Log implementation configured via environment variable - Rust source code

• adep: librust-esaxx-rs-dev

  Wrapping around sentencepiece's esaxxx library - Rust source code

• adep: librust-indicatif-dev

  Progress bar and cli reporting library for Rust - Rust source code

• adep: librust-itertools-dev

  Extra iterator adaptors, iterator methods, free functions, and macros - Rust source code

• adep: librust-libc-dev

  Rust bindings to libc - Rust source code

• adep: librust-macro-rules-attribute-dev

  Use declarative macros in attribute or derive position - Rust source code

• adep: librust-monostate-dev

  Type that deserializes only from one specific value - Rust source code

• adep: librust-ndarray-dev

  N-dimensional array for general elements and for numerics - Rust source code

• adep: librust-numpy-dev

  PyO3-based Rust bindings of the NumPy C-API - Rust source code

• adep: librust-onig-dev

  Bindings for the Oniguruma regex library - Rust source code

• adep: librust-pyo3-async-runtimes-dev

  PyO3 bridges from Rust runtimes to Python's Asyncio library - Rust source code

• adep: librust-pyo3-dev

  Bindings to Python interpreter - Rust source code

• adep: librust-rand-dev

  Randomness functionality, number generators - Rust source code

• adep: librust-rayon-cond-dev

  Experimental iterator wrapper that is conditionally parallel or serial - Rust source code

• adep: librust-rayon-dev

  Simple work-stealing parallelism for Rust - Rust source code

• adep: librust-regex-dev

  Regular expressions for Rust - Rust source code

• adep: librust-regex-syntax-dev

  Regular expression parser - Rust source code

• adep: librust-serde-derive-dev

  Macros 1.1 #[derive(Serialize, Deserialize)] - Rust source code

• adep: librust-serde-dev

  Generic serialization/deserialization framework - Rust source code

• adep: librust-serde-json-dev

  JSON serialization file format - Rust source code

• adep: librust-spm-precompiled-dev

  This crate aims to emulate https://github.com/google/sentencepiece Dart::DoubleArray struct and it's Normalizer - Rust source code

• adep: librust-thiserror-1-dev

  Derive(Error) - Rust source code

• adep: librust-unicode-categories-dev

  Query Unicode category membership for chars - Rust source code

• adep: librust-unicode-normalization-alignments-dev

  Functions for normalization of Unicode strings, including Canonical and Compatible Decomposition and Recomposition, as described in Unicode Standard Annex #15 - Rust source code

• adep: pybuild-plugin-pyproject

  Debian helper tools for packaging Python libraries using PEP517

• adep: python3-all

  package depending on all supported Python 3 runtime versions

• adep: python3-filelock

  platform independent file locking module

• adep: python3-fsspec

  specification that Python filesystems should adhere to (Python 3)

• adep: python3-huggingface-hub

  Official Python client for the Huggingface Hub.

• adep: python3-maturin

  Build and publish crates with pyo3

• adep: python3-packaging

  core utilities for python3 packages

• adep: python3-requests

  elegant and simple HTTP library for Python3, built for human beings

• adep: python3-setuptools

  Python3 Distutils Enhancements

• adep: python3-tqdm

  fast, extensible progress bar for Python 3 and CLI tool

• adep: python3-typing-extensions

  Backported and Experimental Type Hints for Python

• adep: python3-yaml

  YAML parser and emitter for Python3

• adep: rustc

  Rust systems programming language

Download tokenizers

Datei	Größe (in kB)	MD5-Prüfsumme
tokenizers_0.23.1+dfsg-1.dsc	3,1 kB	17072b3932d71c827ea56dd82a2b30ea
tokenizers_0.23.1+dfsg.orig.tar.xz	1.155,4 kB	6baf3ed9bef2c119c2ac88ebc630abff
tokenizers_0.23.1+dfsg-1.debian.tar.xz	5,3 kB	989f527eaa536e6a173039a39f52835c

Quellcode-Depot des Debian-Pakets (VCS: Git)

https://salsa.debian.org/deeplearning-team/tokenizers.git

Quellcode-Depot des Debian-Pakets (browsable)

https://salsa.debian.org/deeplearning-team/tokenizers