noms de paquets descriptions noms de paquets source contenu du paquet toutes les options

[  stonking  ]

Liens pour tokenizers

Ressources Ubuntu :

• Rapports de bogues
• Journal des modifications Ubuntu
• Fichier de licence
• Dépôt de source Debian (Git)

Responsable :

• Ubuntu MOTU Developers (Archive du courrier électronique)

Please consider filing a bug or asking a question via Launchpad before contacting the maintainer directly.

Original Maintainers (usually from Debian):

• Debian Deep Learning Team (Archive du courrier électronique)
• Kohei Sendai
• Mo Zhou

It should generally not be necessary for users to contact the original maintainer.

Les paquets binaires suivants sont compilés à partir de ce paquet source :

python3-tokenizers

Fast State-of-the-Art Tokenizers for Research and Production

Autres paquets associés à tokenizers

build-depends

build-depends-indep

• adep: debhelper-compat (= 13)

  Paquet indisponible

• adep: dh-sequence-python3

  paquet virtuel fourni par dh-python

• adep: cargo

  Rust package manager

• adep: librust-compact-str-dev

  Memory efficient string type - Rust source code

• adep: librust-daachorse-dev

  Daachorse: Double-Array Aho-Corasick - Rust source code

• adep: librust-dary-heap-dev

  D-ary heap - Rust source code

• adep: librust-derive-builder-dev

  Rust macro to automatically implement the builder pattern for arbitrary structs - Rust source code

• adep: librust-env-logger-dev

  Log implementation configured via environment variable - Rust source code

• adep: librust-esaxx-rs-dev

  Wrapping around sentencepiece's esaxxx library - Rust source code

• adep: librust-indicatif-dev

  Progress bar and cli reporting library for Rust - Rust source code

• adep: librust-itertools-dev

  Extra iterator adaptors, iterator methods, free functions, and macros - Rust source code

• adep: librust-libc-dev

  Rust bindings to libc - Rust source code

• adep: librust-macro-rules-attribute-dev

  Use declarative macros in attribute or derive position - Rust source code

• adep: librust-monostate-dev

  Type that deserializes only from one specific value - Rust source code

• adep: librust-ndarray-dev

  N-dimensional array for general elements and for numerics - Rust source code

• adep: librust-numpy-dev

  PyO3-based Rust bindings of the NumPy C-API - Rust source code

• adep: librust-onig-dev

  Bindings for the Oniguruma regex library - Rust source code

• adep: librust-pyo3-async-runtimes-dev

  PyO3 bridges from Rust runtimes to Python's Asyncio library - Rust source code

• adep: librust-pyo3-dev

  Bindings to Python interpreter - Rust source code

• adep: librust-rand-dev

  Randomness functionality, number generators - Rust source code

• adep: librust-rayon-cond-dev

  Experimental iterator wrapper that is conditionally parallel or serial - Rust source code

• adep: librust-rayon-dev

  Simple work-stealing parallelism for Rust - Rust source code

• adep: librust-regex-dev

  Regular expressions for Rust - Rust source code

• adep: librust-regex-syntax-dev

  Regular expression parser - Rust source code

• adep: librust-serde-derive-dev

  Macros 1.1 #[derive(Serialize, Deserialize)] - Rust source code

• adep: librust-serde-dev

  Generic serialization/deserialization framework - Rust source code

• adep: librust-serde-json-dev

  JSON serialization file format - Rust source code

• adep: librust-spm-precompiled-dev

  This crate aims to emulate https://github.com/google/sentencepiece Dart::DoubleArray struct and it's Normalizer - Rust source code

• adep: librust-thiserror-1-dev

  Derive(Error) - Rust source code

• adep: librust-unicode-categories-dev

  Query Unicode category membership for chars - Rust source code

• adep: librust-unicode-normalization-alignments-dev

  Functions for normalization of Unicode strings, including Canonical and Compatible Decomposition and Recomposition, as described in Unicode Standard Annex #15 - Rust source code

• adep: pybuild-plugin-pyproject

  Debian helper tools for packaging Python libraries using PEP517

• adep: python3-all

  package depending on all supported Python 3 runtime versions

• adep: python3-filelock

  platform independent file locking module

• adep: python3-fsspec

  specification that Python filesystems should adhere to (Python 3)

• adep: python3-huggingface-hub

  Official Python client for the Huggingface Hub.

• adep: python3-maturin

  Build and publish crates with pyo3

• adep: python3-packaging

  core utilities for python3 packages

• adep: python3-requests

  elegant and simple HTTP library for Python3, built for human beings

• adep: python3-setuptools

  Python3 Distutils Enhancements

• adep: python3-tqdm

  fast, extensible progress bar for Python 3 and CLI tool

• adep: python3-typing-extensions

  Backported and Experimental Type Hints for Python

• adep: python3-yaml

  YAML parser and emitter for Python3

• adep: rustc

  Rust systems programming language

Download tokenizers

Fichier	Taille (en ko)	Somme MD5
tokenizers_0.23.1+dfsg-1.dsc	3,1 ko	17072b3932d71c827ea56dd82a2b30ea
tokenizers_0.23.1+dfsg.orig.tar.xz	1 155,4 ko	6baf3ed9bef2c119c2ac88ebc630abff
tokenizers_0.23.1+dfsg-1.debian.tar.xz	5,3 ko	989f527eaa536e6a173039a39f52835c

Dépôt Debian des paquets source (VCS: Git)

https://salsa.debian.org/deeplearning-team/tokenizers.git

Dépôt Debian des paquets source (interface web)

https://salsa.debian.org/deeplearning-team/tokenizers