[ stonking ]

Пакет исходного кода: tokenizers (0.23.1+dfsg-1) [universe]

Ссылки для tokenizers

Ресурсы Ubuntu:

Сопровождающий:

Ubuntu MOTU Developers (Почтовый архив)

Please consider filing a bug or asking a question via Launchpad before contacting the maintainer directly.

Original Maintainers (usually from Debian):

Debian Deep Learning Team (Почтовый архив)
Kohei Sendai
Mo Zhou

It should generally not be necessary for users to contact the original maintainer.

Из этого пакета исходного кода собираются следующие двоичные пакеты:

python3-tokenizers: Fast State-of-the-Art Tokenizers for Research and Production

Другие пакеты, относящиеся к tokenizers

для сборки

build-depends-indep

adep: debhelper-compat (= 13)

Пакет недоступен
adep: dh-sequence-python3

виртуальный пакет, предоставляемый dh-python
adep: cargo

Rust package manager
adep: librust-compact-str-dev

Memory efficient string type - Rust source code
adep: librust-daachorse-dev

Daachorse: Double-Array Aho-Corasick - Rust source code
adep: librust-dary-heap-dev

D-ary heap - Rust source code
adep: librust-derive-builder-dev

Rust macro to automatically implement the builder pattern for arbitrary structs - Rust source code
adep: librust-env-logger-dev

Log implementation configured via environment variable - Rust source code
adep: librust-esaxx-rs-dev

Wrapping around sentencepiece's esaxxx library - Rust source code
adep: librust-indicatif-dev

Progress bar and cli reporting library for Rust - Rust source code
adep: librust-itertools-dev

Extra iterator adaptors, iterator methods, free functions, and macros - Rust source code
adep: librust-libc-dev

Rust bindings to libc - Rust source code
adep: librust-macro-rules-attribute-dev

Use declarative macros in attribute or derive position - Rust source code
adep: librust-monostate-dev

Type that deserializes only from one specific value - Rust source code
adep: librust-ndarray-dev

N-dimensional array for general elements and for numerics - Rust source code
adep: librust-numpy-dev

PyO3-based Rust bindings of the NumPy C-API - Rust source code
adep: librust-onig-dev

Bindings for the Oniguruma regex library - Rust source code
adep: librust-pyo3-async-runtimes-dev

PyO3 bridges from Rust runtimes to Python's Asyncio library - Rust source code
adep: librust-pyo3-dev

Bindings to Python interpreter - Rust source code
adep: librust-rand-dev

Randomness functionality, number generators - Rust source code
adep: librust-rayon-cond-dev

Experimental iterator wrapper that is conditionally parallel or serial - Rust source code
adep: librust-rayon-dev

Simple work-stealing parallelism for Rust - Rust source code
adep: librust-regex-dev

Regular expressions for Rust - Rust source code
adep: librust-regex-syntax-dev

Regular expression parser - Rust source code
adep: librust-serde-derive-dev

Macros 1.1 #[derive(Serialize, Deserialize)] - Rust source code
adep: librust-serde-dev

Generic serialization/deserialization framework - Rust source code
adep: librust-serde-json-dev

JSON serialization file format - Rust source code
adep: librust-spm-precompiled-dev

This crate aims to emulate https://github.com/google/sentencepiece Dart::DoubleArray struct and it's Normalizer - Rust source code
adep: librust-thiserror-1-dev

Derive(Error) - Rust source code
adep: librust-unicode-categories-dev

Query Unicode category membership for chars - Rust source code
adep: librust-unicode-normalization-alignments-dev

Functions for normalization of Unicode strings, including Canonical and Compatible Decomposition and Recomposition, as described in Unicode Standard Annex #15 - Rust source code
adep: pybuild-plugin-pyproject

Debian helper tools for packaging Python libraries using PEP517
adep: python3-all

package depending on all supported Python 3 runtime versions
adep: python3-filelock

platform independent file locking module
adep: python3-fsspec

specification that Python filesystems should adhere to (Python 3)
adep: python3-huggingface-hub

Official Python client for the Huggingface Hub.
adep: python3-maturin

Build and publish crates with pyo3
adep: python3-packaging

core utilities for python3 packages
adep: python3-requests

elegant and simple HTTP library for Python3, built for human beings
adep: python3-setuptools

Python3 Distutils Enhancements
adep: python3-tqdm

fast, extensible progress bar for Python 3 and CLI tool
adep: python3-typing-extensions

Backported and Experimental Type Hints for Python
adep: python3-yaml

YAML parser and emitter for Python3
adep: rustc

Rust systems programming language

Download tokenizers

Файл	Размер (в Кб)	Контрольная сумма MD5
tokenizers_0.23.1+dfsg-1.dsc	3,1 Кб	17072b3932d71c827ea56dd82a2b30ea
tokenizers_0.23.1+dfsg.orig.tar.xz	1 155,4 Кб	6baf3ed9bef2c119c2ac88ebc630abff
tokenizers_0.23.1+dfsg-1.debian.tar.xz	5,3 Кб	989f527eaa536e6a173039a39f52835c

Репозиторий пакетов исходного кода Debian (VCS: Git): https://salsa.debian.org/deeplearning-team/tokenizers.git
Репозиторий пакетов исходного кода Debian (доступен просмотр): https://salsa.debian.org/deeplearning-team/tokenizers