PyPI page
Home page
Author:
License:
Apache-2.0
Summary:
SMASHED is a toolkit designed to apply transformations to samples in datasets, such as fields extraction, tokenization, prompting, batching, and more. Supports datasets from Huggingface, torchdata iterables, or simple lists of dictionaries.
Latest version:
0.21.5
Required dependencies:
ftfy
|
glom
|
jinja2
|
necessary
|
numpy
|
platformdirs
|
trouting
Optional dependencies:
autopep8
|
black
|
blingfire
|
boto3
|
datasets
|
dill
|
flake8
|
flake8-pyi
|
flake8-pyproject
|
ipdb
|
ipython
|
isort
|
moto
|
mypy
|
promptsource
|
pytest
|
smart-open
|
smashed
|
torch
|
torchdata
|
transformers
Downloads last day:
133
Downloads last week:
746
Downloads last month:
2,322