o 2j )@sdZddlZddlmZddlZddlmZddlmZddl m Z ddl m Z m Z ddlmZmZd d lmZmZmZmZmZd d lmZgd Z         d'dedeeedBejddfdededeeedBejddfdeeedBejddfdeedBejddfdeedBejddfdeeejddfdeeejd dfd!ed"eeejd#dfd$dfd%d&Z dS)(anContains command to download files from the Hub with the CLI. Usage: hf download --help # Download file hf download gpt2 config.json # Download entire repo hf download fffiloni/zeroscope --repo-type=space --revision=refs/pr/78 # Download repo with filters hf download gpt2 --include="*.safetensors" # Download with token hf download Wauplin/private-model --token=hf_*** # Download quietly (no progress bar, no warnings, only the returned path) hf download gpt2 config.json --quiet # Download to local dir hf download gpt2 --local-dir=./models/gpt2 # Download a subfolder hf download HuggingFaceM4/FineVision art/ --repo-type=dataset # Download using an hf:// URI (repo type, revision and file path are read from the URI) hf download hf://datasets/HuggingFaceM4/FineVision@refs/pr/1/data/train.parquet N) Annotated) constants)snapshot_download)CLIError)DryRunFileInfohf_hub_download) _format_size parse_hf_uri) RepoIdArgRepoTypeRepoTypeOptionalOpt RevisionOptTokenOpt)out)z,hf download meta-llama/Llama-3.2-1B-InstructzGhf download meta-llama/Llama-3.2-1B-Instruct config.json tokenizer.jsonzXhf download meta-llama/Llama-3.2-1B-Instruct --include "*.safetensors" --exclude "*.bin"zGhf download meta-llama/Llama-3.2-1B-Instruct --local-dir ./models/llamaz=hf download HuggingFaceM4/FineVision art/ --repo-type datasetz6hf download hf://datasets/HuggingFaceH4/ultrachat_200kFrepo_id filenamesz>Files to download (e.g. `config.json`, `data/metadata.jsonl`).)help repo_typerevisionincludez;Glob patterns to include from files to download. eg: *.jsonexcludez0Glob patterns to exclude from files to download. cache_dirzDirectory where to save files. local_dirzIf set, the downloaded file will be placed under this directory. Check out https://huggingface.co/docs/huggingface_hub/guides/download#download-files-to-a-local-folder for more details.force_downloadzFIf True, the files will be downloaded even if they are already cached.dry_runzAIf True, perform a dry run without actually downloading the file.token max_workerszEMaximum number of workers to use for downloading files. Default is 8.returnc  s.dur dur tdtjra|durtdd dur*tddt} | jr5td| j} | rCdrC| d7} | j| j | j  | r`r]td| d d | gn|pet j j d ttBttBf f d d }dttBttBd dfdd}||dS)zDownload files from the Hub.NzCannot use both `--local-dir` and `--cache-dir` at the same time. Use `--cache-dir` (or set the HF_HOME environment variable) for shared caching, or `--local-dir` for a one-off download to a specific directory.z3'--repo-type' cannot be used with an 'hf://' URI ('z').z2'--revision' cannot be used with an 'hf://' URI ('zBBuckets are not supported by `hf download`. Use `hf sync` instead./z.Cannot combine a file path in the hf:// URI ('z') with positional filenames .rcsxdurng}dd|D}dd|D}dd|D}t|dkrUdurz2download..run_download..cSsg|] }|ddqS)r z/**)rstripr%r(r(r)r*scSsg|] }|ds|qSr"r#r%r(r(r)r*r+rz$Cannot combine subfolder argument ('z,') with `--include`. Please use `--include "z *"` instead.z,') with `--exclude`. Please use `--include "z*"` with `--exclude` instead.z>Ignoring `--include` since filenames have been explicitly set.z>Ignoring `--exclude` since filenames have been explicitly set.r zhuggingface-cli) rrrfilenamerrrr library_namer) rrrallow_patternsignore_patternsrrrrr.rr)lenrwarningswarnrr)filenames_list subfolderssubfolder_patternsregular_filenamesr/r0 rrrrrrrrr repo_type_strrrr(r) run_downloadsl      zdownload..run_downloadresultc Sst|trtjd|ddSt|tr|g}dd|D}tdt|dt|dttdd |Dd d dt |d d dD}t |dS)N Downloaded)pathcSsg|]}|jr|qSr() will_downloadr&rr(r(r)r*sz3download.._print_result..z[dry-run] Will download z files (out of z ) totalling css|]}|jVqdSN) file_sizer?r(r(r) sz2download.._print_result..r!cSs(g|]}|j|jrt|jnddqS)-)filesize)r-r>rrB)r&infor(r(r)r*s cSs|jSrA)r-)xr(r(r)sz1download.._print_result..)key) isinstancestrrr;rtextr1rsumsortedtable)r;r>itemsr(r(r) _print_results"   zdownload.._print_result)r startswithr HF_PROTOCOLr is_bucket path_in_repor$idtyperr modelvaluerLrlist)rrrrrrrrrrrrurirVr:rRr(r8r)downloadEs67  4Fr]) NNNNNNNFFNr)!__doc__r2typingrtyperhuggingface_hubr"huggingface_hub._snapshot_downloadrhuggingface_hub.errorsrhuggingface_hub.file_downloadrrhuggingface_hub.utilsrr _cli_utilsr r r rr_outputrDOWNLOAD_EXAMPLESr[rLArgumentOptionboolintr]r(r(r(r)s          "(./5