Skip to main content

Toolbelt User Guide

Introduction​

The LabScale Toolbelt is a suite of tools coalesced within a single executable and is found within the LabScale Agent folder at bin/labscale-toolbelt. This utility contains useful tools that are utilized at test time, such as to chunk or split individual files or directory of files into smaller pieces, and/or upload files and directories of files to the LabScale service.

Download & Install​

The Toolbelt is bundled with the LabScale Agent, to download and install the Agent, please refer to the Quick Start Guide.

Command Reference​

The LabScale Toolbelt usage

./bin/labscale-toolbelt [options] command

Options​

The agentctl allows for the following options:

  -help

The help option will display a list of options available to the provided command.

  -debug

The debug option enables extended logging to help debug problems with the agent at run time.

Available Commands​

  help|chunkfile|chunkdir|download|upload|upload-dir

help​

The help subcommand displays some helpful information about usage of the bin/labscale-toolbelt.

tip

bin/labscale-toolbelt help -help Will display the help for all available commands.

  • Examples:
    • To show top level help for the agentctl:
    ./bin/labscale-toolbelt help
    Usage of labscale-tool help|chunkfile|chunkdir|download|upload|upload-dir
    -debug
          Debug logging
    -help
          Show help
    • To show help for a single command within the agentctl:
    ./agentctl adb-discover -help
    Usage of adb-discover:
      -debug
      	Debug logging
      -force
      	Force update to backend
    • To show help for all commands within in the agentctl:
    ./bin/labscale-toolbelt help -help
    Usage of labscale-tool help|chunkfile|chunkdir|download|upload|upload-dir
    Usage of chunkfile [opts] filepath [dest] (options below):
    -always-chunk
          Always rename chunk files even if not chunked
    -debug
          Debug logging
    -help
          Help
    -pad-name
          Pad chunk names for better alphanum sorting
    -preserve-lines
          No split text lines between text files
    -remove
          Remove original unchunked file
    -size value
          Chunk size (default 134217728 bytes)
    Usage of chunkdir [opts] dirpath [dest] (options below):
    -always-chunk
          Always rename chunk files even if not chunked
    -debug
          Debug logging
    -help
          Help
    -pad-name
          Pad chunk names for better alphanum sorting
    -preserve-lines
          No split text lines between text files
    -remove
          Remove original unchunked file
    -size value
          Chunk size (default 134217728 bytes)
    Usage of download [opts] URL (options below):
    -debug
          Debug logging
    -flat
          Use flat cache directory
    -help
          Help
    -output string
          Output location or file (default "/Users/scottvail/Projects/taurus-agent")
    -retries int
          Number of retry on error
    -timeout int
          Timeout in milliseconds
    Usage of upload [opts] filepath [dest] (options below):
    -debug
          Debug logging
    -help
          Help
    -jobId int
          Bind to a job ID
    -retries int
          Number of retries on error
    Usage of upload-dir [opts] dirpath [dest] (options below):
    -debug
          Debug logging
    -help
          Help
    -jobId int
          Bind to a job ID
    -num-threads int
          Number of concurrent uploads (default 5)
    -retries int
          Number of retries on error
    -upload-type string
          Type of upload

chunkfile​

The chunkfile subcommand will break a large file into smaller chunks for faster upload. Each chunk is labelled with a number in the order of its constituent parts, though files that are too small to chunk will retain the original file's name unless the -always-chunk option is set. The default size of the chunks is 128 MB but can be set by the user with the -size option. It also has an option to preserve full lines of text without splitting them between files with the -preserve-lines option.

  • Examples:
    • To split a log file into roughly 1K chunks, but preserve lines and send output files to the /tmp folder:
    ./bin/labscale-toolbelt chunkfile -size 1k -preserve-lines log.txt /tmp/

chunkdir​

The chunkdir subcommand will recursively chunk every file in a directory structure. The same options apply to chunkdir as to chunkfile. Each chunk is labelled with a number in the order of its constituent parts, though files that are too small to chunk will retain the original file's name unless the -always-chunk option is set. The default size of the chunks is 128 MB but can be set by the user with the -size option. It also has the capability to preserve full lines of text without splitting them between files with the -preserve-lines option.

  • Examples:
    • To split all files in myfolder into roughly 1K chunks, but preserve lines and send output files ot the /tmp folder:
    ./bin/labscale-toolbelt chunkdir -size 1k -preserve-lines myfolder /tmp/

upload​

The upload subcommand will upload a file to the team bucket and assign it to a particular Job ID determined by the -jobId option. Downloads can also be automatically retried when encountering network errors with the -retries <num> option. Files uploaded to the team location will appear in the Team's Job Details view for the job identified by the Job ID. These files will appear in the Job Details fiew on the web site.

  • Examples:
    • To upload a file for Job ID 800 and perform 3 retries on any network error:
    ./bin/labscale-toolbelt upload -jobId 800 -retries 3 myfile.txt
    • To upload a file and have it appear within a folder:
    ./bin/labscale-toolbelt upload -jobId 800 myfile.txt some/folder

upload-dir​

The upload-dir subcommand will recursively upload all files within a directory structure to the team bucket and assign them to a particular Job ID determined by the -jobId option. Downloads can also be automatically retried when there are network errors with the -retries <num> option. Files uploaded to the team location will appear in the Team's Job Details view for the job identified by the Job ID.

  • Examples:
    • To upload all files in myfolder for Job ID 800 and perform 3 retries on any network error:
    ./bin/labscale-toolbelt upload -jobId 800 -retries 3 myfolder
    • To upload a folder and have it appear within another folder:
    ./bin/labscale-toolbelt upload -jobId 800 folder some/folder

download​

The download subcommand leverages the LabScale Agent's built-in file cache to reduce duplicate downloading of the same file. It replaces the legacy bin/labscale-agent-dl executable and accepts the same options and arguments.

  • Example:
    • To download the index.html file from the LabScale file service, retry 3 times on any network error, and write output files to /tmp:
    ./bin/labscale-toolbelt download -retries 3 -output '/tmp' 'https://files.labscale.com/index.html'