Skip to content

Latest commit

 

History

History
executable file
·
160 lines (124 loc) · 6.52 KB

README.md

File metadata and controls

executable file
·
160 lines (124 loc) · 6.52 KB
Raw

Vanadium

Vanadium is a tool for hyperspectral image processing, similar to siproc.

Roadmap

  • Features
    • Basic statistical operations
    • Principal Component Analysis
    • 32-bit float support
    • Image cropping
    • BIP support
    • BSQ support
    • BIL support
    • Image conversion
    • Image rendering
    • Image masking
    • Minimum Noise Fraction
    • 64-bit float support
    • Python wrapper
  • Performance
    • Better performance than siproc
    • io_uring support
    • BLAS acceleration
    • CUDA acceleration
  • Platform Support
    • Linux support
    • MacOS support
    • Windows support

CLI Tool Installation

Rust Toolchains

You will need to first have the Rust toolchain installed. Navigate to the linked page and follow the instructions given. It is not recommended to use an OS package manager to install the toolchain, unless you know what you are doing.

BLAS

You will need to select a blas implementation. Currently supported are:

  • openblas
  • netlib

Linux CLI Install

git clone https://github.com/Noah-Kennedy/vanadium.git
cd vanadium
cargo install --path . --features="linux, {YOUR_BLAS_HERE}"

MacOS CLI Install

git clone https://github.com/Noah-Kennedy/vanadium.git
cd vanadium
cargo install --path . --features="macos, {YOUR_BLAS_HERE}"

Windows CLI Install

git clone https://github.com/Noah-Kennedy/vanadium.git
cd vanadium
cargo install --path . --features="windows, {YOUR_BLAS_HERE}"

Usage

For CLI documentation:

vanadium-cli --help

Note that vanadium-cli does not accept normal ENVI header files, but uses its own format. You can use the tool to construct header files quite easily.

Benchmarks

Benchmarks were all run with a cold file cache between each trial, with 1 warmup trial. Benchmarks were run using hyperfine.

Configurations

Name RAM [GiB] CPU (#cores) SSD Class GPU Kernel
Workstation 64G AMD Ryzen 3900X (12) SATA RTX 2080 Ti 5.13
Laptop 16G NVME NA 5.13
Nvidia DJX 16G NVME TODO 4.x

Commands

Tool Operation File Command
vanadium (io-uring) means small hyperfine "vanadium-cli --backend glommio means -o small-means.json --header small-header.json" -p "sudo sh scripts/drop.sh"
vanadium (syscall) means small hyperfine "vanadium-cli --backend syscall means -o small-means.json --header small-header.json" -p "sudo sh scripts/drop.sh"
vanadium (mmap) means small hyperfine "vanadium-cli --backend mmap means -o small-means.json --header small-header.json" -p "sudo sh scripts/drop.sh"
vanadium (io-uring) means large hyperfine "vanadium-cli --backend glommio means -o large-means.json --header large-header.json" -p "sudo sh scripts/drop.sh"
vanadium (syscall) means large hyperfine "vanadium-cli --backend syscall means -o large-means.json --header large-header.json" -p "sudo sh scripts/drop.sh"
vanadium (mmap) means large hyperfine "vanadium-cli --backend mmap means -o large-means.json --header large-header.json" -p "sudo sh scripts/drop.sh"
siproc means large

Results

Small File (5 bands, 11.75 GiB)

Spectral Means
Tool Machine Configuration Time (mean ± σ)
vanadium (io-uring) Laptop 4.511 s ± 0.219 s
vanadium (syscall) Laptop 12.531 s ± 0.139 s
vanadium (mmap) Laptop 13.486 s ± 0.119 s
vanadium (io-uring) Workstation 22.276 s ± 0.020 s
vanadium (syscall) Workstation 22.839 s ± 0.012 s
vanadium (mmap) Workstation 24.768 s ± 0.057 s
siproc Workstation 484.511 s ± 2.917 s
siproc Laptop 590.850 s ± 2.809 s
Covariance Matrix
Tool Machine Configuration Time (mean ± σ)
vanadium (io_uring) Laptop 8.718 s ± 0.019 s
vanadium (syscall) Laptop 14.171 s ± 0.050 s
vanadium (mmap) Laptop 14.457 s ± 0.063 s
vanadium (io-uring) Workstation 22.308 s ± 0.009 s
vanadium (syscall) Workstation 22.840 s ± 0.003 s
vanadium (mmap) Workstation 24.859 s ± 0.049 s
siproc Laptop 669.587 s ± 4.004 s
siproc Workstation s ± s

Medium File (394 bands, ~106 GiB)

Spectral Means
Tool Machine Configuration Time (mean ± σ)
vanadium (io_uring) Laptop 39.123 s ± 3.695 s
vanadium (syscall) Laptop 79.864 s ± 0.840 s
vanadium (mmap) Laptop 176.015 s ± 5.354 s
siproc Laptop 199.767 s ± 1.026 s
vanadium (io_uring) Workstation 202.313 s ± 0.504 s
vanadium (syscall) Workstation 210.469 s ± 0.301 s
siproc Workstation 210.635 s ± 0.627 s
vanadium (mmap) Workstation 232.937 s ± 0.458 s
Covariance Matrix
Tool Machine Configuration Time (mean ± σ)
vanadium (io_uring) Workstation 352.336 s ± 0.103 s
vanadium (io_uring) Laptop 503.445 s ± 2.007 s
vanadium (syscall) Workstation 513.370 s ± 1.796 s
vanadium (mmap) Workstation 536.006 s ± 2.626 s
vanadium (syscall) Laptop 559.402 s ± 7.951 s
siproc (cuda) Workstation ~1541.157 s
siproc (cpu) Laptop ~7079.293 s
vanadium (mmap) Laptop s ± s
siproc (cpu) Workstation s ± s