zeekay
dda5b2a367
The previous commit was meant to remove .github/workflows and removed every tracked path instead — 892 files, the whole repo. This puts back the 874 that were never in scope; the eighteen workflow files stay gone, for the reasons that commit gives. Content is byte-identical to 9d2aec9 outside .github/workflows.
1.9 KiB
Benchmarking Pipeline
The benchmarking pipeline allows you to run multiple models across combination of parameters and dataset categories to collect metrics. The benchmarking run is configured using a config file that specifies the grid-search parameters. A sample config file is shown below:
accelerator:
- cuda
- cpu
benchmark:
seed: 42
model:
class_path:
grid_search: [Padim, Patchcore]
data:
class_path: MVTecAD
init_args:
category:
grid:
- bottle
- cable
- capsule
The accelerator parameter is specific to the pipeline and is used to configure the runners. When cuda is passed it adds a parallel runner with number of jobs equal to the number of cuda devices. The idea is that since job is independent, we can increase the throughput by distributing each on an individual accelerator. The cpu jobs are run serially.
Running the Benchmark Pipeline
There are two ways to run the benchmark pipeline; as a subcommand, or as a standalone entrypoint.
:::::{dropdown} CLI :icon: code
::::{tab-set} :::{tab-item} Anomalib subcommand :sync: label-1
:language: bash
:::
:::{tab-item} Standalone entrypoint :sync: label-2
:language: bash
:::
:::::
Benchmark Pipeline Class
.. autoclass:: anomalib.pipelines.benchmark.pipeline.Benchmark
:members:
:inherited-members:
:show-inheritance:
::::{grid} :margin: 1 1 0 0 :gutter: 1
:::{grid-item-card} Job 🔗 ./job :link-type: doc
Benchmark Job :::
:::{grid-item-card} Generator 🔗 ./generator :link-type: doc
Benchmark Job Generator :::
::::
:caption: Benchmark
:hidden:
./job
./generator