- Python 100%
| .github/workflows | ||
| docs | ||
| src/koogu | ||
| tests | ||
| .gitignore | ||
| CHANGELOG.txt | ||
| HOWTO.md | ||
| LICENSE | ||
| pyproject.toml | ||
| README-upstream.md | ||
| README.md | ||
Koogu
Koogu is a Python package for developing and using Machine Learning (ML) solutions in Animal Bioacoustics.
Fork
This repository is a fork of the upstream Koogu project by Shyam Madhusudhana.
See README-upstream.md for the upstream README.
Upstream documentation:
See also: HOWTO.md.
Install
Dependencies
Install system dependencies:
sudo apt install python3-pip python3-venv python-is-python3
Python
Install to Python taste, such as:
git clone https://spacecruft.org/deepcrayon/koogu
cd koogu/
python -m venv venv
source venv/bin/activate
pip install -U setuptools pip wheel
pip install -e .
Install Tensorflow for either CPU or GPU, depending on the host system:
# CPU
pip install -e .[cpu]
# GPU
pip install -e .[gpu]
Documentation
Documentation build dependencies can be installed, such as:
pip install -e .[docs]
Development
Development dependencies can be installed thusly:
pip install -e .[dev]
Usage
Thusly.
koogu-example
An example script is in src/koogu/example.py.
$ koogu-example
Inference can be run with the example model, such as:
koogu-inference --raw-outputs tmp/raw tmp/model/ tmp/chickadee/recordings-annotations/swift/
koogu-preprocess
$ koogu-preprocess --help
usage: preprocess [-h] [-V] [--whitelist FILE] [--selmap CSVFILE] [--selroot DIRECTORY] [--accept-thld 0-100] [--save-reject-class] [--reject-thld 0-100] [--threads NUM] [--log FILE] [--loglevel {CRITICAL,ERROR,WARNING,INFO,DEBUG}]
[--filetypes EXTN [EXTN ...]] [--maxdur SECONDS]
<CONFIG FILE> <AUDIO SOURCE> <DST DIRECTORY>
Prepare audio data before their conversion to TFRecords.
positional arguments:
<CONFIG FILE> Path to config file.
<AUDIO SOURCE> Path to either a single audio file or to a directory. When a directory, if selection table info is also provided (using 'selmap'), then this must be the root path from which relative paths to audio files in the
selection tables will be resolved. Otherwise, this must be the root directory containing per-class top-level subdirectories which in turn contain audio files.
<DST DIRECTORY> Path to destination directory into which prepared data will be written.
options:
-h, --help show this help message and exit
-V, --version Show version
--whitelist FILE Path to text file containing names (one per line) of whitelisted classes.
Selection tables:
Control which sections of audio files are retained in the output, with the use of Raven selection tables.
--selmap CSVFILE Path to csv file containing one-to-one mappings from audio files to selection table files. Audio filepaths must be relative to <AUDIO_SOURCE>. If selection table files are not absolute paths, use 'selroot' to
specify the root directory path.
--selroot DIRECTORY Path to the root directory containing selection table files. Note that, if this is specified, all selection table paths in 'selmap' file be treated as relative paths.
--accept-thld 0-100 Clips from the source audio files are retained in the output only if the percentage of their temporal overlap with any annotation in a matched selection table is above this threshold value. Default: 90%.
--save-reject-class Enable saving of clips that do not match annotations as 'other' class. Default: False.
--reject-thld 0-100 Clips from the source audio files are retained in the output 'other' class only if the percentage of their temporal overlap with any annotation in a matched selection table is under this threshold value. Default:
0%.
Process control:
--threads NUM Number of threads to spawn for parallel execution (default: as many CPUs).
Logging:
--log FILE Path to file to which logs will be written out.
--loglevel {CRITICAL,ERROR,WARNING,INFO,DEBUG}
Logging level.
Miscellaneous:
--filetypes EXTN [EXTN ...]
Audio file types to restrict processing to. Option is ignored if processing selection tables or a single file. Can specify multiple types separated by whitespaces. By default, will include for processing all
discovered files with the following extensions: .wav, .WAV, .flac, .aif, .mp3
--maxdur SECONDS Maximum duration of an audio file to consider it for processing. Larger files will be ignored. Default: no limit.
koogu-train-eval
$ koogu-train-eval --help
usage: train_and_eval [-h] [-V] [--batch-size NUM] [--epochs NUM] [--epochs-between-evals NUM] [--dropout-rate 0-100] [--learning-rate NUM] [--seed NUM] [--dim-order {NCHW,NHWC}] [--non-augmented-class CLASS [CLASS ...]] [--log FILE]
[--loglevel {CRITICAL,ERROR,WARNING,INFO,DEBUG}]
<DATA DIR> <MODEL DIR> <CONFIG FILE> <ARCHITECTURE>
Train a TF model.
positional arguments:
<DATA DIR> Path to the root directory containing training and validation data.
<MODEL DIR> Path to destination directory into which model-specific contents will be written out.
<CONFIG FILE> Path to config file.
<ARCHITECTURE> Model architecture.
options:
-h, --help show this help message and exit
-V, --version Show version
Training config override:
Overrides settings obtained from the config file.
--batch-size NUM Size to batch the inputs into.
--epochs NUM Number of epochs to train for.
--epochs-between-evals NUM
How often evaluation is to be performed.
--dropout-rate 0-100 Dropout probability (as a percent). Higher value = more regularization.
--learning-rate NUM Static (or initial value of) learning rate.
Miscellaneous:
--seed NUM Seed value (integer) for deterministic shuffling.
--dim-order {NCHW,NHWC}
Dimension ordering of data. 'NCHW' (a.k.a channels first) is [batch, channels, height, width] which is good when training on GPU using cuDNN. 'NHWC' (a.k.a channels last) is [batch, height, width, channels] which
is good when training on CPU. If unspecified, appropriate choice will be made depending on GPU availability.
--non-augmented-class CLASS [CLASS ...]
Name (case sensitive) of the class (like 'Noise' or 'Other') that need not be subject to data augmentation (if enabled). Can specify multiple (separated by whitespaces).
Logging:
--log FILE Path to file to which logs will be written out.
--loglevel {CRITICAL,ERROR,WARNING,INFO,DEBUG}
Logging level.
koogu-inference
$ koogu-inference --help
usage: predict [-h] [-V] [--filetypes EXTN [EXTN ...]] [--recursive] [--channels # [# ...]] [--clip-advance SEC] [--raw-outputs DIR] [--processed-outputs DIR] [--reject-class CLASS [CLASS ...]] [--frequency-info FILE] [--combine-outputs]
[--threshold [0-1]] [--top | --squeeze MIN-DUR | --top-squeeze MIN-DUR] [--scale-scores] [--fetch-threads NUM] [--batch-size NUM] [--show-progress] [--log FILE] [--loglevel {CRITICAL,ERROR,WARNING,INFO,DEBUG}]
<MODEL DIR> <AUDIO SOURCE>
Make inferences using a trained model.
positional arguments:
<MODEL DIR> Path to the directory containing a TensorFlow exported model.
<AUDIO SOURCE> Path to either a single audio file or to a directory. When a directory, all files of the supported filetypes within the specified directory will be processed (use the --recursive flag to process subdirectories as
well).
options:
-h, --help show this help message and exit
-V, --version Show version
Input control:
--filetypes EXTN [EXTN ...]
Audio file types to restrict processing to. Option is ignored if processing a single file. Can specify multiple types separated by whitespaces. By default, will include for processing all discovered files with the
following extensions: .wav, .WAV, .flac, .aif, .mp3
--recursive Process files also in subdirectories of <AUDIO_SOURCE>.
--channels # [# ...] Channels to restrict processing to. List out the desired channel indices, separated with whitespaces. If unspecified, all available channels will beprocessed. Channel indices must be 0-based.
--clip-advance SEC When audio file's contents are broken up into clips, by default the amount of overlap between successive clips is determined by the settings that were in place during model training. Use this flag to alter that,
by setting a different amount (in seconds) of gap (or advance) between successive clips.
Output type(s):
At least one of these must be specified. If multiple audio files are to be processed, as many corresponding output files will be generated, and necessary subdirectories will be created.
--raw-outputs DIR If set, raw outputs from the model will be written out into the specified directory.
--processed-outputs DIR
If set, processed recognition results (Raven selection tables) will be written out into the specified directory. Use options under 'Output control' and 'Post-process control' for further control.
Output control:
These options will have no effect if --processed-outputs is not specified.
--reject-class CLASS [CLASS ...]
Name (case sensitive) of the class (like 'Noise' or 'Other') that must be ignored from the recognition results. The corresponding detections will not be written to the output selection tables. Can specify multiple
(separated by whitespaces).
--frequency-info FILE
Path to a json file containing a dictionary of per-class frequency bounds. If unspecified, the "Low Frequency (Hz)" and "High Frequency (Hz)" fields in the output table will be the same for all classes.
--combine-outputs Enable this to combine recognition results of processing every file within a directory and write them to a single output file. When enabled, the outputs will contain 2 additional fields describing offsets of
detections in the corresponding audio files.
--threshold [0-1] Suppress writing of detections with confidence below this value.
Post-process control:
By default, per-class scores from successive clips are averaged to produce the results. You may choose from one of the below alternative algorithms instead. These options will have no effect if --processed-outputs is not specified.
--top Same algorithm as default, but only considers the top-scoring class for each clip.
--squeeze MIN-DUR An algorithm 'to squeeze together' temporally overlapping regions from successive raw detections will be applied. The 'squeezing' will be restricted to produce detections that are at least 'MIN-DUR' seconds long.
MIN-DUR must be smaller than the duration of the model input.
--top-squeeze MIN-DUR
Same algorithm as --squeeze, but only considers the top-scoring class from each clip.
--scale-scores Enable this to scale the raw scores. Use of this setting is most recommended when the output of a model is based on softmax and the model was trained with training data where each input corresponded to a single
class.
Miscellaneous:
--fetch-threads NUM Number of threads that will fetch audio from files in parallel.
--batch-size NUM Size to batch audio file's clips into (default: 1). Increasing this may improve speed on computers with high RAM.
--show-progress Show progress of processing on screen.
Logging:
--log FILE Path to file to which logs will be written out.
--loglevel {CRITICAL,ERROR,WARNING,INFO,DEBUG}
Logging level.
License
GPLv3.
Unofficial project, not related to upstream projects.
Upstream sources under their respective copyrights.
Copyright © 2025 Jeff Moe.