NatureLM-audio

Audio-Language Foundation Model for Bioacoustics.

• https://spacecruft.org/deepcrayon/NatureLM-audio

Fork

This repository is a fork of:

• https://github.com/earthspecies/NatureLM-audio

The upstream code requires a google API key.

This version is De-googled.

See README-upstream.md for the upstream README.

Install

Note, upstream ships with a .python-version with Python 3.10. So you probably want a version of Python 3.10 installed.

They use uv upstream, so perhaps use that. Install to taste, such as:

python -m venv .venv
source .venv/bin/activate
pip install -U pip uv
uv sync

Usage

Inference

On an Nvidia A6000, uses 16.628Gi VRAM, slightly too much for A16.

The file extensions need to be lower case (e.g. .wav not .WAV).

This perhaps works. Note output_path is a file, not a directory name.

python NatureLM/infer.py --cfg-path configs/inference.yml --audio ../audio/ --query "Caption the audio" --window_length_seconds 10.0 --hop_length_seconds 10.0 --output_path ../out

And/or run with uv:

uv run naturelm infer --cfg-path configs/inference.yml --audio-path ../audio/ --query "Caption the audio" --window-length-seconds 10.0 --hop-length-seconds 10.0 --output-path ../out  

BEANS-Zero inference (~50G dataset, uses 23.073G GPU VRAM, takes ~4.5 hours to run on an Nvidia A6000):

python beans_zero_inference.py --cfg-path configs/inference.yml --data_path EarthSpeciesProject/BEANS-Zero --output_path ../out-NatureLM-audio/beans_zero_eval.jsonl --beans_zero_config_path beans_zero_dataset_config.json

Whence SPICE?

Running beans_zero_inference.py uses the beans-zero package. The Earth Species Project code for that is here:

• https://github.com/earthspecies/beans-zero

That repository contains this blob:

• beans_zero/external/spice/spice-1.0.jar

That ancient blob requires Java 8, which was released in 2014.

The Earth Species Project got the blob from this repo:

• https://github.com/salaniz/pycocoevalcap

That repo is a fork of an earlier repo. The newer fork updates the Python code from old Python 2.7.

• https://github.com/tylin/coco-caption

All three copies of spice-1.0.jar from the above repos have this sha1sum:

• a27d265796c64d167834bcd4b0cf4b7ba6ca11a4

SPICE itself, afaict, comes from the authors of this 2016 paper, "SPICE: Semantic Propositional Image Caption Evaluation"

• https://arxiv.org/abs/1607.08822

That paper links to this archive:

• http://panderson.me/spice

That archive has a link to a SPICE-1.0.zip archive, which contains SPICE-1.0/spice-1.0.jar.

• https://panderson.me/images/SPICE-1.0.zip

The spice-1.0.jar in that archive is 19751099 bytes, identical to the size of the spice-1.0.jar in the beans-zero and earlier archives. But the panderson.me has a different checksum, so it is a different file:

• 925fad3c2030e86c3d99375e55eb2ee15973169d

It is unclear who built the blob used by the Earth Species Project.

The panderson.me site links to this repo:

• https://github.com/peteanderson80/SPICE

This appears to be the actual source code for the blob. The code is under the GNU AGPL v3 license. It was last modified in 2018.

The tylin/coco-caption repository, which is the most likely candidate for who built the blob, uses the BSD 2-Clause license (Simplified BSD license). This may be an issue, because the upstream license is AGPL v3.

The salaniz/pycocoevalcap repository, also uses this BSD license, again not mentioning the original AGPL v3 license.

The Earth Species Project beans-zero repository contains no license file. Neither the README nor the pyproject.toml file make any reference to a license. None of the source code files' headers have a license. Licenses are only mentioned with respect to the datasets used, nothing about the code itself. The paper the repository references is here:

• https://openreview.net/forum?id=hJVdwBpWjt

That paper states:

"To advance bioacoustics research, we release our model weights, benchmark data, and open-source the code for training and benchmark data generation and model training."

That quote is linked to this page:

• https://earthspecies.github.io/naturelm-audio-demo/

Which links to this repository:

• https://github.com/earthspecies/NatureLM-audio

That repository is under the MIT license. I see no mention of the AGPLv3 license.

Without an ancient Java install, running beans-zero will give this error:

ERROR:beans_zero:Error evaluating dataset captioning: [Errno 2] No such file or directory: 'java'

I'm not sure why the Earth Species Project is using this ancient blob.

License

MIT license.

Unofficial project, not related to upstream projects.

Upstream sources under their respective copyrights.

Copyright © 2025 Jeff Moe.