Streaming server for the AniMUL model https://spacecruft.org/deepcrayon/AniMUL-stream

Python 72.8%
Svelte 9.7%
Shell 7.3%
TypeScript 6.8%
CSS 2%
Other 1.4%

Find a file

Jeff Moe e188209cd1 v0.8.1		2026-02-04 12:51:00 -07:00
backend	Fix: use float32 not int16 for vllm	2026-02-04 12:42:32 -07:00
docs	Add screenshot	2026-02-02 21:34:03 -07:00
frontend	Change browser colors	2026-02-03 11:33:13 -07:00
scripts	More config examples	2026-02-03 11:26:18 -07:00
.gitignore	git ignores	2026-02-02 10:57:23 -07:00
.python-version	Python 3.13	2026-02-02 11:41:21 -07:00
CHANGELOG.txt	v0.8.1	2026-02-04 12:51:00 -07:00
LICENSE-apache.txt	Apache 2.0	2026-02-02 10:56:48 -07:00
README.md	Add screenshot	2026-02-02 21:34:03 -07:00

README.md

AniMUL Stream

Real-time audio stream analysis server for the AniMUL model, a fine-tune of Qwen3-Omni trained on nature sounds.

Overview

AniMUL Stream processes live audio streams (SRT/RTMP) and identifies sounds in real-time using the AniMUL multimodal language model. Perfect for:

Wildlife monitoring with forest microphones
Bird call identification
Nature sound logging
Real-time biodiversity surveys

Architecture

Forest Microphones → OBS Studio → MediaMTX → SRT Stream
                                                  ↓
                                        AniMUL Stream Backend
                                                  ↓
                                            vLLM (GPU)
                                                  ↓
                                        WebSocket → Browser

Features

Real-time Processing: Continuous analysis of live audio streams
Sliding Window: Overlapping audio chunks to avoid missing sounds
WebSocket Updates: Instant results delivered to connected clients
Simple UI: Clean web interface for monitoring and control
vLLM Integration: OpenAI-compatible API for the AniMUL model

Quick Start

Prerequisites

Python 3.13+
Node.js 20+
vLLM server with AniMUL-v1 model
SRT audio stream source

Development Setup

# Clone repository
git clone https://spacecruft.org/deepcrayon/AniMUL-stream.git
cd AniMUL-stream

# Backend
cd backend
python3.13 -m venv venv
source venv/bin/activate
pip install -e ".[dev]"
cp ../scripts/animul-stream.env.example .env
uvicorn server.main:app --reload --port 8080

# Frontend (new terminal)
cd frontend
npm install
npm run dev

See docs/QUICKSTART.md for detailed setup instructions.

Documentation

Quick Start Guide - Get running quickly
Deployment Guide - Production deployment
Development Plan - Architecture and design

API

Endpoint	Method	Description
`/api/health`	GET	Health check
`/api/stream`	GET	Stream status
`/api/stream/start`	POST	Start processing
`/api/stream/stop`	POST	Stop processing
`/ws/stream`	WebSocket	Real-time results

AniMUL - Model fine-tuning
AniMUL-server - Single file analysis
AniMUL-v1 - Model on HuggingFace

License

Apache License 2.0 - See LICENSE-apache.txt

Author

Jeff Moe

moe@spacecruft.org

Loveland, Colorado, USA