https://huggingface.co/laion/larger_clap_general with ONNX weights to be compatible with Transformers.js.
Usage (Transformers.js)
If you haven't already, you can install the Transformers.js JavaScript library from NPM using:
npm i @xenova/transformers
Example: Perform zero-shot audio classification with Xenova/larger_clap_general
.
import { pipeline } from '@xenova/transformers';
const classifier = await pipeline('zero-shot-audio-classification', 'Xenova/larger_clap_general');
const audio = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/piano.wav';
const candidate_labels = ['calm piano music', 'heavy metal music'];
const scores = await classifier(audio, candidate_labels);
// [
// { score: 0.9829504489898682, label: 'calm piano music' },
// { score: 0.017049523070454597, label: 'heavy metal music' }
// ]
Example: Compute text embeddings with ClapTextModelWithProjection
.
import { AutoTokenizer, ClapTextModelWithProjection } from '@xenova/transformers';
// Load tokenizer and text model
const tokenizer = await AutoTokenizer.from_pretrained('Xenova/larger_clap_general');
const text_model = await ClapTextModelWithProjection.from_pretrained('Xenova/larger_clap_general');
// Run tokenization
const texts = ['calm piano music', 'heavy metal music'];
const text_inputs = tokenizer(texts, { padding: true, truncation: true });
// Compute embeddings
const { text_embeds } = await text_model(text_inputs);
// Tensor {
// dims: [ 2, 512 ],
// type: 'float32',
// data: Float32Array(1024) [ ... ],
// size: 1024
// }
Example: Compute audio embeddings with ClapAudioModelWithProjection
.
import { AutoProcessor, ClapAudioModelWithProjection, read_audio } from '@xenova/transformers';
// Load processor and audio model
const processor = await AutoProcessor.from_pretrained('Xenova/larger_clap_general');
const audio_model = await ClapAudioModelWithProjection.from_pretrained('Xenova/larger_clap_general');
// Read audio and run processor
const audio = await read_audio('https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/piano.wav');
const audio_inputs = await processor(audio);
// Compute embeddings
const { audio_embeds } = await audio_model(audio_inputs);
// Tensor {
// dims: [ 1, 512 ],
// type: 'float32',
// data: Float32Array(512) [ ... ],
// size: 512
// }
Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using 🤗 Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx
).
- Downloads last month
- 20
Inference API (serverless) does not yet support transformers.js models for this pipeline type.