FluidBufNoveltySlice:
Filter:
Classes (extension) | Libraries > FluidCorpusManipulation

FluidBufNoveltySlice : FluidBufProcessor : FluidServerObject : Object
ExtensionExtension

Buffer-Based Novelty-Based Slicer

Description

A non-realtime slicer using an algorithm assessing novelty in the signal to estimate the slicing points.

A novelty curve is derived from running a kernel across the diagonal of the similarity matrix, and looking for peaks of changes. It implements the seminal results published in 'Automatic Audio Segmentation Using a Measure of Audio Novelty' by J Foote.

The process will return a buffer which contains indices (in samples) of estimated starting points of different slices.

The argument for algorithm can be passed as an integer (see table below), or as one of the following symbols: \spectrum, \mfcc, \chroma, \pitch, or \loudness.

Read more about FluidBufNoveltySlice on the learn platform.

Class Methods

FluidBufNoveltySlice.process(server, source, startFrame: 0, numFrames: -1, startChan: 0, numChans: -1, indices, algorithm: 0, kernelSize: 3, threshold: 0.5, filterSize: 1, minSliceLength: 2, windowSize: 1024, hopSize: -1, fftSize: -1, freeWhenDone: true, action)

FluidBufNoveltySlice.processBlocking(server, source, startFrame: 0, numFrames: -1, startChan: 0, numChans: -1, indices, algorithm: 0, kernelSize: 3, threshold: 0.5, filterSize: 1, minSliceLength: 2, windowSize: 1024, hopSize: -1, fftSize: -1, freeWhenDone: true, action)

Processs the source Buffer on the Server. processBlocking will execute directly in the server command FIFO, whereas process will delegate to a separate worker thread. The latter is generally only worthwhile for longer-running jobs where you don't wish to tie up the server.

Arguments:

server

The Server on which the buffers to be processed are allocated.

source

The buffer to use as the source material to be sliced through novelty identification. The different channels of multichannel buffers will be summed.

startFrame

Where in the srcBuf should the slicing process start, in samples.

Constraints

  • Minimum: 0
numFrames

How many frames should be processed.

startChan

For multichannel srcBuf, which channel should be processed.

Constraints

  • Minimum: 0
numChans

For multichannel srcBuf, how many channels should be summed.

indices

The buffer where the indices (in samples) of the estimated starting points of slices will be written. The first and last points are always the boundary points of the analysis.

algorithm

The feature on which novelty is computed.

0Spectrum – The magnitude of the full spectrum.
1MFCC – 13 Mel-Frequency Cepstrum Coefficients.
2Chroma - The contour of a 12-band chromagram.
3Pitch – The pitch and its confidence.
4Loudness – The true peak and loudness.

kernelSize

The granularity of the window in which the algorithm looks for change, in samples. A small number will be sensitive to short term changes, and a large number should look for long term changes.

Constraints

  • Minimum: 3
  • Maximum: maxkernelSize
  • Snaps to odd numbers
threshold

The normalised threshold, between 0 and 1, on the novelty curve to consider it a segmentation point.

Constraints

  • Minimum: 0
filterSize

The size of a smoothing filter that is applied on the novelty curve. A larger filter size allows for cleaner cuts on very sharp changes.

Constraints

  • Minimum: 1
  • Maximum: maxfilterSize
minSliceLength

The minimum duration of a slice in number of hopSize.

Constraints

  • Minimum: 0
windowSize

The window size. As novelty estimation relies on spectral frames, we need to decide what precision we give it spectrally and temporally. For more information visit https://learn.flucoma.org/learn/fourier-transform/

hopSize

The window hop size. As novelty estimation relies on spectral frames, we need to move the window forward. It can be any size, but low overlap will create audible artefacts.

fftSize

The inner FFT/IFFT size. It should be at least 4 samples long, at least the size of the window, and a power of 2. Making it larger allows an oversampling of the spectral precision.

freeWhenDone

Free the server instance when processing complete. Default true

action

A function to be evaluated once the offline process has finished and all Buffer's instance variables have been updated on the client side. The function will be passed [features] as an argument.

Returns:

An instance of the processor

FluidBufNoveltySlice.kr(source, startFrame: 0, numFrames: -1, startChan: 0, numChans: -1, indices, algorithm: 0, kernelSize: 3, threshold: 0.5, filterSize: 1, minSliceLength: 2, windowSize: 1024, hopSize: -1, fftSize: -1, trig: 1, blocking: 0)

Trigger the equivalent behaviour to processBlocking / process from a Synth. Can be useful for expressing a sequence of buffer and data processing jobs to execute. Note that the work still executes on the server command FIFO (not the audio thread), and it is the caller's responsibility to manage the sequencing, using the done status of the various UGens.

Arguments:

source

The buffer to use as the source material to be sliced through novelty identification. The different channels of multichannel buffers will be summed.

startFrame

Where in the srcBuf should the slicing process start, in samples.

Constraints

  • Minimum: 0
numFrames

How many frames should be processed.

startChan

For multichannel srcBuf, which channel should be processed.

Constraints

  • Minimum: 0
numChans

For multichannel srcBuf, how many channels should be summed.

indices

The buffer where the indices (in samples) of the estimated starting points of slices will be written. The first and last points are always the boundary points of the analysis.

algorithm

The feature on which novelty is computed.

0Spectrum – The magnitude of the full spectrum.
1MFCC – 13 Mel-Frequency Cepstrum Coefficients.
2Chroma - The contour of a 12-band chromagram.
3Pitch – The pitch and its confidence.
4Loudness – The true peak and loudness.

kernelSize

The granularity of the window in which the algorithm looks for change, in samples. A small number will be sensitive to short term changes, and a large number should look for long term changes.

Constraints

  • Minimum: 3
  • Maximum: maxkernelSize
  • Snaps to odd numbers
threshold

The normalised threshold, between 0 and 1, on the novelty curve to consider it a segmentation point.

Constraints

  • Minimum: 0
filterSize

The size of a smoothing filter that is applied on the novelty curve. A larger filter size allows for cleaner cuts on very sharp changes.

Constraints

  • Minimum: 1
  • Maximum: maxfilterSize
minSliceLength

The minimum duration of a slice in number of hopSize.

Constraints

  • Minimum: 0
windowSize

The window size. As novelty estimation relies on spectral frames, we need to decide what precision we give it spectrally and temporally. For more information visit https://learn.flucoma.org/learn/fourier-transform/

hopSize

The window hop size. As novelty estimation relies on spectral frames, we need to move the window forward. It can be any size, but low overlap will create audible artefacts.

fftSize

The inner FFT/IFFT size. It should be at least 4 samples long, at least the size of the window, and a power of 2. Making it larger allows an oversampling of the spectral precision.

trig

A kr signal that will trigger execution

blocking

Whether to execute this process directly on the server command FIFO or delegate to a worker thread. See processBlocking/process for caveats.

Inherited class methods

Instance Methods

.cancel

From superclass: FluidBufProcessor

Cancels non-blocking processing

.wait

From superclass: FluidBufProcessor

When called in the context of a Routine (it won't work otherwise), will block execution until the processor has finished. This can be convinient for writing sequences of processes more linearly than using lots of nested actions.

Inherited instance methods

Examples

helpfile source: /usr/share/SuperCollider/Extensions/FluidCorpusManipulation/HelpSource/Classes/FluidBufNoveltySlice.schelp
link::Classes/FluidBufNoveltySlice::