FluidUMAP | SuperCollider 3.13.0 Help

Description

Reduce the dimensions of a FluidDataSet using the Uniform Manifold Approximation and Projection (UMAP) algorithm.

Performs dimensionality reduction of a FluidDataSet using Uniform Manifold Approximation and Projection (UMAP)

Please refer to https://umap-learn.readthedocs.io/ and https://learn.flucoma.org/reference/umap for more information on the algorithm.

Read more about FluidUMAP on the learn platform.

Class Methods

FluidUMAP.new(server, numDimensions: 2, numNeighbours: 15, minDist: 0.1, iterations: 200, learnRate: 0.1)

Arguments:

server	The Server on which to construct this object
numDimensions	The number of dimensions to reduce to Constraints Minimum: `1`
numNeighbours	The number of neighbours considered by the algorithm to balance local vs global structures to conserve. Low values will prioritise preservation of the local structure while high values will prioritise preservation of the global structure. Constraints Minimum: `1`
minDist	The minimum distance each point is allowed to be from the others in the low dimension space. Low values will make tighter clumps, and higher will spread the points more. Constraints Minimum: `0`
iterations	The number of iterations that the algorithm will go through to optimise the new representation Constraints Minimum: `1`
learnRate	The learning rate of the algorithm, aka how much of the error it uses to estimate the next iteration. Constraints Minimum: `0.0` Maximum: `1.0`

Instance Methods

.minDist = value

Property for minDist. See new

.fitTransform(sourceDataSet, destDataSet, action)

Fit the model to a FluidDataSet and write the new projected data to a destination FluidDataSet.

Arguments:

sourceDataSet	Source data, or the DataSet name
destDataSet	Destination data, or the DataSet name
action	A function to execute when the server has completed running fitTransform

.fit(dataSet, action)

Train this model on a FluidDataSet but don't transform the data

Arguments:

dataSet	A FluidDataSet to analyse
action	A function to execute when the server has completed running fit

.transform(sourceDataSet, destDataSet, action)

Given a trained model, apply the reduction to a source FluidDataSet and write to a destination. Can be the same for both input and output (in-place).

Arguments:

sourceDataSet	Source data, or the DataSet name
destDataSet	Destination data, or the DataSet name
action	A function to execute when the server has completed running transform

.transformPoint(sourceBuffer, destBuffer, action)

Transform a new data point to the reduced number of dimensions using the projection learned from previous fit call to FluidUMAP.

Arguments:

sourceBuffer	A Buffer with the new data point.
destBuffer	A Buffer to contain the dimensionally reduced data point.
action	A function to execute when the server has completed running transformPoint

.cols(action)

The number of columns (dimensions) in this model or dataset / labeset

Arguments:

action

A function to execute when the server has completed running cols

.clear

From superclass: Object

Resets the internal state of the model

Arguments:

(action)

A function to execute when the server has completed running clear

.size(action)

The number of data points (entries / observations) in this model or dataset / labeset

Arguments:

action

A function to execute when the server has completed running size

.load(dict, action)

From superclass: FluidDataObject

Replace the internal state of the object from a Dictionary.

Arguments:

dict
action	A function to execute when the server has completed running load

.dump(action)

From superclass: FluidDataObject

Dump the state of this object as a Dictionary, which will be passed to the action function provided. This object must first be fit``ted before ``dump can be called.

Arguments:

action

A function to execute when the server has completed running dump

.write(filename, action)

From superclass: FluidDataObject

Save the internal state of the object to a JSON file on disk. This object must first be fit before write can be called.

Arguments:

filename	Path of the file to load from
action	A function to execute when the server has completed running write

.read(filename, action)

From superclass: FluidDataObject

Replace the internal state of the object from a JSON file on disk.

Arguments:

filename	Path of the file to load from
action	A function to execute when the server has completed running read

.fitMsg(dataSet)

.fitTransformMsg(sourceDataSet, destDataSet)

.kr(trig, inputBuffer, outputBuffer)

.transformMsg(sourceDataSet, destDataSet)

.transformPointMsg(sourceBuffer, destBuffer)

// Using UMAP we'll reduce this dataset from 26 dimensions (26 MFCC values) down to 2 dimensions so we can plot it
(
~loader = FluidLoadFolder(FluidFilesPath()).play; // load the audio (all the files in the FluCoMa audio files folder!)
~slicePoints = Buffer.read(s,FluidFilesPath("../Data/flucoma_corpus_slices.wav")); // load the slice positions of the audio
~ds_mfcc = FluidDataSet(s).read(FluidFilesPath("../Data/flucoma_corpus_mfcc.json")); // load the pre-analyzed dataset
)

// reduce and plot
// try changing the numNeighbours and minDist to see how it affects the plot.
(
var reduced = FluidDataSet(s);
var tree;
FluidUMAP(s,2,numNeighbours:15,minDist:0.1).fitTransform(~ds_mfcc,reduced);
FluidNormalize(s).fitTransform(reduced,reduced);// normalize so it's easier to plot
tree = FluidKDTree(s).fit(reduced); // use a kdtree to find the point nearest to the mouse position
reduced.dump({
    arg dict;
    var xybuf = Buffer.alloc(s,2);
    defer{
        FluidPlotter(dict:dict,mouseMoveAction:{
            arg view, x, y;
            xybuf.setn(0,[x,y]);
            tree.kNearest(xybuf,1,{
                arg id;
                var index = id.asInteger;
                defer{
                    view.highlight_(id);
                };

{
                    var start = Index.kr(~slicePoints,index);
                    var end = Index.kr(~slicePoints,index+1);
                    var sig = PlayBuf.ar(2,~loader.buffer,BufRateScale.ir(~loader.buffer),1,start)[0];
                    var dur_sec = min((end-start) / SampleRate.ir,1);
                    var env = EnvGen.kr(Env([0,1,1,0],[0.03,dur_sec-0.06,0.03]),doneAction:2);
                    sig.dup * env;
                }.play;
            });
        });
    };
});
)

Reducing a 3 dimensional colour space to 2 dimensional space// make a dataset of 100 random 3 dimensional points (that will later be RGB values) ( ~ds_rgb = FluidDataSet(s).load( Dictionary.newFrom([ "cols",3, "data",Dictionary.newFrom( 100.collect{ arg i; [i,{rrand(0.0,1.0)} ! 3] }.flatten; ) ]) ); ~ds_rgb.print; ) // rather than trying to plot these points in 3 dimensional space, // we'll reduce it to 2 dimensional space so it looks better on a screen // again play around with the arguments numNeighbours and minDist to see // how they affect he spread of the points ( var reduced = FluidDataSet(s); FluidUMAP(s,2,numNeighbours:15,minDist:0.6).fitTransform(~ds_rgb,reduced); FluidNormalize(s).fitTransform(reduced,reduced); ~ds_rgb.dump{ arg rgb; reduced.dump{ arg xy; defer{ var fp = FluidPlotter(dict:xy).pointSizeScale_(2); rgb["data"].keysValuesDo{ arg k, v; fp.pointColor_(k,Color(*v)); }; }; }; }; )

Retrieving values on the server// Using UMAP we'll reduce this dataset from 26 dimensions (26 MFCC values) down to 2 dimensions // wait for it to finish ( ~loader = FluidLoadFolder(FluidFilesPath()).play; // load the audio (all the files in the FluCoMa audio files folder!) ~slicePoints = Buffer.read(s,FluidFilesPath("../Data/flucoma_corpus_slices.wav")); // load the slice positions of the audio ~ds_mfcc = FluidDataSet(s).read(FluidFilesPath("../Data/flucoma_corpus_mfcc.json")); // load the pre-analyzed dataset ~reduced = FluidDataSet(s); ~umap = FluidUMAP(s,2,15,0.1).fitTransform(~ds_mfcc,~reduced); ~normalizer = FluidNormalize(s).fitTransform(~reduced,~reduced,action:{"done".postln;}); ) ( ~reduced.dump{ arg dict; defer{ ~fp = FluidPlotter(dict:dict).addPoint_("current",0.5,0.5,Color.red,2); { // play the original buffer back, but backwardsa and a little faster for some variety var sig = PlayBuf.ar(2,~loader.buffer,-1.3,1,rrand(0,~loader.buffer.numFrames-1),1)[0]; var mfccs = FluidMFCC.kr(sig,26,startCoeff:1); // get mfcc analyses var mfccbuf = LocalBuf(mfccs.numChannels); var reducedbuf = LocalBuf(2); var normbuf = LocalBuf(2); var trig = Impulse.kr(30); var xy; FluidKrToBuf.kr(mfccs,mfccbuf); ~umap.kr(trig,mfccbuf,reducedbuf); // project the mfcc analyses into the umap space ~normalizer.kr(trig,reducedbuf,normbuf); // make sure it's in the normalized space xy = FluidBufToKr.kr(normbuf); SendReply.kr(trig,"/xy",xy); // send back to the language for plotting sig; }.play; OSCdef(\xy,{ arg msg; defer{ ~fp.setPoint_("current",msg[3],msg[4],Color.red,2); // shows where the current mfcc analysis would be }; },"/xy"); } }; )

helpfile source: /usr/share/SuperCollider/Extensions/FluidCorpusManipulation/HelpSource/Classes/FluidUMAP.schelp
link::Classes/FluidUMAP::

FluidUMAP : FluidModelObject : FluidDataObject : FluidServerObject : Object Extension

Description

Class Methods

FluidUMAP.new(server, numDimensions: 2, numNeighbours: 15, minDist: 0.1, iterations: 200, learnRate: 0.1)

Arguments:

Inherited class methods

Instance Methods

.numDimensions

.numDimensions = value

.numNeighbours

.numNeighbours = value

.minDist

.minDist = value

.iterations

.iterations = value

.learnRate

.learnRate = value

.fitTransform(sourceDataSet, destDataSet, action)

Arguments:

.fit(dataSet, action)

Arguments:

.transform(sourceDataSet, destDataSet, action)

Arguments:

.transformPoint(sourceBuffer, destBuffer, action)

Arguments:

.cols(action)

Arguments:

.clear

Arguments:

.size(action)

Arguments:

.load(dict, action)

Arguments:

.dump(action)

Arguments:

.write(filename, action)

Arguments:

.read(filename, action)

Arguments:

Inherited instance methods

Undocumented instance methods

.fitMsg(dataSet)

.fitTransformMsg(sourceDataSet, destDataSet)

.kr(trig, inputBuffer, outputBuffer)

.prGetParams

.transformMsg(sourceDataSet, destDataSet)

.transformPointMsg(sourceBuffer, destBuffer)

Examples

FluidUMAP : FluidModelObject : FluidDataObject : FluidServerObject : Object
Extension