Processing

This guide explains how to add a custom Audio Processing component with MimiCore inside MimiSDK, providing your users with Mimified audio.

Introduction

MimiCore provides the focal point for Mimified Audio Processing via the ProcessingController. While this itself does not do anything much audio related, it provides a flexible interface that communicates parameters and other data between the processing and the rest of the application.

ProcessingController acts as a middle-man, communicating between a ProcessingHandler (which does the actual processing) and external objects that have an interest.

Implementation

Activation

To begin receiving processing related events and data, you must first activate your MimiProcessingHandler. This will be your communication platform with MimiCore and allow you to react to any updates and provide the necessary processing data.

import io.mimi.sdk.core.MimiCore

class CustomAudioProcessor: ProcessingHandler {

    fun init() {
        // Activate the handler.
        MimiCore.processingController.activate(processingHandlerMock)
    }
}

Communication

Once activated, communication can begin between the handler and MimiCore. You will need to implement the ProcessingHandler interface functions.

Applying and loading processing parameters

ProcessingParameter is a sealed class that contains the individual data classes in charge of managing the different processing parameters.

When override fun <T : ProcessingParameter> setParameter(param: T) is called, the processing handler is expected to attempt to apply this value appropriately to the internal audio processing.

When override fun <T : ProcessingParameter> getParameter(param: Class<T>) is called, the processing handler is expected to return the required processing parameter value.

The following snippet showcases a basic implementation of the ProcessingHandler class in charge of applying the audio processing and returning the demanded ProcessingParameters by the MSDK when needed:

    private var processingListener: ProcessingHandler.Listener? = null

    protected val map = arrayOf(
            ProcessingParameter.CanEnable(true),
            ProcessingParameter.Preset(null),
            ProcessingParameter.Intensity.DEFAULT,
            ProcessingParameter.IsEnabled(false),
            ProcessingParameter.Fitting(3)
    ).map { it::class.java to it }.toMap().toMutableMap()

    override fun <T : ProcessingParameter> setParameter(param: T) {
        map[param::class.java] = param
    }

    override fun <T : ProcessingParameter> getParameter(param: Class<T>) = map[param] as T

    override fun setListener(listener: ProcessingHandler.Listener?) {
        processingListener = listener
    }

    open fun didInvalidate() =
            processingListener?.didInvalidate("ProcessingHandler invalidated.")

    open fun didInvalidateParam(param: KClass<ProcessingParameter>, reason: String?) =
            processingListener?.didInvalidate(param.java, reason)
            

But things are absolutely not always as easy and synchronous as pictured above, in that case these methods can also gracefully fail returning an error exception.

Continuous / discrete application of processing parameters

In some cases it is necessary to throttle the amount of parameter updates to a specific handler. This is especially the case if your processing handler has a high latency (i.e. using a network or bluetooth connection). You can configure the processing controller to deliver parameters to the processing handler only after input settles using these configuration variables:

MimiCore.processingController.config.parameterDeliveryMode = ParameterDeliveryModes.DISCRETE
MimiCore.processingController.config.discreteInputSettleTimeMs = 200

parameterDeliveryMode defines the delivery mode which defaults to ParameterDeliveryModes.CONTINUOUS which will result in applying the parameter immediately. ParameterDeliveryModes.DISCRETE will introduce the behavior described above.

discreteInputSettleTimeMs defines the minimum amount of time that the ProcessingController will wait between two consecutive calls to setParameter to actually delivering the parameter to the processing handler. Defaults to 200ms. This value will be obviously irrelevant unless the delivery mode is set to ParameterDeliveryModes.DISCRETE.