Other Verifications of Decomposition Accuracy
Validation using mathematically synthesized signals
This approach uses a signal synthesized from artificially generated MUAPs and firing instances that are designated as the “truth”. The synthesized signal is decomposed and the accuracy is evaluated by comparing the decomposed data with the truth data.
But, the use of a mathematically synthesized signal is a generic test used under artificial circumstances. It requires the presumption that the accuracy results obtained under artificial conditions provide a faithful representation of the decomposition accuracy of a real sEMG signal. This is a difficult point to establish when the mathematically synthesized signal does not contain some of the complexities found in real physiological signals. For example current mathematical models use fabricated MUAPs that are unrealistically smooth because they are generated by models where the depolarization is surrounded by a homogeneous environment as is evidenced by the volume conductor model proposed by Merletti et al (1999), Farina and Merletti (2001) and Lowery et al (2002). The artificial smooth shapes of the MUAPs do not capture and represent the richer irregular details of the shapes of real MUAPs nor do they consider the changes in MUAP shape that occur in normal physiological conditions. In addition, current mathematical models use fabricated motor unit firing instances that are either constructed artificially, as was done by Holobar and Zazula (2007) and Holobar et al (2012), or are derived from a generic model, such as that reported by Fuglevand et al (1993). These approaches fail to integrate known and documented physiological firing behavior such as synchronization and common drive. Yet, they have been used extensively by Holobar and Zazula (2004), Holobar et al (2009) and Holobar et al (2014) as the primary validation of the CKC decomposition algorithm. Because mathematically synthesized models do not directly assess the decomposition of a real sEMG signal the accuracy of individual extracted MUAPTs cannot be measured and remains unknown.
Validation using the two-source test
The test was developed by our own group more than 3 decades ago (Mambrito and De Luca, 1984), and has been used by Holobar et al (2009), Marateb et al (2011), Holobar et al (2011), and Farina, Merletti and Enoka (2014) to justify accurate performance of the CKC decomposition algorithm. The test evaluates the decomposition accuracy by comparing common MUAPTs extracted from two EMG signals recorded simultaneously from two sensors arranged in near proximity.
Although capable of estimating the accuracy of a select few MUAPTs, the two-source test falls far short of providing a comprehensive validation. Only a small fraction of MUAPTs common across two signals can be assessed leaving the accuracy of the remaining MUAPTs untested.
Validation based on predicted/expected results
Validation tests developed by McGill and Marateb (2011) and separately by Parsaei and Stashuk (2013) evaluate decomposition accuracy based on statistical expectations and/or assumptions of motor unit firing behavior.
This is a generic validation approach. It applies assumptions – such as stationary MUAP shapes and independent motor unit firing instances – that do not hold under normal physiological conditions (see detailed explanation in Section 2). Furthermore, the validation is limited to a small class of signals that “should not be too complex” according to McGill and Marateb (2011). But it is in complex portions of the sEMG signal where decomposition errors are most likely to occur. Because this test is unable to function in these regions, it is incapable of providing a useful assessment of the accuracy of the extracted MUAPTs.
Validation based on the action potential pulse-to-noise ratio
Recently, Holobar et al (2014) reported that the pulse-to-noise ratio – a measure of the MUAP signal to noise ratio – can be used to assess the accuracy of decomposition. They used the two-source test to measure the sensitivity (a measure of accuracy) of extracted MUAPTs and plotted the values as a function of the MUAP signal to noise ratio. They claimed a relationship between the two variables indicated the pulse-to-noise ratio was a reliable indicator of decomposition accuracy.
But, the MUAP signal to noise ratio does not measure actual accuracy because it ignores the incidence of identification and location errors that occur throughout the decomposition. Furthermore, the correlations between their measure of accuracy and the pulse-to-noise ratio had R2 values as low as 0.41. In other words, in some contractions, more than 50% of the variance in the accuracy data could not be explained by the pulse-to-noise ratio. The actual accuracy for extracting each MUAPT remains unknown using the pulse-to-noise ratio.