Lumi.cloud > Segmentation Performance
Segmentation Performance
The LumiNE US Augmedit MRI T1 plus contrast Segmentation Function (T1cSF) transforms T1-weighted MR images (MRI-T1) of the head into 3D models of the skin, brain, and ventricles. The LumiNE US MRI T1 segmentation function (T1cSF) can thereby only be used in case of a single intracranial contrast enhancing tumor, diagnosed by a neuroradiologist or a neurosurgeon, with a minimal volume of 2.0 cc (0.1 in3) and a minimal diameter in any direction of 15 mm (0.6 inch), and a maximum volume of 100cc (6.1 in3) and a maximal diameter in any direction of 75 mm (3.0 inch). We performed detailed independent standalone testing of the four binary segmentation models included in T1cSF (Brain, Skin, Tumor and Ventricle) on a ground truth set with representative data from the USA. This USA data was truthed by three board certified neurosurgeons with over 5-year post-fellowship experience.
​
The Dice Score (DSC) and 95% Hausdorff Distance (95% HD) were chosen as evaluation metrics. The Dice score measures the overlap between the predicted segmentation and the ground truth, providing a straightforward way to assess segmentation accuracy. The 95% Hausdorff distance measures the distance between the boundaries of the predicted and actual segmentations, focusing on the worst-case errors while mitigating the impact of outliers. Per structure in Table 1 and 2 the mean and median value from each metric is given.
Collective performance table (US and OUS aggregate)
Table 1. Segmentation performance: T1cSF combined USA and EU test set results in DICE and 95%HD Note that only Dice scores of brain and ventricles were normally distributed so all other metrics should be primarily evaluated using the median value.
USA-specific performance table
Table 2. Segmentation performance: T1cSF USA test set results in DICE and 95%HD Note that only Dice scores of brain and ventricles were normally distributed so all other metrics should be primarily evaluated using the median value.
False Negative and Positive Segmentations
The frequency and regions of common failures (false positive and false negative) of the T1cSF are outlined in Table 3.​​
T1cSF failure risk description
Table 3. T1cSF risk estimation per structure and known failure regions
Possible steps in case of false positive or negative segmentations
These steps may only be evaluated by qualified users:​​
-
Ignore: This is an option for false positives and negatives because they do not impact usability. Especially in the case of loose small positive islands (Figure 1, 2 and 3) this is preferable.
-
Shape Edit: Outline revision is an option in false positive of negative structures. The option is easier in round shapes then in complex shapes. Figure 4 gives an example of a suitable false negative case. The shape edit function is described in Shape Edit (Lumi.cloud) and Shape Edit (HL App).
-
Stop using LumiNE US: If no successful segmentation is possible for structures needed for surgical preparation. ​
Figure 1, 2 and 3 show most frequent examples of false positive segmentations.
Figure 4 shows an example of a false negative tumor segmentation, which can be improved with the Shape Edit feature
Figure 1: False positive tumor prediction in red around the right sided vertebral artery/C1 level. (yellow arrow). (this patient has a supratentorial glioblastoma which is not visible in these slides) These small islands do not hamper view and are best ignored.
Figure 2: False positive tumor prediction in the transversal sagittal sinus and torcular region. These small islands do not hamper view (this patient has a supratentorial glioblastoma which is not visible in these slides) and are best ignored.
Figure 3: Ventricle segmentation. Ground truth in green, prediction in red. Note the small false positive elements around the aqueduct and in the prepontine cisterns. (yellow arrow) Given these regions do not have a relation with the precuneal tumor ignoring is a valid option.
Figure 4: Tumor segmentation showing the ground truth in green and the prediction in red. The transversal, coronal and axial direction of the source Dicom are centered on the right parietal tumor. The contrast uptake of the tumor is low and therefore there is a significant false negative part. Editing the shape could be a good option in this patient.
