Wrist fracture detection using self-supervised learning methodology

Fracture detection approaches

Deep learning has gained significant attention in medical imaging due to its ability to extract complex image features automatically. Deep learning algorithms, particularly convolutional neural networks (CNNs), have revolutionized medical image analysis. These algorithms can automatically extract complex features from medical images making them suitable for fracture detection tasks.^[9]

Several approaches have been proposed for fracture detection using conventional methods. A computer-aided diagnosis system for detecting wrist fractures from radiographic images was developed achieving an accuracy of 86.4% using traditional machine learning techniques including support vector machines (SVMs) and decision trees.^[10]

A fractal-based method for rib fracture detection in CT scans was introduced by Yao et al. with the calculation of the fractal dimension of the fractured ribs resulting in a sensitivity of 93%.^[11] In 2016, a method based on a CNN for detecting fractures in wrist radiographs was proposed in a study achieving an accuracy of 92.5%.^[12] In 2020, a CNN was utilized for identifying hip fractures in radiographic images leading to an accuracy of 94.2% and a precision of 96.3%.^[13]

These approaches demonstrate the potential of supervised learning in fracture detection but highlight the reliance on labeled data, which may limit scalability and generalization. Furthermore, while effective, these methods often require handcrafted features and extensive manual labeling making them less scalable and potentially less accurate in complex cases.

These deep-learning approaches typically require large amounts of labeled data for training, which can be a limiting factor in the medical domain due to the need for expert annotations. Training supervised learning models requires expensive data labeling, and even then, they can struggle with challenges such as poor generalization, false relationships, and vulnerabilities to manipulation.^[14]

Self-supervised learning

Self-supervised learning has gained traction as an alternative approach to leverage the abundance of unlabeled data for training deep models. Self-supervised learning offers a solution to the labeled data scarcity problem. By leveraging the inherent structure or information within the unlabeled data, self-supervised learning algorithms can train deep learning models in a semi-supervised or unsupervised manner.^[15]

One popular self-supervised learning technique is contrastive learning, which learns representations by maximizing agreement between augmented views of the same image and minimizing agreement between views from different images.^[15] It aims to embed augmented versions of the same sample close to each other while pushing away embeddings from different samples. By formulating pretext tasks that generate supervisory signals from the data itself, self-supervised learning enables the models to learn meaningful representations without manual annotations.^[14] This approach could address the scarcity and cost of labeled fracture datasets.

Furthermore, integrating self-supervised learning with transfer learning techniques has demonstrated even more significant performance gains. Leveraging pre-trained models on large-scale datasets, the learned representations can transfer the learned representations to the fracture detection task with limited labeled data.^[16] This transfer-learning approach has shown improved fracture detection accuracy and efficiency, particularly in scenarios with limited annotated fracture images.

Recent studies have explored applying self-supervised learning techniques for fracture detection in medical imaging. Cho et al. proposed a self-supervised framework based on contrastive learning to detect fractures in radiographic images.^[17] Their results demonstrated improved fracture detection performance compared to traditional methods. Similarly, another study utilized a self-supervised approach based on rotation prediction to detect fractures in CT scans achieving competitive results.^[18] Doersch et al. introduced the concept of pretext tasks, such as predicting image rotations, to learn representations from natural images.^[19] Inspired by this, recent studies have applied self-supervised learning to medical images for various tasks.

A self-supervised approach for skin lesion classification using dermoscopic images was introduced achieving competitive results compared to supervised methods.^[20] Furthermore, self-supervised learning study for lung nodule detection in CT scans leads to improved performance over traditional supervised models.^[21] These studies highlight the potential of self-supervised learning to capture clinically relevant features without extensive manual annotation.

Dataset

In this study, we used the pediatric trauma wrist radiograph dataset from Kaggle and GRAZPERDWRI-DX dataset to train a machine-learning model to detect wrist fractures. The Kaggle dataset contains 20,327 annotated images, which were divided into a training set of 16,327 images and a testing set of 4000 images.

The GRAZPEDWRI-DX dataset is a large and diverse collection of pediatric wrist radiographic images that are publicly available for free download. The dataset consists of 20,327 images from 6091 patients who were treated at the University Hospital Graz between 2008 and 2018. The images were annotated by pediatric radiologists, and the dataset is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. GRAZPEDWRI-DX is used for backbone training, and the Kaggle dataset is used for model training.

Our method harnesses the power of self-supervised learning algorithms, precisely the contrastive learning approach. Simplified contrastive learning (SimCLR) is a way to teach a computer to understand images without showing it any labeled examples. It does this by training the computer to find image similarities and differences. The learned representations can then be used for other tasks, such as image classification, without having to train the computer from scratch.

SimCLR consists of two main components:

1. Base encoder network: This network extracts features from the input images.

2. Contrastive loss function: This function trains the model by maximizing the similarity between the features of augmented versions of the same image and minimizing the similarity between the features of different images.

Base encoder networks such as ResNet-18 are typically pre-trained CNNs [Figures 1 and 2].

Data pre-processing

As a part of the pre-processing phase, uniformity is ensured by resizing all images to dimensions of 512 × 512. Normalization is employed to standardize pixel values across the dataset, which aids in enhancing model performance during training and evaluation.

Close

Figure 1:

Self-supervised methodology – The unlabeled images are trained using ResNet-18 using a projection head to develop a 128d feature vector.

Export to PPT

Data augmentation and pair generation

To enrich the diversity of our dataset, we employ data augmentation techniques, which encompass a range of random transformations such as rotations, translations, flips, and brightness adjustments. These augmented versions of the images serve as inputs to the subsequent stages of our approach.

For training, we generate positive and negative image pairs. Creating this paired dataset constitutes a pivotal step in our self-supervised learning framework. Positive pairs consist of augmented images originating from the same underlying image, capturing its inherent features under diverse transformations. Negative pairs are formed by combining augmented images from different original images. This dataset of image pairs is subsequently utilized to train our fracture detection model.

Architecture

We employ a ResNet-18 deep CNN as the backbone for feature extraction [Figure 3]. The SimCLR model consists of a ResNet-18 feature backbone (with the classification head removed) and a linear projection head. After training the contrastive model, we would discard the projection head and add an appropriate classification head for the downstream task. The training procedure relies heavily on producing augmented pairs for each input data point. Those pairs, called positive samples, are the basis for the contrastive learning objective, where we optimize our model to produce representations that achieve high similarity between the positive pairs and low similarity with all the other augmented data points (negatives).

SimCLR projection head

We incorporate the SimCLR algorithm’s projection head mechanism to enhance feature extraction [Figure 4]. The projection head is integrated into the backbone neural network, ResNet-18 architecture. This augmented backbone learns to extract pertinent features from radiographic images, effectively representing both fractured and non-fractured images. The Cosine similarity function is calculated as

sim: Cosine similarity, u and v: Vectors whose similarity to be found, U and V: Matrices that hold the vectors u and v as rows, T: Transpose operation.

The application of SimCLR involves augmenting the dataset of fractured reports with specific transformations. Pairs of augmented data points are then created, and the SimCLR algorithm is employed to learn and maximize the similarity of pairs from the same class while minimizing the similarity of pairs from different classes. This learning process ensures that representations are robust and discriminative, aligning well with fracture detection requirements.

Close

Figure 2:

Supervised fine tuning – The labeled images use the learned representations to detect fracture and non-fracture images.

Export to PPT

Close

Figure 3:

Architecture of self-supervised learning – Data collection, data augmentation, and pair generation followed by contrastive learning for feature extraction to classify fracture and not-fracture images.

Export to PPT

Close

Figure 4:

Simplified contrastive learning – Project head is used to augment images. The transformed images using base encoder ResNet-18 to develop 128d feature vectors.

Export to PPT

The learned representations from the SimCLR algorithm serve as the basis for training a fracture detection classifier. The classifier’s training is executed on a separate held-out test set containing fractured images.

Loss function

SimCLR uses a contrastive loss function to measure the similarity between two views of an image. The goal is to maximize the similarity between positive pairs (two views of the same image) and minimize the similarity between negative pairs (two views of different images). The model’s objective is achieved through a contrastive loss function – NT-Xent Loss, which penalizes errors in predicting similarities. The NT-Xent loss function for a positive pair is calculated as

l: Loss value, i and j: Indexes referring to a specific positive pair of examples within a batch, sim: Cosine similarity function, exp: Exponential function, z: Vectors produced by the neural network for each image in the batch, N: Total number of images in a batch, T: Temperature hyperparameter, k: Index that iterates over all the examples in a batch (of size 2N) excluding the current positive example.

Model training

The model is trained using a stochastic gradient descent (SGD) optimizer, which is an iterative algorithm that updates the model parameters by moving in the direction of the negative gradient of the loss function. During training, the weights of the network are adjusted based on the contrastive loss. This loss function measures both the similarity between positive pairs (representing related data points) and the dissimilarity between negative pairs (representing unrelated data points). By minimizing the contrastive loss, the network learns to encode similar data points close together in the feature space, while pushing dissimilar data points apart. This process ultimately helps the network learn meaningful representations of the data.

Accuracy charts

Accuracy charts are used to visualize the performance of a model on a given task. The charts are explained below [Figures 5 and 6].

Area under the receiver operating characteristics (AUROC)

The classification head was trained for 40 epochs. The entire model was then trained from epoch 40 to 50 results.

Fine tuning

After training a neural network on a supervised task, the base network and projection head is fine-tuned using labeled data for a specific task. This helps to adapt the learned representations to the new task. The network is fine-tuned on labeled data to learn the specific features of fractures that are important for detection.

Close

Figure 5:

Plots of accuracy versus the number of epochs obtained while training a linear classifier on labeled training data.

Export to PPT

Close

Figure 6:

Plots of area under the receiver operating characteristics (AUROC) versus the number of epochs obtained while training a linear classifier on labeled training data.

Export to PPT

Close

Figure 7:

Confusion matrix for bone fracture detection.

Export to PPT

Evaluation metrics

The evaluation measures of the models are calculated using accuracy, precision, recall, F1-score, AUC, and ROC [Figure 7]. These metrics are evaluated based on the confusion matrix of each model.

These metrics comprehensively assess the model’s performance across key fracture detection accuracy and robustness aspects. The confusion matrix is a tool that provides a detailed overview of the performance of a classifier model. Accuracy can be misleading, especially for imbalanced datasets. The figure illustrates the different types of predictions (true positive, true negative, false positive, and false negative) in the context of bone fracture detection.

Accuracy is the percentage of correct predictions made by a model, calculated by dividing the number of correct predictions by the total number of predictions.

TP: True positive, TN: True negative, FP: False positive, FN: False negative.

Recall, also known as sensitivity, is the percentage of positive cases correctly identified by a model, calculated by dividing the number of true positives by the total number of positive cases.

The F1 score is a metric that combines precision and recall into a single measure of model performance. It is calculated by taking the harmonic mean of precision and recall, which gives equal weight to both metrics.

F1 - Score: Metric used to evaluate performance of a model.

Hyper parameters

• Learning rate: 0.001–0.01

• Optimizer: Stochastic gradient descent

• Loss function: NT-Xent loss

• Similarity: Cosine similarity function

• Number of epochs: 50

• Batch size: 32

• Model architecture: ResNet-18

In our experiments, the proposed algorithm consistently achieves noteworthy results: an accuracy of 94.10%, a specificity of 93.21%, and a recall of 95.03%. This surpasses the performance benchmarks of conventional fracture detection methods.

Observations

• Self-supervised learning is a promising approach for fracture detection. Self-supervised learning methods have shown promise in detecting fractures in various medical images, including radiographs.

• Self-supervised learning methods are more scalable and cost-effective than traditional methods. Self-supervised learning methods do not require labeled data, which can be expensive and time-consuming to collect.

• Self-supervised learning methods can be used to automate fracture detection. This could free up radiologists to focus on more complex cases and help to reduce the time it takes to diagnose fractures.

• Self-supervised learning methods can be used to improve the accuracy of fracture detection.

• Self-supervised learning methods can learn representations of medical images that are more robust to variations in pose and contrast than traditional methods, leading to earlier diagnosis and treatment of fractures and improving patient outcomes.

However, challenges such as dataset variability, generalizability to different imaging modalities, and interpretability of learned features still need to be addressed. These studies suggest that self-supervised learning is a promising approach for fracture detection. However, more research is needed to improve the accuracy and robustness of self-supervised learning methods for fracture detection.

Limitations

• Limited data understanding: Self-supervised models learn from unlabeled data, which may lack the specific context or medical expertise needed to identify and classify fractures accurately. This can lead to misinterpretations and missed diagnoses.

• False positive/negative rates: Without the guidance of labeled data, self-supervised models can struggle to differentiate between fractures, leading to both false positives (misdiagnosing fractures) and false negatives (missing actual fractures).

• Generalization issues: Models trained on unlabeled data may not generalize well to unseen scenarios or variations in fracture presentation.

The research provides recommendations for future improvements and potential research directions.

Future direction

Further research and exploration of self-supervised learning methods hold great potential for advancing fracture detection systems, enhancing diagnostic accuracy, and improving patient outcomes. Future research should focus on investigating different pretext tasks and architectural designs that best suit the task of fracture detection. In addition, transfer learning techniques that leverage self-supervised features for downstream tasks could further enhance the clinical applicability of self-supervised learning in fracture detection.