| Title: | Neural Networks for Medical Image Processing |
|---|---|
| Description: | Deep learning tools for medical image processing, interfacing with the 'ANTsR' package and Advanced Normalization Tools ('ANTs'). |
| Authors: | Nicholas J Tustison [aut, cre] (ORCID: <https://orcid.org/0000-0001-9418-5103>), Brian B Avants [aut] (ORCID: <https://orcid.org/0000-0002-4212-3362>), John Muschelli [ctb] (ORCID: <https://orcid.org/0000-0001-6469-1750>) |
| Maintainer: | Nicholas J Tustison <[email protected]> |
| License: | Apache License (>=2.0) |
| Version: | 1.1 |
| Built: | 2026-10-30 07:32:52 UTC |
| Source: | https://github.com/ANTsX/ANTsRNet |
Anchor box layer for SSD architecture (2-D).
a 5-D tensor with shape
In the last dimension, the first 4 values correspond to the
2-D coordinates of the bounding boxes and the other 4 are the variances.
anchorBoxGenerator <- AnchorBoxLayer2D$new( imageSize,
scale, nextScale, aspectRatios = c( '1:1', '2:1', '1:2' ),
variances = 1.0 )
anchorBoxGenerator$call( x, mask = NULL )
anchorBoxGenerator$compute_output_shape( input_shape )
A process object.
size of the input image.
scale of each box (in pixels).
next scale of each box (in pixels).
vector describing the geometries of the anchor boxes for this layer.
a list of 4 floats > 0 with scaling factors for the encoded predicted box coordinates. A variance value of 1.0 would apply no scaling at all to the predictions, while values in (0,1) upscale the encoded predictions and values greater than 1.0 downscale the encoded predictions. Defaults to 1.0.
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
x = AnchorBoxLayer2D$new(imageSize = c(20, 20), scale = 2, nextScale = 2) x$build()x = AnchorBoxLayer2D$new(imageSize = c(20, 20), scale = 2, nextScale = 2) x$build()
Anchor box layer for SSD architecture (3-D).
a 6-D tensor with shape
In the last dimension, the first 6 values correspond to the
3-D coordinates of the bounding boxes and the other 6 are the variances.
anchorBoxGenerator <- AnchorBoxLayer3D$new( imageSize,
scale, nextScale, aspectRatios = c( '1:1:1', '2:1:1', '1:2:1', '1:1:2' ),
variances = 1.0 )
anchorBoxGenerator$call( x, mask = NULL )
anchorBoxGenerator$compute_output_shape( input_shape )
A process object.
size of the input image.
scale of each box (in pixels).
next scale of each box (in pixels).
vector describing the geometries of the anchor boxes for this layer.
a list of 6 floats > 0 with scaling factors for the encoded predicted box coordinates. A variance value of 1.0 would apply no scaling at all to the predictions, while values in (0,1) upscale the encoded predictions and values greater than 1.0 downscale the encoded predictions. Defaults to 1.0.
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
x = AnchorBoxLayer3D$new(imageSize = c(20, 20, 20), scale = 2, nextScale = 2) x$build()x = AnchorBoxLayer3D$new(imageSize = c(20, 20, 20), scale = 2, nextScale = 2) x$build()
Helper function for applying a pretrained deep back projection model. Apply a patch-wise trained network to perform super-resolution. Can be applied to variable sized inputs. Warning: This function may be better used on CPU unless the GPU can accommodate the full image size. Warning 2: The global intensity range (min to max) of the output will match the input where the range is taken over all channels.
applyDeepBackProjectionModel( image, model, targetRange = c(-127.5, 127.5), batchSize = 32, regressionOrder = NA, verbose = FALSE )applyDeepBackProjectionModel( image, model, targetRange = c(-127.5, 127.5), batchSize = 32, regressionOrder = NA, verbose = FALSE )
image |
input image. |
model |
pretrained model or filename (cf |
targetRange |
a vector defining the |
batchSize |
batch size used for the prediction call. |
regressionOrder |
if specified, then apply the function
|
verbose |
If |
super-resolution image upscaled to resolution specified by the network.
Avants BB
## Not run: image <- applyDeepBackProjectionModel( ri( 1 ), getPretrainedNetwork( "dbpn4x" ) ) ## End(Not run)## Not run: image <- applyDeepBackProjectionModel( ri( 1 ), getPretrainedNetwork( "dbpn4x" ) ) ## End(Not run)
Apply pretrained super-resolution network
applySuperResolutionModel( image, model, targetRange, batch_size = 32, linmatchOrder, mask, verbose = FALSE )applySuperResolutionModel( image, model, targetRange, batch_size = 32, linmatchOrder, mask, verbose = FALSE )
image |
input image |
model |
model object or filename see |
targetRange |
a vector defining min max of each the input image, eg -127.5, 127.5. Output images will be scaled back to original intensity. This range should match the mapping used in the training of the network. |
batch_size |
for prediction call |
linmatchOrder |
if not missing, then apply |
mask |
restrict intensity rescaling parameters within the mask |
verbose |
If |
Apply a patch-wise trained network to perform super-resolution. Can be applied to variable sized inputs. Warning: This function may be better used on CPU unless the GPU can accommodate the full image size. Warning 2: The global intensity range (min to max) of the output will match the input where the range is taken over all channels.
image upscaled to resolution provided by network
Avants BB
## Not run: library(ANTsRCore) library(keras) orig_img = antsImageRead( getANTsRData( "r16" ) ) # input needs to be 48x48 img = resampleImage(orig_img, resampleParams = rep(256/48, 2)) model = getPretrainedNetwork( "dbpn4x" ) simg <- applySuperResolutionModel(img, model = model) plot(orig_img) plot(img) plot(simg) ## End(Not run)## Not run: library(ANTsRCore) library(keras) orig_img = antsImageRead( getANTsRData( "r16" ) ) # input needs to be 48x48 img = resampleImage(orig_img, resampleParams = rep(256/48, 2)) model = getPretrainedNetwork( "dbpn4x" ) simg <- applySuperResolutionModel(img, model = model) plot(orig_img) plot(img) plot(simg) ## End(Not run)
Apply pretrained super-resolution network by stitching together patches.
applySuperResolutionModelPatch( image, model, targetRange, lowResolutionPatchSize = 128, strideLength = 16, batch_size = 32, mask, verbose = FALSE )applySuperResolutionModelPatch( image, model, targetRange, lowResolutionPatchSize = 128, strideLength = 16, batch_size = 32, mask, verbose = FALSE )
image |
input image |
model |
model object or filename see |
targetRange |
a vector defining min max of each the input image, eg -127.5, 127.5. Output images will be scaled back to original intensity. This range should match the mapping used in the training of the network. |
lowResolutionPatchSize |
size of patches to upsample |
strideLength |
voxel/pixel steps between patches |
batch_size |
for prediction call |
mask |
restrict intensity rescaling parameters within the mask |
verbose |
If |
Apply a patch-wise trained network to perform super-resolution. Can be applied to variable sized inputs. Warning: This function may be better used on CPU unless the GPU can accommodate the full patch size. Warning 2: The global intensity range (min to max) of the output will match the input where the range is taken over all channels.
image upscaled to resolution provided by network
Avants BB
## Not run: library(ANTsRCore) library( keras ) orig_img = antsImageRead( getANTsRData( "r16" ) ) # input needs to be 48x48 model = createDeepBackProjectionNetworkModel2D( list(NULL,NULL, 1) ) img = resampleImage(orig_img, resampleParams = rep(256/48, 2)) simg <- applySuperResolutionModelPatch( img, model = model, lowResolutionPatchSize = 8, strideLength = 2) simgm <- applySuperResolutionModelPatch( img, mask = getMask( img ), model = model, lowResolutionPatchSize = 8, strideLength = 2) plot( orig_img ) plot( img ) plot( simg ) plot( simgm ) ## End(Not run)## Not run: library(ANTsRCore) library( keras ) orig_img = antsImageRead( getANTsRData( "r16" ) ) # input needs to be 48x48 model = createDeepBackProjectionNetworkModel2D( list(NULL,NULL, 1) ) img = resampleImage(orig_img, resampleParams = rep(256/48, 2)) simg <- applySuperResolutionModelPatch( img, model = model, lowResolutionPatchSize = 8, strideLength = 2) simgm <- applySuperResolutionModelPatch( img, mask = getMask( img ), model = model, lowResolutionPatchSize = 8, strideLength = 2) plot( orig_img ) plot( img ) plot( simg ) plot( simgm ) ## End(Not run)
Helper function for applying a pretrained super resolution model. Apply a patch-wise trained network to perform super-resolution. Can be applied to variable sized inputs. Warning: This function may be better used on CPU unless the GPU can accommodate the full image size. Warning 2: The global intensity range (min to max) of the output will match the input where the range is taken over all channels.
applySuperResolutionModelToImage( image, model, targetRange = c(-127.5, 127.5), batchSize = 32, regressionOrder = NA, verbose = FALSE )applySuperResolutionModelToImage( image, model, targetRange = c(-127.5, 127.5), batchSize = 32, regressionOrder = NA, verbose = FALSE )
image |
input image. |
model |
pretrained model or filename (cf |
targetRange |
a vector defining the |
batchSize |
batch size used for the prediction call. |
regressionOrder |
if specified, then apply the function
|
verbose |
If |
super-resolution image upscaled to resolution specified by the network.
Avants BB
## Not run: image <- applySuperResolutionModelToImage( ri( 1 ), getPretrainedNetwork( "dbpn4x" ) ) ## End(Not run)## Not run: image <- applySuperResolutionModelToImage( ri( 1 ), getPretrainedNetwork( "dbpn4x" ) ) ## End(Not run)
Perform arterial lesion segmentation using U-net.
arterialLesionSegmentation(image, verbose = FALSE)arterialLesionSegmentation(image, verbose = FALSE)
image |
input histology image. |
verbose |
print progress. |
segmentation and probability images
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "image.nii.gz" ) output <- arterialLesionSegmentation( image ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "image.nii.gz" ) output <- arterialLesionSegmentation( image ) ## End(Not run)
Attention augmentation layer (2-D)
output of tensor shape batchSize, height, width, depthOfQueries.
number of filters for queries.
number of filters for values.
number of attention heads to use. It is required that depthOfQueries/numberOfHeads > 0.
whether or not to use relative encodings.
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
Attention layer (2-D)
output of tensor shape.
number of channels.
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
Attention layer (3-D)
output of tensor shape.
number of channels.
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
The function will generate a deformable transformation (via exponential map) from a basis of deformations.
basisWarp( deformationBasis, betaParameters, numberOfCompositions = 2, spatialSmoothing = 0 )basisWarp( deformationBasis, betaParameters, numberOfCompositions = 2, spatialSmoothing = 0 )
deformationBasis |
list containing deformationBasis set |
betaParameters |
vector containing deformationBasis set parameters |
numberOfCompositions |
integer greater than or equal to one |
spatialSmoothing |
spatial smoothing for generated deformation |
list of fields
Avants BB
randomImageTransformParametersBatchGenerator
## Not run: library( ANTsR ) i1 = ri( 1 ) %>% resampleImage( 4 ) i2 = ri( 2 ) %>% resampleImage( 4 ) reg = antsRegistration( i1, i2, 'SyN' ) w = composeTransformsToField( i1, reg$fwd ) bw = basisWarp( list( w, w ), c( 0.25, 0.25 ), 2, 0 ) bwApp = applyAntsrTransformToImage( bw, i2, i1 ) bw = basisWarp( list( w, w ), c( 0.25, 0.25 )*(-1.0), 2, 0 ) # inverse bwApp = applyAntsrTransformToImage( bw, i1, i2 ) ## End(Not run)## Not run: library( ANTsR ) i1 = ri( 1 ) %>% resampleImage( 4 ) i2 = ri( 2 ) %>% resampleImage( 4 ) reg = antsRegistration( i1, i2, 'SyN' ) w = composeTransformsToField( i1, reg$fwd ) bw = basisWarp( list( w, w ), c( 0.25, 0.25 ), 2, 0 ) bwApp = applyAntsrTransformToImage( bw, i2, i1 ) bw = basisWarp( list( w, w ), c( 0.25, 0.25 )*(-1.0), 2, 0 ) # inverse bwApp = applyAntsrTransformToImage( bw, i1, i2 ) ## End(Not run)
Note: Assumption is that y_true is not a one-hot representation of the segmentation batch. For use with e.g., sigmoid activation.
binary_dice_coefficient(y_true, y_pred, smoothingFactor = 0)binary_dice_coefficient(y_true, y_pred, smoothingFactor = 0)
y_true |
True labels (Tensor) |
y_pred |
Predictions (Tensor of the same shape as |
smoothingFactor |
parameter for smoothing the metric. |
Dice value (negative)
Tustison NJ
library( ANTsR ) library( ANTsRNet ) library( keras ) model <- createUnetModel2D( c( 64, 64, 1 ) ) dice_loss <- binary_dice_coefficient( smoothingFactor = 0.1 ) model %>% compile( loss = dice_loss, optimizer = optimizer_adam( lr = 0.0001 ) ) rm(model); gc()library( ANTsR ) library( ANTsRNet ) library( keras ) model <- createUnetModel2D( c( 64, 64, 1 ) ) dice_loss <- binary_dice_coefficient( smoothingFactor = 0.1 ) model %>% compile( loss = dice_loss, optimizer = optimizer_adam( lr = 0.0001 ) ) rm(model); gc()
https://pubmed.ncbi.nlm.nih.gov/33080507/
binary_surface_loss(y_true, y_pred, dimensionality = 3L)binary_surface_loss(y_true, y_pred, dimensionality = 3L)
y_true |
True labels (Tensor) |
y_pred |
Predictions (Tensor of the same shape as |
ported from this implementation:
https://github.com/LIVIAETS/boundary-loss/blob/master/keras_loss.py
Note: Assumption is that y_true is a one-hot representation of the segmentation batch. The background (label 0) should be included but is not used in the calculation.
function value
Tustison NJ
library( ANTsRNet ) library( keras ) model <- createUnetModel2D( c( 64, 64, 1 ), numberOfOutputs = 2 ) model %>% compile( loss = binary_surface_loss, optimizer = optimizer_adam( lr = 0.0001 ), metrics = "accuracy" ) ######################################## # # Run in isolation # library( ANTsR ) r16 <- antsImageRead( getANTsRData( "r16" ) ) r16seg <- kmeansSegmentation( r16, 3 )$segmentation r16array <- array( data = as.array( r16seg ), dim = c( 1, dim( r16seg ) ) ) r16tensor <- tensorflow::tf$convert_to_tensor( encodeUnet( r16array, c( 0, 1, 2, 3 ) ) ) r64 <- antsImageRead( getANTsRData( "r64" ) ) r64seg <- kmeansSegmentation( r64, 3 )$segmentation r64array <- array( data = as.array( r64seg ), dim = c( 1, dim( r64seg ) ) ) r64tensor <- tensorflow::tf$convert_to_tensor( encodeUnet( r64array, c( 0, 1, 2, 3 ) ) ) surface_loss <- binary_surface_loss( r16tensor, r64tensor, dimensionality = 2L ) loss_value <- surface_loss( r16tensor, r64tensor )$numpy()library( ANTsRNet ) library( keras ) model <- createUnetModel2D( c( 64, 64, 1 ), numberOfOutputs = 2 ) model %>% compile( loss = binary_surface_loss, optimizer = optimizer_adam( lr = 0.0001 ), metrics = "accuracy" ) ######################################## # # Run in isolation # library( ANTsR ) r16 <- antsImageRead( getANTsRData( "r16" ) ) r16seg <- kmeansSegmentation( r16, 3 )$segmentation r16array <- array( data = as.array( r16seg ), dim = c( 1, dim( r16seg ) ) ) r16tensor <- tensorflow::tf$convert_to_tensor( encodeUnet( r16array, c( 0, 1, 2, 3 ) ) ) r64 <- antsImageRead( getANTsRData( "r64" ) ) r64seg <- kmeansSegmentation( r64, 3 )$segmentation r64array <- array( data = as.array( r64seg ), dim = c( 1, dim( r64seg ) ) ) r64tensor <- tensorflow::tf$convert_to_tensor( encodeUnet( r64array, c( 0, 1, 2, 3 ) ) ) surface_loss <- binary_surface_loss( r16tensor, r64tensor, dimensionality = 2L ) loss_value <- surface_loss( r16tensor, r64tensor )$numpy()
Estimate BrainAge from a T1-weighted MR image using the DeepBrainNet architecture and weights described here:
brainAge( image, doPreprocessing = TRUE, numberOfSimulations = 0, sdAffine = 0.01, verbose = TRUE )brainAge( image, doPreprocessing = TRUE, numberOfSimulations = 0, sdAffine = 0.01, verbose = TRUE )
image |
input 3-D T1-weighted brain image. |
doPreprocessing |
boolean dictating whether prescribed preprocessing is performed (brain extraction, bias correction, normalization to template). |
numberOfSimulations |
number of random affine perturbations to transform the input. |
sdAffine |
define the standard deviation of the affine transformation parameter for the simulations. |
verbose |
print progress. |
https://github.com/vishnubashyam/DeepBrainNet
and described in the following article:
https://pubmed.ncbi.nlm.nih.gov/32591831/
Preprocessing on the training data consisted of:
n4 bias correction,
brain extraction, and
affine registration to MNI.
The input T1 should undergo the same steps. If the input T1 is the raw
T1, these steps can be performed by the internal preprocessing, i.e. set
doPreprocessing = TRUE
predicted age and binned confidence values
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1w_image.nii.gz" ) estimatedBrainAge <- brainAge( image ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1w_image.nii.gz" ) estimatedBrainAge <- brainAge( image ) ## End(Not run)
Perform T1, FA, or bold brain extraction using a U-net architecture training data. "NoBrainer" is also possible where brain extraction uses U-net and FreeSurfer training data ported from the
brainExtraction( image, modality = c("t1", "t1.v0", "t1.v1", "t1nobrainer", "t1combined", "t1threetissue", "t1hemi", "t1lobes", "t2", "t2.v0", "t2star", "flair", "flair.v0", "bold", "bold.v0", "fa", "fa.v0", "mra", "t1t2infant", "t1infant", "t2infant"), verbose = FALSE )brainExtraction( image, modality = c("t1", "t1.v0", "t1.v1", "t1nobrainer", "t1combined", "t1threetissue", "t1hemi", "t1lobes", "t2", "t2.v0", "t2star", "flair", "flair.v0", "bold", "bold.v0", "fa", "fa.v0", "mra", "t1t2infant", "t1infant", "t2infant"), verbose = FALSE )
image |
input 3-D brain image (or list of images for multi-modal scenarios). |
modality |
image type. Options include:
|
verbose |
print progress. |
https://github.com/neuronets/nobrainer-models
brain probability mask (ANTsR image)
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1w_image.nii.gz" ) probabilityMask <- brainExtraction( image, modality = "t1" ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1w_image.nii.gz" ) probabilityMask <- brainExtraction( image, modality = "t1" ) ## End(Not run)
Perform MRA-TOF vessel segmentation. Training data taken from the (https://data.kitware.com/#item/58a372e48d777f0721a64dc9).
brainMraVesselSegmentation( mra, mask = NULL, predictionBatchSize = 16, patchStrideLength = 32, verbose = FALSE )brainMraVesselSegmentation( mra, mask = NULL, predictionBatchSize = 16, patchStrideLength = 32, verbose = FALSE )
mra |
input mra image. |
mask |
input binary mask which defines the patch extraction. If not supplied, one is estimated. |
predictionBatchSize |
Control memory usage for prediction. More consequential for GPU-usage. |
patchStrideLength |
3-D vector or int. Dictates the stride length for accumulating predicting patches. |
verbose |
print progress. |
Probability image.
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) ## End(Not run)
Perform brain tumor probabilistic segmentation given pre-aligned FLAIR, T1, T1 contrast, and T2 images. Note that the underlying model is 3-D and requires images to be of > 64 voxels in each dimension.
brainTumorSegmentation( flair, t1, t1Contrast, t2, predictionBatchSize = 16, patchStrideLength = 32, doPreprocessing = TRUE, verbose = FALSE )brainTumorSegmentation( flair, t1, t1Contrast, t2, predictionBatchSize = 16, patchStrideLength = 32, doPreprocessing = TRUE, verbose = FALSE )
flair |
input 3-D FLAIR brain image (not skull-stripped). |
t1 |
input 3-D T1-weighted brain image (not skull-stripped). |
t1Contrast |
input 3-D T1-weighted contrast brain image (not skull-stripped). |
t2 |
input 3-D T2-weighted brain image (not skull-stripped). |
predictionBatchSize |
Control memory usage for prediction. More consequential for GPU-usage. |
patchStrideLength |
3-D vector or int. Dictates the stride length for accumulating predicting patches. |
doPreprocessing |
perform n4 bias correction, intensity truncation, brain extraction. |
verbose |
print progress. |
Brain tumor segmentation probability images (4 tumor tissue types).
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) t1 <- antsImageRead( "t1.nii.gz" ) flair <- antsImageRead( "flair.nii.gz" ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) t1 <- antsImageRead( "t1.nii.gz" ) flair <- antsImageRead( "flair.nii.gz" ) ## End(Not run)
The negative of the categorical focal loss discussed in this paper:
categorical_focal_gain(y_true, y_pred, gamma = 2, alpha = 0.25)categorical_focal_gain(y_true, y_pred, gamma = 2, alpha = 0.25)
y_true |
True labels (Tensor) |
y_pred |
Predictions (Tensor of the same shape as |
gamma |
focusing parameter for modulating factor (1-p). Default = 2.0. |
alpha |
weighing factor in balanced cross entropy. Default = 0.25. |
https://arxiv.org/pdf/1708.02002.pdf
and ported from this implementation:
https://github.com/umbertogriffo/focal-loss-keras/blob/master/losses.py
Used to handle imbalanced classes.
function value
Tustison NJ
library( ANTsRNet ) library( keras ) model <- createUnetModel2D( c( 64, 64, 1 ) ) metric_categorical_focal_gain <- custom_metric( "categorical_focal_gain", categorical_focal_gain( alpha = 0.25, gamma = 2.0 ) ) model %>% compile( loss = categorical_focal_loss( alpha = 0.25, gamma = 2.0 ), optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( metric_categorical_focal_gain ) )library( ANTsRNet ) library( keras ) model <- createUnetModel2D( c( 64, 64, 1 ) ) metric_categorical_focal_gain <- custom_metric( "categorical_focal_gain", categorical_focal_gain( alpha = 0.25, gamma = 2.0 ) ) model %>% compile( loss = categorical_focal_loss( alpha = 0.25, gamma = 2.0 ), optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( metric_categorical_focal_gain ) )
The categorical focal loss discussed in this paper:
categorical_focal_loss(y_true, y_pred, gamma = 2, alpha = 0.25)categorical_focal_loss(y_true, y_pred, gamma = 2, alpha = 0.25)
y_true |
True labels (Tensor) |
y_pred |
Predictions (Tensor of the same shape as |
gamma |
focusing parameter for modulating factor (1-p). Default = 2.0. |
alpha |
weighing factor in balanced cross entropy. Default = 0.25. |
https://arxiv.org/pdf/1708.02002.pdf
and ported from this implementation:
https://github.com/umbertogriffo/focal-loss-keras/blob/master/losses.py
Used to handle imbalanced classes .
function value
Tustison NJ
library( ANTsRNet ) library( keras ) model <- createUnetModel2D( c( 64, 64, 1 ) ) metric_categorical_focal_gain <- custom_metric( "categorical_focal_gain", categorical_focal_gain( alpha = 0.25, gamma = 2.0 ) ) model %>% compile( loss = categorical_focal_loss( alpha = 0.25, gamma = 2.0 ), optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( metric_categorical_focal_gain ) )library( ANTsRNet ) library( keras ) model <- createUnetModel2D( c( 64, 64, 1 ) ) metric_categorical_focal_gain <- custom_metric( "categorical_focal_gain", categorical_focal_gain( alpha = 0.25, gamma = 2.0 ) ) model %>% compile( loss = categorical_focal_loss( alpha = 0.25, gamma = 2.0 ), optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( metric_categorical_focal_gain ) )
Perform cerebellum segmentation using a modification of the set of MaGET cerebellum atlases — https://www.cobralab.ca/cerebellum-lobules.
cerebellumMorphology( t1, cerebellumMask = NULL, computeThicknessImage = FALSE, doPreprocessing = TRUE, verbose = FALSE )cerebellumMorphology( t1, cerebellumMask = NULL, computeThicknessImage = FALSE, doPreprocessing = TRUE, verbose = FALSE )
t1 |
raw or preprocessed 3-D T1-weighted whole head image. |
cerebellumMask |
Option for initialization. If not specified, the cerebellum ROI is determined using ANTsXNet brain_extraction followed by registration to a template. |
computeThicknessImage |
Compute KellyKapowski thickness image of the gray matter. |
doPreprocessing |
Perform N4 bias correction and spatiall normalize to template space. |
verbose |
print progress. |
https://www.nature.com/articles/s41598-024-59440-6
The tissue labeling is as follows:
Label 1 :CSF
Label 2 :Gray matter
Label 3 :White matter
The parcellation labeling is as follows:
Label 1 :L_I_II
Label 2 :L_III
Label 3 :L_IV
Label 4 :L_V
Label 5 :L_VI
Label 6 :L_Crus_I
Label 7 :L_Crus_II
Label 8 :L_VIIB
Label 9 :L_VIIIA
Label 10 :L_VIIIB
Label 11 :L_IX
Label 12 :L_X
Label 101:R_I_II
Label 102:R_III
Label 103:R_IV
Label 104:R_V
Label 105:R_VI
Label 106:R_Crus_I
Label 107:R_Crus_II
Label 108:R_VIIB
Label 109:R_VIIIA
Label 110:R_VIIIB
Label 111:R_IX
Label 112:R_X
List consisting of the multiple segmentation images and probability images for each label and foreground. Optional thickness image.
Tustison NJ, Tustison MG
## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1.nii.gz" ) cereb <- cerebellumMorphology( image ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1.nii.gz" ) cereb <- cerebellumMorphology( image ) ## End(Not run)
Check the correctness of image orientation, i.e., flipped left-right, up-down, or both. If True, attempts to correct before returning corrected image. Otherwise it returns NULL.
checkXrayLungOrientation(image, verbose = FALSE)checkXrayLungOrientation(image, verbose = FALSE)
image |
input 3-D lung image. |
verbose |
print progress. |
segmentation and probability images
Tustison NJ
## Not run: library( ANTsRNet ) ## End(Not run)## Not run: library( ANTsRNet ) ## End(Not run)
ANTsXNet reproduction of https://arxiv.org/pdf/1711.05225.pdf. This includes our own network architecture and training (including data augmentation).
chexnet( image, lungMask = NULL, checkImageOrientation = FALSE, useANTsXNetVariant = TRUE, includeTuberculosisDiagnosis = FALSE, verbose = FALSE )chexnet( image, lungMask = NULL, checkImageOrientation = FALSE, useANTsXNetVariant = TRUE, includeTuberculosisDiagnosis = FALSE, verbose = FALSE )
image |
input 3-D lung image. |
lungMask |
ANTsImage. If None is specified, one is estimated. |
checkImageOrientation |
Check the correctness of image orientation, i.e., flipped left-right, up-down, or both. If TRUE, attempts to correct before prediction. |
useANTsXNetVariant |
Use an extension of the original chexnet approach by adding a left/right lung masking and including those two masked regions in the red and green channels, respectively. |
includeTuberculosisDiagnosis |
Include the output of an additional network trained on TB data but using the ANTsXNet variant chexnet data as the initial weights. |
verbose |
print progress. |
Includes a network for checking the correctness of image orientation, i.e., flipped left-right, up-down, or both.
Disease categories: Atelectasis Cardiomegaly Consolidation Edema Effusion Emphysema Fibrosis Hernia Infiltration Mass No Finding Nodule Pleural Thickening Pneumonia Pneumothorax
classification scores for each of the 14 or 15 categories.
Tustison NJ
## Not run: library( ANTsRNet ) ## End(Not run)## Not run: library( ANTsRNet ) ## End(Not run)
Perform CIT168 segmentation in T1 images using Pauli atlas (CIT168) labels described in https://pubmed.ncbi.nlm.nih.gov/29664465/
cit168Labeling(t1, verbose = FALSE)cit168Labeling(t1, verbose = FALSE)
t1 |
input ANTs image (3D). |
verbose |
Print progress to the screen. |
The labeling is as follows:
Label 1: BN_STR_Pu_Left
Label 2: BN_STR_Ca_Left
Label 3: BN_STR_NAC_Left (not modeled)
Label 4: EXA_Left (not modeled)
Label 5: BN_GP_GPe_Left
Label 6: BN_GP_GPi_Left
Label 7: MTg_SN_SNc_Left
Label 8: MTg_RN_Left
Label 9: MTg_SN_SNc_Left
Label 10: MTg_VTR_PBP_Left (not modeled)
Label 11: MTg_VTR_VTA_Left (not modeled)
Label 12: BN_GP_VeP_Left (not modeled)
Label 13: THM_ETH_HN_Left (not modeled)
Label 14: Die_HTH_Left (not modeled)
Label 15: Die_HTH_MN_Left (not modeled)
Label 16: Die_STH_Left (not modeled)
Label 17: BN_STR_Pu_Right
Label 18: BN_STR_Ca_Right
Label 19: BN_STR_NAC_Right (not modeled)
Label 20: EXA_Right (not modeled)
Label 21: BN_GP_GPe_Right
Label 22: BN_GP_GPi_Right
Label 23: MTg_SN_SNc_Right
Label 24: MTg_RN_Right
Label 25: MTg_SN_SNr_Right
Label 26: MTg_VTR_PBP_Right (not modeled)
Label 27: MTg_VTR_VTA_Right (not modeled)
Label 28: BN_GP_VeP_Right (not modeled)
Label 29: THM_ETH_HN_Right (not modeled)
Label 30: Die_HTH_Right (not modeled)
Label 31: Die_HTH_MN_Right (not modeled)
Label 32: Die_STH_Right (not modeled)
Label 33: ReferenceRegion_Left
Label 34: ReferenceRegion_Right
Preprocessing consists of:
n4 bias correction
brain extraction
ANTs segmentation label image.
Tustison NJ
## Not run: library( ANTsR ) library( ANTsRNet ) image <- antsImageRead( getANTsRData( "r16" ) ) seg <- cit168Labeling( image ) ## End(Not run)## Not run: library( ANTsR ) library( ANTsRNet ) image <- antsImageRead( getANTsRData( "r16" ) ) seg <- cit168Labeling( image ) ## End(Not run)
Described here:
claustrumSegmentation( t1, doPreprocessing = TRUE, useEnsemble = TRUE, verbose = FALSE )claustrumSegmentation( t1, doPreprocessing = TRUE, useEnsemble = TRUE, verbose = FALSE )
t1 |
input 3-D T1-weighted brain image. |
doPreprocessing |
perform n4 bias correction, denoising? |
useEnsemble |
boolean to check whether to use all 3 sets of weights. |
verbose |
print progress. |
\url{https://pubmed.ncbi.nlm.nih.gov/34520080/}
with the implementation available at:
\url{https://github.com/hongweilibran/claustrum_multi_view}
claustrum probability image
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1.nii.gz" ) probabilityMask <- claustrumSegmentation( image ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1.nii.gz" ) probabilityMask <- claustrumSegmentation( image ) ## End(Not run)
Clustering layer for Deep Embedded Clustering
clustering layer
number of clusters.
parameter
name
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
model = ClusteringLayer$new(numberOfClusters = 2) ## Not run: model$build(c(20, 20)) ## End(Not run)model = ClusteringLayer$new(numberOfClusters = 2) ## Not run: model$build(c(20, 20)) ## End(Not run)
Contextual attention layer for generative image inpainting described in
Jiahui Yu, et al., Generative Image Inpainting with Contextual Attention, CVPR 2018.
available here:
\code{https://arxiv.org/abs/1801.07892}
output tensor with the same shape as the input.
layer <- ContextualAttentionLayer2D$new( scale ) layer$call( x, mask = NULL ) layer$build( input_shape ) layer$compute_output_shape( input_shape )
A process object.
feature scale. Default = 20
$initialize instantiates a new class.
$build
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
x = ContextualAttentionLayer2D$new() x$build()x = ContextualAttentionLayer2D$new() x$build()
Contextual attention layer for generative image inpainting described in
Jiahui Yu, et al., Generative Image Inpainting with Contextual Attention, CVPR 2018.
available here:
\code{https://arxiv.org/abs/1801.07892}
output tensor with the same shape as the input.
layer <- ContextualAttentionLayer3D$new( scale ) layer$call( x, mask = NULL ) layer$build( input_shape ) layer$compute_output_shape( input_shape )
A process object.
$initialize instantiates a new class.
$build
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
x = ContextualAttentionLayer3D$new() x$build()x = ContextualAttentionLayer3D$new() x$build()
Function for converting box coordinates to/from min/max representation from/to centroids/width
convertCoordinates(boxes, type = c("minmax2centroids", "centroids2minmax"))convertCoordinates(boxes, type = c("minmax2centroids", "centroids2minmax"))
boxes |
A vector or 2-D array where each row corresponds to a single box consisting of the format (xmin,xmax,ymin,ymax) or (centerx,centery,width,height) for 2-D vs. (xmin,xmax,ymin,ymax,zmin,zmax) or (centerx,centery,centerz,width,height,depth) for 3-D. |
type |
either |
a vector or 2-D array with the converted coordinates
Tustison NJ
vec2d = c(1, 10, 1, 10) convertCoordinates(vec2d) convertCoordinates(vec2d, type = "centroids2minmax") convertCoordinates(rbind( vec2d, vec2d, vec2d)) convertCoordinates(rbind( vec2d, vec2d, vec2d), "centroids2minmax") vec3d = c(1, 10, 1, 10, 1, 10) convertCoordinates(vec3d) convertCoordinates(vec3d, type = "centroids2minmax") convertCoordinates(rbind( vec3d, vec3d, vec3d)) convertCoordinates(rbind( vec3d, vec3d, vec3d), type = "centroids2minmax")vec2d = c(1, 10, 1, 10) convertCoordinates(vec2d) convertCoordinates(vec2d, type = "centroids2minmax") convertCoordinates(rbind( vec2d, vec2d, vec2d)) convertCoordinates(rbind( vec2d, vec2d, vec2d), "centroids2minmax") vec3d = c(1, 10, 1, 10, 1, 10) convertCoordinates(vec3d) convertCoordinates(vec3d, type = "centroids2minmax") convertCoordinates(rbind( vec3d, vec3d, vec3d)) convertCoordinates(rbind( vec3d, vec3d, vec3d), type = "centroids2minmax")
Utility function to calculate the output shape of a convolutional filter along a single dimension. It's based on the python keras utility located here
convOutputLength( inputLength, filterSize, stride, dilation = 1, padding = c("same", "valid", "full", "causal") )convOutputLength( inputLength, filterSize, stride, dilation = 1, padding = c("same", "valid", "full", "causal") )
inputLength |
input size along a single dimension. |
filterSize |
kernel size along a single dimension. |
stride |
stride length along a single dimension. |
dilation |
dilation rate along a single dimension. |
padding |
type of padding. Can be "same", "valid", "full", or "causal". |
https://github.com/keras-team/keras/blob/master/keras/utils/conv_utils.py#L118-L141
but I can't see to locate it's R analog so it's reproduced explicitly in ANTsRNet.
the output size along a single dimension.
Tustison NJ
library( ANTsRNet ) outputSize <- convOutputLength( 256, filterSize = 3, stride = 2, padding = "same" ) testthat::expect_identical( outputSize, 128 )library( ANTsRNet ) outputSize <- convOutputLength( 256, filterSize = 3, stride = 2, padding = "same" ) testthat::expect_identical( outputSize, 128 )
Perform KellyKapowski cortical thickness using deepAtropos for
segmentation. Description concerning implementaiton and evaluation:
corticalThickness(t1, verbose = FALSE)corticalThickness(t1, verbose = FALSE)
t1 |
input 3-D unprocessed T1-weighted brain image |
verbose |
print progress. |
https://www.medrxiv.org/content/10.1101/2020.10.19.20215392v1
Cortical thickness image and segmentation probability images.
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1w_image.nii.gz" ) kk <- corticalThickness( image ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1w_image.nii.gz" ) kk <- corticalThickness( image ) ## End(Not run)
Creates a keras model of the AlexNet deep learning architecture for image recognition based on the paper
createAlexNetModel2D( inputImageSize, numberOfOutputs = 1000, numberOfDenseUnits = 4096, dropoutRate = 0, mode = c("classification", "regression"), batch_size = NULL )createAlexNetModel2D( inputImageSize, numberOfOutputs = 1000, numberOfDenseUnits = 4096, dropoutRate = 0, mode = c("classification", "regression"), batch_size = NULL )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Number of segmentation labels. |
numberOfDenseUnits |
number of dense units. |
dropoutRate |
optional regularization parameter between |
mode |
'classification' or 'regression'. |
batch_size |
batch size to pass to first layer |
A. Krizhevsky, and I. Sutskever, and G. Hinton. ImageNet Classification with Deep Convolutional Neural Networks.
available here:
http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
This particular implementation was influenced by the following python implementation:
https://github.com/duggalrahul/AlexNet-Experiments-Keras/
https://github.com/lunardog/convnets-keras/
an AlexNet keras model
Tustison NJ
library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:10,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:10], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createAlexNetModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) gc() rm(mnistData); gc() # Comment out the rest due to travis build constraints # track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, # epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model # testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) # predictedData <- model %>% predict( X_testSmall, verbose = 1 ) rm(model); gc()library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:10,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:10], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createAlexNetModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) gc() rm(mnistData); gc() # Comment out the rest due to travis build constraints # track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, # epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model # testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) # predictedData <- model %>% predict( X_testSmall, verbose = 1 ) rm(model); gc()
Creates a keras model of the AlexNet deep learning architecture for image recognition based on the paper
createAlexNetModel3D( inputImageSize, numberOfOutputs = 1000, numberOfDenseUnits = 4096, dropoutRate = 0, mode = c("classification", "regression"), batch_size = NULL )createAlexNetModel3D( inputImageSize, numberOfOutputs = 1000, numberOfDenseUnits = 4096, dropoutRate = 0, mode = c("classification", "regression"), batch_size = NULL )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Number of segmentation labels. |
numberOfDenseUnits |
number of dense units. |
dropoutRate |
optional regularization parameter between |
mode |
'classification' or 'regression'. Default = 'classification'. |
batch_size |
batch size to pass to first layer |
A. Krizhevsky, and I. Sutskever, and G. Hinton. ImageNet Classification with Deep Convolutional Neural Networks.
available here:
http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
This particular implementation was influenced by the following python implementation:
https://github.com/duggalrahul/AlexNet-Experiments-Keras/
https://github.com/lunardog/convnets-keras/
an AlexNet keras model
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:100,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:100], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size rm(mnistData); gc() inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createAlexNetModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) gc() track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, epochs = 2, batch_size = 20, shuffle = TRUE, validation_split = 0.25 ) # Now test the model testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) predictedData <- model %>% predict( X_testSmall, verbose = 1 ) rm(model); gc() ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:100,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:100], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size rm(mnistData); gc() inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createAlexNetModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) gc() track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, epochs = 2, batch_size = 20, shuffle = TRUE, validation_split = 0.25 ) # Now test the model testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) predictedData <- model %>% predict( X_testSmall, verbose = 1 ) rm(model); gc() ## End(Not run)
Builds an autoencoder based on the specified array definining the number of units in the encoding branch. Ported to Keras R from the Keras python implementation here:
createAutoencoderModel( numberOfUnitsPerLayer, activation = "relu", initializer = "glorot_uniform" )createAutoencoderModel( numberOfUnitsPerLayer, activation = "relu", initializer = "glorot_uniform" )
numberOfUnitsPerLayer |
vector defining the number of units in the encoding branch |
activation |
activation type for the dense layers |
initializer |
initializer type for the dense layers |
https://github.com/XifengGuo/DEC-keras
two models: the encoder and auto-encoder
Tustison NJ
library( ANTsRNet ) library( keras ) ae <- createAutoencoderModel( c( 784, 500, 500, 2000, 10 ) )library( ANTsRNet ) library( keras ) ae <- createAutoencoderModel( c( 784, 500, 500, 2000, 10 ) )
Builds a convolutional autoencoder based on the specified array definining the number of units in the encoding branch. Ported to Keras R from the Keras python implementation here:
createConvolutionalAutoencoderModel2D( inputImageSize, numberOfFiltersPerLayer = c(32, 64, 128, 10), convolutionKernelSize = c(5, 5), deconvolutionKernelSize = c(5, 5) )createConvolutionalAutoencoderModel2D( inputImageSize, numberOfFiltersPerLayer = c(32, 64, 128, 10), convolutionKernelSize = c(5, 5), deconvolutionKernelSize = c(5, 5) )
inputImageSize |
vector definining spatial dimensions + channels |
numberOfFiltersPerLayer |
vector defining the number of convolutional filters in the encoding branch per layer |
convolutionKernelSize |
kernel size fo the convolutional filters |
deconvolutionKernelSize |
kernel size fo the convolutional transpose filters |
https://github.com/XifengGuo/DCEC
two models: the convolutional encoder and convolutional auto-encoder
Tustison NJ
library( ANTsRNet ) library( keras ) ae <- createConvolutionalAutoencoderModel2D( c( 32, 32, 1 ) )library( ANTsRNet ) library( keras ) ae <- createConvolutionalAutoencoderModel2D( c( 32, 32, 1 ) )
Builds a convolutional autoencoder based on the specified array definining the number of units in the encoding branch. Ported to Keras R from the Keras python implementation here:
createConvolutionalAutoencoderModel3D( inputImageSize, numberOfFiltersPerLayer = c(32, 64, 128, 10), convolutionKernelSize = c(5, 5, 5), deconvolutionKernelSize = c(5, 5, 5) )createConvolutionalAutoencoderModel3D( inputImageSize, numberOfFiltersPerLayer = c(32, 64, 128, 10), convolutionKernelSize = c(5, 5, 5), deconvolutionKernelSize = c(5, 5, 5) )
inputImageSize |
vector definining spatial dimensions + channels |
numberOfFiltersPerLayer |
vector defining the number of convolutional filters in the encoding branch per layer |
convolutionKernelSize |
kernel size fo the convolutional filters |
deconvolutionKernelSize |
kernel size fo the convolutional transpose filters |
https://github.com/XifengGuo/DCEC
two models: the convolutional encoder and convolutional auto-encoder
Tustison NJ
library( ANTsRNet ) library( keras ) ae <- createConvolutionalAutoencoderModel2D( c( 32, 32, 1 ) )library( ANTsRNet ) library( keras ) ae <- createConvolutionalAutoencoderModel2D( c( 32, 32, 1 ) )
Creates a keras model of the deep back-project network for image super resolution. More information is provided at the authors' website:
createDeepBackProjectionNetworkModel2D( inputImageSize, numberOfOutputs = 1, numberOfBaseFilters = 64, numberOfFeatureFilters = 256, numberOfBackProjectionStages = 7, convolutionKernelSize = c(12, 12), strides = c(8, 8), lastConvolution = c(3, 3), numberOfLossFunctions = 1 )createDeepBackProjectionNetworkModel2D( inputImageSize, numberOfOutputs = 1, numberOfBaseFilters = 64, numberOfFeatureFilters = 256, numberOfBackProjectionStages = 7, convolutionKernelSize = c(12, 12), strides = c(8, 8), lastConvolution = c(3, 3), numberOfLossFunctions = 1 )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). |
numberOfOutputs |
number of outputs (e.g., 3 for RGB images). |
numberOfBaseFilters |
number of base filters. |
numberOfFeatureFilters |
number of feature filters. |
numberOfBackProjectionStages |
number of up-/down-projection stages. This number includes the final up block. |
convolutionKernelSize |
kernel size for certain convolutional layers. This
and |
strides |
strides for certain convolutional layers. This and the
|
lastConvolution |
the kernel size for the last convolutional layer |
numberOfLossFunctions |
the number of data targets, e.g. 2 for 2 targets |
\url{https://www.toyota-ti.ac.jp/Lab/Denshi/iim/members/muhammad.haris/projects/DBPN.html}
with the paper available here:
\url{https://arxiv.org/abs/1803.02735}
This particular implementation was influenced by the following keras (python) implementation:
\url{https://github.com/rajatkb/DBPN-Keras}
with help from the original author's Caffe and Pytorch implementations:
\url{https://github.com/alterzero/DBPN-caffe}
\url{https://github.com/alterzero/DBPN-Pytorch}
a keras model defining the deep back-projection network.
Tustison NJ
model = createDeepBackProjectionNetworkModel2D(c(25, 25, 1)) rm(model); gc()model = createDeepBackProjectionNetworkModel2D(c(25, 25, 1)) rm(model); gc()
Creates a keras model of the deep back-project network for image super resolution. More information is provided at the authors' website:
createDeepBackProjectionNetworkModel3D( inputImageSize, numberOfOutputs = 1, numberOfBaseFilters = 64, numberOfFeatureFilters = 256, numberOfBackProjectionStages = 7, convolutionKernelSize = c(12, 12, 12), strides = c(8, 8, 8), lastConvolution = c(3, 3, 3), numberOfLossFunctions = 1 )createDeepBackProjectionNetworkModel3D( inputImageSize, numberOfOutputs = 1, numberOfBaseFilters = 64, numberOfFeatureFilters = 256, numberOfBackProjectionStages = 7, convolutionKernelSize = c(12, 12, 12), strides = c(8, 8, 8), lastConvolution = c(3, 3, 3), numberOfLossFunctions = 1 )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). |
numberOfOutputs |
number of outputs (e.g., 3 for RGB images). |
numberOfBaseFilters |
number of base filters. |
numberOfFeatureFilters |
number of feature filters. |
numberOfBackProjectionStages |
number of up-/down-projection stages. This number includes the final up block. |
convolutionKernelSize |
kernel size for certain convolutional layers. This
and |
strides |
strides for certain convolutional layers. This and the
|
lastConvolution |
the kernel size for the last convolutional layer |
numberOfLossFunctions |
the number of data targets, e.g. 2 for 2 targets |
\url{https://www.toyota-ti.ac.jp/Lab/Denshi/iim/members/muhammad.haris/projects/DBPN.html}
with the paper available here:
\url{https://arxiv.org/abs/1803.02735}
This particular implementation was influenced by the following keras (python) implementation:
\url{https://github.com/rajatkb/DBPN-Keras}
with help from the original author's Caffe and Pytorch implementations:
\url{https://github.com/alterzero/DBPN-caffe}
\url{https://github.com/alterzero/DBPN-Pytorch}
a keras model defining the deep back-projection network.
Tustison NJ
model = createDeepBackProjectionNetworkModel3D(c(25, 25, 25, 1)) rm(model); gc()model = createDeepBackProjectionNetworkModel3D(c(25, 25, 25, 1)) rm(model); gc()
Creates a keras model of the expanded image super resolution deep learning framework based on the following python implementation:
createDeepDenoiseSuperResolutionModel2D( inputImageSize, layers = 2, lowestResolution = 64, convolutionKernelSize = c(3, 3), poolSize = c(2, 2), strides = c(2, 2) )createDeepDenoiseSuperResolutionModel2D( inputImageSize, layers = 2, lowestResolution = 64, convolutionKernelSize = c(3, 3), poolSize = c(2, 2), strides = c(2, 2) )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
layers |
number of architecture layers. |
lowestResolution |
number of filters at the beginning and end of the architecture. |
convolutionKernelSize |
2-D vector defining the kernel size during the encoding path |
poolSize |
2-D vector defining the region for each pooling layer. |
strides |
2-D vector describing the stride length in each direction. |
\url{https://github.com/titu1994/Image-Super-Resolution}
a keras model for image super resolution
Tustison NJ
## Not run: createDeepDenoiseSuperResolutionModel2D(c(256L, 256L, 3L)) ## End(Not run)## Not run: createDeepDenoiseSuperResolutionModel2D(c(256L, 256L, 3L)) ## End(Not run)
Creates a keras model of the expanded image super resolution deep learning framework based on the following python implementation:
createDeepDenoiseSuperResolutionModel3D( inputImageSize, layers = 2, lowestResolution = 64, convolutionKernelSize = c(3, 3, 3), poolSize = c(2, 2, 2), strides = c(2, 2, 2) )createDeepDenoiseSuperResolutionModel3D( inputImageSize, layers = 2, lowestResolution = 64, convolutionKernelSize = c(3, 3, 3), poolSize = c(2, 2, 2), strides = c(2, 2, 2) )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
layers |
number of architecture layers. |
lowestResolution |
number of filters at the beginning and end of the architecture. |
convolutionKernelSize |
3-D vector defining the kernel size during the encoding path |
poolSize |
3-D vector defining the region for each pooling layer. |
strides |
3-D vector describing the stride length in each direction. |
\url{https://github.com/titu1994/Image-Super-Resolution}
a keras model for image super resolution
Tustison NJ
## Not run: createDeepDenoiseSuperResolutionModel3D(c(256L, 256L, 45L, 1L)) ## End(Not run)## Not run: createDeepDenoiseSuperResolutionModel3D(c(256L, 256L, 45L, 1L)) ## End(Not run)
2-D implementation of the denoising autoencoder image super resolution architecture.
createDenoisingAutoEncoderSuperResolutionModel2D( inputImageSize, convolutionKernelSizes = list(c(3, 3), c(5, 5)), numberOfEncodingLayers = 2, numberOfFilters = 64 )createDenoisingAutoEncoderSuperResolutionModel2D( inputImageSize, convolutionKernelSizes = list(c(3, 3), c(5, 5)), numberOfEncodingLayers = 2, numberOfFilters = 64 )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
convolutionKernelSizes |
a 2-element list of 2-D vectors specifying the kernel size at each convolution layer. The first element is the kernel size of the encoding layers and the 2nd element is the kernel size of the final convolution layer. |
numberOfEncodingLayers |
the number of encoding layers. |
numberOfFilters |
the number of filters for each encoding layer. |
a keras model for image super resolution
Tustison NJ
createDenoisingAutoEncoderSuperResolutionModel2D(c( 28, 28, 1 ))createDenoisingAutoEncoderSuperResolutionModel2D(c( 28, 28, 1 ))
3-D implementation of the denoising autoencoder image super resolution architecture.
createDenoisingAutoEncoderSuperResolutionModel3D( inputImageSize, convolutionKernelSizes = list(c(3, 3, 3), c(5, 5, 5)), numberOfEncodingLayers = 2, numberOfFilters = 64 )createDenoisingAutoEncoderSuperResolutionModel3D( inputImageSize, convolutionKernelSizes = list(c(3, 3, 3), c(5, 5, 5)), numberOfEncodingLayers = 2, numberOfFilters = 64 )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
convolutionKernelSizes |
a 2-element list of 3-D vectors specifying the kernel size at each convolution layer. The first element is the kernel size of the encoding layers and the 2nd element is the kernel size of the final convolution layer. |
numberOfEncodingLayers |
the number of encoding layers. |
numberOfFilters |
the number of filters for each encoding layer. |
a keras model for image super resolution
Tustison NJ
createDenoisingAutoEncoderSuperResolutionModel3D(c( 28, 28, 28, 1 )) gc()createDenoisingAutoEncoderSuperResolutionModel3D(c( 28, 28, 28, 1 )) gc()
Simple multilayer dense network.
createDenseModel( inputVectorSize, numberOfFiltersAtBaseLayer = 512, numberOfLayers = 2, mode = "classification", numberOfOutputs = 1000 )createDenseModel( inputVectorSize, numberOfFiltersAtBaseLayer = 512, numberOfLayers = 2, mode = "classification", numberOfOutputs = 1000 )
inputVectorSize |
Specifies the length of the input vector. |
numberOfFiltersAtBaseLayer |
Number of filters at the initial dense layer. This number is halved for each subsequent layer. |
numberOfLayers |
Number of dense layers defining the model. |
mode |
'classification' or 'regression'. |
numberOfOutputs |
Specifies number of units in final layer |
a keras model
Tustison NJ
library( ANTsRNet ) model <- createDenseModel( 166 )library( ANTsRNet ) model <- createDenseModel( 166 )
Creates a keras model of the DenseNet deep learning architecture for image recognition based on the paper
createDenseNetModel2D( inputImageSize, numberOfOutputs = 1000, numberOfFilters = 16, depth = 7, numberOfDenseBlocks = 1, growthRate = 12, dropoutRate = 0.2, weightDecay = 1e-04, mode = "classification" )createDenseNetModel2D( inputImageSize, numberOfOutputs = 1000, numberOfFilters = 16, depth = 7, numberOfDenseBlocks = 1, growthRate = 12, dropoutRate = 0.2, weightDecay = 1e-04, mode = "classification" )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Specifies number of units in final layer |
numberOfFilters |
number of filters |
depth |
number of layers—must be equal to 3 * N + 4 where N is an integer (default = 7). |
numberOfDenseBlocks |
number of dense blocks to add to the end (default = 1). |
growthRate |
number of filters to add for each dense block layer (default = 12). |
dropoutRate |
= per drop out layer rate (default = 0.2). |
weightDecay |
= weight decay (default = 1e-4). |
mode |
'classification' or 'regression'. Default = 'classification'. |
G. Huang, Z. Liu, K. Weinberger, and L. van der Maaten. Densely Connected Convolutional Networks Networks
available here:
https://arxiv.org/abs/1608.06993
This particular implementation was influenced by the following python implementation:
https://github.com/tdeboissiere/DeepLearningImplementations/blob/master/DenseNet/densenet.py
an DenseNet keras model
Tustison NJ
library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:100,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:100], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createDenseNetModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) # Comment out the rest due to travis build constraints # track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, # epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model # testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) # predictedData <- model %>% predict( X_testSmall, verbose = 1 )library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:100,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:100], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createDenseNetModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) # Comment out the rest due to travis build constraints # track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, # epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model # testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) # predictedData <- model %>% predict( X_testSmall, verbose = 1 )
Creates a keras model of the DenseNet deep learning architecture for image recognition based on the paper
createDenseNetModel3D( inputImageSize, numberOfOutputs = 1000, numberOfFilters = 16, depth = 7, numberOfDenseBlocks = 1, growthRate = 12, dropoutRate = 0.2, weightDecay = 1e-04, mode = "classification" )createDenseNetModel3D( inputImageSize, numberOfOutputs = 1000, numberOfFilters = 16, depth = 7, numberOfDenseBlocks = 1, growthRate = 12, dropoutRate = 0.2, weightDecay = 1e-04, mode = "classification" )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Number of segmentation labels. |
numberOfFilters |
number of filters |
depth |
number of layers—must be equal to 3 * N + 4 where N is an integer (default = 7). |
numberOfDenseBlocks |
number of dense blocks to add to the end (default = 1). |
growthRate |
number of filters to add for each dense block layer (default = 12). |
dropoutRate |
= per drop out layer rate (default = 0.2) |
weightDecay |
= weight decay (default = 1e-4) |
mode |
'classification' or 'regression'. Default = 'classification'. |
G. Huang, Z. Liu, K. Weinberger, and L. van der Maaten. Densely Connected Convolutional Networks Networks
available here:
https://arxiv.org/abs/1608.06993
This particular implementation was influenced by the following python implementation:
https://github.com/tdeboissiere/DeepLearningImplementations/blob/master/DenseNet/densenet.py
an DenseNet keras model
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:10,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:10], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createDenseNetModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) predictedData <- model %>% predict( X_testSmall, verbose = 1 ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:10,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:10], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createDenseNetModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) predictedData <- model %>% predict( X_testSmall, verbose = 1 ) ## End(Not run)
Creates a keras model of the dense U-net deep learning architecture for image segmentation
createDenseUnetModel2D( inputImageSize, numberOfOutputs = 1L, numberOfLayersPerDenseBlock = c(6, 12, 36, 24), growthRate = 48, initialNumberOfFilters = 96, reductionRate = 0, depth = 7, dropoutRate = 0, weightDecay = 1e-04, mode = c("classification", "regression") )createDenseUnetModel2D( inputImageSize, numberOfOutputs = 1L, numberOfLayersPerDenseBlock = c(6, 12, 36, 24), growthRate = 48, initialNumberOfFilters = 96, reductionRate = 0, depth = 7, dropoutRate = 0, weightDecay = 1e-04, mode = c("classification", "regression") )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). |
numberOfOutputs |
Meaning depends on the |
numberOfLayersPerDenseBlock |
number of dense blocks per layer. |
growthRate |
number of filters to add for each dense block layer (default = 48). |
initialNumberOfFilters |
number of filters at the beginning (default = 96). |
reductionRate |
reduction factor of transition blocks |
depth |
number of layers—must be equal to 3 * N + 4 where N is an integer (default = 7). |
dropoutRate |
drop out layer rate (default = 0.2). |
weightDecay |
weight decay (default = 1e-4). |
mode |
A switch to determine the activation function to use.
If |
X. Li, H. Chen, X. Qi, Q. Dou, C.-W. Fu, P.-A. Heng. H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation from CT Volumes
available here:
https://arxiv.org/pdf/1709.07330.pdf
with the author's implementation available at:
https://github.com/xmengli999/H-DenseUNet
an DenseUnet keras model
Tustison NJ
library( ANTsR ) library( ANTsRNet ) library( keras ) imageIDs <- c( "r16", "r27", "r30", "r62", "r64", "r85" ) trainingBatchSize <- length( imageIDs ) # Perform simple 3-tissue segmentation. segmentationLabels <- c( 1, 2, 3 ) numberOfLabels <- length( segmentationLabels ) initialization <- paste0( 'KMeans[', numberOfLabels, ']' ) domainImage <- antsImageRead( getANTsRData( imageIDs[1] ) ) X_train <- array( data = NA, dim = c( trainingBatchSize, dim( domainImage ), 1 ) ) Y_train <- array( data = NA, dim = c( trainingBatchSize, dim( domainImage ) ) ) images <- list() segmentations <- list() for( i in seq_len( trainingBatchSize ) ) { cat( "Processing image", imageIDs[i], "\n" ) image <- antsImageRead( getANTsRData( imageIDs[i] ) ) mask <- getMask( image ) segmentation <- atropos( image, mask, initialization )$segmentation X_train[i,,, 1] <- as.array( image ) Y_train[i,,] <- as.array( segmentation ) } Y_train <- encodeUnet( Y_train, segmentationLabels ) # Perform a simple normalization X_train <- ( X_train - mean( X_train ) ) / sd( X_train ) # Create the model model <- createDenseUnetModel2D( c( dim( domainImage ), 1L ), numberOfOutputs = numberOfLabels ) metric_multilabel_dice_coefficient <- custom_metric( "multilabel_dice_coefficient", multilabel_dice_coefficient ) loss_dice <- function( y_true, y_pred ) { -multilabel_dice_coefficient(y_true, y_pred) } attr(loss_dice, "py_function_name") <- "multilabel_dice_coefficient" model %>% compile( loss = loss_dice, optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( metric_multilabel_dice_coefficient, metric_categorical_crossentropy ) ) # Comment out the rest due to travis build constraints # Fit the model # checkpoint_file = tempfile(fileext = ".h5") # track <- model %>% fit( X_train, Y_train, # epochs = 5, batch_size = 4, verbose = 1, shuffle = TRUE, # callbacks = list( # callback_model_checkpoint( checkpoint_file, # monitor = 'val_loss', save_best_only = TRUE ), # callback_reduce_lr_on_plateau( monitor = "val_loss", factor = 0.1 ) # ), # validation_split = 0.2 ) # Save the model and/or save the model weights # save_model_hdf5( model, filepath = 'unetModel.h5' ) # save_model_weights_hdf5( unetModel, filepath = 'unetModelWeights.h5' ) ) rm(model); gc()library( ANTsR ) library( ANTsRNet ) library( keras ) imageIDs <- c( "r16", "r27", "r30", "r62", "r64", "r85" ) trainingBatchSize <- length( imageIDs ) # Perform simple 3-tissue segmentation. segmentationLabels <- c( 1, 2, 3 ) numberOfLabels <- length( segmentationLabels ) initialization <- paste0( 'KMeans[', numberOfLabels, ']' ) domainImage <- antsImageRead( getANTsRData( imageIDs[1] ) ) X_train <- array( data = NA, dim = c( trainingBatchSize, dim( domainImage ), 1 ) ) Y_train <- array( data = NA, dim = c( trainingBatchSize, dim( domainImage ) ) ) images <- list() segmentations <- list() for( i in seq_len( trainingBatchSize ) ) { cat( "Processing image", imageIDs[i], "\n" ) image <- antsImageRead( getANTsRData( imageIDs[i] ) ) mask <- getMask( image ) segmentation <- atropos( image, mask, initialization )$segmentation X_train[i,,, 1] <- as.array( image ) Y_train[i,,] <- as.array( segmentation ) } Y_train <- encodeUnet( Y_train, segmentationLabels ) # Perform a simple normalization X_train <- ( X_train - mean( X_train ) ) / sd( X_train ) # Create the model model <- createDenseUnetModel2D( c( dim( domainImage ), 1L ), numberOfOutputs = numberOfLabels ) metric_multilabel_dice_coefficient <- custom_metric( "multilabel_dice_coefficient", multilabel_dice_coefficient ) loss_dice <- function( y_true, y_pred ) { -multilabel_dice_coefficient(y_true, y_pred) } attr(loss_dice, "py_function_name") <- "multilabel_dice_coefficient" model %>% compile( loss = loss_dice, optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( metric_multilabel_dice_coefficient, metric_categorical_crossentropy ) ) # Comment out the rest due to travis build constraints # Fit the model # checkpoint_file = tempfile(fileext = ".h5") # track <- model %>% fit( X_train, Y_train, # epochs = 5, batch_size = 4, verbose = 1, shuffle = TRUE, # callbacks = list( # callback_model_checkpoint( checkpoint_file, # monitor = 'val_loss', save_best_only = TRUE ), # callback_reduce_lr_on_plateau( monitor = "val_loss", factor = 0.1 ) # ), # validation_split = 0.2 ) # Save the model and/or save the model weights # save_model_hdf5( model, filepath = 'unetModel.h5' ) # save_model_weights_hdf5( unetModel, filepath = 'unetModelWeights.h5' ) ) rm(model); gc()
Creates a keras model of the dense U-net deep learning architecture for image segmentation
createDenseUnetModel3D( inputImageSize, numberOfOutputs = 1L, numberOfLayersPerDenseBlock = c(3, 4, 12, 8), growthRate = 48, initialNumberOfFilters = 96, reductionRate = 0, depth = 7, dropoutRate = 0, weightDecay = 1e-04, mode = c("classification", "regression") )createDenseUnetModel3D( inputImageSize, numberOfOutputs = 1L, numberOfLayersPerDenseBlock = c(3, 4, 12, 8), growthRate = 48, initialNumberOfFilters = 96, reductionRate = 0, depth = 7, dropoutRate = 0, weightDecay = 1e-04, mode = c("classification", "regression") )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). |
numberOfOutputs |
Meaning depends on the |
numberOfLayersPerDenseBlock |
number of dense blocks per layer. |
growthRate |
number of filters to add for each dense block layer (default = 48). |
initialNumberOfFilters |
number of filters at the beginning (default = 96). |
reductionRate |
reduction factor of transition blocks |
depth |
number of layers—must be equal to 3 * N + 4 where N is an integer (default = 7). |
dropoutRate |
drop out layer rate (default = 0.2). |
weightDecay |
weight decay (default = 1e-4). |
mode |
A switch to determine the activation function to use.
If |
X. Li, H. Chen, X. Qi, Q. Dou, C.-W. Fu, P.-A. Heng. H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation from CT Volumes
available here:
https://arxiv.org/pdf/1709.07330.pdf
with the author's implementation available at:
https://github.com/xmengli999/H-DenseUNet
an DenseUnet keras model
Tustison NJ
library( ANTsRNet ) library( keras ) model <- createDenseUnetModel3D( c( 64, 64, 64, 1 ) ) metric_multilabel_dice_coefficient <- custom_metric( "multilabel_dice_coefficient", multilabel_dice_coefficient ) loss_dice <- function( y_true, y_pred ) { -multilabel_dice_coefficient(y_true, y_pred) } attr(loss_dice, "py_function_name") <- "multilabel_dice_coefficient" model %>% compile( loss = loss_dice, optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( metric_multilabel_dice_coefficient, metric_categorical_crossentropy ) ) print( model ) rm(model); gc()library( ANTsRNet ) library( keras ) model <- createDenseUnetModel3D( c( 64, 64, 64, 1 ) ) metric_multilabel_dice_coefficient <- custom_metric( "multilabel_dice_coefficient", multilabel_dice_coefficient ) loss_dice <- function( y_true, y_pred ) { -multilabel_dice_coefficient(y_true, y_pred) } attr(loss_dice, "py_function_name") <- "multilabel_dice_coefficient" model %>% compile( loss = loss_dice, optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( metric_multilabel_dice_coefficient, metric_categorical_crossentropy ) ) print( model ) rm(model); gc()
Creates a keras model of the expanded image super resolution deep learning framework based on EDSR.
createEnhancedDeepSuperResolutionModel2D( inputImageSize, convolutionKernelSize = c(3, 3), numberOfFilters = 256, numberOfResidualBlocks = 32, scale = 2, numberOfLossFunctions = 1, numberOfOutputChannels = 1, doBatchNormalization = FALSE, interpolation = c("bilinear", "nearest", "conv") )createEnhancedDeepSuperResolutionModel2D( inputImageSize, convolutionKernelSize = c(3, 3), numberOfFilters = 256, numberOfResidualBlocks = 32, scale = 2, numberOfLossFunctions = 1, numberOfOutputChannels = 1, doBatchNormalization = FALSE, interpolation = c("bilinear", "nearest", "conv") )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
convolutionKernelSize |
a vector specifying the kernel size for convolution. |
numberOfFilters |
the number of filters for each encoding layer. |
numberOfResidualBlocks |
the number of residual blocks. |
scale |
the upsampling amount, 2, 4 or 8 |
numberOfLossFunctions |
the number of data targets, e.g. 2 for 2 targets |
numberOfOutputChannels |
the number of ouput channels |
doBatchNormalization |
boolean for include BN in the residual blocks |
interpolation |
nearest, bilinear or conv for upscaling block |
a keras model for EDSR image super resolution
Tustison NJ, Avants BB
model = createEnhancedDeepSuperResolutionModel2D(c( 28, 28, 1 )) rm(model); gc() model = createEnhancedDeepSuperResolutionModel2D(c( 28, 28, 1 ), doBatchNormalization = TRUE, interpolation = "conv", scale = 4) rm(model); gc() model = createEnhancedDeepSuperResolutionModel2D(c( 28, 28, 1 ), doBatchNormalization = TRUE, numberOfLossFunctions = 2, interpolation = "conv", scale = 8) rm(model); gc()model = createEnhancedDeepSuperResolutionModel2D(c( 28, 28, 1 )) rm(model); gc() model = createEnhancedDeepSuperResolutionModel2D(c( 28, 28, 1 ), doBatchNormalization = TRUE, interpolation = "conv", scale = 4) rm(model); gc() model = createEnhancedDeepSuperResolutionModel2D(c( 28, 28, 1 ), doBatchNormalization = TRUE, numberOfLossFunctions = 2, interpolation = "conv", scale = 8) rm(model); gc()
Creates a keras model of the expanded image super resolution deep learning framework based on the following python implementation:
createExpandedSuperResolutionModel2D( inputImageSize, convolutionKernelSizes = list(c(9, 9), c(1, 1), c(3, 3), c(5, 5), c(5, 5)), numberOfFilters = c(64, 32, 32, 32) )createExpandedSuperResolutionModel2D( inputImageSize, convolutionKernelSizes = list(c(9, 9), c(1, 1), c(3, 3), c(5, 5), c(5, 5)), numberOfFilters = c(64, 32, 32, 32) )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
convolutionKernelSizes |
a list of 2-D vectors specifying the kernel size at each convolution layer. Default values are the same as given in the original paper. The length of kernel size vectors must be 1 greater than the vector length of the number of filters. |
numberOfFilters |
a vector containing the number of filters for each convolutional layer. Default values are the same as given in the original paper. |
\url{https://github.com/titu1994/Image-Super-Resolution}
a keras model for image super resolution
Tustison NJ
createExpandedSuperResolutionModel2D(c( 100, 100, 1 ))createExpandedSuperResolutionModel2D(c( 100, 100, 1 ))
Creates a keras model of the expanded image super resolution deep learning framework based on the following python implementation:
createExpandedSuperResolutionModel3D( inputImageSize, convolutionKernelSizes = list(c(9, 9, 9), c(1, 1, 1), c(3, 3, 3), c(5, 5, 5), c(5, 5, 5)), numberOfFilters = c(64, 32, 32, 32) )createExpandedSuperResolutionModel3D( inputImageSize, convolutionKernelSizes = list(c(9, 9, 9), c(1, 1, 1), c(3, 3, 3), c(5, 5, 5), c(5, 5, 5)), numberOfFilters = c(64, 32, 32, 32) )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
convolutionKernelSizes |
a list of 3-D vectors specifying the kernel size at each convolution layer. Default values are the same as given in the original paper. The length of kernel size vectors must be 1 greater than the vector length of the number of filters. |
numberOfFilters |
a vector containing the number of filters for each convolutional layer. Default values are the same as given in the original paper. |
\url{https://github.com/titu1994/Image-Super-Resolution}
a keras model for image super resolution
Tustison NJ
createExpandedSuperResolutionModel3D(c( 100, 100, 100, 1 ))createExpandedSuperResolutionModel3D(c( 100, 100, 100, 1 ))
Creates a keras model of the Vgg deep learning architecture for image recognition based on the paper
createFullyConvolutionalVggModel2D( inputImageSize, layers = c(1, 2, 3, 4, 4), lowestResolution = 64, convolutionKernelSize = c(3, 3), poolSize = c(2, 2), strides = c(2, 2), dropoutRate = 0, style = 19, activation = "relu" )createFullyConvolutionalVggModel2D( inputImageSize, layers = c(1, 2, 3, 4, 4), lowestResolution = 64, convolutionKernelSize = c(3, 3), poolSize = c(2, 2), strides = c(2, 2), dropoutRate = 0, style = 19, activation = "relu" )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
layers |
a vector determining the number of filters defined at for each layer. |
lowestResolution |
number of filters at the beginning. |
convolutionKernelSize |
2-d vector definining the kernel size during the encoding path |
poolSize |
2-d vector defining the region for each pooling layer. |
strides |
2-d vector describing the stride length in each direction. |
dropoutRate |
float between 0 and 1 to use between dense layers. |
style |
|
activation |
activation function for convolutional layer. |
K. Simonyan and A. Zisserman, Very Deep Convolutional Networks for Large-Scale Image Recognition
available here:
\url{https://arxiv.org/abs/1409.1556}
This particular implementation was influenced by the following python implementation:
\url{https://gist.github.com/baraldilorenzo/8d096f48a1be4a2d660d}
a VGG keras model
Tustison NJ
Creates a keras model of the Vgg deep learning architecture for image recognition based on the paper
createFullyConvolutionalVggModel3D( inputImageSize, layers = c(1, 2, 3, 4, 4), lowestResolution = 64, convolutionKernelSize = c(3, 3, 3), poolSize = c(2, 2, 2), strides = c(2, 2, 2), dropoutRate = 0, style = 19, activation = "relu" )createFullyConvolutionalVggModel3D( inputImageSize, layers = c(1, 2, 3, 4, 4), lowestResolution = 64, convolutionKernelSize = c(3, 3, 3), poolSize = c(2, 2, 2), strides = c(2, 2, 2), dropoutRate = 0, style = 19, activation = "relu" )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
layers |
a vector determining the number of 'filters' defined at for each layer. |
lowestResolution |
number of filters at the beginning. |
convolutionKernelSize |
3-d vector definining the kernel size during the encoding path |
poolSize |
3-d vector defining the region for each pooling layer. |
strides |
3-d vector describing the stride length in each direction. |
dropoutRate |
float between 0 and 1 to use between dense layers. |
style |
|
activation |
activation function for convolutional layer. |
K. Simonyan and A. Zisserman, Very Deep Convolutional Networks for Large-Scale Image Recognition
available here:
\url{https://arxiv.org/abs/1409.1556}
This particular implementation was influenced by the following python implementation:
\url{https://gist.github.com/baraldilorenzo/8d096f48a1be4a2d660d}
a VGG keras model
Tustison NJ
Creates a keras model of the GoogLeNet deep learning architecture for image recognition based on the paper
createGoogLeNetModel2D( inputImageSize, numberOfOutputs = 1000, mode = c("classification", "regression") )createGoogLeNetModel2D( inputImageSize, numberOfOutputs = 1000, mode = c("classification", "regression") )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Specifies number of units in final layer |
mode |
'classification' or 'regression'. |
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, Going Deeper with Convolutions C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. Rethinking the Inception Architecture for Computer Vision
available here:
https://arxiv.org/abs/1409.4842
https://arxiv.org/abs/1512.00567
This particular implementation was influenced by the following python implementation:
https://github.com/fchollet/deep-learning-models/blob/master/inception_v3.py
a GoogLeNet keras model
Tustison NJ
library( ANTsRNet ) library( keras ) library( ANTsR ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly. # We also need to resample since the native mnist data size does # not fit with GoogLeNet parameters. resampledImageSize <- c( 100, 100 ) numberOfTrainingData <- 10 numberOfTestingData <- 5 X_trainSmall <- as.array( resampleImage( as.antsImage( mnistData$train$x[1:numberOfTrainingData,,] ), c( numberOfTrainingData, resampledImageSize ), TRUE ) ) X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:numberOfTrainingData], numberOfLabels ) X_testSmall <- as.array( resampleImage( as.antsImage( mnistData$test$x[1:numberOfTestingData,,] ), c( numberOfTestingData, resampledImageSize ), TRUE ) ) X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:numberOfTestingData], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createGoogLeNetModel2D( inputImageSize = c( resampledImageSize, 1 ), numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) # Comment out the rest due to travis build constraints # track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, # epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model # testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) # predictedData <- model %>% predict( X_testSmall, verbose = 1 ) rm(model); gc() model <- createGoogLeNetModel2D( inputImageSize = c( resampledImageSize, 1 ), numberOfOutputs = 2 ) rm(model); gc() model <- createGoogLeNetModel2D( inputImageSize = c( resampledImageSize, 1 ), mode = "regression" ) rm(model); gc()library( ANTsRNet ) library( keras ) library( ANTsR ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly. # We also need to resample since the native mnist data size does # not fit with GoogLeNet parameters. resampledImageSize <- c( 100, 100 ) numberOfTrainingData <- 10 numberOfTestingData <- 5 X_trainSmall <- as.array( resampleImage( as.antsImage( mnistData$train$x[1:numberOfTrainingData,,] ), c( numberOfTrainingData, resampledImageSize ), TRUE ) ) X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:numberOfTrainingData], numberOfLabels ) X_testSmall <- as.array( resampleImage( as.antsImage( mnistData$test$x[1:numberOfTestingData,,] ), c( numberOfTestingData, resampledImageSize ), TRUE ) ) X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:numberOfTestingData], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createGoogLeNetModel2D( inputImageSize = c( resampledImageSize, 1 ), numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) # Comment out the rest due to travis build constraints # track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, # epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model # testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) # predictedData <- model %>% predict( X_testSmall, verbose = 1 ) rm(model); gc() model <- createGoogLeNetModel2D( inputImageSize = c( resampledImageSize, 1 ), numberOfOutputs = 2 ) rm(model); gc() model <- createGoogLeNetModel2D( inputImageSize = c( resampledImageSize, 1 ), mode = "regression" ) rm(model); gc()
Creates a keras model implementation of the u-net architecture described here:
createHippMapp3rUnetModel3D(inputImageSize, doFirstNetwork = TRUE)createHippMapp3rUnetModel3D(inputImageSize, doFirstNetwork = TRUE)
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). |
doFirstNetwork |
boolean dictating if the model built should be the first (initial) network or second (refinement) network. |
\url{https://onlinelibrary.wiley.com/doi/pdf/10.1002/hbm.24811}
with the implementation available here:
\url{https://github.com/mgoubran/HippMapp3r}
a u-net keras model
Tustison NJ
## Not run: model1 <- createHippMapp3rUnetModel3D( c( 160, 160, 128, 1 ), doFirstNetwork = TRUE ) model2 <- createHippMapp3rUnetModel3D( c( 112, 112, 64, 1 ), doFirstNetwork = FALSE ) json_config <- model_to_json( model1 ) writeLines( json_config, "/Users/ntustison/Desktop/model1_config.json" ) ## End(Not run)## Not run: model1 <- createHippMapp3rUnetModel3D( c( 160, 160, 128, 1 ), doFirstNetwork = TRUE ) model2 <- createHippMapp3rUnetModel3D( c( 112, 112, 64, 1 ), doFirstNetwork = FALSE ) json_config <- model_to_json( model1 ) writeLines( json_config, "/Users/ntustison/Desktop/model1_config.json" ) ## End(Not run)
Creates a keras model implementation of the u-net architecture described here:
createHyperMapp3rUnetModel3D(inputImageSize, dataFormat = "channels_last")createHyperMapp3rUnetModel3D(inputImageSize, dataFormat = "channels_last")
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). |
dataFormat |
One of "channels_first" or "channels_last". We do this for this specific architecture as the original weights were saved in "channels_first" format. |
\url{https://pubmed.ncbi.nlm.nih.gov/35088930/}
with the implementation available here:
\url{https://github.com/mgoubran/HyperMapp3r}
a u-net keras model
Tustison NJ
## Not run: model <- createHyperMapp3rUnetModel3D( c( 2, 224, 224, 224 ) ) ## End(Not run)## Not run: model <- createHyperMapp3rUnetModel3D( c( 2, 224, 224, 224 ) ) ## End(Not run)
Implementation of the U-net architecture for hypothalamus segmentation described in
createHypothalamusUnetModel3D(inputImageSize)createHypothalamusUnetModel3D(inputImageSize)
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions only (number of channels is 1). |
https://pubmed.ncbi.nlm.nih.gov/32853816/
and ported from the original implementation:
https://github.com/BBillot/hypothalamus_seg
The network has is characterized by the following parameters:
3 resolution levels: 24 —> 48 —> 96 filters
convolution: kernel size: c(3, 3, 3), activation: 'elu'
pool size: c(2, 2, 2)
a u-net keras model
Tustison NJ
# Simple examples, must run successfully and quickly. These will be tested. library( ANTsRNet ) model <- createHypothalamusUnetModel3d( c( 160, 160, 160 ) ) print( model )# Simple examples, must run successfully and quickly. These will be tested. library( ANTsRNet ) model <- createHypothalamusUnetModel3d( c( 160, 160, 160 ) ) print( model )
Creates a keras model of the image super resolution deep learning framework. based on the paper available here:
createImageSuperResolutionModel2D( inputImageSize, convolutionKernelSizes = list(c(9, 9), c(1, 1), c(5, 5)), numberOfFilters = c(64, 32) )createImageSuperResolutionModel2D( inputImageSize, convolutionKernelSizes = list(c(9, 9), c(1, 1), c(5, 5)), numberOfFilters = c(64, 32) )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
convolutionKernelSizes |
a list of 2-D vectors specifying the kernel size at each convolution layer. Default values are the same as given in the original paper. The length of kernel size vectors must be 1 greater than the vector length of the number of filters. |
numberOfFilters |
a vector containing the number of filters for each convolutional layer. Default values are the same as given in the original paper. |
\url{https://arxiv.org/pdf/1501.00092}
This particular implementation is based on the following python implementation:
\url{https://github.com/titu1994/Image-Super-Resolution}
a keras model for image super resolution
Tustison NJ
createImageSuperResolutionModel2D(c( 100, 100, 1 )) gc()createImageSuperResolutionModel2D(c( 100, 100, 1 )) gc()
Creates a keras model of the image super resolution deep learning framework. based on the paper available here:
createImageSuperResolutionModel3D( inputImageSize, convolutionKernelSizes = list(c(9, 9, 9), c(1, 1, 1), c(5, 5, 5)), numberOfFilters = c(64, 32) )createImageSuperResolutionModel3D( inputImageSize, convolutionKernelSizes = list(c(9, 9, 9), c(1, 1, 1), c(5, 5, 5)), numberOfFilters = c(64, 32) )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
convolutionKernelSizes |
a list of 3-D vectors specifying the kernel size at each convolution layer. Default values are the same as given in the original paper. The length of kernel size vectors must be 1 greater than the vector length of the number of filters. |
numberOfFilters |
a vector containing the number of filters for each convolutional layer. Default values are the same as given in the original paper. |
\url{https://arxiv.org/pdf/1501.00092}
This particular implementation is based on the following python implementation:
\url{https://github.com/titu1994/Image-Super-Resolution}
a keras model for image super resolution
Tustison NJ
createImageSuperResolutionModel3D(c( 100, 100, 100, 1 ))createImageSuperResolutionModel3D(c( 100, 100, 100, 1 ))
Creates a keras model implementation of the u-net architecture avaialable here:
createNoBrainerUnetModel3D(inputImageSize)createNoBrainerUnetModel3D(inputImageSize)
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). |
\url{https://github.com/neuronets/nobrainer/}
a u-net keras model
Tustison NJ
library( ANTsRNet ) model <- createNoBrainerUnetModel3D( list( NULL, NULL, NULL, 1 ) )library( ANTsRNet ) model <- createNoBrainerUnetModel3D( list( NULL, NULL, NULL, 1 ) )
\url{https://arxiv.org/abs/1804.07723}
createPartialConvolutionUnetModel2D( inputImageSize, numberOfPriors = 0, numberOfFilters = c(64, 128, 256, 512, 512, 512, 512, 512), kernelSize = c(7, 5, 5, 3, 3, 3, 3, 3), usePartialConv = TRUE )createPartialConvolutionUnetModel2D( inputImageSize, numberOfPriors = 0, numberOfFilters = c(64, 128, 256, 512, 512, 512, 512, 512), kernelSize = c(7, 5, 5, 3, 3, 3, 3, 3), usePartialConv = TRUE )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfPriors |
Specify tissue priors for use during the decoding branch. Default = 0. |
numberOfFilters |
vector explicitly setting the number of filters at each layer. Defaults to number used in the paper. |
kernelSize |
single scalar or tuple of same length as the number of filters. Specifies the kernel size schedule for the encoding path. Defaults to the kernel sizes used in the paper. |
usePartialConv |
boolean. Testing. Switch between vanilla convolution layers and partial convolution layers. |
a u-net keras model
Tustison NJ
\url{https://arxiv.org/abs/1804.07723}
createPartialConvolutionUnetModel3D( inputImageSize, numberOfPriors = 0, numberOfFilters = c(64, 128, 256, 512, 512, 512, 512, 512), kernelSize = c(7, 5, 5, 3, 3, 3, 3, 3), usePartialConv = TRUE )createPartialConvolutionUnetModel3D( inputImageSize, numberOfPriors = 0, numberOfFilters = c(64, 128, 256, 512, 512, 512, 512, 512), kernelSize = c(7, 5, 5, 3, 3, 3, 3, 3), usePartialConv = TRUE )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfPriors |
Specify tissue priors for use during the decoding branch. Default = 0. |
numberOfFilters |
vector explicitly setting the number of filters at each layer. Defaults to number used in the paper. |
kernelSize |
single scalar or tuple of same length as the number of filters. Specifies the kernel size schedule for the encoding path. Defaults to the kernel sizes used in the paper. |
usePartialConv |
boolean. Testing. Switch between vanilla convolution layers and partial convolution layers. |
a u-net keras model
Tustison NJ
Creates a keras model of the ResNet deep learning architecture for image classification. The paper is available here:
createResNetModel2D( inputImageSize, inputScalarsSize = 0, numberOfOutputs = 1000, layers = 1:4, residualBlockSchedule = c(3, 4, 6, 3), lowestResolution = 64, cardinality = 1, squeezeAndExcite = FALSE, mode = c("classification", "sigmoid", "regression") )createResNetModel2D( inputImageSize, inputScalarsSize = 0, numberOfOutputs = 1000, layers = 1:4, residualBlockSchedule = c(3, 4, 6, 3), lowestResolution = 64, cardinality = 1, squeezeAndExcite = FALSE, mode = c("classification", "sigmoid", "regression") )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
inputScalarsSize |
Optional integer specifying the size of the input vector for scalars that get concatenated to the fully connected layer at the end of the network. |
numberOfOutputs |
Specifies number of units in final layer |
layers |
a vector determining the number of 'filters' defined at for each layer. |
residualBlockSchedule |
vector defining the how many residual blocks repeats. |
lowestResolution |
number of filters at the initial layer. |
cardinality |
perform ResNet (cardinality = 1) or ResNeXt (cardinality != 1 but powers of 2—try '32' ) |
squeezeAndExcite |
boolean to add the squeeze-and-excite block variant. |
mode |
'classification', 'sigmoid' or 'regression'. |
https://arxiv.org/abs/1512.03385
This particular implementation was influenced by the following python implementation:
https://gist.github.com/mjdietzx/0cb95922aac14d446a6530f87b3a04ce
an ResNet keras model
Tustison NJ
library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:10,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:10], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createResNetModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) # Comment out the rest due to travis build constraints # track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, # epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model # testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) # predictedData <- model %>% predict( X_testSmall, verbose = 1 )library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:10,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:10], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createResNetModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) # Comment out the rest due to travis build constraints # track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, # epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model # testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) # predictedData <- model %>% predict( X_testSmall, verbose = 1 )
Creates a keras model of the ResNet deep learning architecture for image classification. The paper is available here:
createResNetModel3D( inputImageSize, inputScalarsSize = 0, numberOfOutputs = 1000, layers = 1:4, residualBlockSchedule = c(3, 4, 6, 3), lowestResolution = 64, cardinality = 1, squeezeAndExcite = FALSE, mode = c("classification", "regression") )createResNetModel3D( inputImageSize, inputScalarsSize = 0, numberOfOutputs = 1000, layers = 1:4, residualBlockSchedule = c(3, 4, 6, 3), lowestResolution = 64, cardinality = 1, squeezeAndExcite = FALSE, mode = c("classification", "regression") )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
inputScalarsSize |
Optional integer specifying the size of the input vector for scalars that get concatenated to the fully connected layer at the end of the network. |
numberOfOutputs |
Number of segmentation labels. |
layers |
a vector determining the number of 'filters' defined at for each layer. |
residualBlockSchedule |
vector defining the how many residual blocks repeats. |
lowestResolution |
number of filters at the initial layer. |
cardinality |
perform ResNet (cardinality = 1) or ResNeXt (cardinality != 1 but powers of 2—try '32' ) |
squeezeAndExcite |
boolean to add the squeeze-and-excite block variant. |
mode |
'classification', 'sigmoid' or 'regression'. |
https://arxiv.org/abs/1512.03385
This particular implementation was influenced by the following python implementation:
https://gist.github.com/mjdietzx/0cb95922aac14d446a6530f87b3a04ce
an ResNet keras model
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:10,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:10], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createResNetModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) predictedData <- model %>% predict( X_testSmall, verbose = 1 ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:10,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:10], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createResNetModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) predictedData <- model %>% predict( X_testSmall, verbose = 1 ) ## End(Not run)
Creates a keras model of the expanded image super resolution deep learning framework based on the following python implementation:
createResNetSuperResolutionModel2D( inputImageSize, convolutionKernelSize = c(3, 3), numberOfFilters = 64, numberOfResidualBlocks = 5, numberOfResNetBlocks = 1 )createResNetSuperResolutionModel2D( inputImageSize, convolutionKernelSize = c(3, 3), numberOfFilters = 64, numberOfResidualBlocks = 5, numberOfResNetBlocks = 1 )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
convolutionKernelSize |
a vector specifying the kernel size for convolution. |
numberOfFilters |
the number of filters for each encoding layer. |
numberOfResidualBlocks |
the number of residual blocks. |
numberOfResNetBlocks |
the number of resNet blocks. Each block will double the upsampling amount. |
\url{https://github.com/titu1994/Image-Super-Resolution}
a keras model for ResNet image super resolution
Tustison NJ
createResNetSuperResolutionModel2D(c(256, 256, 3))createResNetSuperResolutionModel2D(c(256, 256, 3))
Creates a keras model of the expanded image super resolution deep learning framework based on the following python implementation:
createResNetSuperResolutionModel3D( inputImageSize, convolutionKernelSize = c(3, 3, 3), numberOfFilters = 64, numberOfResidualBlocks = 5, numberOfResNetBlocks = 1 )createResNetSuperResolutionModel3D( inputImageSize, convolutionKernelSize = c(3, 3, 3), numberOfFilters = 64, numberOfResidualBlocks = 5, numberOfResNetBlocks = 1 )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
convolutionKernelSize |
a vector specifying the kernel size for convolution. |
numberOfFilters |
the number of filters for each encoding layer. |
numberOfResidualBlocks |
the number of residual blocks. |
numberOfResNetBlocks |
the number of resNet blocks. Each block will double the upsampling amount. |
\url{https://github.com/titu1994/Image-Super-Resolution}
a keras model for ResNet image super resolution
Tustison NJ
inputImageSize = c(256, 256, 30, 1) createResNetSuperResolutionModel3D(inputImageSize)inputImageSize = c(256, 256, 30, 1) createResNetSuperResolutionModel3D(inputImageSize)
Creates a keras model of the ResNet deep learning architecture for image classification with a spatial transformer network (STN) layer. The paper is available here:
createResNetWithSpatialTransformerNetworkModel2D( inputImageSize, numberOfOutputs = 1000, layers = 1:4, residualBlockSchedule = c(3, 4, 6, 3), lowestResolution = 64, cardinality = 1, numberOfSpatialTransformerUnits = 50, resampledSize = c(64, 64), mode = c("classification", "regression") )createResNetWithSpatialTransformerNetworkModel2D( inputImageSize, numberOfOutputs = 1000, layers = 1:4, residualBlockSchedule = c(3, 4, 6, 3), lowestResolution = 64, cardinality = 1, numberOfSpatialTransformerUnits = 50, resampledSize = c(64, 64), mode = c("classification", "regression") )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Specifies number of units in final layer |
layers |
a vector determining the number of 'filters' defined at for each layer. |
residualBlockSchedule |
vector defining the how many residual blocks repeats. |
lowestResolution |
number of filters at the initial layer. |
cardinality |
perform ResNet (cardinality = 1) or ResNeXt (cardinality != 1 but powers of 2—try '32' ) |
numberOfSpatialTransformerUnits |
number of units in the dense layer. |
resampledSize |
output image size of the spatial transformer network. |
mode |
'classification' or 'regression'. |
https://arxiv.org/abs/1512.03385
an STN + ResNet keras model
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:10,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:10], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createResNetWithSpatialTransformerNetworkModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) # Comment out the rest due to travis build constraints # track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, # epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model # testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) # predictedData <- model %>% predict( X_testSmall, verbose = 1 ) rm(model); gc() ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:10,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:10], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createResNetWithSpatialTransformerNetworkModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) # Comment out the rest due to travis build constraints # track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, # epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model # testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) # predictedData <- model %>% predict( X_testSmall, verbose = 1 ) rm(model); gc() ## End(Not run)
Creates a keras model of the ResNet deep learning architecture for image classification with a spatial transformer network (STN) layer. The paper is available here:
createResNetWithSpatialTransformerNetworkModel3D( inputImageSize, numberOfOutputs = 1000, layers = 1:4, residualBlockSchedule = c(3, 4, 6, 3), lowestResolution = 64, cardinality = 1, resampledSize = c(64, 64, 64), numberOfSpatialTransformerUnits = 50, mode = c("classification", "regression") )createResNetWithSpatialTransformerNetworkModel3D( inputImageSize, numberOfOutputs = 1000, layers = 1:4, residualBlockSchedule = c(3, 4, 6, 3), lowestResolution = 64, cardinality = 1, resampledSize = c(64, 64, 64), numberOfSpatialTransformerUnits = 50, mode = c("classification", "regression") )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Number of segmentation labels. |
layers |
a vector determining the number of 'filters' defined at for each layer. |
residualBlockSchedule |
vector defining the how many residual blocks repeats. |
lowestResolution |
number of filters at the initial layer. |
cardinality |
perform ResNet (cardinality = 1) or ResNeXt (cardinality != 1 but powers of 2—try '32' ) |
resampledSize |
output image size of the spatial transformer network. |
numberOfSpatialTransformerUnits |
number of units in the dense layer. |
mode |
'classification' or 'regression'. Default = 'classification'. |
https://arxiv.org/abs/1512.03385
an ResNet keras model
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:10,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:10], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createResNetWithSpatialTransformerNetworkModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) predictedData <- model %>% predict( X_testSmall, verbose = 1 ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:10,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:10], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createResNetWithSpatialTransformerNetworkModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) predictedData <- model %>% predict( X_testSmall, verbose = 1 ) ## End(Not run)
Creates a keras model of the U-net + ResNet deep learning architecture for image segmentation and regression with the paper available here:
createResUnetModel2D( inputImageSize, numberOfOutputs = 1, numberOfFiltersAtBaseLayer = 32, bottleNeckBlockDepthSchedule = c(3, 4), convolutionKernelSize = c(3, 3), deconvolutionKernelSize = c(2, 2), dropoutRate = 0, weightDecay = 1e-04, mode = c("classification", "regression") )createResUnetModel2D( inputImageSize, numberOfOutputs = 1, numberOfFiltersAtBaseLayer = 32, bottleNeckBlockDepthSchedule = c(3, 4), convolutionKernelSize = c(3, 3), deconvolutionKernelSize = c(2, 2), dropoutRate = 0, weightDecay = 1e-04, mode = c("classification", "regression") )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Meaning depends on the |
numberOfFiltersAtBaseLayer |
number of filters at the beginning and end
of the |
bottleNeckBlockDepthSchedule |
vector that provides the encoding layer schedule for the number of bottleneck blocks per long skip connection. |
convolutionKernelSize |
2-d vector defining the kernel size during the encoding path |
deconvolutionKernelSize |
2-d vector defining the kernel size during the decoding |
dropoutRate |
float between 0 and 1 to use between dense layers. |
weightDecay |
weighting parameter for L2 regularization of the kernel weights of the convolution layers. Default = 0.0. |
mode |
'classification' or 'regression'. |
\url{https://arxiv.org/abs/1608.04117}
This particular implementation was ported from the following python implementation:
\url{https://github.com/veugene/fcn_maker/}
a res/u-net keras model
Tustison NJ
library( ANTsR ) library( ANTsRNet ) library( keras ) imageIDs <- c( "r16", "r27", "r30", "r62", "r64", "r85" ) trainingBatchSize <- length( imageIDs ) # Perform simple 3-tissue segmentation. segmentationLabels <- c( 1, 2, 3 ) numberOfLabels <- length( segmentationLabels ) initialization <- paste0( 'KMeans[', numberOfLabels, ']' ) domainImage <- antsImageRead( getANTsRData( imageIDs[1] ) ) X_train <- array( data = NA, dim = c( trainingBatchSize, dim( domainImage ), 1 ) ) Y_train <- array( data = NA, dim = c( trainingBatchSize, dim( domainImage ) ) ) images <- list() segmentations <- list() for( i in seq_len( trainingBatchSize ) ) { cat( "Processing image", imageIDs[i], "\n" ) image <- antsImageRead( getANTsRData( imageIDs[i] ) ) mask <- getMask( image ) segmentation <- atropos( image, mask, initialization )$segmentation X_train[i,,, 1] <- as.array( image ) Y_train[i,,] <- as.array( segmentation ) } Y_train <- encodeUnet( Y_train, segmentationLabels ) # Perform a simple normalization X_train <- ( X_train - mean( X_train ) ) / sd( X_train ) # Create the model model <- createResUnetModel2D( c( dim( domainImage ), 1 ), numberOfOutputs = numberOfLabels ) rm(domainImage); gc() metric_multilabel_dice_coefficient <- custom_metric( "multilabel_dice_coefficient", multilabel_dice_coefficient ) loss_dice <- function( y_true, y_pred ) { -multilabel_dice_coefficient(y_true, y_pred) } attr(loss_dice, "py_function_name") <- "multilabel_dice_coefficient" model %>% compile( loss = loss_dice, optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( metric_multilabel_dice_coefficient, metric_categorical_crossentropy ) ) # Comment out the rest due to travis build constraints # Fit the model # track <- model %>% fit( X_train, Y_train, # epochs = 100, batch_size = 4, verbose = 1, shuffle = TRUE, # callbacks = list( # callback_model_checkpoint( "resUnetModelInterimWeights.h5", # monitor = 'val_loss', save_best_only = TRUE ), # callback_reduce_lr_on_plateau( monitor = "val_loss", factor = 0.1 ) # ), # validation_split = 0.2 ) rm(X_train); gc() rm(Y_train); gc() # Save the model and/or save the model weights # save_model_hdf5( model, filepath = 'resUnetModel.h5' ) # save_model_weights_hdf5( unetModel, filepath = 'resUnetModelWeights.h5' ) ) rm(model); gc()library( ANTsR ) library( ANTsRNet ) library( keras ) imageIDs <- c( "r16", "r27", "r30", "r62", "r64", "r85" ) trainingBatchSize <- length( imageIDs ) # Perform simple 3-tissue segmentation. segmentationLabels <- c( 1, 2, 3 ) numberOfLabels <- length( segmentationLabels ) initialization <- paste0( 'KMeans[', numberOfLabels, ']' ) domainImage <- antsImageRead( getANTsRData( imageIDs[1] ) ) X_train <- array( data = NA, dim = c( trainingBatchSize, dim( domainImage ), 1 ) ) Y_train <- array( data = NA, dim = c( trainingBatchSize, dim( domainImage ) ) ) images <- list() segmentations <- list() for( i in seq_len( trainingBatchSize ) ) { cat( "Processing image", imageIDs[i], "\n" ) image <- antsImageRead( getANTsRData( imageIDs[i] ) ) mask <- getMask( image ) segmentation <- atropos( image, mask, initialization )$segmentation X_train[i,,, 1] <- as.array( image ) Y_train[i,,] <- as.array( segmentation ) } Y_train <- encodeUnet( Y_train, segmentationLabels ) # Perform a simple normalization X_train <- ( X_train - mean( X_train ) ) / sd( X_train ) # Create the model model <- createResUnetModel2D( c( dim( domainImage ), 1 ), numberOfOutputs = numberOfLabels ) rm(domainImage); gc() metric_multilabel_dice_coefficient <- custom_metric( "multilabel_dice_coefficient", multilabel_dice_coefficient ) loss_dice <- function( y_true, y_pred ) { -multilabel_dice_coefficient(y_true, y_pred) } attr(loss_dice, "py_function_name") <- "multilabel_dice_coefficient" model %>% compile( loss = loss_dice, optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( metric_multilabel_dice_coefficient, metric_categorical_crossentropy ) ) # Comment out the rest due to travis build constraints # Fit the model # track <- model %>% fit( X_train, Y_train, # epochs = 100, batch_size = 4, verbose = 1, shuffle = TRUE, # callbacks = list( # callback_model_checkpoint( "resUnetModelInterimWeights.h5", # monitor = 'val_loss', save_best_only = TRUE ), # callback_reduce_lr_on_plateau( monitor = "val_loss", factor = 0.1 ) # ), # validation_split = 0.2 ) rm(X_train); gc() rm(Y_train); gc() # Save the model and/or save the model weights # save_model_hdf5( model, filepath = 'resUnetModel.h5' ) # save_model_weights_hdf5( unetModel, filepath = 'resUnetModelWeights.h5' ) ) rm(model); gc()
Creates a keras model of the U-net + ResNet deep learning architecture for image segmentation and regression with the paper available here:
createResUnetModel3D( inputImageSize, numberOfOutputs = 1, numberOfFiltersAtBaseLayer = 32, bottleNeckBlockDepthSchedule = c(3, 4), convolutionKernelSize = c(3, 3, 3), deconvolutionKernelSize = c(2, 2, 2), dropoutRate = 0, weightDecay = 1e-04, mode = c("classification", "regression") )createResUnetModel3D( inputImageSize, numberOfOutputs = 1, numberOfFiltersAtBaseLayer = 32, bottleNeckBlockDepthSchedule = c(3, 4), convolutionKernelSize = c(3, 3, 3), deconvolutionKernelSize = c(2, 2, 2), dropoutRate = 0, weightDecay = 1e-04, mode = c("classification", "regression") )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Meaning depends on the |
numberOfFiltersAtBaseLayer |
number of filters at the beginning and end
of the |
bottleNeckBlockDepthSchedule |
vector that provides the encoding layer schedule for the number of bottleneck blocks per long skip connection. |
convolutionKernelSize |
2-d vector defining the kernel size during the encoding path |
deconvolutionKernelSize |
2-d vector defining the kernel size during the decoding |
dropoutRate |
float between 0 and 1 to use between dense layers. |
weightDecay |
weighting parameter for L2 regularization of the kernel weights of the convolution layers. Default = 0.0. |
mode |
'classification' or 'regression'. |
\url{https://arxiv.org/abs/1608.04117}
This particular implementation was ported from the following python implementation:
\url{https://github.com/veugene/fcn_maker/}
a res/u-net keras model
Tustison NJ
library( ANTsRNet ) library( keras ) model <- createResUnetModel3D( c( 64, 64, 64, 1 ) ) metric_multilabel_dice_coefficient <- custom_metric( "multilabel_dice_coefficient", multilabel_dice_coefficient ) loss_dice <- function( y_true, y_pred ) { -multilabel_dice_coefficient(y_true, y_pred) } attr(loss_dice, "py_function_name") <- "multilabel_dice_coefficient" model %>% compile( loss = loss_dice, optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( metric_multilabel_dice_coefficient, metric_categorical_crossentropy ) ) print( model )library( ANTsRNet ) library( keras ) model <- createResUnetModel3D( c( 64, 64, 64, 1 ) ) metric_multilabel_dice_coefficient <- custom_metric( "multilabel_dice_coefficient", multilabel_dice_coefficient ) loss_dice <- function( y_true, y_pred ) { -multilabel_dice_coefficient(y_true, y_pred) } attr(loss_dice, "py_function_name") <- "multilabel_dice_coefficient" model %>% compile( loss = loss_dice, optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( metric_multilabel_dice_coefficient, metric_categorical_crossentropy ) ) print( model )
Creates a keras model implementation of the model:
createRmnetGenerator(numberOfChannels = 3)createRmnetGenerator(numberOfChannels = 3)
\url{https://github.com/Jireh-Jam/R-MNet-Inpainting-keras}
a keras model
Tustison NJ
library( ANTsRNet ) model <- createRmnetGenerator()library( ANTsRNet ) model <- createRmnetGenerator()
Publications:
* PVS: https://pubmed.ncbi.nlm.nih.gov/34262443/ * WMH: https://pubmed.ncbi.nlm.nih.gov/38050769/
createShivaUnetModel3D(numberOfModalities = 1, numberOfOutputs = 1)createShivaUnetModel3D(numberOfModalities = 1, numberOfOutputs = 1)
numberOfModalities |
Specifies number of channels in the architecture. |
numberOfOutputs |
Specifies the number of outputs per voxel. Determines final activation function (1 = sigmoid, >1 = softmax). |
with respective GitHub repositories:
* PVS: https://github.com/pboutinaud/SHIVA_PVS * WMH: https://github.com/pboutinaud/SHIVA_WMH
a u-net keras model
Tustison NJ
## Not run: model <- createShivaUnetModel3D() ## End(Not run)## Not run: model <- createShivaUnetModel3D() ## End(Not run)
Creates a keras model of the spatial transformer network:
createSimpleClassificationWithSpatialTransformerNetworkModel2D( inputImageSize, resampledSize = c(30, 30), numberOfClassificationLabels = 10 )createSimpleClassificationWithSpatialTransformerNetworkModel2D( inputImageSize, resampledSize = c(30, 30), numberOfClassificationLabels = 10 )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
resampledSize |
resampled size of the transformed input images. |
numberOfClassificationLabels |
Number of classes. |
\url{https://arxiv.org/abs/1506.02025}
based on the following python Keras model:
\url{https://github.com/oarriaga/STN.keras/blob/master/src/models/STN.py}
a keras model
Tustison NJ
library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:100,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:100], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) ## Not run: model <- createSimpleClassificationWithSpatialTransformerNetworkModel2D( inputImageSize = inputImageSize, resampledSize = c( 30, 30 ), numberOfClassificationLabels = numberOfLabels ) ## End(Not run)library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:100,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:100], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) ## Not run: model <- createSimpleClassificationWithSpatialTransformerNetworkModel2D( inputImageSize = inputImageSize, resampledSize = c( 30, 30 ), numberOfClassificationLabels = numberOfLabels ) ## End(Not run)
Creates a keras model of the spatial transformer network:
createSimpleClassificationWithSpatialTransformerNetworkModel3D( inputImageSize, resampledSize = c(30, 30, 30), numberOfClassificationLabels = 10 )createSimpleClassificationWithSpatialTransformerNetworkModel3D( inputImageSize, resampledSize = c(30, 30, 30), numberOfClassificationLabels = 10 )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
resampledSize |
resampled size of the transformed input images. |
numberOfClassificationLabels |
Number of classes. |
\url{https://arxiv.org/abs/1506.02025}
based on the following python Keras model:
\url{https://github.com/oarriaga/STN.keras/blob/master/src/models/STN.py}
a keras model
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:100,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:100], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createSimpleClassificationWithSpatialTransformerNetworkModel2D( inputImageSize = inputImageSize, resampledSize = c( 30, 30 ), numberOfClassificationLabels = numberOfLabels ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:100,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:100], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createSimpleClassificationWithSpatialTransformerNetworkModel2D( inputImageSize = inputImageSize, resampledSize = c( 30, 30 ), numberOfClassificationLabels = numberOfLabels ) ## End(Not run)
Creates a keras model implementation of the Simple Fully Convolutional Network model from the FMRIB group:
createSimpleFullyConvolutionalNeuralNetworkModel3D( inputImageSize, numberOfFiltersPerLayer = c(32, 64, 128, 256, 256, 64), numberOfBins = 40, dropoutRate = 0.5, doExperimentalVariant = FALSE )createSimpleFullyConvolutionalNeuralNetworkModel3D( inputImageSize, numberOfFiltersPerLayer = c(32, 64, 128, 256, 256, 64), numberOfBins = 40, dropoutRate = 0.5, doExperimentalVariant = FALSE )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). |
numberOfFiltersPerLayer |
number of filters for the convolutional layers |
numberOfBins |
number of bins for final softmax output. |
dropoutRate |
dropout rate before final convolution layer. |
\url{https://github.com/ha-ha-ha-han/UKBiobank_deep_pretrain}
a SCFN keras model
Tustison NJ
library( ANTsRNet ) model <- createSimpleFullyConvolutionalNeuralNetworkModel3D( list( NULL, NULL, NULL, 1 ) )library( ANTsRNet ) model <- createSimpleFullyConvolutionalNeuralNetworkModel3D( list( NULL, NULL, NULL, 1 ) )
Creates a keras model of the SSD 7 deep learning architecture for object detection based on the paper
createSsd7Model2D( inputImageSize, numberOfOutputs, minScale = 0.08, maxScale = 0.96, aspectRatiosPerLayer = list(c("1:1", "2:1", "1:2"), c("1:1", "2:1", "1:2"), c("1:1", "2:1", "1:2"), c("1:1", "2:1", "1:2")), variances = rep(1, 4) )createSsd7Model2D( inputImageSize, numberOfOutputs, minScale = 0.08, maxScale = 0.96, aspectRatiosPerLayer = list(c("1:1", "2:1", "1:2"), c("1:1", "2:1", "1:2"), c("1:1", "2:1", "1:2"), c("1:1", "2:1", "1:2")), variances = rep(1, 4) )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Number of classification labels. Needs to include the background as one of the labels. |
minScale |
The smallest scaling factor for the size of the anchor boxes as a fraction of the shorter side of the input images. |
maxScale |
The largest scaling factor for the size of the anchor boxes as a fraction of the shorter side of the input images. All scaling factors between the smallest and the largest are linearly interpolated. |
aspectRatiosPerLayer |
A list containing one aspect ratio list for
each predictor layer. #' each predictor layer. The default lists follows
the original implementation except each aspect ratio is defined as a
character string (e.g., |
variances |
A list of 4 floats > 0 with scaling factors for the encoded
predicted box coordinates. A variance value of 1.0 would apply no scaling at
all to the predictions, while values in (0, 1) upscale the encoded predictions
and values greater than 1.0 downscale the encoded predictions. Defaults to
|
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C-Y. Fu, A. Berg. SSD: Single Shot MultiBox Detector.
available here:
\url{https://arxiv.org/abs/1512.02325}
This particular implementation was influenced by the following python and R implementations:
\url{https://github.com/pierluigiferrari/ssd_keras}
\url{https://github.com/gsimchoni/ssdkeras}
an SSD keras model
Tustison NJ
tensorflow::tf$compat$v1$disable_eager_execution() createSsd7Model2D(c(250, 250, 3), 2) createSsd7Model3D(c(250, 250, 250, 3), 2)tensorflow::tf$compat$v1$disable_eager_execution() createSsd7Model2D(c(250, 250, 3), 2) createSsd7Model3D(c(250, 250, 250, 3), 2)
Creates a keras model of the SSD 7 deep learning architecture for object detection based on the paper
createSsd7Model3D( inputImageSize, numberOfOutputs, minScale = 0.08, maxScale = 0.96, aspectRatiosPerLayer = list(c("1:1:1", "2:1:1", "1:2:1", "1:1:2"), c("1:1:1", "2:1:1", "1:2:1", "1:1:2"), c("1:1:1", "2:1:1", "1:2:1", "1:1:2"), c("1:1:1", "2:1:1", "1:2:1", "1:1:2")), variances = rep(1, 6) )createSsd7Model3D( inputImageSize, numberOfOutputs, minScale = 0.08, maxScale = 0.96, aspectRatiosPerLayer = list(c("1:1:1", "2:1:1", "1:2:1", "1:1:2"), c("1:1:1", "2:1:1", "1:2:1", "1:1:2"), c("1:1:1", "2:1:1", "1:2:1", "1:1:2"), c("1:1:1", "2:1:1", "1:2:1", "1:1:2")), variances = rep(1, 6) )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Number of classification labels. Needs to include the background as one of the labels. |
minScale |
The smallest scaling factor for the size of the anchor boxes as a fraction of the shorter side of the input images. |
maxScale |
The largest scaling factor for the size of the anchor boxes as a fraction of the shorter side of the input images. All scaling factors between the smallest and the largest are linearly interpolated. |
aspectRatiosPerLayer |
A list containing one aspect ratio list for
each predictor layer. The default lists follows the original
implementation except each aspect ratio is defined as a character string
(e.g., |
variances |
A list of 6 floats > 0 with scaling factors for the encoded
predicted box coordinates. A variance value of 1.0 would apply no scaling at
all to the predictions, while values in (0, 1) upscale the encoded predictions
and values greater than 1.0 downscale the encoded predictions. Defaults to
|
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C-Y. Fu, A. Berg. SSD: Single Shot MultiBox Detector.
available here:
\url{https://arxiv.org/abs/1512.02325}
This particular implementation was influenced by the following python and R implementations:
\url{https://github.com/pierluigiferrari/ssd_keras}
\url{https://github.com/gsimchoni/ssdkeras}
an SSD keras model
Tustison NJ
## Not run: createSsd7Model3D(c(256, 256, 100, 3), 2) ## End(Not run)## Not run: createSsd7Model3D(c(256, 256, 100, 3), 2) ## End(Not run)
Creates a keras model of the SSD deep learning architecture for object detection based on the paper
createSsdModel2D( inputImageSize, numberOfOutputs, l2Regularization = 5e-04, minScale = 0.1, maxScale = 0.9, aspectRatiosPerLayer = list(c("1:1", "2:1", "1:2"), c("1:1", "2:1", "1:2", "3:1", "1:3"), c("1:1", "2:1", "1:2", "3:1", "1:3"), c("1:1", "2:1", "1:2", "3:1", "1:3"), c("1:1", "2:1", "1:2"), c("1:1", "2:1", "1:2")), variances = rep(1, 4), style = 300 )createSsdModel2D( inputImageSize, numberOfOutputs, l2Regularization = 5e-04, minScale = 0.1, maxScale = 0.9, aspectRatiosPerLayer = list(c("1:1", "2:1", "1:2"), c("1:1", "2:1", "1:2", "3:1", "1:3"), c("1:1", "2:1", "1:2", "3:1", "1:3"), c("1:1", "2:1", "1:2", "3:1", "1:3"), c("1:1", "2:1", "1:2"), c("1:1", "2:1", "1:2")), variances = rep(1, 4), style = 300 )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Number of classification labels. Needs to include the background as one of the labels. |
l2Regularization |
The L2-regularization rate. Default = 0.0005. |
minScale |
The smallest scaling factor for the size of the anchor boxes as a fraction of the shorter side of the input images. |
maxScale |
The largest scaling factor for the size of the anchor boxes as a fraction of the shorter side of the input images. All scaling factors between the smallest and the largest are linearly interpolated. |
aspectRatiosPerLayer |
A list containing one aspect ratio list for
each predictor layer. The default lists follows the original
implementation except each aspect ratio is defined as a character string
(e.g. |
variances |
A list of 4 floats > 0 with scaling factors for the encoded predicted box coordinates. A variance value of 1.0 would apply no scaling at all to the predictions, while values in (0,1) upscale the encoded predictions and values greater than 1.0 downscale the encoded predictions. Defaults to 1.0. |
style |
300 or 512 |
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C-Y. Fu, A. Berg. SSD: Single Shot MultiBox Detector.
available here:
https://arxiv.org/abs/1512.02325
This particular implementation was influenced by the following python and R implementations:
https://github.com/pierluigiferrari/ssd_keras
https://github.com/rykov8/ssd_keras
https://github.com/gsimchoni/ssdkeras
an SSD keras model
Tustison NJ
Creates a keras model of the SSD deep learning architecture for object detection based on the paper
createSsdModel3D( inputImageSize, numberOfOutputs, l2Regularization = 5e-04, minScale = 0.1, maxScale = 0.9, aspectRatiosPerLayer = list(c("1:1:1", "2:1:1", "1:2:1", "1:1:2"), c("1:1:1", "2:1:1", "1:2:1", "1:1:2", "3:1:1", "1:3:1", "1:1:3"), c("1:1:1", "2:1:1", "1:2:1", "1:1:2", "3:1:1", "1:3:1", "1:1:3"), c("1:1:1", "2:1:1", "1:2:1", "1:1:2", "3:1:1", "1:3:1", "1:1:3"), c("1:1:1", "2:1:1", "1:2:1", "1:1:2"), c("1:1:1", "2:1:1", "1:2:1", "1:1:2")), variances = rep(1, 6), style = 300 )createSsdModel3D( inputImageSize, numberOfOutputs, l2Regularization = 5e-04, minScale = 0.1, maxScale = 0.9, aspectRatiosPerLayer = list(c("1:1:1", "2:1:1", "1:2:1", "1:1:2"), c("1:1:1", "2:1:1", "1:2:1", "1:1:2", "3:1:1", "1:3:1", "1:1:3"), c("1:1:1", "2:1:1", "1:2:1", "1:1:2", "3:1:1", "1:3:1", "1:1:3"), c("1:1:1", "2:1:1", "1:2:1", "1:1:2", "3:1:1", "1:3:1", "1:1:3"), c("1:1:1", "2:1:1", "1:2:1", "1:1:2"), c("1:1:1", "2:1:1", "1:2:1", "1:1:2")), variances = rep(1, 6), style = 300 )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Number of classification labels. Needs to include the background as one of the labels. |
l2Regularization |
The L2-regularization rate. Default = 0.0005. |
minScale |
The smallest scaling factor for the size of the anchor boxes as a fraction of the shorter side of the input images. |
maxScale |
The largest scaling factor for the size of the anchor boxes as a fraction of the shorter side of the input images. All scaling factors between the smallest and the largest are linearly interpolated. |
aspectRatiosPerLayer |
A list containing one aspect ratio list for
each predictor layer. The default lists follows the original
implementation except each aspect ratio is defined as a character string
(e.g. |
variances |
A list of 6 floats > 0 with scaling factors for the encoded predicted box coordinates. A variance value of 1.0 would apply no scaling at all to the predictions, while values in (0,1) upscale the encoded predictions and values greater than 1.0 downscale the encoded predictions. Defaults to 1.0. |
style |
300 or 512 |
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C-Y. Fu, A. Berg. SSD: Single Shot MultiBox Detector.
available here:
\url{https://arxiv.org/abs/1512.02325}
This particular implementation was influenced by the following python and R implementations:
https://github.com/pierluigiferrari/ssd_keras
https://github.com/rykov8/ssd_keras
https://github.com/gsimchoni/ssdkeras
an SSD keras model
Tustison NJ
Creates a keras model implementation of the u-net architecture in the 2017 MICCAI WMH challenge by the sysu_medial team described here:
createSysuMediaUnetModel2D(inputImageSize, anatomy = c("wmh", "claustrum"))createSysuMediaUnetModel2D(inputImageSize, anatomy = c("wmh", "claustrum"))
inputImageSize |
Used for specifying the input tensor shape.
This will be |
anatomy |
"wmh" or "claustrum". |
\url{https://pubmed.ncbi.nlm.nih.gov/30125711/}
or targeting the claustrum:
\url{https://arxiv.org/abs/2008.03465}
with the original implementations available at
\url{https://github.com/hongweilibran/wmh_ibbmTum}
and
\url{https://github.com/hongweilibran/claustrum_multi_view},
respectively.
a u-net keras model
Tustison NJ
## Not run: model <- createSysuMediaUnetModel2D( c( 200, 200, 1 ) ) ## End(Not run)## Not run: model <- createSysuMediaUnetModel2D( c( 200, 200, 1 ) ) ## End(Not run)
3-D variant of the sysu_media U-net architecture
createSysuMediaUnetModel3D( inputImageSize, numberOfFilters = NULL, anatomy = "wmh" )createSysuMediaUnetModel3D( inputImageSize, numberOfFilters = NULL, anatomy = "wmh" )
inputImageSize |
Used for specifying the input tensor shape.
This will be |
numberOfFilters |
specifies schedule for the encoding/decoding layers. |
anatomy |
"wmh" or "claustrum". Vestigial option from the original 2-D version. |
a u-net keras model
Tustison NJ
## Not run: model <- createSysuMediaUnetModel3D( c( 64, 64, 64, 2 ), numberOfFilters = c( 64, 128, 256, 512 ) ) ## End(Not run)## Not run: model <- createSysuMediaUnetModel3D( c( 64, 64, 64, 2 ), numberOfFilters = c( 64, 128, 256, 512 ) ) ## End(Not run)
Creates a keras model of the U-net deep learning architecture for image segmentation and regression. More information is provided at the authors' website:
createUnetModel2D( inputImageSize, numberOfOutputs = 2, scalarOutputSize = 0, scalarOutputActivation = "relu", numberOfLayers = 4, numberOfFiltersAtBaseLayer = 32, numberOfFilters = NULL, convolutionKernelSize = c(3, 3), deconvolutionKernelSize = c(2, 2), poolSize = c(2, 2), strides = c(2, 2), dropoutRate = 0, weightDecay = 0, mode = c("classification", "regression", "sigmoid"), additionalOptions = NA )createUnetModel2D( inputImageSize, numberOfOutputs = 2, scalarOutputSize = 0, scalarOutputActivation = "relu", numberOfLayers = 4, numberOfFiltersAtBaseLayer = 32, numberOfFilters = NULL, convolutionKernelSize = c(3, 3), deconvolutionKernelSize = c(2, 2), poolSize = c(2, 2), strides = c(2, 2), dropoutRate = 0, weightDecay = 0, mode = c("classification", "regression", "sigmoid"), additionalOptions = NA )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Meaning depends on the |
scalarOutputSize |
if greater than 0, a global average pooling from each encoding layer is concatenated to a dense layer as a secondary output. |
scalarOutputActivation |
activation for nonzero output scalar. |
numberOfLayers |
number of encoding/decoding layers. |
numberOfFiltersAtBaseLayer |
number of filters at the beginning and end
of the |
numberOfFilters |
vector explicitly setting the number of filters at
each layer. One can either set this or |
convolutionKernelSize |
2-d vector defining the kernel size during the encoding path. |
deconvolutionKernelSize |
2-d vector defining the kernel size during the decoding. |
poolSize |
2-d vector defining the region for each pooling layer. |
strides |
2-d vector describing the stride length in each direction. |
dropoutRate |
float between 0 and 1 to use between dense layers. |
weightDecay |
weighting parameter for L2 regularization of the kernel weights of the convolution layers. Default = 0.0. |
mode |
'classification' or 'regression' or 'sigmoid'. |
additionalOptions |
string or vector of strings specifying specific configuration add-ons/tweaks:
|
\url{https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/}
with the paper available here:
\url{https://arxiv.org/abs/1505.04597}
This particular implementation was influenced by the following python implementation:
\url{https://github.com/joelthelion/ultrasound-nerve-segmentation}
a u-net keras model
Tustison NJ
library( ANTsR ) library( ANTsRNet ) library( keras ) imageIDs <- c( "r16", "r27", "r30", "r62", "r64", "r85" ) trainingBatchSize <- length( imageIDs ) # Perform simple 3-tissue segmentation. segmentationLabels <- c( 1, 2, 3 ) numberOfLabels <- length( segmentationLabels ) initialization <- paste0( 'KMeans[', numberOfLabels, ']' ) domainImage <- antsImageRead( getANTsRData( imageIDs[1] ) ) X_train <- array( data = NA, dim = c( trainingBatchSize, dim( domainImage ), 1 ) ) Y_train <- array( data = NA, dim = c( trainingBatchSize, dim( domainImage ) ) ) images <- list() segmentations <- list() for( i in seq_len( trainingBatchSize ) ) { cat( "Processing image", imageIDs[i], "\n" ) image <- antsImageRead( getANTsRData( imageIDs[i] ) ) mask <- getMask( image ) segmentation <- atropos( image, mask, initialization )$segmentation X_train[i,,, 1] <- as.array( image ) Y_train[i,,] <- as.array( segmentation ) } Y_train <- encodeUnet( Y_train, segmentationLabels ) # Perform a simple normalization X_train <- ( X_train - mean( X_train ) ) / sd( X_train ) # Create the model model <- createUnetModel2D( c( dim( domainImage ), 1 ), numberOfOutputs = numberOfLabels ) metric_multilabel_dice_coefficient <- custom_metric( "multilabel_dice_coefficient", multilabel_dice_coefficient ) loss_dice <- function( y_true, y_pred ) { -multilabel_dice_coefficient(y_true, y_pred) } attr(loss_dice, "py_function_name") <- "multilabel_dice_coefficient" model %>% compile( loss = loss_dice, optimizer = optimizer_adam( lr = 0.0001 ), metrics = metric_multilabel_dice_coefficient ) # Comment out the rest due to travis build constraints # Fit the model # track <- model %>% fit( X_train, Y_train, # epochs = 100, batch_size = 5, verbose = 1, shuffle = TRUE, # callbacks = list( # callback_model_checkpoint( "unetModelInterimWeights.h5", # monitor = 'val_loss', save_best_only = TRUE ), # callback_reduce_lr_on_plateau( monitor = "val_loss", factor = 0.1 ) # ), # validation_split = 0.2 ) # Save the model and/or save the model weights # save_model_hdf5( model, filepath = 'unetModel.h5' ) # save_model_weights_hdf5( unetModel, filepath = 'unetModelWeights.h5' ) )library( ANTsR ) library( ANTsRNet ) library( keras ) imageIDs <- c( "r16", "r27", "r30", "r62", "r64", "r85" ) trainingBatchSize <- length( imageIDs ) # Perform simple 3-tissue segmentation. segmentationLabels <- c( 1, 2, 3 ) numberOfLabels <- length( segmentationLabels ) initialization <- paste0( 'KMeans[', numberOfLabels, ']' ) domainImage <- antsImageRead( getANTsRData( imageIDs[1] ) ) X_train <- array( data = NA, dim = c( trainingBatchSize, dim( domainImage ), 1 ) ) Y_train <- array( data = NA, dim = c( trainingBatchSize, dim( domainImage ) ) ) images <- list() segmentations <- list() for( i in seq_len( trainingBatchSize ) ) { cat( "Processing image", imageIDs[i], "\n" ) image <- antsImageRead( getANTsRData( imageIDs[i] ) ) mask <- getMask( image ) segmentation <- atropos( image, mask, initialization )$segmentation X_train[i,,, 1] <- as.array( image ) Y_train[i,,] <- as.array( segmentation ) } Y_train <- encodeUnet( Y_train, segmentationLabels ) # Perform a simple normalization X_train <- ( X_train - mean( X_train ) ) / sd( X_train ) # Create the model model <- createUnetModel2D( c( dim( domainImage ), 1 ), numberOfOutputs = numberOfLabels ) metric_multilabel_dice_coefficient <- custom_metric( "multilabel_dice_coefficient", multilabel_dice_coefficient ) loss_dice <- function( y_true, y_pred ) { -multilabel_dice_coefficient(y_true, y_pred) } attr(loss_dice, "py_function_name") <- "multilabel_dice_coefficient" model %>% compile( loss = loss_dice, optimizer = optimizer_adam( lr = 0.0001 ), metrics = metric_multilabel_dice_coefficient ) # Comment out the rest due to travis build constraints # Fit the model # track <- model %>% fit( X_train, Y_train, # epochs = 100, batch_size = 5, verbose = 1, shuffle = TRUE, # callbacks = list( # callback_model_checkpoint( "unetModelInterimWeights.h5", # monitor = 'val_loss', save_best_only = TRUE ), # callback_reduce_lr_on_plateau( monitor = "val_loss", factor = 0.1 ) # ), # validation_split = 0.2 ) # Save the model and/or save the model weights # save_model_hdf5( model, filepath = 'unetModel.h5' ) # save_model_weights_hdf5( unetModel, filepath = 'unetModelWeights.h5' ) )
Creates a keras model of the U-net deep learning architecture for image segmentation and regression. More information is provided at the authors' website:
createUnetModel3D( inputImageSize, numberOfOutputs = 2, scalarOutputSize = 0, scalarOutputActivation = "relu", numberOfLayers = 4, numberOfFiltersAtBaseLayer = 32, numberOfFilters = NULL, convolutionKernelSize = c(3, 3, 3), deconvolutionKernelSize = c(2, 2, 2), poolSize = c(2, 2, 2), strides = c(2, 2, 2), dropoutRate = 0, weightDecay = 0, mode = c("classification", "regression", "sigmoid"), additionalOptions = NA )createUnetModel3D( inputImageSize, numberOfOutputs = 2, scalarOutputSize = 0, scalarOutputActivation = "relu", numberOfLayers = 4, numberOfFiltersAtBaseLayer = 32, numberOfFilters = NULL, convolutionKernelSize = c(3, 3, 3), deconvolutionKernelSize = c(2, 2, 2), poolSize = c(2, 2, 2), strides = c(2, 2, 2), dropoutRate = 0, weightDecay = 0, mode = c("classification", "regression", "sigmoid"), additionalOptions = NA )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Meaning depends on the |
scalarOutputSize |
if greater than 0, a global average pooling from each encoding layer is concatenated to a dense layer as a secondary output. |
scalarOutputActivation |
activation for nonzero output scalar. |
numberOfLayers |
number of encoding/decoding layers. |
numberOfFiltersAtBaseLayer |
number of filters at the beginning and end
of the |
numberOfFilters |
vector explicitly setting the number of filters at
each layer. One can either set this or |
convolutionKernelSize |
3-d vector defining the kernel size during the encoding path. |
deconvolutionKernelSize |
3-d vector defining the kernel size during the decoding. |
poolSize |
3-d vector defining the region for each pooling layer. |
strides |
3-d vector describing the stride length in each direction. |
dropoutRate |
float between 0 and 1 to use between dense layers. |
weightDecay |
weighting parameter for L2 regularization of the kernel weights of the convolution layers. Default = 0.0. |
mode |
'classification' or 'regression' or 'sigmoid'. |
additionalOptions |
string or vector of strings specifying specific configuration add-ons/tweaks:
|
\url{https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/}
with the paper available here:
\url{https://arxiv.org/abs/1505.04597}
This particular implementation was influenced by the following python implementation:
\url{https://github.com/joelthelion/ultrasound-nerve-segmentation}
a u-net keras model
Tustison NJ
# Simple examples, must run successfully and quickly. These will be tested. library( ANTsRNet ) library( keras ) model <- createUnetModel3D( c( 64, 64, 64, 1 ) ) metric_multilabel_dice_coefficient <- custom_metric( "multilabel_dice_coefficient", multilabel_dice_coefficient ) loss_dice <- function( y_true, y_pred ) { -multilabel_dice_coefficient(y_true, y_pred) } attr(loss_dice, "py_function_name") <- "multilabel_dice_coefficient" model %>% compile( loss = loss_dice, optimizer = optimizer_adam( lr = 0.0001 ), metrics = metric_multilabel_dice_coefficient ) print( model )# Simple examples, must run successfully and quickly. These will be tested. library( ANTsRNet ) library( keras ) model <- createUnetModel3D( c( 64, 64, 64, 1 ) ) metric_multilabel_dice_coefficient <- custom_metric( "multilabel_dice_coefficient", multilabel_dice_coefficient ) loss_dice <- function( y_true, y_pred ) { -multilabel_dice_coefficient(y_true, y_pred) } attr(loss_dice, "py_function_name") <- "multilabel_dice_coefficient" model %>% compile( loss = loss_dice, optimizer = optimizer_adam( lr = 0.0001 ), metrics = metric_multilabel_dice_coefficient ) print( model )
Creates a keras model of the Vgg deep learning architecture for image recognition based on the paper
createVggModel2D( inputImageSize, numberOfOutputs = 1000, layers = c(1, 2, 3, 4, 4), lowestResolution = 64, convolutionKernelSize = c(3, 3), poolSize = c(2, 2), strides = c(2, 2), numberOfDenseUnits = 4096, dropoutRate = 0, style = 19, mode = c("classification", "regression") )createVggModel2D( inputImageSize, numberOfOutputs = 1000, layers = c(1, 2, 3, 4, 4), lowestResolution = 64, convolutionKernelSize = c(3, 3), poolSize = c(2, 2), strides = c(2, 2), numberOfDenseUnits = 4096, dropoutRate = 0, style = 19, mode = c("classification", "regression") )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Number of outputs in the final layer |
layers |
a vector determining the number of filters defined at for each layer. |
lowestResolution |
number of filters at the beginning. |
convolutionKernelSize |
2-d vector definining the kernel size during the encoding path |
poolSize |
2-d vector defining the region for each pooling layer. |
strides |
2-d vector describing the stride length in each direction. |
numberOfDenseUnits |
integer for the number of units in the last layers. |
dropoutRate |
float between 0 and 1 to use between dense layers. |
style |
|
mode |
'classification' or 'regression'. |
K. Simonyan and A. Zisserman, Very Deep Convolutional Networks for Large-Scale Image Recognition
available here:
\url{https://arxiv.org/abs/1409.1556}
This particular implementation was influenced by the following python implementation:
\url{https://gist.github.com/baraldilorenzo/8d096f48a1be4a2d660d}
a VGG keras model
Tustison NJ
library( ANTsRNet ) library( keras ) library( ANTsR ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly. # We also need to resample since the native mnist data size does # not fit with GoogLeNet parameters. resampledImageSize <- c( 100, 100 ) numberOfTrainingData <- 10 numberOfTestingData <- 5 X_trainSmall <- as.array( resampleImage( as.antsImage( mnistData$train$x[1:numberOfTrainingData,,] ), c( numberOfTrainingData, resampledImageSize ), TRUE ) ) X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:numberOfTrainingData], numberOfLabels ) X_testSmall <- as.array( resampleImage( as.antsImage( mnistData$test$x[1:numberOfTestingData,,] ), c( numberOfTestingData, resampledImageSize ), TRUE ) ) X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:numberOfTestingData], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createVggModel2D( inputImageSize = c( resampledImageSize, 1 ), numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) # Comment out the rest due to travis build constraints # track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, # epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model # testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) # predictedData <- model %>% predict( X_testSmall, verbose = 1 )library( ANTsRNet ) library( keras ) library( ANTsR ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly. # We also need to resample since the native mnist data size does # not fit with GoogLeNet parameters. resampledImageSize <- c( 100, 100 ) numberOfTrainingData <- 10 numberOfTestingData <- 5 X_trainSmall <- as.array( resampleImage( as.antsImage( mnistData$train$x[1:numberOfTrainingData,,] ), c( numberOfTrainingData, resampledImageSize ), TRUE ) ) X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:numberOfTrainingData], numberOfLabels ) X_testSmall <- as.array( resampleImage( as.antsImage( mnistData$test$x[1:numberOfTestingData,,] ), c( numberOfTestingData, resampledImageSize ), TRUE ) ) X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:numberOfTestingData], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createVggModel2D( inputImageSize = c( resampledImageSize, 1 ), numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) # Comment out the rest due to travis build constraints # track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, # epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model # testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) # predictedData <- model %>% predict( X_testSmall, verbose = 1 )
Creates a keras model of the Vgg deep learning architecture for image recognition based on the paper
createVggModel3D( inputImageSize, numberOfOutputs = 1000, layers = c(1, 2, 3, 4, 4), lowestResolution = 64, convolutionKernelSize = c(3, 3, 3), poolSize = c(2, 2, 2), strides = c(2, 2, 2), numberOfDenseUnits = 4096, dropoutRate = 0, style = 19, mode = c("classification", "regression") )createVggModel3D( inputImageSize, numberOfOutputs = 1000, layers = c(1, 2, 3, 4, 4), lowestResolution = 64, convolutionKernelSize = c(3, 3, 3), poolSize = c(2, 2, 2), strides = c(2, 2, 2), numberOfDenseUnits = 4096, dropoutRate = 0, style = 19, mode = c("classification", "regression") )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Number of segmentation labels. |
layers |
a vector determining the number of 'filters' defined at for each layer. |
lowestResolution |
number of filters at the beginning. |
convolutionKernelSize |
3-d vector definining the kernel size during the encoding path |
poolSize |
3-d vector defining the region for each pooling layer. |
strides |
3-d vector describing the stride length in each direction. |
numberOfDenseUnits |
integer for the number of units in the last layers. |
dropoutRate |
float between 0 and 1 to use between dense layers. |
style |
|
mode |
'classification' or 'regression'. |
K. Simonyan and A. Zisserman, Very Deep Convolutional Networks for Large-Scale Image Recognition
available here:
\url{https://arxiv.org/abs/1409.1556}
This particular implementation was influenced by the following python implementation:
\url{https://gist.github.com/baraldilorenzo/8d096f48a1be4a2d660d}
a VGG keras model
Tustison NJ
# Simple examples, must run successfully and quickly. These will be tested. ## Not run: library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:100,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:100], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createVggModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, epochs = 2, batch_size = 20, shuffle = TRUE, validation_split = 0.25 ) # Now test the model testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) predictedData <- model %>% predict( X_testSmall, verbose = 1 ) ## End(Not run)# Simple examples, must run successfully and quickly. These will be tested. ## Not run: library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:100,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:100], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createVggModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, epochs = 2, batch_size = 20, shuffle = TRUE, validation_split = 0.25 ) # Now test the model testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) predictedData <- model %>% predict( X_testSmall, verbose = 1 ) ## End(Not run)
Creates a keras model of the Wide ResNet deep learning architecture for image classification/regression. The paper is available here:
createWideResNetModel2D( inputImageSize, numberOfOutputs = 1000, depth = 2, width = 1, residualBlockSchedule = c(16, 32, 64), poolSize = c(8, 8), dropoutRate = 0, weightDecay = 5e-04, mode = "classification" )createWideResNetModel2D( inputImageSize, numberOfOutputs = 1000, depth = 2, width = 1, residualBlockSchedule = c(16, 32, 64), poolSize = c(8, 8), dropoutRate = 0, weightDecay = 5e-04, mode = "classification" )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Number of outputs in the final layer |
depth |
integer determining the depth of the network. Related to the
actual number of layers by the |
width |
integer determining the width of the network. Default = 1. |
residualBlockSchedule |
vector determining the number of filters
per convolutional block. Default = |
poolSize |
pool size for final average pooling layer. Default = c( 8, 8 ). |
dropoutRate |
Dropout percentage. Default = 0.0. |
weightDecay |
weight for l2 regularizer in convolution layers. Default = 0.0005. |
mode |
'classification' or 'regression'. Default = 'classification'. |
https://arxiv.org/abs/1512.03385
This particular implementation was influenced by the following python implementation:
https://github.com/titu1994/Wide-Residual-Networks
a Wide ResNet keras model
Tustison NJ
library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:10,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:10], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createWideResNetModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) # Comment out the rest due to travis build constraints # track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, # epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model # testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) # predictedData <- model %>% predict( X_testSmall, verbose = 1 ) rm(model); gc()library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:10,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:10], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createWideResNetModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) # Comment out the rest due to travis build constraints # track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, # epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model # testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) # predictedData <- model %>% predict( X_testSmall, verbose = 1 ) rm(model); gc()
Creates a keras model of the Wide ResNet deep learning architecture for image classification/regression. The paper is available here:
createWideResNetModel3D( inputImageSize, numberOfOutputs = 1000, depth = 2, width = 1, residualBlockSchedule = c(16, 32, 64), poolSize = c(8, 8, 8), dropoutRate = 0, weightDecay = 5e-04, mode = c("classification", "regression") )createWideResNetModel3D( inputImageSize, numberOfOutputs = 1000, depth = 2, width = 1, residualBlockSchedule = c(16, 32, 64), poolSize = c(8, 8, 8), dropoutRate = 0, weightDecay = 5e-04, mode = c("classification", "regression") )
inputImageSize |
Used for specifying the input tensor shape. The shape (or dimension) of that tensor is the image dimensions followed by the number of channels (e.g., red, green, and blue). The batch size (i.e., number of training images) is not specified a priori. |
numberOfOutputs |
Number of classification labels. |
depth |
integer determining the depth of the ntwork. Related to the
actual number of layers by the |
width |
integer determining the width of the network. Default = 1. |
residualBlockSchedule |
vector determining the number of filters
per convolutional block. Default = |
poolSize |
pool size for final average pooling layer. Default = c( 8, 8, 8 ). |
dropoutRate |
Dropout percentage. Default = 0.0. |
weightDecay |
weight for l2 regularizer in convolution layers. Default = 0.0005. |
mode |
'classification' or 'regression'. |
https://arxiv.org/abs/1512.03385
This particular implementation was influenced by the following python implementation:
https://github.com/titu1994/Wide-Residual-Networks
a Wide ResNet keras model
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:10,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:10], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createWideResNetModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) # Comment out the rest due to travis build constraints # track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, # epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model # testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) # predictedData <- model %>% predict( X_testSmall, verbose = 1 ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) mnistData <- dataset_mnist() numberOfLabels <- 10 # Extract a small subset for something that can run quickly X_trainSmall <- mnistData$train$x[1:10,,] X_trainSmall <- array( data = X_trainSmall, dim = c( dim( X_trainSmall ), 1 ) ) Y_trainSmall <- to_categorical( mnistData$train$y[1:10], numberOfLabels ) X_testSmall <- mnistData$test$x[1:10,,] X_testSmall <- array( data = X_testSmall, dim = c( dim( X_testSmall ), 1 ) ) Y_testSmall <- to_categorical( mnistData$test$y[1:10], numberOfLabels ) # We add a dimension of 1 to specify the channel size inputImageSize <- c( dim( X_trainSmall )[2:3], 1 ) model <- createWideResNetModel2D( inputImageSize = inputImageSize, numberOfOutputs = numberOfLabels ) model %>% compile( loss = 'categorical_crossentropy', optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( 'categorical_crossentropy', 'accuracy' ) ) # Comment out the rest due to travis build constraints # track <- model %>% fit( X_trainSmall, Y_trainSmall, verbose = 1, # epochs = 1, batch_size = 2, shuffle = TRUE, validation_split = 0.5 ) # Now test the model # testingMetrics <- model %>% evaluate( X_testSmall, Y_testSmall ) # predictedData <- model %>% predict( X_testSmall, verbose = 1 ) ## End(Not run)
Cycle generative adverserial network from the paper:
https://arxiv.org/pdf/1703.10593
and ported from the Keras (python) implementation:
https://github.com/eriklindernoren/Keras-GAN/blob/master/cyclegan/cyclegan.py
$initialize instantiates a new class and builds the
generator and discriminator.
$buildGeneratorbuild generator.
$buildGeneratorbuild discriminator.
Tustison NJ
library( keras ) library( ANTsRNet ) keras::backend()$clear_session() ganModel <- CycleGanModel$new( inputImageSize = c( 128, 128, 3 ) ) ganModel$buildGenerator()library( keras ) library( ANTsRNet ) keras::backend()$clear_session() ganModel <- CycleGanModel$new( inputImageSize = c( 128, 128, 3 ) ) ganModel$buildGenerator()
Function for translating the predictions from the SSD model output to boxes, (centerx, centery, width, height), for subsequent usage.
decodeSsd2D( yPredicted, imageSize, confidenceThreshold = 0.5, overlapThreshold = 0.45 )decodeSsd2D( yPredicted, imageSize, confidenceThreshold = 0.5, overlapThreshold = 0.45 )
yPredicted |
The predicted output produced by the SSD model expected to
be an array of shape ( |
imageSize |
2-D vector specifying the spatial domain of the input images. |
confidenceThreshold |
Float between 0 and 1. The minimum classification value required for a given box to be considered a "positive prediction." A lower value will result in better recall while a higher value yields higher precision results. Default = 0.5. |
overlapThreshold |
'NULL' or a float between 0 and 1. If 'NULL' then
no non-maximum suppression will be performed. Otherwise, a greedy non-
maximal suppression is performed following confidence thresholding. In
other words all boxes with Jaccard similarities > |
This particular implementation was heavily influenced by the following python and R implementations:
\url{https://github.com/pierluigiferrari/ssd_keras}
\url{https://github.com/rykov8/ssd_keras}
\url{https://github.com/gsimchoni/ssdkeras}
a list of length batchSize where each element comprises a 2-D
array where each row describes a single box using the following six elements
(classId, confidenceValue, xmin, xmax, ymin, ymax)
Tustison NJ
Function for translating the predictions from the SSD model output to boxes, (centerx, centery, width, height), for subsequent usage.
decodeSsd3D( yPredicted, imageSize, confidenceThreshold = 0.5, overlapThreshold = 0.45 )decodeSsd3D( yPredicted, imageSize, confidenceThreshold = 0.5, overlapThreshold = 0.45 )
yPredicted |
The predicted output produced by the SSD model expected to
be an array of shape ( |
imageSize |
3-D vector specifying the spatial domain of the input images. |
confidenceThreshold |
Float between 0 and 1. The minimum classification value required for a given box to be considered a "positive prediction." A lower value will result in better recall while a higher value yields higher precision results. Default = 0.5. |
overlapThreshold |
|
This particular implementation was heavily influenced by the following python and R implementations:
\url{https://github.com/pierluigiferrari/ssd_keras}
\url{https://github.com/rykov8/ssd_keras}
\url{https://github.com/gsimchoni/ssdkeras}
a list of length batchSize where each element comprises a 2-D
array where each row describes a single box using the following six elements
(classId, confidenceValue, xmin, xmax, ymin, ymax, zmin, zmax).
Tustison NJ
Perform Atropos-style six tissue segmentation using deep learning
deepAtropos( t1, doPreprocessing = TRUE, useSpatialPriors = 1, doDenoising = TRUE, verbose = FALSE )deepAtropos( t1, doPreprocessing = TRUE, useSpatialPriors = 1, doDenoising = TRUE, verbose = FALSE )
t1 |
raw or preprocessed 3-D T1-weighted brain image. |
doPreprocessing |
perform preprocessing. See description above. |
useSpatialPriors |
Use MNI spatial tissue priors (0 or 1). Currently, only '0' (no priors) and '1' (cerebellar prior only) are the only two options. Default is 1. |
doDenoising |
Activate denoising within preprocessing (default TRUE). |
verbose |
print progress. |
The labeling is as follows:
Label 0:background
Label 1:CSF
Label 2:gray matter
Label 3:white matter
Label 4:deep gray matter
Label 5:brain stem
Label 6:cerebellum
Preprocessing on the training data consisted of:
n4 bias correction,
denoising,
brain extraction, and
affine registration to MNI.
The input T1 should undergo the same steps. If the input T1 is the raw
T1, these steps can be performed by the internal preprocessing, i.e. set
doPreprocessing = TRUE
list consisting of the segmentation image and probability images for each label.
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1.nii.gz" ) results <- deepAtropos( image ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1.nii.gz" ) results <- deepAtropos( image ) ## End(Not run)
Deep convolutional generative adverserial network from the paper:
https://arxiv.org/abs/1511.06434
and ported from the Keras (python) implementation:
https://github.com/eriklindernoren/Keras-GAN/blob/master/dcgan/dcgan.py
$initialize instantiates a new class and builds the
generator and discriminator.
$buildGeneratorbuild generator.
$buildGeneratorbuild discriminator.
Tustison NJ
library( keras ) library( ANTsRNet ) keras::backend()$clear_session() # Let's use the mnist data set. mnist <- dataset_mnist() numberOfTrainingData <- length( mnist$train$y ) inputImageSize <- c( dim( mnist$train$x[1,,] ), 1 ) x <- array( data = mnist$train$x / 255, dim = c( numberOfTrainingData, inputImageSize ) ) y <- mnist$train$y numberOfClusters <- length( unique( mnist$train$y ) ) # Instantiate the DCEC model ganModel <- DeepConvolutionalGanModel$new( inputImageSize = inputImageSize, latentDimension = 100 ) ganModel$train( x, numberOfEpochs = 2 )library( keras ) library( ANTsRNet ) keras::backend()$clear_session() # Let's use the mnist data set. mnist <- dataset_mnist() numberOfTrainingData <- length( mnist$train$y ) inputImageSize <- c( dim( mnist$train$x[1,,] ), 1 ) x <- array( data = mnist$train$x / 255, dim = c( numberOfTrainingData, inputImageSize ) ) y <- mnist$train$y numberOfClusters <- length( unique( mnist$train$y ) ) # Instantiate the DCEC model ganModel <- DeepConvolutionalGanModel$new( inputImageSize = inputImageSize, latentDimension = 100 ) ganModel$train( x, numberOfEpochs = 2 )
Deep embedded clustering (DEC) model class
array describing the auteoencoder.
number of clusters.
parameter
initializer for autoencoder
$initialize instantiates a new class.
$pretrain
$loadWeights
$extractFeatures
$predictClusterLabels
$targetDistribution
$compile
$fit
Tustison NJ
## Not run: library( keras ) library( ANTsRNet ) fmnist <- dataset_fashion_mnist() numberOfTrainingData <- length( fmnist$train$y ) numberOfTestingData <- length( fmnist$test$y ) numberOfPixels <- prod( dim( fmnist$test$x[1,,] ) ) fmnist$train$xreshaped <- array_reshape( fmnist$train$x, dim = c( numberOfTrainingData, numberOfPixels ), order = "C" ) fmnist$test$xreshaped <- array_reshape( fmnist$train$x, dim = c( numberOfTrainingData, numberOfPixels ), order = "C" ) x <- rbind( fmnist$test$xreshaped, fmnist$train$xreshaped ) / 255 y <- c( fmnist$test$y, fmnist$train$y ) numberOfClusters <- length( unique( fmnist$train$y ) ) initializer <- initializer_variance_scaling( scale = 1/3, mode = 'fan_in', distribution = 'uniform' ) pretrainOptimizer <- optimizer_sgd( lr = 1.0, momentum = 0.9 ) decModel <- DeepEmbeddedClusteringModel$new( numberOfUnitsPerLayer = c( numberOfPixels, 500, 500, 2000, 10 ), numberOfClusters = numberOfClusters, initializer = initializer ) modelWeightsFile <- "decAutoencoderModelWeights.h5" if( ! file.exists( modelWeightsFile ) ) { decModel$pretrain( x = x, optimizer = optimizer_sgd( lr = 1.0, momentum = 0.9 ), epochs = 300L, batchSize = 256L ) save_model_weights_hdf5( decModel$autoencoder, modelWeightsFile ) } else { load_model_weights_hdf5( decModel$autoencoder, modelWeightsFile ) } decModel$compile( optimizer = optimizer_sgd( lr = 1.0, momentum = 0.9 ), loss = 'kld' ) yPredicted <- decModel$fit( x, maxNumberOfIterations = 2e4, batchSize = 256, tolerance = 1e-3, updateInterval = 10 ) ## End(Not run)## Not run: library( keras ) library( ANTsRNet ) fmnist <- dataset_fashion_mnist() numberOfTrainingData <- length( fmnist$train$y ) numberOfTestingData <- length( fmnist$test$y ) numberOfPixels <- prod( dim( fmnist$test$x[1,,] ) ) fmnist$train$xreshaped <- array_reshape( fmnist$train$x, dim = c( numberOfTrainingData, numberOfPixels ), order = "C" ) fmnist$test$xreshaped <- array_reshape( fmnist$train$x, dim = c( numberOfTrainingData, numberOfPixels ), order = "C" ) x <- rbind( fmnist$test$xreshaped, fmnist$train$xreshaped ) / 255 y <- c( fmnist$test$y, fmnist$train$y ) numberOfClusters <- length( unique( fmnist$train$y ) ) initializer <- initializer_variance_scaling( scale = 1/3, mode = 'fan_in', distribution = 'uniform' ) pretrainOptimizer <- optimizer_sgd( lr = 1.0, momentum = 0.9 ) decModel <- DeepEmbeddedClusteringModel$new( numberOfUnitsPerLayer = c( numberOfPixels, 500, 500, 2000, 10 ), numberOfClusters = numberOfClusters, initializer = initializer ) modelWeightsFile <- "decAutoencoderModelWeights.h5" if( ! file.exists( modelWeightsFile ) ) { decModel$pretrain( x = x, optimizer = optimizer_sgd( lr = 1.0, momentum = 0.9 ), epochs = 300L, batchSize = 256L ) save_model_weights_hdf5( decModel$autoencoder, modelWeightsFile ) } else { load_model_weights_hdf5( decModel$autoencoder, modelWeightsFile ) } decModel$compile( optimizer = optimizer_sgd( lr = 1.0, momentum = 0.9 ), loss = 'kld' ) yPredicted <- decModel$fit( x, maxNumberOfIterations = 2e4, batchSize = 256, tolerance = 1e-3, updateInterval = 10 ) ## End(Not run)
Perform hippocampal/entorhinal segmentation in T1 and T1/T2 images using labels from Mike Yassa's lab—https://faculty.sites.uci.edu/myassa/
deepFlash( t1, t2 = NULL, whichParcellation = "yassa", doPreprocessing = TRUE, useRankIntensity = TRUE, verbose = FALSE )deepFlash( t1, t2 = NULL, whichParcellation = "yassa", doPreprocessing = TRUE, useRankIntensity = TRUE, verbose = FALSE )
t1 |
raw or preprocessed 3-D T1-weighted brain image. |
t2 |
optional raw or preprocessed 3-D T2-weighted brain image. |
whichParcellation |
string — "yassa". See above label descriptions. |
doPreprocessing |
perform preprocessing. See description above. |
useRankIntensity |
If false, use histogram matching with cropped template ROI. Otherwise, use a rank intensity transform on the cropped ROI. Only for 'yassa' parcellation. |
verbose |
print progress. |
https://www.nature.com/articles/s41598-024-59440-6
The labeling is as follows:
Label 0 :background
Label 5 :left aLEC
Label 6 :right aLEC
Label 7 :left pMEC
Label 8 :right pMEC
Label 9 :left perirhinal
Label 10:right perirhinal
Label 11:left parahippocampal
Label 12:right parahippocampal
Label 13:left DG/CA2/CA3/CA4
Label 14:right DG/CA2/CA3/CA4
Label 15:left CA1
Label 16:right CA1
Label 17:left subiculum
Label 18:right subiculum
Preprocessing on the training data consisted of:
n4 bias correction,
affine registration to deep flash template.
which is performed on the input images if doPreprocessing = TRUE.
list consisting of the segmentation image and probability images for each label.
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1.nii.gz" ) results <- deepFlash( image ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1.nii.gz" ) results <- deepFlash( image ) ## End(Not run)
Desikan-Killiany-Tourville labeling. https://pubmed.ncbi.nlm.nih.gov/23227001/
desikanKillianyTourvilleLabeling( t1, doPreprocessing = TRUE, returnProbabilityImages = FALSE, doLobarParcellation = FALSE, version = 0, verbose = FALSE )desikanKillianyTourvilleLabeling( t1, doPreprocessing = TRUE, returnProbabilityImages = FALSE, doLobarParcellation = FALSE, version = 0, verbose = FALSE )
t1 |
raw or preprocessed 3-D T1-weighted brain image. |
doPreprocessing |
perform preprocessing. See description above. |
returnProbabilityImages |
whether to return the two sets of probability images for the inner and outer labels. |
doLobarParcellation |
perform lobar parcellation (also divided by hemisphere). |
version |
which version. |
verbose |
print progress. |
Perform DKT labeling using deep learning. Description available here: https://mindboggle.readthedocs.io/en/latest/labels.html The labeling is as follows:
Outer labels:
Label 1002:left caudal anterior cingulate
Label 1003:left caudal middle frontal
Label 1005:left cuneus
Label 1006:left entorhinal
Label 1007:left fusiform
Label 1008:left inferior parietal
Label 1009:left inferior temporal
Label 1010:left isthmus cingulate
Label 1011:left lateral occipital
Label 1012:left lateral orbitofrontal
Label 1013:left lingual
Label 1014:left medial orbitofrontal
Label 1015:left middle temporal
Label 1016:left parahippocampal
Label 1017:left paracentral
Label 1018:left pars opercularis
Label 1019:left pars orbitalis
Label 1020:left pars triangularis
Label 1021:left pericalcarine
Label 1022:left postcentral
Label 1023:left posterior cingulate
Label 1024:left precentral
Label 1025:left precuneus
Label 1026:left rostral anterior cingulate
Label 1027:left rostral middle frontal
Label 1028:left superior frontal
Label 1029:left superior parietal
Label 1030:left superior temporal
Label 1031:left supramarginal
Label 1034:left transverse temporal
Label 1035:left insula
Label 2002:right caudal anterior cingulate
Label 2003:right caudal middle frontal
Label 2005:right cuneus
Label 2006:right entorhinal
Label 2007:right fusiform
Label 2008:right inferior parietal
Label 2009:right inferior temporal
Label 2010:right isthmus cingulate
Label 2011:right lateral occipital
Label 2012:right lateral orbitofrontal
Label 2013:right lingual
Label 2014:right medial orbitofrontal
Label 2015:right middle temporal
Label 2016:right parahippocampal
Label 2017:right paracentral
Label 2018:right pars opercularis
Label 2019:right pars orbitalis
Label 2020:right pars triangularis
Label 2021:right pericalcarine
Label 2022:right postcentral
Label 2023:right posterior cingulate
Label 2024:right precentral
Label 2025:right precuneus
Label 2026:right rostral anterior cingulate
Label 2027:right rostral middle frontal
Label 2028:right superior frontal
Label 2029:right superior parietal
Label 2030:right superior temporal
Label 2031:right supramarginal
Label 2034:right transverse temporal
Label 2035:right insula
Performing the lobar parcellation is based on the FreeSurfer division described here:
https://surfer.nmr.mgh.harvard.edu/fswiki/CorticalParcellation
Frontal lobe:
Label 1002:left caudal anterior cingulate
Label 1003:left caudal middle frontal
Label 1012:left lateral orbitofrontal
Label 1014:left medial orbitofrontal
Label 1017:left paracentral
Label 1018:left pars opercularis
Label 1019:left pars orbitalis
Label 1020:left pars triangularis
Label 1024:left precentral
Label 1026:left rostral anterior cingulate
Label 1027:left rostral middle frontal
Label 1028:left superior frontal
Label 2002:right caudal anterior cingulate
Label 2003:right caudal middle frontal
Label 2012:right lateral orbitofrontal
Label 2014:right medial orbitofrontal
Label 2017:right paracentral
Label 2018:right pars opercularis
Label 2019:right pars orbitalis
Label 2020:right pars triangularis
Label 2024:right precentral
Label 2026:right rostral anterior cingulate
Label 2027:right rostral middle frontal
Label 2028:right superior frontal
Parietal:
Label 1008:left inferior parietal
Label 1010:left isthmus cingulate
Label 1022:left postcentral
Label 1023:left posterior cingulate
Label 1025:left precuneus
Label 1029:left superior parietal
Label 1031:left supramarginal
Label 2008:right inferior parietal
Label 2010:right isthmus cingulate
Label 2022:right postcentral
Label 2023:right posterior cingulate
Label 2025:right precuneus
Label 2029:right superior parietal
Label 2031:right supramarginal
Temporal:
Label 1006:left entorhinal
Label 1007:left fusiform
Label 1009:left inferior temporal
Label 1015:left middle temporal
Label 1016:left parahippocampal
Label 1030:left superior temporal
Label 1034:left transverse temporal
Label 2006:right entorhinal
Label 2007:right fusiform
Label 2009:right inferior temporal
Label 2015:right middle temporal
Label 2016:right parahippocampal
Label 2030:right superior temporal
Label 2034:right transverse temporal
Occipital:
Label 1005:left cuneus
Label 1011:left lateral occipital
Label 1013:left lingual
Label 1021:left pericalcarine
Label 2005:right cuneus
Label 2011:right lateral occipital
Label 2013:right lingual
Label 2021:right pericalcarine
Other outer labels:
Label 1035:left insula
Label 2035:right insula
For the original DKT version (version 0), the set of inner labels are also included.
Inner labels:
Label 4:left lateral ventricle
Label 5:left inferior lateral ventricle
Label 6:left cerebellem exterior
Label 7:left cerebellum white matter
Label 10:left thalamus proper
Label 11:left caudate
Label 12:left putamen
Label 13:left pallidium
Label 14:3rd ventricle
Label 15:4th ventricle
Label 16:brain stem
Label 17:left hippocampus
Label 18:left amygdala
Label 24:CSF
Label 25:left lesion
Label 26:left accumbens area
Label 28:left ventral DC
Label 30:left vessel
Label 43:right lateral ventricle
Label 44:right inferior lateral ventricle
Label 45:right cerebellum exterior
Label 46:right cerebellum white matter
Label 49:right thalamus proper
Label 50:right caudate
Label 51:right putamen
Label 52:right palladium
Label 53:right hippocampus
Label 54:right amygdala
Label 57:right lesion
Label 58:right accumbens area
Label 60:right ventral DC
Label 62:right vessel
Label 72:5th ventricle
Label 85:optic chasm
Label 91:left basal forebrain
Label 92:right basal forebrain
Label 630:cerebellar vermal lobules I-V
Label 631:cerebellar vermal lobules VI-VII
Label 632:cerebellar vermal lobules VIII-X
Preprocessing on the training data consisted of:
n4 bias correction,
denoising,
brain extraction, and
affine registration to MNI.
The input T1 should undergo the same steps. If the input T1 is the raw
T1, these steps can be performed by the internal preprocessing, i.e. set
doPreprocessing = TRUE
list consisting of the segmentation image and probability images for each label.
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1.nii.gz" ) results <- desikanKillianyTourvilleLabeling( image ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1.nii.gz" ) results <- desikanKillianyTourvilleLabeling( image ) ## End(Not run)
Renders boxes on objects within rasterized images.
drawRectangles( image, boxes, boxColors = "red", confidenceValues = NULL, captions = NULL )drawRectangles( image, boxes, boxColors = "red", confidenceValues = NULL, captions = NULL )
image |
standard image using something like jpeg::readJPEG. |
boxes |
a data frame or comprising where each row has the format: xmin, xmax, ymin, ymax. |
boxColors |
Optional scalar or vector of length = |
confidenceValues |
Optional vector of length = |
captions |
Optional vector of length = |
Tustison NJ
jpg = ANTsRCore::getANTsRData("r16") if (requireNamespace("jpeg", quietly = TRUE)) { image = jpeg::readJPEG(jpg) vec2d = c(1, 10, 1, 10) drawRectangles(image, vec2d) vec2d = rbind( c(1, 10, 1, 10), c(20, 40, 30, 40) ) drawRectangles(image, vec2d) }jpg = ANTsRCore::getANTsRData("r16") if (requireNamespace("jpeg", quietly = TRUE)) { image = jpeg::readJPEG(jpg) vec2d = c(1, 10, 1, 10) drawRectangles(image, vec2d) vec2d = rbind( c(1, 10, 1, 10), c(20, 40, 30, 40) ) drawRectangles(image, vec2d) }
Efficient attention layer (2-D)
output of tensor shape.
number of filters for F and G layers.
number of filters for H. If = NA, only use filter F for efficiency.
pool_size in max pool layer.
concatenate final layer with input. Alternatively, add.
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
BB Avants, NJ Tustison
Efficient attention layer (3-D)
output of tensor shape.
number of filters for F and G layers.
number of filters for H. If = NA, only use filter F for efficiency.
pool_size in max pool layer.
concatenate final layer with input. Alternatively, add.
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
BB Avants, NJ Tustison
Perform functional lung segmentation using hyperpolarized gases.
elBicho(ventilationImage, mask, useCoarseSlicesOnly = TRUE, verbose = FALSE)elBicho(ventilationImage, mask, useCoarseSlicesOnly = TRUE, verbose = FALSE)
ventilationImage |
input ventilation image |
mask |
input mask image |
useCoarseSlicesOnly |
if |
verbose |
print progress. |
https://pubmed.ncbi.nlm.nih.gov/30195415/
ventilation segmentation and corresponding probability images
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "flair.nii.gz" ) probabilityMask <-sysuMediaWmhSegmentation( image ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "flair.nii.gz" ) probabilityMask <-sysuMediaWmhSegmentation( image ) ## End(Not run)
Function for translating the min/max ground truth box coordinates to something expected by the SSD network. This is a SSD-specific analog for keras::to_categorical(). For each image in the batch, we compare the ground truth boxes for that image with all the anchor boxes. If the overlap measure exceeds a specific threshold, we write the ground truth box coordinates and class to the specific position of the matched anchor box. Note that the background class will be assigned to all the anchor boxes for which there was no match with any ground truth box. However, an exception to this are the anchor boxes whose overlap measure is higher that the specified negative threshold.
encodeSsd2D( groundTruthLabels, anchorBoxes, imageSize, variances = rep(1, 4), foregroundThreshold = 0.5, backgroundThreshold = 0.2 )encodeSsd2D( groundTruthLabels, anchorBoxes, imageSize, variances = rep(1, 4), foregroundThreshold = 0.5, backgroundThreshold = 0.2 )
groundTruthLabels |
A list of length |
anchorBoxes |
a list of 2-D arrays where each element comprises the anchor boxes for a specific aspect ratios layer. The row of each 2-D array comprises a single box specified in the form (xmin,xmax,ymin,ymax). |
imageSize |
2-D vector specifying the spatial domain of the input images. |
variances |
A list of 4 floats > 0 with scaling factors (actually it's
not factors but divisors to be precise) for the encoded predicted box
coordinates. A variance value of 1.0 would apply no scaling at all to the
predictions, while values in |
foregroundThreshold |
float between 0 and 1 determining the min threshold
for matching an anchor box with a ground truth box and, thus, labeling an anchor
box as a non-background class. If an anchor box exceeds the |
backgroundThreshold |
float between 0 and 1 determining the max threshold
for labeling an anchor box as |
This particular implementation was heavily influenced by the following python and R implementations:
\url{https://github.com/pierluigiferrari/ssd_keras}
\url{https://github.com/rykov8/ssd_keras}
\url{https://github.com/gsimchoni/ssdkeras}
a 3-D array of shape (batchSize, numberOfBoxes,
numberOfClasses + 4 + 4 + 4)
where the additional 4's along the third dimension correspond to the 4 predicted box coordinate offsets, the 4 coordinates for the anchor boxes, and the 4 variance values.
Tustison NJ
Function for translating the min/max ground truth box coordinates to something expected by the SSD network. This is a SSD-specific analog for keras::to_categorical(). For each image in the batch, we compare the ground truth boxes for that image with all the anchor boxes. If the overlap measure exceeds a specific threshold, we write the ground truth box coordinates and class to the specific position of the matched anchor box. Note that the background class will be assigned to all the anchor boxes for which there was no match with any ground truth box. However, an exception to this are the anchor boxes whose overlap measure is higher that the specified negative threshold.
encodeSsd3D( groundTruthLabels, anchorBoxes, imageSize, variances = rep(1, 6), foregroundThreshold = 0.5, backgroundThreshold = 0.2 )encodeSsd3D( groundTruthLabels, anchorBoxes, imageSize, variances = rep(1, 6), foregroundThreshold = 0.5, backgroundThreshold = 0.2 )
groundTruthLabels |
A list of length |
anchorBoxes |
a list of 2-D arrays where each element comprises the anchor boxes for a specific aspect ratios layer. The row of each 2-D array comprises a single box specified in the for (xmin,xmax,ymin,ymax,zmin,zmax). |
imageSize |
3-D vector specifying the spatial domain of the input images. |
variances |
A list of 6 floats > 0 with scaling factors (actually it's
not factors but divisors to be precise) for the encoded predicted box
coordinates. A variance value of 1.0 would apply no scaling at all to the
predictions, while values in (0,1) upscale the encoded predictions and
values greater than 1.0 downscale the encoded predictions. These are the same
variances used to construct the model. Default =
|
foregroundThreshold |
float between 0 and 1 determining the min threshold
for matching an anchor box with a ground truth box and, thus, labeling an anchor
box as a non-background class. If an anchor box exceeds the |
backgroundThreshold |
float between 0 and 1 determining the max threshold
for labeling an anchor box as |
This particular implementation was heavily influenced by the following python and R implementations:
\url{https://github.com/pierluigiferrari/ssd_keras}
\url{https://github.com/rykov8/ssd_keras}
\url{https://github.com/gsimchoni/ssdkeras}
a 3-D array of shape (batchSize, numberOfBoxes,
numberOfClasses + 6 + 6 + 6)
where the additional 6's along the third dimension correspond to
the 6 predicted box coordinate offsets, the 6 coordinates for
the anchor boxes, and the 6 variance values.
Tustison NJ
Adapted from
https://github.com/keras-team/keras-io/blob/master/examples/generative/ddpm.py
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
This function returns the current cache directory used by ANTsXNet to store downloaded templates and model weights. If no directory has been set, it initializes it to the default path "~/.keras/ANTsXNet".
getANTsXNetCacheDirectory()getANTsXNetCacheDirectory()
A character string specifying the path to the cache directory.
Download data such as prefabricated templates and spatial priors.
getANTsXNetData( fileId = c("show", "biobank", "croppedMni152", "croppedMni152Priors", "deepFlashPriors", "deepFlashTemplateT1", "deepFlashTemplateT1SkullStripped", "deepFlashTemplateT2", "deepFlashTemplateT2SkullStripped", "deepFlashTemplate2T1SkullStripped", "deepFlashTemplate2Labels", "mprage_hippmapp3r", "protonLungTemplate", "ctLungTemplate", "luna16LungPriors", "xrayLungPriors", "priorDktLabels", "priorDeepFlashLeftLabels", "priorDeepFlashRightLabels", "protonLobePriors", "S_template3", "S_template3_brain", "adni", "ixi", "kirby", "mni152", "nki", "nki10", "oasis", "magetTemplate", "magetTemplateBrainMask", "magetCerebellumTemplate", "magetCerebellumTemplatePriors", "magetCerebellumxTemplate0GenericAffine", "mraTemplate", "mraTemplateBrainMask", "mraTemplateVesselPrior", "bsplineT2MouseTemplate", "bsplineT2MouseTemplateBrainMask", "DevCCF_P56_MRI_T2_50um", "DevCCF_P56_MRI_T2_50um_BrainParcellationNickMask", "DevCCF_P56_MRI_T2_50um_BrainParcellationTctMask", "DevCCF_P04_STPT_50um", "DevCCF_P04_STPT_50um_BrainParcellationJayMask", "hcpaT1Template", "hcpaT2Template", "hcpaFATemplate", "hcpyaT1Template", "hcpyaT2Template", "hcpyaFATemplate", "hcpyaTemplateBrainMask", "hcpyaTemplateBrainSegmentation", "hcpinterT1Template", "hcpinterT2Template", "hcpinterFATemplate", "hcpinterTemplateBrainMask", "hcpinterTemplateBrainSegmentation", "CIT168_T1w_700um_pad_adni", "CIT168_basal_forebrain_adni", "det_atlas_25_pad_LR_adni"), targetFileName )getANTsXNetData( fileId = c("show", "biobank", "croppedMni152", "croppedMni152Priors", "deepFlashPriors", "deepFlashTemplateT1", "deepFlashTemplateT1SkullStripped", "deepFlashTemplateT2", "deepFlashTemplateT2SkullStripped", "deepFlashTemplate2T1SkullStripped", "deepFlashTemplate2Labels", "mprage_hippmapp3r", "protonLungTemplate", "ctLungTemplate", "luna16LungPriors", "xrayLungPriors", "priorDktLabels", "priorDeepFlashLeftLabels", "priorDeepFlashRightLabels", "protonLobePriors", "S_template3", "S_template3_brain", "adni", "ixi", "kirby", "mni152", "nki", "nki10", "oasis", "magetTemplate", "magetTemplateBrainMask", "magetCerebellumTemplate", "magetCerebellumTemplatePriors", "magetCerebellumxTemplate0GenericAffine", "mraTemplate", "mraTemplateBrainMask", "mraTemplateVesselPrior", "bsplineT2MouseTemplate", "bsplineT2MouseTemplateBrainMask", "DevCCF_P56_MRI_T2_50um", "DevCCF_P56_MRI_T2_50um_BrainParcellationNickMask", "DevCCF_P56_MRI_T2_50um_BrainParcellationTctMask", "DevCCF_P04_STPT_50um", "DevCCF_P04_STPT_50um_BrainParcellationJayMask", "hcpaT1Template", "hcpaT2Template", "hcpaFATemplate", "hcpyaT1Template", "hcpyaT2Template", "hcpyaFATemplate", "hcpyaTemplateBrainMask", "hcpyaTemplateBrainSegmentation", "hcpinterT1Template", "hcpinterT2Template", "hcpinterFATemplate", "hcpinterTemplateBrainMask", "hcpinterTemplateBrainSegmentation", "CIT168_T1w_700um_pad_adni", "CIT168_basal_forebrain_adni", "det_atlas_25_pad_LR_adni"), targetFileName )
fileId |
one of the permitted file ids or pass "show" to list all valid possibilities. Note that most require internet access to download. |
targetFileName |
optional target filename |
filename string
See https://figshare.com/authors/Nick_Tustison/441144 or https://figshare.com/authors/Brian_Avants/418551 for some more descriptions
Avants BB
## Not run: net <- getANTsXNetData("biobank") ## End(Not run)## Not run: net <- getANTsXNetData("biobank") ## End(Not run)
Ported from:
getMixtureDensityLossFunction(outputDimension, numberOfMixtures)getMixtureDensityLossFunction(outputDimension, numberOfMixtures)
outputDimension |
output dimension |
numberOfMixtures |
number of mixture components |
https://github.com/cpmpercussion/keras-mdn-layer/
a function providing the mean square error accuracy
Tustison NJ
library( keras )library( keras )
Ported from:
getMixtureDensityMseAccuracyFunction(outputDimension, numberOfMixtures)getMixtureDensityMseAccuracyFunction(outputDimension, numberOfMixtures)
outputDimension |
output dimension |
numberOfMixtures |
number of mixture components |
https://github.com/cpmpercussion/keras-mdn-layer/
a function providing the mean square error accuracy
Tustison NJ
library( keras )library( keras )
Ported from:
getMixtureDensitySamplingFunction(outputDimension, numberOfMixtures)getMixtureDensitySamplingFunction(outputDimension, numberOfMixtures)
outputDimension |
output dimension |
numberOfMixtures |
number of mixture components |
https://github.com/cpmpercussion/keras-mdn-layer/
a function for sampling a mixture density
Tustison NJ
library( keras )library( keras )
Downloads pretrained network/weights.
getPretrainedNetwork( fileId = c("show", "antsxnetWmh", "antsxnetWmhOr", "arterialLesionWeibinShi", "brainAgeGender", "brainAgeFmrib", "brainAgeDeepBrainNet", "brainExtraction", "brainExtractionT1", "brainExtractionT1v1", "brainExtractionRobustT1", "brainExtractionT2", "brainExtractionRobustT2", "brainExtractionRobustT2Star", "brainExtractionFLAIR", "brainExtractionRobustFLAIR", "brainExtractionBOLD", "brainExtractionRobustBOLD", "brainExtractionFA", "brainExtractionMra", "brainExtractionRobustFA", "brainExtractionNoBrainer", "brainExtractionInfantT1T2", "brainExtractionInfantT1", "brainExtractionInfantT2", "brainExtractionBrainWeb20", "brainExtractionT1Hemi", "brainExtractionT1Lobes", "brainSegmentation", "brainSegmentationPatchBased", "bratsStage1", "bratsStage2", "cerebellumWhole", "cerebellumTissue", "cerebellumLabels", "claustrum_axial_0", "claustrum_axial_1", "claustrum_axial_2", "claustrum_coronal_0", "claustrum_coronal_1", "claustrum_coronal_2", "ctHumanLung", "dbpn4x", "deepFlash", "deepFlashLeftT1", "deepFlashRightT1", "deepFlashLeftBoth", "deepFlashRightBoth", "deepFlashLeftT1Hierarchical", "deepFlashRightT1Hierarchical", "deepFlashLeftBothHierarchical", "deepFlashRightBothHierarchical", "deepFlashLeftT1Hierarchical_ri", "deepFlashRightT1Hierarchical_ri", "deepFlashLeftBothHierarchical_ri", "deepFlashRightBothHierarchical_ri", "deepFlash2LeftT1Hierarchical", "deepFlash2RightT1Hierarchical", "deepFlashLeft8", "deepFlashRight8", "deepFlashLeft16", "deepFlashRight16", "deepFlashLeft16new", "deepFlashRight16new", "denoising", "dktInner", "dktOuter", "dktOuterWithSpatialPriors", "DesikanKillianyTourvilleOuter", "HarvardOxfordAtlasSubcortical", "elBicho", "ex5_coronal_weights", "ex5_sagittal_weights", "allen_brain_mask_weights", "functionalLungMri", "hippMapp3rInitial", "hippMapp3rRefine", "hyperMapp3r", "hypothalamus", "inpainting_sagittal_rmnet_weights", "inpainting_coronal_rmnet_weights", "inpainting_axial_rmnet_weights", "inpainting_axial_rmnet_flair_weights", "inpainting_coronal_rmnet_flair_weights", "inpainting_sagittal_rmnet_flair_weights", "koniqMBCS", "koniqMS", "koniqMS2", "koniqMS3", "lesion_whole_brain", "lungCtWithPriorsSegmentationWeights", "maskLobes", "mriSuperResolution", "mriModalityClassification", "mraVesselWeights_160", "mouseMriBrainExtraction", "mouseT2wBrainExtraction3D", "mouseT2wBrainParcellation3DNick", "mouseT2wBrainParcellation3DTct", "mouseSTPTBrainParcellation3DJay", "pvs_shiva_t1_0", "pvs_shiva_t1_1", "pvs_shiva_t1_2", "pvs_shiva_t1_3", "pvs_shiva_t1_4", "pvs_shiva_t1_5", "pvs_shiva_t1_flair_0", "pvs_shiva_t1_flair_1", "pvs_shiva_t1_flair_2", "pvs_shiva_t1_flair_3", "pvs_shiva_t1_flair_4", "wmh_shiva_flair_0", "wmh_shiva_flair_1", "wmh_shiva_flair_2", "wmh_shiva_flair_3", "wmh_shiva_flair_4", "wmh_shiva_t1_flair_0", "wmh_shiva_t1_flair_1", "wmh_shiva_t1_flair_2", "wmh_shiva_t1_flair_3", "wmh_shiva_t1_flair_4", "protonLungMri", "protonLobes", "pulmonaryArteryWeights", "pulmonaryAirwayWeights", "sixTissueOctantBrainSegmentation", "sixTissueOctantBrainSegmentationWithPriors1", "sixTissueOctantBrainSegmentationWithPriors2", "sysuMediaWmhFlairOnlyModel0", "sysuMediaWmhFlairOnlyModel1", "sysuMediaWmhFlairOnlyModel2", "sysuMediaWmhFlairT1Model0", "sysuMediaWmhFlairT1Model1", "sysuMediaWmhFlairT1Model2", "tidsQualityAssessment", "xrayLungOrientation", "xrayLungExtraction", "chexnetClassificationModel", "chexnetClassificationANTsXNetModel", "tb_antsxnet_model", "wholeHeadInpaintingFLAIR", "wholeHeadInpaintingPatchBasedT1", "wholeHeadInpaintingPatchBasedFLAIR", "wholeTumorSegmentationT2Flair", "wholeLungMaskFromVentilation", "DeepAtroposHcpT1Weights", "DeepAtroposHcpT1T2Weights", "DeepAtroposHcpT1FAWeights", "DeepAtroposHcpT1T2FAWeights", "sig_smallshort_train_1x1x2_1chan_featgraderL6_best_mdl", "sig_smallshort_train_1x1x2_1chan_featvggL6_best_mdl", "sig_smallshort_train_1x1x3_1chan_featgraderL6_best_mdl", "sig_smallshort_train_1x1x3_1chan_featvggL6_best_mdl", "sig_smallshort_train_1x1x4_1chan_featgraderL6_best_mdl", "sig_smallshort_train_1x1x4_1chan_featvggL6_best_mdl", "sig_smallshort_train_1x1x6_1chan_featvggL6_best_mdl", "sig_smallshort_train_2x2x2_1chan_featgraderL6_best_mdl", "sig_smallshort_train_2x2x2_1chan_featvggL6_best_mdl", "sig_smallshort_train_2x2x4_1chan_featgraderL6_best_mdl", "sig_smallshort_train_2x2x4_1chan_featvggL6_best_mdl", "deep_nbm_rank", "deepCIT168", "deepCIT168_sn", "resnet_grader"), targetFileName )getPretrainedNetwork( fileId = c("show", "antsxnetWmh", "antsxnetWmhOr", "arterialLesionWeibinShi", "brainAgeGender", "brainAgeFmrib", "brainAgeDeepBrainNet", "brainExtraction", "brainExtractionT1", "brainExtractionT1v1", "brainExtractionRobustT1", "brainExtractionT2", "brainExtractionRobustT2", "brainExtractionRobustT2Star", "brainExtractionFLAIR", "brainExtractionRobustFLAIR", "brainExtractionBOLD", "brainExtractionRobustBOLD", "brainExtractionFA", "brainExtractionMra", "brainExtractionRobustFA", "brainExtractionNoBrainer", "brainExtractionInfantT1T2", "brainExtractionInfantT1", "brainExtractionInfantT2", "brainExtractionBrainWeb20", "brainExtractionT1Hemi", "brainExtractionT1Lobes", "brainSegmentation", "brainSegmentationPatchBased", "bratsStage1", "bratsStage2", "cerebellumWhole", "cerebellumTissue", "cerebellumLabels", "claustrum_axial_0", "claustrum_axial_1", "claustrum_axial_2", "claustrum_coronal_0", "claustrum_coronal_1", "claustrum_coronal_2", "ctHumanLung", "dbpn4x", "deepFlash", "deepFlashLeftT1", "deepFlashRightT1", "deepFlashLeftBoth", "deepFlashRightBoth", "deepFlashLeftT1Hierarchical", "deepFlashRightT1Hierarchical", "deepFlashLeftBothHierarchical", "deepFlashRightBothHierarchical", "deepFlashLeftT1Hierarchical_ri", "deepFlashRightT1Hierarchical_ri", "deepFlashLeftBothHierarchical_ri", "deepFlashRightBothHierarchical_ri", "deepFlash2LeftT1Hierarchical", "deepFlash2RightT1Hierarchical", "deepFlashLeft8", "deepFlashRight8", "deepFlashLeft16", "deepFlashRight16", "deepFlashLeft16new", "deepFlashRight16new", "denoising", "dktInner", "dktOuter", "dktOuterWithSpatialPriors", "DesikanKillianyTourvilleOuter", "HarvardOxfordAtlasSubcortical", "elBicho", "ex5_coronal_weights", "ex5_sagittal_weights", "allen_brain_mask_weights", "functionalLungMri", "hippMapp3rInitial", "hippMapp3rRefine", "hyperMapp3r", "hypothalamus", "inpainting_sagittal_rmnet_weights", "inpainting_coronal_rmnet_weights", "inpainting_axial_rmnet_weights", "inpainting_axial_rmnet_flair_weights", "inpainting_coronal_rmnet_flair_weights", "inpainting_sagittal_rmnet_flair_weights", "koniqMBCS", "koniqMS", "koniqMS2", "koniqMS3", "lesion_whole_brain", "lungCtWithPriorsSegmentationWeights", "maskLobes", "mriSuperResolution", "mriModalityClassification", "mraVesselWeights_160", "mouseMriBrainExtraction", "mouseT2wBrainExtraction3D", "mouseT2wBrainParcellation3DNick", "mouseT2wBrainParcellation3DTct", "mouseSTPTBrainParcellation3DJay", "pvs_shiva_t1_0", "pvs_shiva_t1_1", "pvs_shiva_t1_2", "pvs_shiva_t1_3", "pvs_shiva_t1_4", "pvs_shiva_t1_5", "pvs_shiva_t1_flair_0", "pvs_shiva_t1_flair_1", "pvs_shiva_t1_flair_2", "pvs_shiva_t1_flair_3", "pvs_shiva_t1_flair_4", "wmh_shiva_flair_0", "wmh_shiva_flair_1", "wmh_shiva_flair_2", "wmh_shiva_flair_3", "wmh_shiva_flair_4", "wmh_shiva_t1_flair_0", "wmh_shiva_t1_flair_1", "wmh_shiva_t1_flair_2", "wmh_shiva_t1_flair_3", "wmh_shiva_t1_flair_4", "protonLungMri", "protonLobes", "pulmonaryArteryWeights", "pulmonaryAirwayWeights", "sixTissueOctantBrainSegmentation", "sixTissueOctantBrainSegmentationWithPriors1", "sixTissueOctantBrainSegmentationWithPriors2", "sysuMediaWmhFlairOnlyModel0", "sysuMediaWmhFlairOnlyModel1", "sysuMediaWmhFlairOnlyModel2", "sysuMediaWmhFlairT1Model0", "sysuMediaWmhFlairT1Model1", "sysuMediaWmhFlairT1Model2", "tidsQualityAssessment", "xrayLungOrientation", "xrayLungExtraction", "chexnetClassificationModel", "chexnetClassificationANTsXNetModel", "tb_antsxnet_model", "wholeHeadInpaintingFLAIR", "wholeHeadInpaintingPatchBasedT1", "wholeHeadInpaintingPatchBasedFLAIR", "wholeTumorSegmentationT2Flair", "wholeLungMaskFromVentilation", "DeepAtroposHcpT1Weights", "DeepAtroposHcpT1T2Weights", "DeepAtroposHcpT1FAWeights", "DeepAtroposHcpT1T2FAWeights", "sig_smallshort_train_1x1x2_1chan_featgraderL6_best_mdl", "sig_smallshort_train_1x1x2_1chan_featvggL6_best_mdl", "sig_smallshort_train_1x1x3_1chan_featgraderL6_best_mdl", "sig_smallshort_train_1x1x3_1chan_featvggL6_best_mdl", "sig_smallshort_train_1x1x4_1chan_featgraderL6_best_mdl", "sig_smallshort_train_1x1x4_1chan_featvggL6_best_mdl", "sig_smallshort_train_1x1x6_1chan_featvggL6_best_mdl", "sig_smallshort_train_2x2x2_1chan_featgraderL6_best_mdl", "sig_smallshort_train_2x2x2_1chan_featvggL6_best_mdl", "sig_smallshort_train_2x2x4_1chan_featgraderL6_best_mdl", "sig_smallshort_train_2x2x4_1chan_featvggL6_best_mdl", "deep_nbm_rank", "deepCIT168", "deepCIT168_sn", "resnet_grader"), targetFileName )
fileId |
one of the permitted file ids or pass "show" to list all valid possibilities. Note that most require internet access to download. |
targetFileName |
optional target filename |
filename string
See https://figshare.com/authors/Nick_Tustison/441144 or https://figshare.com/authors/Brian_Avants/418551 for some more descriptions
Avants BB
## Not run: net <- getPretrainedNetwork( "dbpn4x" ) ## End(Not run)## Not run: net <- getPretrainedNetwork( "dbpn4x" ) ## End(Not run)
A fast and simple metric that correlates to perceptual quality
GMSD(x, y)GMSD(x, y)
x |
input image. |
y |
input image. |
scalar
Avants BB
library( ANTsR ) r16 <- antsImageRead( getANTsRData( 'r16' ) ) r85 <- antsImageRead( getANTsRData( 'r85' ) ) value <- GMSD( r16, r85 )library( ANTsR ) r16 <- antsImageRead( getANTsRData( 'r16' ) ) r85 <- antsImageRead( getANTsRData( 'r85' ) ) value <- GMSD( r16, r85 )
Perform HOA labeling using deep learning and data from Morphology" number: "NIH NIMH 5R01MH112748-04". Repository: https://github.com/HOA-2/SubcorticalParcellations'
harvardOxfordAtlasLabeling(t1, doPreprocessing = TRUE, verbose = FALSE)harvardOxfordAtlasLabeling(t1, doPreprocessing = TRUE, verbose = FALSE)
t1 |
raw or preprocessed 3-D T1-weighted brain image. |
doPreprocessing |
perform preprocessing. See description above. |
verbose |
print progress. |
The labeling is as follows:
Label 1:Lateral Ventricle Left
Label 2:Lateral Ventricle Right
Label 3:CSF
Label 4:Third Ventricle
Label 5:Fourth Ventricle
Label 6:5th Ventricle
Label 7:Nucleus Accumbens Left
Label 8:Nucleus Accumbens Right
Label 9:Caudate Left
Label 10:Caudate Right
Label 11:Putamen Left
Label 12:Putamen Right
Label 13:Globus Pallidus Left
Label 14:Globus Pallidus Right
Label 15:Brainstem
Label 16:Thalamus Left
Label 17:Thalamus Right
Label 18:Inferior Horn of the Lateral Ventricle Left
Label 19:Inferior Horn of the Lateral Ventricle Right
Label 20:Hippocampal Formation Left
Label 21:Hippocampal Formation Right
Label 22:Amygdala Left
Label 23:Amygdala Right
Label 24:Optic Chiasm
Label 25:VDC Anterior Left
Label 26:VDC Anterior Right
Label 27:VDC Posterior Left
Label 28:VDC Posterior Right
Label 29:Cerebellar Cortex Left
Label 30:Cerebellar Cortex Right
Label 31:Cerebellar White Matter Left
Label 32:Cerebellar White Matter Right
Preprocessing on the training data consisted of:
n4 bias correction,
brain extraction, and
affine registration to HCP.
The input T1 should undergo the same steps. If the input T1 is the raw
T1, these steps can be performed by the internal preprocessing, i.e. set
doPreprocessing = TRUE
list consisting of the segmentation image and probability images for each label.
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1.nii.gz" ) results <- harvardOxfordAtlasLabeling( image ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1.nii.gz" ) results <- harvardOxfordAtlasLabeling( image ) ## End(Not run)
Perform HippMapp3r (hippocampal) segmentation described in
hippMapp3rSegmentation(t1, doPreprocessing = TRUE, verbose = FALSE)hippMapp3rSegmentation(t1, doPreprocessing = TRUE, verbose = FALSE)
doPreprocessing |
perform preprocessing. See description above. |
verbose |
print progress. |
image |
input 3-D T1-weighted brain image. |
https://www.ncbi.nlm.nih.gov/pubmed/31609046
with models and architecture ported from
https://github.com/AICONSlab/HippMapp3r
Additional documentation and attribution resources found at
https://hippmapp3r.readthedocs.io/en/latest/
Preprocessing consists of:
n4 bias correction and
brain extraction.
The input T1 should undergo the same steps. If the input T1 is the raw
T1, these steps can be performed by the internal preprocessing, i.e. set
doPreprocessing = TRUE
labeled hippocampal mask (ANTsR image)
Tustison NJ
## Not run: library( ANTsRNet ) library( ANTsRNet ) library( keras ) url <- "https://github.com/mgoubran/HippMapp3r/blob/master/data/test_case/mprage.nii.gz?raw=true" imageFile <- "head.nii.gz" download.file( url, imageFile ) image <- antsImageRead( imageFile ) imageN4 <- n4BiasFieldCorrection( image, verbose = TRUE ) segmentation <- hippMapp3rSegmentation( imageN4, verbose = TRUE ) ## End(Not run)## Not run: library( ANTsRNet ) library( ANTsRNet ) library( keras ) url <- "https://github.com/mgoubran/HippMapp3r/blob/master/data/test_case/mprage.nii.gz?raw=true" imageFile <- "head.nii.gz" download.file( url, imageFile ) image <- antsImageRead( imageFile ) imageN4 <- n4BiasFieldCorrection( image, verbose = TRUE ) segmentation <- hippMapp3rSegmentation( imageN4, verbose = TRUE ) ## End(Not run)
Perform HyperMapp3r (white matter hyperintensity) segmentation described in
hyperMapp3rSegmentation( t1, flair, doPreprocessing = TRUE, numberOfMonteCarloIterations = 30, verbose = FALSE )hyperMapp3rSegmentation( t1, flair, doPreprocessing = TRUE, numberOfMonteCarloIterations = 30, verbose = FALSE )
t1 |
input 3-D t1-weighted MR image. Assumed to be aligned with the flair. |
flair |
input 3-D flair MR image. Assumed to be aligned with the t1. |
doPreprocessing |
perform preprocessing. See description above. |
verbose |
print progress. |
https://pubmed.ncbi.nlm.nih.gov/35088930/
with models and architecture ported from
https://github.com/AICONSlab/HyperMapp3r
Additional documentation and attribution resources found at
https://hypermapp3r.readthedocs.io/en/latest/
Preprocessing consists of:
n4 bias correction and
brain extraction.
The input T1 should undergo the same steps. If the input T1 is the raw
T1, these steps can be performed by the internal preprocessing, i.e. set
doPreprocessing = TRUE
white matter hyperintensity probability mask
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) wmh <- hyperMapp3rSegmentation( image, verbose = TRUE ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) wmh <- hyperMapp3rSegmentation( image, verbose = TRUE ) ## End(Not run)
Described here:
hypothalamusSegmentation(t1, verbose = FALSE)hypothalamusSegmentation(t1, verbose = FALSE)
t1 |
input 3-D T1-weighted brain image. |
verbose |
print progress. |
\url{https://pubmed.ncbi.nlm.nih.gov/32853816/}
with the implementation available at:
\url{https://github.com/BBillot/hypothalamus_seg}
hypothalamic subunit segmentation and probability images
Label 1: left anterior-inferior
Label 2: left anterior-superior
Label 3: left posterior
Label 4: left tubular inferior
Label 5: left tubular superior
Label 6: right anterior-inferior
Label 7: right anterior-superior
Label 8: right posterior
Label 9: right tubular inferior
Label 10: right tubular superior
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1.nii.gz" ) hypo <- hypothalamusSegmentation( image ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1.nii.gz" ) hypo <- hypothalamusSegmentation( image ) ## End(Not run)
Improved Wasserstein generative adverserial network (with gradient penalty) from the paper:
https://arxiv.org/abs/1704.00028
and ported from the Keras (python) implementation:
https://github.com/eriklindernoren/Keras-GAN/blob/master/wgan_gp/wgan_gp.py
$initialize instantiates a new class and builds the
generator and critic.
$buildGeneratorbuild generator.
$buildGeneratorbuild critic.
Tustison NJ
library( keras ) library( ANTsRNet ) keras::backend()$clear_session() # Let's use the mnist data set. mnist <- dataset_mnist() numberOfTrainingData <- length( mnist$train$y ) inputImageSize <- c( dim( mnist$train$x[1,,] ), 1 ) x <- array( data = mnist$train$x / 255, dim = c( numberOfTrainingData, inputImageSize ) ) y <- mnist$train$y numberOfClusters <- length( unique( mnist$train$y ) ) # Instantiate the WGAN model ganModel <- ImprovedWassersteinGanModel$new( inputImageSize = inputImageSize, latentDimension = 100 ) ## Not run: ganModel$train( x, numberOfEpochs = 2 ) ## End(Not run) tryCatch({tensorflow::tf$compat$v1$enable_eager_execution()}, silent = TRUE, error = function(e) {})library( keras ) library( ANTsRNet ) keras::backend()$clear_session() # Let's use the mnist data set. mnist <- dataset_mnist() numberOfTrainingData <- length( mnist$train$y ) inputImageSize <- c( dim( mnist$train$x[1,,] ), 1 ) x <- array( data = mnist$train$x / 255, dim = c( numberOfTrainingData, inputImageSize ) ) y <- mnist$train$y numberOfClusters <- length( unique( mnist$train$y ) ) # Instantiate the WGAN model ganModel <- ImprovedWassersteinGanModel$new( inputImageSize = inputImageSize, latentDimension = 100 ) ## Not run: ganModel$train( x, numberOfEpochs = 2 ) ## End(Not run) tryCatch({tensorflow::tf$compat$v1$enable_eager_execution()}, silent = TRUE, error = function(e) {})
Original generative adverserial network (GAN) model from the paper:
https://arxiv.org/abs/1801.07892
and ported from the (TensorFlow) implementation:
https://github.com/JiahuiYu/generative_inpainting
$initialize instantiates a new class and builds the
generator and discriminator.
$buildGeneratorbuild generator.
$buildGeneratorbuild discriminator.
Tustison NJ
x = InpaintingDeepFillModel$new(c( 28, 28, 1 )) ## Not run: x$buildNetwork() ## End(Not run)x = InpaintingDeepFillModel$new(c( 28, 28, 1 )) ## Not run: x$buildNetwork() ## End(Not run)
Creates an instance normalization layer as described in the paper
https://arxiv.org/abs/1701.02096
with the implementation ported from the following python implementation
an instance normalization layer
Integer specifying which axis should be normalized, typically
the feature axis. For example, after a Conv2D layer with
channels_first, set axis = 2. Setting axis=-1L will
normalize all values in each instance of the batch. Axis 1
is the batch dimension for tensorflow backend so we throw an
error if axis = 1.
Small float added to the variance to avoid dividing by 0.
If TRUE, add beta offset to normalized tensor.
If TRUE, multiply by gamma.
Intializer for the beta weight.
Intializer for the gamma weight.
Regularizer for the beta weight.
Regularizer for the gamma weight.
Optional constraint for the beta weight.
Optional constraint for the gamma weight.
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
InstanceNormalizationLayer$new() InstanceNormalizationLayer$new(axis = 2L) testthat::expect_error(InstanceNormalizationLayer$new(axis = 1L))InstanceNormalizationLayer$new() InstanceNormalizationLayer$new(axis = 2L) testthat::expect_error(InstanceNormalizationLayer$new(axis = 1L))
Function for determinining the Jaccard or iou (intersection over union) similarity measure between two sets of boxes.
jaccardSimilarity(boxes1, boxes2)jaccardSimilarity(boxes1, boxes2)
boxes1 |
A 2-D array where each row corresponds to a single box consisting of the format (xmin,xmax,ymin,ymax) or (xmin,xmax,ymin,ymax,zmin,zmax) |
boxes2 |
A 2-D array where each row corresponds to a single box consisting of the format (xmin,xmax,ymin,ymax) or (xmin,xmax,ymin,ymax,zmin,zmax) |
the Jaccard simliarity
Tustison NJ
vec2d = c(1, 10, 1, 10) vec2d_2 = c(1, 8, 1, 5) jaccardSimilarity(vec2d, vec2d_2) vec3d = c(1, 10, 1, 10, 1, 10) vec3d_2 = c(1, 8, 1, 5, 1, 10) jaccardSimilarity(vec3d, vec3d_2)vec2d = c(1, 10, 1, 10) vec2d_2 = c(1, 8, 1, 5) jaccardSimilarity(vec2d, vec2d_2) vec3d = c(1, 10, 1, 10, 1, 10) vec3d_2 = c(1, 8, 1, 5, 1, 10) jaccardSimilarity(vec3d, vec3d_2)
L2 2-D normalization layer for SSD300/512 architecture described in
Wei Liu, Andrew Rabinovich, and Alexander C. Berg. ParseNet: Looking Wider to See Better.
available here:
\code{https://arxiv.org/abs/1506.04579}
output tensor with the same shape as the input.
layer <- L2NormalizationLayer2D$new( scale ) layer$call( x, mask = NULL ) layer$build( input_shape ) layer$compute_output_shape( input_shape )
A process object.
feature scale. Default = 20
$initialize instantiates a new class.
$build
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
x = L2NormalizationLayer2D$new() ## Not run: x$build(input_shape = c(20, 20, 20, 3)) ## End(Not run)x = L2NormalizationLayer2D$new() ## Not run: x$build(input_shape = c(20, 20, 20, 3)) ## End(Not run)
L2 3-D normalization layer for SSD300/512 architecture described in
Wei Liu, Andrew Rabinovich, and Alexander C. Berg. ParseNet: Looking Wider to See Better.
available here:
\code{https://arxiv.org/abs/1506.04579}
output tensor with the same shape as the input.
layer <- L2NormalizationLayer3D$new( scale ) layer$call( x, mask = NULL ) layer$build( input_shape ) layer$compute_output_shape( input_shape )
A process object.
feature scale. Default = 20
$initialize instantiates a new class.
$build
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
x = L2NormalizationLayer3D$new() ## Not run: x$build(input_shape = c(20, 20, 20, 20, 4)) ## End(Not run)x = L2NormalizationLayer3D$new() ## Not run: x$build(input_shape = c(20, 20, 20, 20, 4)) ## End(Not run)
Creates a log softmax layer
layer_activation_log_softmax(object, axis = -1, trainable = TRUE)layer_activation_log_softmax(object, axis = -1, trainable = TRUE)
axis |
Integer specifying which axis. |
trainable |
Whether the layer weights will be updated during training. |
a keras layer tensor
Tustison NJ
Wraps a custom layer for the SSD network
layer_anchor_box_2d( object, imageSize, scale, nextScale, aspectRatios, variances, name = NULL, trainable = TRUE ) layer_anchor_box_3d( object, imageSize, scale, nextScale, aspectRatios, variances, name = NULL, trainable = TRUE )layer_anchor_box_2d( object, imageSize, scale, nextScale, aspectRatios, variances, name = NULL, trainable = TRUE ) layer_anchor_box_3d( object, imageSize, scale, nextScale, aspectRatios, variances, name = NULL, trainable = TRUE )
object |
Object to compose layer with. This is either a keras::keras_model_sequential to add the layer to, or another Layer which this layer will call. |
imageSize |
size of the image, passed to |
scale |
box scale, passed to |
nextScale |
box scale, passed to |
aspectRatios |
list of ratios used for the boxes,
passed to |
variances |
list of variances, passed to |
name |
The name of the layer |
trainable |
logical indicating if it is trainable or not |
a keras layer tensor
Wraps the AttentionLayer2D taken from the following python implementation
layer_attention_2d( object, numberOfChannels, doGoogleBrainVersion = TRUE, trainable = TRUE )layer_attention_2d( object, numberOfChannels, doGoogleBrainVersion = TRUE, trainable = TRUE )
object |
Object to compose layer with. This is either a keras::keras_model_sequential to add the layer to or another Layer which this layer will call. |
numberOfChannels |
numberOfChannels |
doGoogleBrainVersion |
boolean. Variant described at second url. |
trainable |
Whether the layer weights will be updated during training. |
https://stackoverflow.com/questions/50819931/self-attention-gan-in-keras https://github.com/taki0112/Self-Attention-GAN-Tensorflow
based on the following paper:
https://arxiv.org/abs/1805.08318
a keras layer tensor
## Not run: library( keras ) library( ANTsRNet ) inputShape <- c( 100, 100, 3 ) input <- layer_input( shape = inputShape ) numberOfFilters <- 64 outputs <- input %>% layer_conv_2d( filters = numberOfFilters, kernel_size = 2 ) outputs <- outputs %>% layer_attention_2d( numberOfFilters ) model <- keras_model( inputs = input, outputs = outputs ) ## End(Not run)## Not run: library( keras ) library( ANTsRNet ) inputShape <- c( 100, 100, 3 ) input <- layer_input( shape = inputShape ) numberOfFilters <- 64 outputs <- input %>% layer_conv_2d( filters = numberOfFilters, kernel_size = 2 ) outputs <- outputs %>% layer_attention_2d( numberOfFilters ) model <- keras_model( inputs = input, outputs = outputs ) ## End(Not run)
Wraps the AttentionLayer3D taken from the following python implementation
layer_attention_3d( object, numberOfChannels, doGoogleBrainVersion = TRUE, trainable = TRUE )layer_attention_3d( object, numberOfChannels, doGoogleBrainVersion = TRUE, trainable = TRUE )
object |
Object to compose layer with. This is either a keras::keras_model_sequential to add the layer to or another Layer which this layer will call. |
numberOfChannels |
numberOfChannels |
doGoogleBrainVersion |
boolean. Variant described at second url. |
trainable |
Whether the layer weights will be updated during training. |
https://stackoverflow.com/questions/50819931/self-attention-gan-in-keras https://github.com/taki0112/Self-Attention-GAN-Tensorflow
based on the following paper:
https://arxiv.org/abs/1805.08318
a keras layer tensor
## Not run: library( keras ) library( ANTsRNet ) inputShape <- c( 100, 100, 100, 3 ) input <- layer_input( shape = inputShape ) numberOfFilters <- 64 outputs <- input %>% layer_conv_3d( filters = numberOfFilters, kernel_size = 2 ) outputs <- outputs %>% layer_attention_3d( numberOfFilters ) model <- keras_model( inputs = input, outputs = outputs ) ## End(Not run)## Not run: library( keras ) library( ANTsRNet ) inputShape <- c( 100, 100, 100, 3 ) input <- layer_input( shape = inputShape ) numberOfFilters <- 64 outputs <- input %>% layer_conv_3d( filters = numberOfFilters, kernel_size = 2 ) outputs <- outputs %>% layer_attention_3d( numberOfFilters ) model <- keras_model( inputs = input, outputs = outputs ) ## End(Not run)
Wraps the AttentionAugmentation2D layer.
layer_attention_augmentation_2d( object, depthOfQueries, depthOfValues, numberOfHeads, isRelative, trainable = TRUE )layer_attention_augmentation_2d( object, depthOfQueries, depthOfValues, numberOfHeads, isRelative, trainable = TRUE )
object |
Object to compose layer with. This is either a keras::keras_model_sequential to add the layer to, or another Layer which this layer will call. |
depthOfQueries |
number of filters for queries. |
depthOfValues |
number of filters for values. |
numberOfHeads |
number of attention heads to use. It is required
that |
isRelative |
whether or not to use relative encodings. |
trainable |
Whether the layer weights will be updated during training. |
a keras layer tensor
Creates a 2-D attention augmented convolutional layer as described in the paper
layer_attention_augmented_convolution_block_2d( inputLayer, numberOfOutputFilters, kernelSize = c(3, 3), strides = c(1, 1), depthOfQueries = 0.2, depthOfValues = 0.2, numberOfAttentionHeads = 8, useRelativeEncodings = TRUE )layer_attention_augmented_convolution_block_2d( inputLayer, numberOfOutputFilters, kernelSize = c(3, 3), strides = c(1, 1), depthOfQueries = 0.2, depthOfValues = 0.2, numberOfAttentionHeads = 8, useRelativeEncodings = TRUE )
inputLayer |
input keras layer. |
numberOfOutputFilters |
number of output filters. |
kernelSize |
convolution kernel size. |
strides |
convolution strides. |
depthOfQueries |
Defines the number of filters for the queries or |
depthOfValues |
Defines the number of filters for the values or |
numberOfAttentionHeads |
number of attention heads. Note that
|
useRelativeEncodings |
boolean for whether to use relative encodings (default = TRUE). |
https://arxiv.org/abs/1904.09925
with the implementation ported from the following repository
https://github.com/titu1994/keras-attention-augmented-convs
a keras tensor
Tustison NJ
Contextual attention layer for generative image inpainting described in
layer_contextual_attention_2d( object, kernelSize = 3L, stride = 1L, dilationRate = 1L, fusionKernelSize = 0L, name = NULL, trainable = FALSE ) layer_contextual_attention_3d( object, kernelSize = 3L, stride = 1L, dilationRate = 1L, fusionKernelSize = 0L, name = NULL, trainable = FALSE )layer_contextual_attention_2d( object, kernelSize = 3L, stride = 1L, dilationRate = 1L, fusionKernelSize = 0L, name = NULL, trainable = FALSE ) layer_contextual_attention_3d( object, kernelSize = 3L, stride = 1L, dilationRate = 1L, fusionKernelSize = 0L, name = NULL, trainable = FALSE )
object |
Object to compose layer with. This is either a keras::keras_model_sequential to add the layer to, or another Layer which this layer will call. |
kernelSize |
integer specifying convolution size |
stride |
integer for specifyingstride length for sampling the tensor |
dilationRate |
ingeger specifying dilation |
fusionKernelSize |
Enhance saliency of large patches |
name |
The name of the layer |
trainable |
Whether the layer weights will be updated during training. |
Jiahui Yu, et al., Generative Image Inpainting with Contextual Attention, CVPR 2018.
available here:
\code{https://arxiv.org/abs/1801.07892}
a keras layer tensor
layer_contextual_attention_2d() layer_contextual_attention_3d() keras::keras_model_sequential() %>% layer_contextual_attention_2d(fusionKernelSize = 2) keras::keras_model_sequential() %>% layer_contextual_attention_3d()layer_contextual_attention_2d() layer_contextual_attention_3d() keras::keras_model_sequential() %>% layer_contextual_attention_2d(fusionKernelSize = 2) keras::keras_model_sequential() %>% layer_contextual_attention_3d()
Wraps the EfficientAttentionLayer2D modified from the following python implementation
layer_efficient_attention_2d( object, numberOfFiltersFG = 4L, numberOfFiltersH = 8L, kernelSize = 1L, poolSize = 2L, doConcatenateFinalLayers = FALSE, trainable = TRUE )layer_efficient_attention_2d( object, numberOfFiltersFG = 4L, numberOfFiltersH = 8L, kernelSize = 1L, poolSize = 2L, doConcatenateFinalLayers = FALSE, trainable = TRUE )
object |
Object to compose layer with. This is either a keras::keras_model_sequential to add the layer to or another Layer which this layer will call. |
numberOfFiltersFG |
number of filters for F and G layers. |
numberOfFiltersH |
number of filters for H. If |
kernelSize |
kernel size in convolution layer. |
poolSize |
pool size in max pool layer. |
doConcatenateFinalLayers |
concatenate final layer with input. Alternatively, add. Default = FALSE |
https://github.com/taki0112/Self-Attention-GAN-Tensorflow
based on the following paper:
https://arxiv.org/abs/1805.08318
a keras layer tensor
## Not run: library( keras ) library( ANTsRNet ) inputShape <- c( 100, 100, 3 ) input <- layer_input( shape = inputShape ) numberOfFiltersFG <- 64L outputs <- input %>% layer_efficient_attention_2d( numberOfFiltersFG ) model <- keras_model( inputs = input, outputs = outputs ) ## End(Not run)## Not run: library( keras ) library( ANTsRNet ) inputShape <- c( 100, 100, 3 ) input <- layer_input( shape = inputShape ) numberOfFiltersFG <- 64L outputs <- input %>% layer_efficient_attention_2d( numberOfFiltersFG ) model <- keras_model( inputs = input, outputs = outputs ) ## End(Not run)
Wraps the EfficientAttentionLayer3D modified from the following python implementation
layer_efficient_attention_3d( object, numberOfFiltersFG = 4L, numberOfFiltersH = 8L, kernelSize = 1L, poolSize = 2L, doConcatenateFinalLayers = FALSE, trainable = TRUE )layer_efficient_attention_3d( object, numberOfFiltersFG = 4L, numberOfFiltersH = 8L, kernelSize = 1L, poolSize = 2L, doConcatenateFinalLayers = FALSE, trainable = TRUE )
object |
Object to compose layer with. This is either a keras::keras_model_sequential to add the layer to or another Layer which this layer will call. |
numberOfFiltersFG |
number of filters for F and G layers. |
numberOfFiltersH |
number of filters for H. If |
kernelSize |
kernel size in convolution layer. |
poolSize |
pool size in max pool layer. |
doConcatenateFinalLayers |
concatenate final layer with input. Alternatively, add. Default = FALSE |
https://github.com/taki0112/Self-Attention-GAN-Tensorflow
based on the following paper:
https://arxiv.org/abs/1805.08318
a keras layer tensor
## Not run: library( keras ) library( ANTsRNet ) inputShape <- c( 100, 100, 100, 3 ) input <- layer_input( shape = inputShape ) numberOfFiltersFG <- 64L outputs <- input %>% layer_efficient_attention_3d( numberOfFiltersFG ) model <- keras_model( inputs = input, outputs = outputs ) ## End(Not run)## Not run: library( keras ) library( ANTsRNet ) inputShape <- c( 100, 100, 100, 3 ) input <- layer_input( shape = inputShape ) numberOfFiltersFG <- 64L outputs <- input %>% layer_efficient_attention_3d( numberOfFiltersFG ) model <- keras_model( inputs = input, outputs = outputs ) ## End(Not run)
Creates an instance normalization layer
layer_instance_normalization( object, axis = NULL, epsilon = 0.001, center = TRUE, scale = TRUE, betaInitializer = "zeros", gammaInitializer = "ones", betaRegularizer = NULL, gammaRegularizer = NULL, betaConstraint = NULL, gammaConstraint = NULL, trainable = TRUE )layer_instance_normalization( object, axis = NULL, epsilon = 0.001, center = TRUE, scale = TRUE, betaInitializer = "zeros", gammaInitializer = "ones", betaRegularizer = NULL, gammaRegularizer = NULL, betaConstraint = NULL, gammaConstraint = NULL, trainable = TRUE )
object |
Object to compose layer with. This is either a keras::keras_model_sequential to add the layer to, or another Layer which this layer will call. |
axis |
Integer specifying which axis should be normalized, typically
the feature axis. For example, after a Conv2D layer with
|
epsilon |
Small float added to the variance to avoid dividing by 0. |
center |
If TRUE, add |
scale |
If TRUE, multiply by |
betaInitializer |
Intializer for the beta weight. |
gammaInitializer |
Intializer for the gamma weight. |
betaRegularizer |
Regularizer for the beta weight. |
gammaRegularizer |
Regularizer for the gamma weight. |
betaConstraint |
Optional constraint for the beta weight. |
gammaConstraint |
Optional constraint for the gamma weight. |
trainable |
Whether the layer weights will be updated during training. |
a keras layer tensor
Tustison NJ
Wraps a custom layer for the SSD network
layer_l2_normalization_2d(object, scale = 20, name = NULL, trainable = TRUE) layer_l2_normalization_3d(object, scale = 20, name = NULL, trainable = TRUE)layer_l2_normalization_2d(object, scale = 20, name = NULL, trainable = TRUE) layer_l2_normalization_3d(object, scale = 20, name = NULL, trainable = TRUE)
object |
Object to compose layer with. This is either a keras::keras_model_sequential to add the layer to, or another Layer which this layer will call. |
scale |
box scale |
name |
The name of the layer |
trainable |
Whether the layer weights will be updated during training. |
a keras layer tensor
Wraps a custom mixture density layer.
layer_mixture_density( object, outputDimension, numberOfMixtures, trainable = TRUE )layer_mixture_density( object, outputDimension, numberOfMixtures, trainable = TRUE )
object |
Object to compose layer with. This is either a keras::keras_model_sequential to add the layer to, or another Layer which this layer will call. |
outputDimension |
output dimension |
numberOfMixtures |
number of Gaussians used to model the function |
trainable |
Whether the layer weights will be updated during training. |
a keras layer tensor
## Not run: model <- keras_model_sequential() inputShape = c(20, 20, 1) model = model %>% layer_conv_2d(filters = 32, kernel_size = c(3,3), activation = 'relu', input_shape = inputShape) model %>% layer_mixture_density(outputDimension = c(18, 18), numberOfMixtures = 3) ## End(Not run)## Not run: model <- keras_model_sequential() inputShape = c(20, 20, 1) model = model %>% layer_conv_2d(filters = 32, kernel_size = c(3,3), activation = 'relu', input_shape = inputShape) model %>% layer_mixture_density(outputDimension = c(18, 18), numberOfMixtures = 3) ## End(Not run)
Creates an 2D partial convolution layer
layer_partial_conv_2d( object, filters, kernelSize, strides = c(1L, 1L), padding = "valid", dataFormat = "channels_last", dilationRate = c(1L, 1L), activation = NULL, kernelInitializer = "glorot_uniform", biasInitializer = "zeros", kernelRegularizer = NULL, biasRegularizer = NULL, activityRegularizer = NULL, useBias = TRUE, trainable = TRUE )layer_partial_conv_2d( object, filters, kernelSize, strides = c(1L, 1L), padding = "valid", dataFormat = "channels_last", dilationRate = c(1L, 1L), activation = NULL, kernelInitializer = "glorot_uniform", biasInitializer = "zeros", kernelRegularizer = NULL, biasRegularizer = NULL, activityRegularizer = NULL, useBias = TRUE, trainable = TRUE )
object |
Object to compose layer with. This is either a keras::keras_model_sequential to add the layer to, or another Layer which this layer will call. |
filters |
number of filters |
kernelSize |
kernel size |
strides |
strides |
padding |
padding |
dataFormat |
format |
dilationRate |
dilate rate |
activation |
activation |
kernelInitializer |
kernel initializer |
biasInitializer |
bias initializer |
kernelRegularizer |
kernel regularizer |
biasRegularizer |
bias regularizer |
activityRegularizer |
activity regularizer |
useBias |
use bias |
trainable |
Whether the layer weights will be updated during training. |
a keras layer tensor
Tustison NJ
Creates an 3D partial convolution layer
layer_partial_conv_3d( object, filters, kernelSize, strides = c(1L, 1L, 1L), padding = "valid", dataFormat = "channels_last", dilationRate = c(1L, 1L, 1L), activation = NULL, kernelInitializer = "glorot_uniform", biasInitializer = "zeros", kernelRegularizer = NULL, biasRegularizer = NULL, activityRegularizer = NULL, useBias = TRUE, trainable = TRUE )layer_partial_conv_3d( object, filters, kernelSize, strides = c(1L, 1L, 1L), padding = "valid", dataFormat = "channels_last", dilationRate = c(1L, 1L, 1L), activation = NULL, kernelInitializer = "glorot_uniform", biasInitializer = "zeros", kernelRegularizer = NULL, biasRegularizer = NULL, activityRegularizer = NULL, useBias = TRUE, trainable = TRUE )
object |
Object to compose layer with. This is either a keras::keras_model_sequential to add the layer to, or another Layer which this layer will call. |
filters |
number of filters |
kernelSize |
kernel size |
strides |
strides |
padding |
padding |
dataFormat |
format |
dilationRate |
dilate rate |
activation |
activation |
kernelInitializer |
kernel initializer |
biasInitializer |
bias initializer |
kernelRegularizer |
kernel regularizer |
biasRegularizer |
bias regularizer |
activityRegularizer |
activity regularizer |
useBias |
use bias |
trainable |
Whether the layer weights will be updated during training. |
a keras layer tensor
Tustison NJ
Resamples a spatial tensor based on the specified shape and interpolation type.
layer_resample_tensor_2d( object, shape, interpolationType = c("nearestNeighbor", "nearest", "linear", "bilinear", "cubic", "bicubic"), name = NULL, trainable = FALSE )layer_resample_tensor_2d( object, shape, interpolationType = c("nearestNeighbor", "nearest", "linear", "bilinear", "cubic", "bicubic"), name = NULL, trainable = FALSE )
object |
Object to compose layer with. This is either a keras::keras_model_sequential to add the layer to, or another Layer which this layer will call. |
shape |
vector or list of length 2 specifying the shape of the output tensor. |
interpolationType |
type of interpolation for resampling. Can be
|
name |
The name of the layer |
trainable |
Whether the layer weights will be updated during training. |
a keras layer tensor
Tustison NJ
Resamples a spatial tensor based on the specified shape and interpolation type.
layer_resample_tensor_3d( object, shape, interpolationType = c("nearestNeighbor", "nearest", "linear", "bilinear", "cubic", "bicubic"), name = NULL, trainable = FALSE )layer_resample_tensor_3d( object, shape, interpolationType = c("nearestNeighbor", "nearest", "linear", "bilinear", "cubic", "bicubic"), name = NULL, trainable = FALSE )
object |
Object to compose layer with. This is either a keras::keras_model_sequential to add the layer to, or another Layer which this layer will call. |
shape |
vector or list of length 3 specifying the shape of the output tensor. |
interpolationType |
type of interpolation for resampling. Can be
|
name |
The name of the layer |
trainable |
Whether the layer weights will be updated during training. |
a keras layer tensor
Tustison NJ
Resamples a spatial tensor based on a target tensor and interpolation type.
layer_resample_tensor_to_target_tensor_2d( object, interpolationType = c("nearestNeighbor", "nearest", "linear", "bilinear", "cubic", "bicubic"), name = NULL, trainable = FALSE )layer_resample_tensor_to_target_tensor_2d( object, interpolationType = c("nearestNeighbor", "nearest", "linear", "bilinear", "cubic", "bicubic"), name = NULL, trainable = FALSE )
object |
Object to compose layer with. This is either a keras::keras_model_sequential to add the layer to, or another Layer which this layer will call. |
interpolationType |
type of interpolation for resampling. Can be
|
name |
The name of the layer |
trainable |
Whether the layer weights will be updated during training. |
a keras layer tensor
Tustison NJ
Resamples a spatial tensor based on the specified shape and interpolation type.
layer_resample_tensor_to_target_tensor_3d( object, interpolationType = c("nearestNeighbor", "nearest", "linear", "bilinear", "cubic", "bicubic"), name = NULL, trainable = FALSE )layer_resample_tensor_to_target_tensor_3d( object, interpolationType = c("nearestNeighbor", "nearest", "linear", "bilinear", "cubic", "bicubic"), name = NULL, trainable = FALSE )
object |
Object to compose layer with. This is either a keras::keras_model_sequential to add the layer to, or another Layer which this layer will call. |
interpolationType |
type of interpolation for resampling. Can be
|
name |
The name of the layer |
trainable |
Whether the layer weights will be updated during training. |
a keras layer tensor
Tustison NJ
Wraps a custom spatial transformer layer
layer_spatial_transformer_2d( object, resampledSize, transformType = "affine", interpolatorType = "linear", name = NULL ) layer_spatial_transformer_3d( object, resampledSize, transformType = "affine", interpolatorType = "linear", name = NULL )layer_spatial_transformer_2d( object, resampledSize, transformType = "affine", interpolatorType = "linear", name = NULL ) layer_spatial_transformer_3d( object, resampledSize, transformType = "affine", interpolatorType = "linear", name = NULL )
object |
Object to compose layer with. This is either a keras::keras_model_sequential to add the layer to, or another Layer which this layer will call. |
resampledSize |
size of the output in voxels |
transformType |
the spatial transform |
interpolatorType |
interpolation used for the sampling |
name |
The name of the layer |
a keras layer tensor
## Not run: model <- keras_model_sequential() input_shape = c(20, 20, 1) model = model %>% layer_conv_2d(filters = 32, kernel_size = c(3,3), activation = 'relu', input_shape = input_shape) model %>% layer_spatial_transformer_2d(resampledSize = c(50, 50)) model %>% layer_spatial_transformer_2d(resampledSize = c(50, 50, 1)) ## End(Not run)## Not run: model <- keras_model_sequential() input_shape = c(20, 20, 1) model = model %>% layer_conv_2d(filters = 32, kernel_size = c(3,3), activation = 'relu', input_shape = input_shape) model %>% layer_spatial_transformer_2d(resampledSize = c(50, 50)) model %>% layer_spatial_transformer_2d(resampledSize = c(50, 50, 1)) ## End(Not run)
Perform in-painting for whole-head MRI
lesionSegmentation(t1, doPreprocessing = TRUE, verbose = FALSE)lesionSegmentation(t1, doPreprocessing = TRUE, verbose = FALSE)
t1 |
input 3-D T1 brain image (not skull-stripped). |
doPreprocessing |
Perform n4 bias correction, intensity truncation, brain extraction. |
verbose |
print progress. |
inpainted image
Tustison NJ
regression between two image intensity spaces
linMatchIntensity(fromImg, toImg, polyOrder = 1, truncate = TRUE, mask)linMatchIntensity(fromImg, toImg, polyOrder = 1, truncate = TRUE, mask)
fromImg |
image whose intensity function we will match to the |
toImg |
defines the reference intensity function. |
polyOrder |
of polynomial fit. default is none or just linear fit. |
truncate |
boolean which turns on/off the clipping of intensity. |
mask |
mask the matching region |
the fromImg matched to the toImg
Avants BB
library(ANTsRCore) sourceImage <- antsImageRead( getANTsRData( "r16" ) ) referenceImage <- antsImageRead( getANTsRData( "r64" ) ) matchedImage <- linMatchIntensity( sourceImage, referenceImage )library(ANTsRCore) sourceImage <- antsImageRead( getANTsRData( "r16" ) ) referenceImage <- antsImageRead( getANTsRData( "r64" ) ) matchedImage <- linMatchIntensity( sourceImage, referenceImage )
Creates a log softmax layer taken from
https://github.com/tensorflow/tensorflow/pull/25514/files
a log softmax layer
Integer specifying the axis.
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
Perform KellyKapowski cortical thickness longitudinally using deepAtropos
for segmentation of the derived single-subject template. It takes inspiration from
the work described here:
longitudinalCorticalThickness( t1s, initialTemplate = "oasis", numberOfIterations = 1, refinementTransform = "antsRegistrationSyNQuick[a]", verbose = FALSE )longitudinalCorticalThickness( t1s, initialTemplate = "oasis", numberOfIterations = 1, refinementTransform = "antsRegistrationSyNQuick[a]", verbose = FALSE )
t1s |
input list of 3-D unprocessed T1-weighted brain images from a single subject |
initialTemplate |
input image to define the orientation of the SST. Can be a string
(see |
numberOfIterations |
Defines the number of iterations for refining the SST. |
refinementTransform |
Transform for defining the refinement registration transform.
See options in |
verbose |
print progress. |
https://pubmed.ncbi.nlm.nih.gov/31356207/
List consisting of the SST, and a (sub-)list for each subject consisting of the preprocessed image, cortical thickness image, segmentation probability images, and affine mapping to the SST.
Tustison NJ, Avants BB
## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1w_image.nii.gz" ) kk <- corticalThickness( image ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1w_image.nii.gz" ) kk <- corticalThickness( image ) ## End(Not run)
Creates an R6 class object for use with the SSD deep learning architecture based on the paper
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C-Y. Fu, A. Berg. SSD: Single Shot MultiBox Detector.
available here:
\url{https://arxiv.org/abs/1512.02325}
an SSD loss function
ssdLoss <- LossSSD$new( dimension = 2L, backgroundRatio = 3L, minNumberOfBackgroundBoxes = 0L, alpha = 1.0, numberOfClassificationLabels ) ssdLoss$smooth_l1_loss( y_true, y_pred ) ssdLoss$log_loss( y_true, y_pred ) ssdLoss$compute_loss( y_true, y_pred )
A process object.
image dimensionality.
The maximum ratio of background to foreround for weighting in the loss function. Is rounded to the nearest integer. Default is 3.
The minimum number of background boxes to use in loss computation per batch. Should reflect a value in proportion to the batch size. Default is 0.
Weighting factor for the localization loss in total loss computation.
number of classes including background.
$smooth_l1_loss smooth loss
$log_loss log loss
$compute_loss computes total loss.
Tustison NJ
Perform pulmonary artery segmentation. Training data taken from the EXACT09 challenge (Lo, Pechin, et al. "Extraction of airways from CT (EXACT'09)." https://pubmed.ncbi.nlm.nih.gov/22855226/)
lungAirwaySegmentation( ct, lungMask = NULL, predictionBatchSize = 16, patchStrideLength = 32, verbose = FALSE )lungAirwaySegmentation( ct, lungMask = NULL, predictionBatchSize = 16, patchStrideLength = 32, verbose = FALSE )
ct |
input 3-D ct image. |
lungMask |
input binary lung mask which defines the patch extraction (label 1 = left lung, label 2 = right lung, label 3 = main airway). If not supplied, one is estimated. |
predictionBatchSize |
Control memory usage for prediction. More consequential for GPU-usage. |
patchStrideLength |
3-D vector or int. Dictates the stride length for accumulating predicting patches. |
verbose |
print progress. |
Probability image.
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) ## End(Not run)
Perform lung extraction.
lungExtraction( image, modality = c("proton", "protonLobes", "maskLobes", "ct", "ventilation", "xray"), verbose = FALSE )lungExtraction( image, modality = c("proton", "protonLobes", "maskLobes", "ct", "ventilation", "xray"), verbose = FALSE )
image |
input 3-D lung image. |
modality |
image type. Options include "proton", "ct", or "ventilation". |
verbose |
print progress. |
segmentation and probability images
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "lung.nii.gz" ) output <- lungExtraction( image, modality = "proton" ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "lung.nii.gz" ) output <- lungExtraction( image, modality = "proton" ) ## End(Not run)
Perform pulmonary artery segmentation. Training data taken from the PARSE2022 challenge (Luo, Gongning, et al. "Efficient automatic segmentation for multi-level pulmonary arteries: The PARSE challenge." https://arxiv.org/abs/2304.03708).
lungPulmonaryArterySegmentation( ct, lungMask = NULL, predictionBatchSize = 16, patchStrideLength = 32, verbose = FALSE )lungPulmonaryArterySegmentation( ct, lungMask = NULL, predictionBatchSize = 16, patchStrideLength = 32, verbose = FALSE )
ct |
input 3-D ct image. |
lungMask |
input binary lung mask which defines the patch extraction. If not supplied, one is estimated. |
predictionBatchSize |
Control memory usage for prediction. More consequential for GPU-usage. |
patchStrideLength |
3-D vector or int. Dictates the stride length for accumulating predicting patches. |
verbose |
print progress. |
Probability image.
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) ## End(Not run)
Mean absolute error of a single image or between two images.
MAE(x, y = NULL)MAE(x, y = NULL)
x |
input image. |
y |
input image. |
the mean absolute error
Avants BB (from redr)
library( ANTsR ) r16 <- antsImageRead( getANTsRData( 'r16' ) ) r85 <- antsImageRead( getANTsRData( 'r85' ) ) maeValue <- MAE( r16, r85 )library( ANTsR ) r16 <- antsImageRead( getANTsRData( 'r16' ) ) r85 <- antsImageRead( getANTsRData( 'r85' ) ) maeValue <- MAE( r16, r85 )
Function for masked mean-squared error
masked_mse_error(y_true, y_pred, maskExcludeValue = 1)masked_mse_error(y_true, y_pred, maskExcludeValue = 1)
y_true |
True labels (Tensor) |
y_pred |
Predictions (Tensor of the same shape as |
masked mse value
Tustison NJ
https://jmlr.csail.mit.edu/papers/volume13/gretton12a/gretton12a.pdf
maximum_mean_discrepancy(y_true, y_pred, sigma = 1)maximum_mean_discrepancy(y_true, y_pred, sigma = 1)
y_true |
True labels (Tensor) |
y_pred |
Predictions (Tensor of the same shape as |
mmd value
Tustison NJ
Ported from:
mixture_density_network_softmax(logits, temperature = 1)mixture_density_network_softmax(logits, temperature = 1)
logits |
input of logits/mixture weights to adjust |
temperature |
the temperature for to adjust the distribution (default 1.0) |
https://github.com/cpmpercussion/keras-mdn-layer/
softmax loss value
Tustison NJ
library( keras )library( keras )
Mixture density network layer
A Keras Layer
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Create custom density layers for each parameter of the mixed Gaussians. (mu, sigma, pi). I could not get the approach from the original implementation to work:
https://github.com/cpmpercussion/keras-mdn-layer/blob/master/mdn/init.py#L28-L73
where the author used the keras dense layers to create the custom MDN layer and assign the trainable weights directly thus circumventing the add_weight() function. Instead, I recreated dense layer functionality using the keras definition here:
https://github.com/keras-team/keras/blob/master/keras/layers/core.py#L796-L937
Tustison NJ
MixtureDensityNetworkLayer$new(outputDimension = c(50, 48), numberOfMixtures = 3)MixtureDensityNetworkLayer$new(outputDimension = c(50, 48), numberOfMixtures = 3)
Perform brain extraction of mouse MRI
mouseBrainExtraction( image, modality = c("t2", "ex5coronal", "ex5sagittal"), returnIsotropicOutput = FALSE, whichAxis = 2, verbose = FALSE )mouseBrainExtraction( image, modality = c("t2", "ex5coronal", "ex5sagittal"), returnIsotropicOutput = FALSE, whichAxis = 2, verbose = FALSE )
image |
input 3-D brain image (or list of images for multi-modal scenarios). |
modality |
image type. Options include:
|
returnIsotropicOutput |
The network actually learns an interpolating function specific to the mouse brain. Setting this to true, the output images are returned isotropically resampled. |
whichAxis |
Specify direction for ex5 modalities.. |
verbose |
print progress. |
brain probability mask
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "brain.nii.gz" ) probabilityMask <- mouseBrainExtraction( image, modality = "t2" ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "brain.nii.gz" ) probabilityMask <- mouseBrainExtraction( image, modality = "t2" ) ## End(Not run)
Perform brain extraction of mouse T2 MRI
mouseBrainParcellation( image, mask = NULL, returnIsotropicOutput = FALSE, whichParcellation = c("nick", "jay", "tct"), verbose = FALSE )mouseBrainParcellation( image, mask = NULL, returnIsotropicOutput = FALSE, whichParcellation = c("nick", "jay", "tct"), verbose = FALSE )
image |
input 3-D brain image (or list of images for multi-modal scenarios). |
mask |
Brain mask. If not specified, one is estimated using ANTsXNet mouse brain extraction. |
returnIsotropicOutput |
The network actually learns an interpolating function specific to the mouse brain. Setting this to true, the output images are returned isotropically resampled. |
whichParcellation |
Brain parcellation type:
|
verbose |
print progress. |
brain probability mask
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "brain.nii.gz" ) parcellation <- mouseBrainParcellation( image, modality = "t2" ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "brain.nii.gz" ) parcellation <- mouseBrainParcellation( image, modality = "t2" ) ## End(Not run)
Perform KellyKapowski cortical thickness
mouseCorticalThickness( t2, mask = NULL, returnIsotropicOutput = FALSE, verbose = FALSE )mouseCorticalThickness( t2, mask = NULL, returnIsotropicOutput = FALSE, verbose = FALSE )
t2 |
input 3-D unprocessed T2-weighted mouse brain image |
mask |
Brain mask. If not specified, one is estimated using ANTsXNet mouse brain extraction. |
returnIsotropicOutput |
The network actually learns an interpolating function specific to the mouse brain. Setting this to true, the output images are returned isotropically resampled. |
verbose |
print progress. |
Cortical thickness image and segmentation probability images.
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1w_image.nii.gz" ) kk <- corticalThickness( image ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "t1w_image.nii.gz" ) kk <- corticalThickness( image ) ## End(Not run)
Predict MRI modality type (whole-head only). Modalities:
T1
T2
FLAIR
T2Star
Mean DWI
Mean Bold
ASL perfusion
mriModalityClassification(image, verbose = FALSE)mriModalityClassification(image, verbose = FALSE)
image |
raw 3-D MRI whole head image. |
verbose |
print progress. |
classification data frame
Tustison NJ
## Not run: library( ANTsRNet ) image <- antsImageRead( getANTsXNetData( "mprageHippmap3r" ) ) classification <- mriModalityClassification( image ) ## End(Not run)## Not run: library( ANTsRNet ) image <- antsImageRead( getANTsXNetData( "mprageHippmap3r" ) ) classification <- mriModalityClassification( image ) ## End(Not run)
Perform super-resolution of MRI data using deep back projection network. Work described in
https://www.medrxiv.org/content/10.1101/2023.02.02.23285376v1
with the GitHub repo located at https://github.com/stnava/siq
Note that some preprocessing possibilities for the input includes:
Truncate intensity (see ants.iMath(..., 'TruncateIntensity', ...)
mriSuperResolution( image, expansionFactor = c(1, 1, 2), feature = "vgg", targetRange = c(1, 0), polyOrder = "hist", verbose = FALSE )mriSuperResolution( image, expansionFactor = c(1, 1, 2), feature = "vgg", targetRange = c(1, 0), polyOrder = "hist", verbose = FALSE )
image |
magnetic resonance image |
expansionFactor |
Specifies the increase in resolution per
dimension. Possibilities include: |
feature |
"grader" or "vgg" |
targetRange |
Range for applySuperResolutionModel. |
polyOrder |
int or the string 'hist'. Parameter for regression matching or specification of histogram matching. |
verbose |
print progress. |
super-resolution image.
Avants BB
## Not run: library( ANTsRNet ) image <- antsImageRead( "t1.nii.gz" ) imageSr <- mriSuperResolution( image ) ## End(Not run)## Not run: library( ANTsRNet ) image <- antsImageRead( "t1.nii.gz" ) imageSr <- mriSuperResolution( image ) ## End(Not run)
Mean square error of a single image or between two images.
MSE(x, y = NULL)MSE(x, y = NULL)
x |
input image. |
y |
input image. |
the mean squared error
Avants BB (from redr)
library( ANTsR ) r16 <- antsImageRead( getANTsRData( 'r16' ) ) r85 <- antsImageRead( getANTsRData( 'r85' ) ) mseValue <- MSE( r16, r85 )library( ANTsR ) r16 <- antsImageRead( getANTsRData( 'r16' ) ) r85 <- antsImageRead( getANTsRData( 'r85' ) ) mseValue <- MSE( r16, r85 )
Note: Assumption is that y_true is a one-hot representation of the segmentation batch. The background (label 0) should be included but is not used in the calculation.
multilabel_dice_coefficient( y_true, y_pred, dimensionality = 3L, smoothingFactor = 0 )multilabel_dice_coefficient( y_true, y_pred, dimensionality = 3L, smoothingFactor = 0 )
y_true |
True labels (Tensor) |
y_pred |
Predictions (Tensor of the same shape as |
dimensionality |
image dimension. |
smoothingFactor |
parameter for smoothing the metric. |
Dice value (negative)
Tustison NJ
library( ANTsR ) library( ANTsRNet ) library( keras ) model <- createUnetModel2D( c( 64, 64, 1 ) ) dice_loss <- multilabel_dice_coefficient( smoothingFactor = 0.1 ) model %>% compile( loss = dice_loss, optimizer = optimizer_adam( lr = 0.0001 ) ) ######################################## # # Run in isolation # library( ANTsR ) r16 <- antsImageRead( getANTsRData( "r16" ) ) r16seg <- kmeansSegmentation( r16, 3 )$segmentation r16array <- array( data = as.array( r16seg ), dim = c( 1, dim( r16seg ) ) ) r16tensor <- tensorflow::tf$convert_to_tensor( encodeUnet( r16array, c( 0, 1, 2, 3 ) ) ) r64 <- antsImageRead( getANTsRData( "r64" ) ) r64seg <- kmeansSegmentation( r64, 3 )$segmentation r64array <- array( data = as.array( r64seg ), dim = c( 1, dim( r64seg ) ) ) r64tensor <- tensorflow::tf$convert_to_tensor( encodeUnet( r64array, c( 0, 1, 2, 3 ) ) ) dice_loss <- multilabel_dice_coefficient( r16tensor, r64tensor, dimensionality = 2L ) loss_value <- dice_loss( r16tensor, r64tensor )$numpy() # Compare with # overlap_value <- labelOverlapMeasures( r16seg, r64seg )$MeanOverlap[1] rm(model); gc()library( ANTsR ) library( ANTsRNet ) library( keras ) model <- createUnetModel2D( c( 64, 64, 1 ) ) dice_loss <- multilabel_dice_coefficient( smoothingFactor = 0.1 ) model %>% compile( loss = dice_loss, optimizer = optimizer_adam( lr = 0.0001 ) ) ######################################## # # Run in isolation # library( ANTsR ) r16 <- antsImageRead( getANTsRData( "r16" ) ) r16seg <- kmeansSegmentation( r16, 3 )$segmentation r16array <- array( data = as.array( r16seg ), dim = c( 1, dim( r16seg ) ) ) r16tensor <- tensorflow::tf$convert_to_tensor( encodeUnet( r16array, c( 0, 1, 2, 3 ) ) ) r64 <- antsImageRead( getANTsRData( "r64" ) ) r64seg <- kmeansSegmentation( r64, 3 )$segmentation r64array <- array( data = as.array( r64seg ), dim = c( 1, dim( r64seg ) ) ) r64tensor <- tensorflow::tf$convert_to_tensor( encodeUnet( r64array, c( 0, 1, 2, 3 ) ) ) dice_loss <- multilabel_dice_coefficient( r16tensor, r64tensor, dimensionality = 2L ) loss_value <- dice_loss( r16tensor, r64tensor )$numpy() # Compare with # overlap_value <- labelOverlapMeasures( r16seg, r64seg )$MeanOverlap[1] rm(model); gc()
Perform CH13 and Nucleus Basalis of Meynert (NBM) segmentation in T1 images using Avants labels.
nbmLabeling(t1, verbose = FALSE)nbmLabeling(t1, verbose = FALSE)
t1 |
input ANTs image (3D). |
verbose |
Print progress to the screen. |
The labeling is as follows:
Label 1: CH13_left
Label 2: CH13_right
Label 3: NBM_left_ant
Label 4: NBM_left_mid
Label 5: NBM_left_pos
Label 6: NBM_right_ant
Label 7: NBM_right_mid
Label 8: NBM_right_pos
Preprocessing consists of:
n4 bias correction
brain extraction
A list containing the ANTs segmentation label image and probability images.
Tustison NJ
## Not run: library( ANTsR ) library( ANTsRNet ) image <- antsImageRead( getANTsRData( "r16" ) ) seg <- nbmLabeling( image ) ## End(Not run)## Not run: library( ANTsR ) library( ANTsRNet ) image <- antsImageRead( getANTsRData( "r16" ) ) seg <- nbmLabeling( image ) ## End(Not run)
The popular neural style transfer described here:
neuralStyleTransfer( contentImage, styleImages, initialCombinationImage = NULL, numberOfIterations = 10, learningRate = 1, totalVariationWeight = 8.5e-05, contentWeight = 0.025, styleImageWeights = 1, contentLayerNames = c("block5_conv2"), styleLayerNames = "all", contentMask = NULL, styleMasks = NULL, useShiftedActivations = TRUE, useChainedInference = TRUE, verbose = FALSE, outputPrefix = NULL )neuralStyleTransfer( contentImage, styleImages, initialCombinationImage = NULL, numberOfIterations = 10, learningRate = 1, totalVariationWeight = 8.5e-05, contentWeight = 0.025, styleImageWeights = 1, contentLayerNames = c("block5_conv2"), styleLayerNames = "all", contentMask = NULL, styleMasks = NULL, useShiftedActivations = TRUE, useChainedInference = TRUE, verbose = FALSE, outputPrefix = NULL )
contentImage |
ANTs image (1 or 3-component). Content (or base) image. |
styleImages |
ANTsImage or list of ANTsImages as the style (or reference) image. |
initialCombinationImage |
ANTsImage (1 or 3-component). Starting point for the optimization. Allows one to start from the output from a previous run. Otherwise, start from the content image. Note that the original paper starts with a noise image. |
numberOfIterations |
Number of gradient steps taken during optimization. |
learningRate |
Parameter for Adam optimization. |
totalVariationWeight |
A penalty on the regularization term to keep the features of the output image locally coherent. |
contentWeight |
Weight of the content layers in the optimization function. |
styleImageWeights |
float or vector of floats. Weights of the style term in the optimization function for each style image. Can either specify a single scalar to be used for all the images or one for each image. The style term computes the sum of the L2 norm between the Gram matrices of the different layers (using ImageNet-trained VGG) of the style and content images. |
contentLayerNames |
vector of strings. Names of VGG layers from which to compute the content loss. |
styleLayerNames |
vector of strings. Names of VGG layers from which to compute the style loss. If "all", the layers used are c('block1_conv1', 'block1_conv2', 'block2_conv1', 'block2_conv2', 'block3_conv1', 'block3_conv2', 'block3_conv3', 'block3_conv4', 'block4_conv1', 'block4_conv2', 'block4_conv3', 'block4_conv4', 'block5_conv1', 'block5_conv2', 'block5_conv3', 'block5_conv4'). This is a proposed improvement from https://arxiv.org/abs/1605.04603. In the original implementation, the layers used are: c('block1_conv1', 'block2_conv1', block3_conv1', 'block4_conv1', 'block5_conv1'). |
contentMask |
an ANTsImage mask to specify the region for content consideration. |
styleMasks |
ANTsImage masks to specify the region for style consideration. |
useShiftedActivations |
boolean to determine whether or not to use shifted activations in calculating the Gram matrix (improvement mentioned in https://arxiv.org/abs/1605.04603). |
useChainedInference |
boolean corresponding to another proposed improvement from https://arxiv.org/abs/1605.04603. |
verbose |
boolean to print progress to the screen. |
outputPrefix |
If specified, outputs a png image to disk at each iteration. |
https://arxiv.org/abs/1508.06576 and https://arxiv.org/abs/1605.04603
and taken from François Chollet's implementation
https://keras.io/examples/generative/neural_style_transfer/
and titu1994's modifications:
https://github.com/titu1994/Neural-Style-Transfer
in order to possibly modify and experiment with medical images.
ANTs 3-component image.
Tustison, NJ
## Not run: library( ANTsRNet ) ## End(Not run)## Not run: library( ANTsRNet ) ## End(Not run)
Creates 2D partial convolution layer as described in the paper
https://arxiv.org/abs/1804.07723
with the implementation ported from the following python implementation
https://github.com/MathiasGruber/PConv-Keras
a partial convolution layer 2D
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
Creates 3D partial convolution layer as described in the paper
https://arxiv.org/abs/1804.07723
with the implementation ported from the following python implementation
https://github.com/MathiasGruber/PConv-Keras
a partial convolution layer 3D
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
Function to calculate peak-signal-to-noise ratio.
peak_signal_to_noise_ratio(y_true, y_pred)peak_signal_to_noise_ratio(y_true, y_pred)
y_true |
True labels (Tensor) |
y_pred |
Predictions (Tensor of the same shape as |
PSNR value
Tustison NJ
library( ANTsRNet ) library( keras ) model <- createUnetModel2D( c( 64, 64, 1 ) ) metric_peak_signal_to_noise_ratio <- custom_metric( "peak_signal_to_noise_ratio", peak_signal_to_noise_ratio ) model %>% compile( loss = loss_categorical_crossentropy, optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( metric_peak_signal_to_noise_ratio ) )library( ANTsRNet ) library( keras ) model <- createUnetModel2D( c( 64, 64, 1 ) ) metric_peak_signal_to_noise_ratio <- custom_metric( "peak_signal_to_noise_ratio", peak_signal_to_noise_ratio ) model %>% compile( loss = loss_categorical_crossentropy, optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( metric_peak_signal_to_noise_ratio ) )
Function for Pearson correlation coefficient.
pearson_correlation_coefficient(y_true, y_pred)pearson_correlation_coefficient(y_true, y_pred)
y_true |
True labels (Tensor) |
y_pred |
Predictions (Tensor of the same shape as |
Correlation
Tustison NJ
library( ANTsRNet ) library( keras ) model <- createUnetModel2D( c( 64, 64, 1 ) ) metric_pearson_correlation_coefficient <- custom_metric( "pearson_correlation_coefficient", pearson_correlation_coefficient ) model %>% compile( loss = loss_categorical_crossentropy, optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( metric_pearson_correlation_coefficient ) )library( ANTsRNet ) library( keras ) model <- createUnetModel2D( c( 64, 64, 1 ) ) metric_pearson_correlation_coefficient <- custom_metric( "pearson_correlation_coefficient", pearson_correlation_coefficient ) model %>% compile( loss = loss_categorical_crossentropy, optimizer = optimizer_adam( lr = 0.0001 ), metrics = c( metric_pearson_correlation_coefficient ) )
Various preprocessing steps that have been previously described in various papers including the cortical thickness pipeline:
preprocessBrainImage( image, truncateIntensity = c(0.01, 0.99), brainExtractionModality = NULL, templateTransformType = NULL, template = "biobank", doBiasCorrection = TRUE, returnBiasField = FALSE, doDenoising = TRUE, intensityMatchingType = NULL, referenceImage = NULL, intensityNormalizationType = NULL, verbose = TRUE )preprocessBrainImage( image, truncateIntensity = c(0.01, 0.99), brainExtractionModality = NULL, templateTransformType = NULL, template = "biobank", doBiasCorrection = TRUE, returnBiasField = FALSE, doDenoising = TRUE, intensityMatchingType = NULL, referenceImage = NULL, intensityNormalizationType = NULL, verbose = TRUE )
image |
input t1-weighted brain MRI |
truncateIntensity |
2-element vector giving the low and high quantiles for intensity truncation. |
brainExtractionModality |
string or NULL. Perform brain extraction using antsxnet tools. One of "t1", "t1v0", "t1nobrainer", "t1combined", "flair", "t2", "bold", "fa", "t1infant", "t2infant", "t1threetissue", or NULL. |
templateTransformType |
see Details in help for |
template |
an ANTs image (not skull-stripped). Other premade templates are "biobank" and "croppedMni152". |
doBiasCorrection |
boolean for performing N4 bias field correction. |
returnBiasField |
if TRUE, return bias field as an additional output without bias correcting the preprocessed image. |
doDenoising |
boolean for performing non-local means denoising. |
intensityMatchingType |
Either "regression" or "histogram". Only is
performed if |
referenceImage |
reference image for intensity matching. |
intensityNormalizationType |
Either rescale the intensities to 0,1
(i.e., "01") or zero-mean, unit variance (i.e., "0mean"). If |
verbose |
print progress to the screen. |
\url{https://www.ncbi.nlm.nih.gov/pubmed/24879923}
preprocessed image and, optionally, the brain mask, bias field, and template transforms.
Tustison NJ, Avants BB
library( ANTsR ) library( ANTsRNet ) image <- antsImageRead( getANTsRData( "r16" ) ) preprocessedImage <- preprocessBrainImage( image, truncateIntensity = c( 0.01, 0.99 ), doBiasCorrection = TRUE, doDenoising = TRUE, intensityNormalizationType = "01", verbose = FALSE )library( ANTsR ) library( ANTsRNet ) image <- antsImageRead( getANTsRData( "r16" ) ) preprocessedImage <- preprocessBrainImage( image, truncateIntensity = c( 0.01, 0.99 ), doBiasCorrection = TRUE, doDenoising = TRUE, intensityNormalizationType = "01", verbose = FALSE )
Peak signal-to-noise ratio between two images.
PSNR(x, y)PSNR(x, y)
x |
input image. |
y |
input image. |
the peak signal-to-noise ratio
Avants BB
library( ANTsR ) r16 <- antsImageRead( getANTsRData( 'r16' ) ) r85 <- antsImageRead( getANTsRData( 'r85' ) ) psnrValue <- PSNR( r16, r85 )library( ANTsR ) r16 <- antsImageRead( getANTsRData( 'r16' ) ) r85 <- antsImageRead( getANTsRData( 'r85' ) ) psnrValue <- PSNR( r16, r85 )
The function will apply rigid, affine or deformable maps to an input set of training images. The reference image domain defines the space in which this happens.
randomImageTransformAugmentation( imageDomain, predictorImageList, outcomeImageList, n = 8, typeOfTransform = "Affine", interpolator = c("linear", "nearestNeighbor"), sdAffine = 1, nControlPoints = 100, spatialSmoothing = 3, composeToField = FALSE, numberOfCompositions = 4, deformationBasis, directoryName, imageDomainY, normalization = "none" )randomImageTransformAugmentation( imageDomain, predictorImageList, outcomeImageList, n = 8, typeOfTransform = "Affine", interpolator = c("linear", "nearestNeighbor"), sdAffine = 1, nControlPoints = 100, spatialSmoothing = 3, composeToField = FALSE, numberOfCompositions = 4, deformationBasis, directoryName, imageDomainY, normalization = "none" )
imageDomain |
defines default spatial domain for images. NOTE: if the
input images do not match the spatial domain of the domain image, we
internally resample the target to the domain. This may have unexpected
consequences if you are not aware of this. This operation will test
|
predictorImageList |
list of lists of image predictors |
outcomeImageList |
optional list of image outcomes |
n |
number of simulations to run |
typeOfTransform |
one of the following options
|
interpolator |
nearestNeighbor or linear (string) for predictor and outcome images respectively |
sdAffine |
roughly controls deviation from identity matrix |
nControlPoints |
number of control points for simulated deformation |
spatialSmoothing |
spatial smoothing for simulated deformation |
composeToField |
defaults to FALSE, will return deformation fields otherwise i.e. maps any transformation to a single deformation field. |
numberOfCompositions |
integer greater than or equal to one |
deformationBasis |
list containing deformationBasis set |
directoryName |
where to write to disk (optional) |
imageDomainY |
optional spatial domain for outcome images. |
normalization |
optional intensity normalization either none, standardize or 01. |
list (if no directory set) or boolean for success, failure
Avants BB
randomImageTransformBatchGenerator
library( ANTsR ) i1 = antsImageRead( getANTsRData( "r16" ) ) i2 = antsImageRead( getANTsRData( "r64" ) ) s1 = thresholdImage( i1, "Otsu", 3 ) s2 = thresholdImage( i2, "Otsu", 3 ) rand = randomImageTransformAugmentation( i1, list( list(i1), list(i2) ), list( s1, s2 ) )library( ANTsR ) i1 = antsImageRead( getANTsRData( "r16" ) ) i2 = antsImageRead( getANTsRData( "r64" ) ) s1 = thresholdImage( i1, "Otsu", 3 ) s2 = thresholdImage( i2, "Otsu", 3 ) rand = randomImageTransformAugmentation( i1, list( list(i1), list(i2) ), list( s1, s2 ) )
This R6 class can be used to generate affine and other transformations of an
input image predictor and outcome population. It currently works for single
predictor and single outcome modalities but will be extended in the future.
The class calls randomImageTransformAugmentation.
bgen = randomImageTransformBatchGenerator$new( ... ) bgen$generate( batchSize = 32L )
imageList List of lists where the embedded list contains k images.
outcomeImageList List of outcome images.
transformType random transform type to generate;
one of the following options
c("Translation","Rigid","ScaleShear","Affine","Deformation",
"AffineAndDeformation")
imageDomain defines the spatial domain for all images.
NOTE: if the input images do not match the spatial domain of the domain
image, we internally resample the target to the domain. This may have
unexpected consequences if you are not aware of this.
This operation will test
antsImagePhysicalSpaceConsistency then call
resampleImageToTarget upon failure.
sdAffine roughly controls deviation from identity matrix
nControlPoints number of control points for simulated deformation
spatialSmoothing spatial smoothing for simulated deformation
toCategorical boolean vector denoting whether the outcome class is categorical or not
imageDomainY optional domain for outcome images
normalization optional intensity normalization either none, standardize or 01
$new() Initialize the class in default empty or filled form.
$generate generate the batch of samples with given batch size
randomImageTransformAugmentation
library( ANTsR ) i1 = antsImageRead( getANTsRData( "r16" ) ) i2 = antsImageRead( getANTsRData( "r64" ) ) s1 = thresholdImage( i1, "Otsu", 3 ) s2 = thresholdImage( i2, "Otsu", 3 ) # see ANTsRNet randomImageTransformAugmentation predictors = list( list(i1), list(i2), list(i1), list(i2) ) outcomes = list( s1, s2, s1, s2 ) trainingData <- randomImageTransformBatchGenerator$new( imageList = predictors, outcomeImageList = outcomes, transformType = "Affine", imageDomain = i1, toCategorical = TRUE ) testBatchGenFunction = trainingData$generate( 2 ) myout = testBatchGenFunction( )library( ANTsR ) i1 = antsImageRead( getANTsRData( "r16" ) ) i2 = antsImageRead( getANTsRData( "r64" ) ) s1 = thresholdImage( i1, "Otsu", 3 ) s2 = thresholdImage( i2, "Otsu", 3 ) # see ANTsRNet randomImageTransformAugmentation predictors = list( list(i1), list(i2), list(i1), list(i2) ) outcomes = list( s1, s2, s1, s2 ) trainingData <- randomImageTransformBatchGenerator$new( imageList = predictors, outcomeImageList = outcomes, transformType = "Affine", imageDomain = i1, toCategorical = TRUE ) testBatchGenFunction = trainingData$generate( 2 ) myout = testBatchGenFunction( )
The function will apply rigid, affine or deformable maps to an input set of training images. The reference image domain defines the space in which this happens. The outcome here is the transform parameters themselves. This is intended for use with low-dimensional transformations.
randomImageTransformParametersAugmentation( imageDomain, predictorImageList, n = 8, typeOfTransform = "Affine", interpolator = "linear", spatialSmoothing = 3, numberOfCompositions = 4, deformationBasis, txParamMeans, txParamSDs, center = FALSE )randomImageTransformParametersAugmentation( imageDomain, predictorImageList, n = 8, typeOfTransform = "Affine", interpolator = "linear", spatialSmoothing = 3, numberOfCompositions = 4, deformationBasis, txParamMeans, txParamSDs, center = FALSE )
imageDomain |
defines the spatial domain for all images. NOTE: if the
input images do not match the spatial domain of the domain image, we
internally resample the target to the domain. This may have unexpected
consequences if you are not aware of this. This operation will test
|
predictorImageList |
list of lists of image predictors |
n |
number of simulations to run |
typeOfTransform |
one of the following options
|
interpolator |
nearestNeighbor or linear (string) for predictor images |
spatialSmoothing |
spatial smoothing for simulated deformation |
numberOfCompositions |
integer greater than or equal to one |
deformationBasis |
list containing deformationBasis set or a matrix if the basis is low-dimensional i.e. affine |
txParamMeans |
list containing deformationBasis set means |
txParamSDs |
list containing deformationBasis standard deviations |
center |
center the parameters before passing as ground truth output |
list of transformed images and transform parameters
Avants BB
randomImageTransformParametersBatchGenerator
library( ANTsR ) i1 = antsImageRead( getANTsRData( "r16" ) ) i2 = antsImageRead( getANTsRData( "r64" ) ) s1 = thresholdImage( i1, "Otsu", 3 ) s2 = thresholdImage( i2, "Otsu", 3 ) rand = randomImageTransformParametersAugmentation( i1, list( i1, i2 ), txParamMeans=c(1,0,0,1), txParamSDs=diag(4)*0.01 )library( ANTsR ) i1 = antsImageRead( getANTsRData( "r16" ) ) i2 = antsImageRead( getANTsRData( "r64" ) ) s1 = thresholdImage( i1, "Otsu", 3 ) s2 = thresholdImage( i2, "Otsu", 3 ) rand = randomImageTransformParametersAugmentation( i1, list( i1, i2 ), txParamMeans=c(1,0,0,1), txParamSDs=diag(4)*0.01 )
This R6 class can be used to generate parameters to affine and other
transformations applied to an input image population.
The class calls randomImageTransformParametersAugmentation.
bgen = randomImageTransformParametersBatchGenerator$new( ... ) bgen$generate( batchSize = 32L )
imageDomain defines the spatial domain for all images.
imageList List contains k images.
transformType random transform type to generate;
one of the following options
c("Translation","Rigid","ScaleShear","Affine","DeformationBasis" )
NOTE: if the input images do not match the spatial domain of the domain
image, we internally resample the target to the domain. This may have
unexpected consequences if you are not aware of this.
This operation will test
antsImagePhysicalSpaceConsistency then call
resampleImageToTarget upon failure.
spatialSmoothing spatial smoothing for simulated deformation
numberOfCompositions number of compositions
deformationBasis list of basis deformations
txParamMeans vector of basis deformations means
txParamSDs vector of basis deformations sds
center center the parameters before passing to Y
$new() Initialize the class in default empty or filled form.
$generate generate the batch of samples with given batch size
randomImageTransformParametersAugmentation
library( ANTsR ) i1 = antsImageRead( getANTsRData( "r16" ) ) i2 = antsImageRead( getANTsRData( "r64" ) ) s1 = thresholdImage( i1, "Otsu", 3 ) s2 = thresholdImage( i2, "Otsu", 3 ) # see ANTsRNet randomImageTransformAugmentation predictors = list( i1, i2, i2, i1 ) trainingData <- randomImageTransformParametersBatchGenerator$new( imageList = predictors, transformType = "Affine", imageDomain = i1, txParamMeans=c(1,0,0,1,0,0), txParamSDs=diag(6)*0.01 ) testBatchGenFunction = trainingData$generate( 2 ) myout = testBatchGenFunction( )library( ANTsR ) i1 = antsImageRead( getANTsRData( "r16" ) ) i2 = antsImageRead( getANTsRData( "r64" ) ) s1 = thresholdImage( i1, "Otsu", 3 ) s2 = thresholdImage( i2, "Otsu", 3 ) # see ANTsRNet randomImageTransformAugmentation predictors = list( i1, i2, i2, i1 ) trainingData <- randomImageTransformParametersBatchGenerator$new( imageList = predictors, transformType = "Affine", imageDomain = i1, txParamMeans=c(1,0,0,1,0,0), txParamSDs=diag(6)*0.01 ) testBatchGenFunction = trainingData$generate( 2 ) myout = testBatchGenFunction( )
Creates a lambda layer which interpolates/resizes an input tensor based on the specified shape
a resampled version of the input tensor
A 2-D vector specifying the new shape.
Type of interpolation. Can be
'nearestNeighbor', 'nearestNeighbor',
'linear', 'bilinear',
'cubic', or 'bicubic'
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
Creates a lambda layer which interpolates/resizes an input tensor based on the specified shape
a resampled version of the input tensor
A 3-D vector specifying the new shape.
Type of interpolation. Can be
'nearestNeighbor', 'nearestNeighbor',
'linear', 'bilinear',
'cubic', or 'bicubic'
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
Creates a lambda layer which interpolates/resizes an input tensor based on the specified target tensor
a resampled version of the input tensor
tensor of desired size.
Type of interpolation. Can be
'nearestNeighbor', 'nearestNeighbor',
'linear', 'bilinear',
'cubic', or 'bicubic'
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
Creates a lambda layer which interpolates/resizes an input tensor based on the specified target tensor
a resampled version of the input tensor
Type of interpolation. Can be
'nearestNeighbor', 'nearestNeighbor',
'linear', 'bilinear',
'cubic', or 'bicubic'
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
Ported from:
sampleFromCategoricalDistribution(distribution)sampleFromCategoricalDistribution(distribution)
distribution |
input categorical distribution from which to sample. |
https://github.com/cpmpercussion/keras-mdn-layer/
a single sample
Tustison NJ
library( keras )library( keras )
Ported from:
sampleFromOutput( parameters, outputDimension, numberOfMixtures, temperature = 1, sigmaTemperature = 1 )sampleFromOutput( parameters, outputDimension, numberOfMixtures, temperature = 1, sigmaTemperature = 1 )
parameters |
vector parameter to split |
outputDimension |
output dimension |
numberOfMixtures |
number of mixture components |
temperature |
the temperature for to adjust the distribution
(default 1.0), passed to |
sigmaTemperature |
multipler to |
https://github.com/cpmpercussion/keras-mdn-layer/
a single sample
Tustison NJ
library( keras )library( keras )
Custom scale layer
A keras layer
integer specifying which axis to normalize.
momentum value used for computation of the exponential average of the mean and standard deviation.
Dimensionality of the input (integer) not including the samples axis.
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
library(keras) inputImageSize = c( 256L, 256L, 1L ) inputs <- keras::layer_input( shape = inputImageSize ) outputs <- inputs %>% layer_zero_padding_2d( padding = c( 3L, 3L ) ) layer_scale = ANTsRNet:::layer_scale outputs = outputs %>% layer_scale(axis = -1L) lay = ScaleLayer$new() ## Not run: lay$build(input_shape = inputImageSize) ## End(Not run)library(keras) inputImageSize = c( 256L, 256L, 1L ) inputs <- keras::layer_input( shape = inputImageSize ) outputs <- inputs %>% layer_zero_padding_2d( padding = c( 3L, 3L ) ) layer_scale = ANTsRNet:::layer_scale outputs = outputs %>% layer_scale(axis = -1L) lay = ScaleLayer$new() ## Not run: lay$build(input_shape = inputImageSize) ## End(Not run)
This function sets the cache directory used by ANTsXNet to store downloaded templates and model weights. If the specified directory does not exist, it will be created.
setANTsXNetCacheDirectory(path)setANTsXNetCacheDirectory(path)
path |
A character string specifying the path to the cache directory. |
Set ANTXNet deterministic behavior
setANTsXNetDeterministic(on = TRUE, seedValue = 123)setANTsXNetDeterministic(on = TRUE, seedValue = 123)
on |
Set deterministic behavior. |
seedValue |
Assign random seed value. |
Perform segmentation of perivascular (PVS) or Vircho-Robin spaces (VRS). https://pubmed.ncbi.nlm.nih.gov/34262443/ with the original implementation available here: https://github.com/pboutinaud/SHIVA_PVS
shivaPvsSegmentation( t1, flair = NULL, whichModel = "all", doPreprocessing = TRUE, verbose = FALSE )shivaPvsSegmentation( t1, flair = NULL, whichModel = "all", doPreprocessing = TRUE, verbose = FALSE )
t1 |
input 3-D T1-weighted brain image. |
flair |
(Optional) input 3-D FLAIR brain image (aligned to T1 image). |
whichModel |
integer or string. Several models were trained for the case of T1-only or T1/FLAIR image pairs. One can use a specific single trained model or the average of the entire ensemble. I.e., options are: * For T1-only: 0, 1, 2, 3, 4, 5. * For T1/FLAIR: 0, 1, 2, 3, 4. * Or "all" for using the entire ensemble. |
doPreprocessing |
perform n4 bias correction, intensity truncation, brain extraction. |
verbose |
print progress. |
probabilistic image.
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) t1 <- antsImageRead( "t1.nii.gz" ) flair <- antsImageRead( "flair.nii.gz" ) results <- wmhSegmentation( t1, flair ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) t1 <- antsImageRead( "t1.nii.gz" ) flair <- antsImageRead( "flair.nii.gz" ) results <- wmhSegmentation( t1, flair ) ## End(Not run)
Perform segmentation of white matter hyperintensities. https://pubmed.ncbi.nlm.nih.gov/38050769/ with the original implementation available here: https://github.com/pboutinaud/SHIVA_WMH
shivaWmhSegmentation( flair, t1 = NULL, whichModel = "all", doPreprocessing = TRUE, verbose = FALSE )shivaWmhSegmentation( flair, t1 = NULL, whichModel = "all", doPreprocessing = TRUE, verbose = FALSE )
flair |
input 3-D FLAIR brain image. |
t1 |
(Optional) input 3-D T1-weighted brain image (aligned to FLAIR image). |
whichModel |
integer or string. Several models were trained for the case of T1-only or T1/FLAIR image pairs. One can use a specific single trained model or the average of the entire ensemble. I.e., options are: * For T1-only: 0, 1, 2, 3, 4, 5. * For T1/FLAIR: 0, 1, 2, 3, 4. * Or "all" for using the entire ensemble. |
doPreprocessing |
perform n4 bias correction, intensity truncation, brain extraction. |
verbose |
print progress. |
probabilistic image.
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) t1 <- antsImageRead( "t1.nii.gz" ) flair <- antsImageRead( "flair.nii.gz" ) results <- wmhSegmentation( t1, flair ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) t1 <- antsImageRead( "t1.nii.gz" ) flair <- antsImageRead( "flair.nii.gz" ) results <- wmhSegmentation( t1, flair ) ## End(Not run)
Spatial transformer layer (2-D)
resampled batch images.
list of size 2 where the first element are the images. The second element are the weights.
size of the resampled output images.
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
resampledSize = c(30L, 30L) model = SpatialTransformerLayer2D$new(resampledSize) model$initialize(resampledSize) testthat::expect_error(model$initialize(5)) model$compute_output_shape(input_shape = c(25, 25))resampledSize = c(30L, 30L) model = SpatialTransformerLayer2D$new(resampledSize) model$initialize(resampledSize) testthat::expect_error(model$initialize(5)) model$compute_output_shape(input_shape = c(25, 25))
Spatial transfomer layer (3-D)
resampled batch images.
list of size 2 where the first element are the images. The second element are the weights.
size of the resampled output images.
$initialize instantiates a new class.
$call main body.
$compute_output_shape computes the output shape.
Tustison NJ
model = SpatialTransformerLayer3D$new(c(30L, 30L, 30L)) model$compute_output_shape(input_shape = c(25, 25, 25))model = SpatialTransformerLayer3D$new(c(30L, 30L, 30L)) model$compute_output_shape(input_shape = c(25, 25, 25))
Ported from:
splitMixtureParameters(parameters, outputDimension, numberOfMixtures)splitMixtureParameters(parameters, outputDimension, numberOfMixtures)
parameters |
vector parameter to split |
outputDimension |
output dimension |
numberOfMixtures |
number of mixture components |
https://github.com/cpmpercussion/keras-mdn-layer/
separate mixture parameters
Tustison NJ
library( keras )library( keras )
Implementation of the SSI quantity for two images proposed in
SSIM(x, y, K = c(0.01, 0.03))SSIM(x, y, K = c(0.01, 0.03))
x |
input image. |
y |
input image. |
K |
vector of length 2 which contain SSI parameters meant to stabilize the formula in case of weak denominators. |
Z. Wang, A.C. Bovik, H.R. Sheikh, E.P. Simoncelli. "Image quality assessment: from error visibility to structural similarity". IEEE TIP. 13 (4): 600–612.
the structural similarity index
Avants BB
library( ANTsR ) r16 <- antsImageRead( getANTsRData( 'r16' ) ) r85 <- antsImageRead( getANTsRData( 'r85' ) ) ssimValue <- SSIM( r16, r85 )library( ANTsR ) r16 <- antsImageRead( getANTsRData( 'r16' ) ) r85 <- antsImageRead( getANTsRData( 'r85' ) ) ssimValue <- SSIM( r16, r85 )
Super resolution generative adverserial network from the paper:
https://arxiv.org/abs/1609.04802
and ported from the Keras (python) implementation:
https://github.com/eriklindernoren/Keras-GAN/blob/master/srgan/srgan.py
$initialize instantiates a new class and builds the
generator and discriminator.
$buildGeneratorbuild generator.
$buildGeneratorbuild discriminator.
Tustison NJ
## Not run: library( keras ) library( ANTsRNet ) keras::backend()$clear_session() ganModel <- SuperResolutionGanModel$new( lowResolutionImageSize = c( 112, 112, 3 ) ) testthat::expect_error({ ganModel <- SuperResolutionGanModel$new( lowResolutionImageSize = c( 64, 65, 3 ) ) }) ## End(Not run)## Not run: library( keras ) library( ANTsRNet ) keras::backend()$clear_session() ganModel <- SuperResolutionGanModel$new( lowResolutionImageSize = c( 112, 112, 3 ) ) testthat::expect_error({ ganModel <- SuperResolutionGanModel$new( lowResolutionImageSize = c( 64, 65, 3 ) ) }) ## End(Not run)
Perform WMH segmentation using the winning submission in the MICCAI 2017 challenge by the sysu_media team using FLAIR or T1/FLAIR. The MICCAI challenge is discussed in
sysuMediaWmhSegmentation(flair, t1 = NULL, useEnsemble = TRUE, verbose = FALSE)sysuMediaWmhSegmentation(flair, t1 = NULL, useEnsemble = TRUE, verbose = FALSE)
flair |
input 3-D FLAIR brain image. |
t1 |
input 3-D T1-weighted brain image (assumed to be aligned to the flair, if specified). |
useEnsemble |
boolean to check whether to use all 3 sets of weights. |
verbose |
print progress. |
https://pubmed.ncbi.nlm.nih.gov/30908194/
with the sysu_media's team entry is discussed in
https://pubmed.ncbi.nlm.nih.gov/30125711/
with the original implementation available here:
\url{https://github.com/hongweilibran/wmh_ibbmTum}
The original implementation used global thresholding as a quick brain extraction approach. Due to possible generalization difficulties, we leave such post-processing steps to the user. For brain or white matter masking see functions brainExtraction or deepAtropos, respectively.
WMH segmentation probability image
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "flair.nii.gz" ) probabilityMask <-sysuMediaWmhSegmentation( image ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) image <- antsImageRead( "flair.nii.gz" ) probabilityMask <-sysuMediaWmhSegmentation( image ) ## End(Not run)
Supervised grader / scoring of T1-weighted brain MRI.
t1Grader(image, verbose = FALSE)t1Grader(image, verbose = FALSE)
image |
raw 3-D T1 brain image. |
verbose |
print progress to the screen. |
data frame with letter grade and numeric score predictions.
Tustison NJ
## Not run: library( ANTsRNet ) image <- antsImageRead( getANTsXNetData( "mprageHippmap3r" ) ) gradeDf <- t1Grader( image ) ## End(Not run)## Not run: library( ANTsRNet ) image <- antsImageRead( getANTsXNetData( "mprageHippmap3r" ) ) gradeDf <- t1Grader( image ) ## End(Not run)
Use a ResNet architecture to estimate image quality in 2D or 3D using subjective QC image databases described in
tidNeuralImageAssessment( image, mask, patchSize = 101L, strideLength, paddingSize = 0L, dimensionsToPredict = 1L, whichModel = "tidsQualityAssessment", imageScaling, doPatchScaling = FALSE, verbose = FALSE )tidNeuralImageAssessment( image, mask, patchSize = 101L, strideLength, paddingSize = 0L, dimensionsToPredict = 1L, whichModel = "tidsQualityAssessment", imageScaling, doPatchScaling = FALSE, verbose = FALSE )
image |
the input image. Either 2D or 3D. |
mask |
optional mask for designating calculation ROI. |
patchSize |
integer (prime) number for patch size; 101 is good. otherwise,
choose |
strideLength |
optional value to speed up computation (typically less than patch size). Integer or vector of image dimension length. |
paddingSize |
positive or negative integer (or vector of image dimension length) for (de)padding to remove edge effects. |
dimensionsToPredict |
if image dimension is 3, this parameter specifies which dimension(s) should be used for prediction. If more than one dimension is specified, the results are averaged. |
whichModel |
model type e.g. string tidsQualityAssessment, koniqMS, koniqMS2 or koniqMS3 where
the former predicts mean opinion score (MOS) and MOS standard deviation and
the latter koniq models predict mean opinion score (MOS) and sharpness. One
may also directly pass a tensorflow model here. In this case, we assume that
the input image is scaled by the |
imageScaling |
a two-vector where the first value is the multiplier and
the second value the subtractor so each image will be scaled as
|
doPatchScaling |
boolean controlling whether each patch is scaled or |
verbose |
print progress. |
https://www.sciencedirect.com/science/article/pii/S0923596514001490
or
https://doi.org/10.1109/TIP.2020.2967829
where the image assessment is either "global", i.e., a single number or an image
based on the specified patch size. In the 3-D case, neighboring slices are used
for each estimate. Note that parameters should be kept as consistent as possible
in order to enable comparison. Patch size should be roughly 1/12th to 1/4th of
image size to enable locality. A global estimate can be gained by setting
patchSize = "global".
list of QC results predicting both both human rater's mean and standard deviation of the MOS ("mean opinion scores") or sharpness depending on the selected network. Both aggregate and spatial scores are returned, the latter in the form of an image.
Avants BB
## Not run: image <- antsImageRead( getANTsRData( "r16" ) ) mask <- getMask( image ) tid <- tidNeuralImageAssessment( image, mask = mask, patchSize = 101L, strideLength = 7L, paddingSize = 0L ) plot( image, tid$MOS, alpha = 0.5) cat( "mean MOS = ", tid$MOS.mean, "\n" ) cat( "sd MOS = ", tid$MOS.standardDeviationMean, "\n" ) ## End(Not run)## Not run: image <- antsImageRead( getANTsRData( "r16" ) ) mask <- getMask( image ) tid <- tidNeuralImageAssessment( image, mask = mask, patchSize = 101L, strideLength = 7L, paddingSize = 0L ) plot( image, tid$MOS, alpha = 0.5) cat( "mean MOS = ", tid$MOS.mean, "\n" ) cat( "sd MOS = ", tid$MOS.standardDeviationMean, "\n" ) ## End(Not run)
Trains a variational autoencoding with a convolutional network. This is followed by k-means clustering to produce a segmentation and probabilities.
uvaSeg( image, model, k, mask, returnProbabilities = FALSE, batchSize = 1028, standardize = TRUE, verbose = FALSE )uvaSeg( image, model, k, mask, returnProbabilities = FALSE, batchSize = 1028, standardize = TRUE, verbose = FALSE )
image |
input image |
model |
the model output from |
k |
number of clusters or cluster centers |
mask |
defining output segmentation space |
returnProbabilities |
boolean |
batchSize |
for the prediction |
standardize |
boolean controlling whether patches are standardized |
verbose |
boolean |
segmentation and probability images are output
Avants BB
## Not run: library(ANTsR) img <- ri( 1 ) %>% resampleImage( c(4,4) ) %>% iMath( "Normalize" ) mask = randomMask( getMask( img ), 50 ) patch = getNeighborhoodInMask( img, mask, c(3,3), boundary.condition = "NA" ) uvaSegModel = uvaSegTrain( patch, k = 6 ) tarImg = ri( 3 ) %>% resampleImage( c(4,4) ) uvaSegmentation = uvaSeg(tarImg, uvaSegModel, k = 3, getMask( tarImg ) ) ## End(Not run)## Not run: library(ANTsR) img <- ri( 1 ) %>% resampleImage( c(4,4) ) %>% iMath( "Normalize" ) mask = randomMask( getMask( img ), 50 ) patch = getNeighborhoodInMask( img, mask, c(3,3), boundary.condition = "NA" ) uvaSegModel = uvaSegTrain( patch, k = 6 ) tarImg = ri( 3 ) %>% resampleImage( c(4,4) ) uvaSegmentation = uvaSeg(tarImg, uvaSegModel, k = 3, getMask( tarImg ) ) ## End(Not run)
Trains a variational autoencoding with a convolutional network. This is followed by k-means clustering to produce a segmentation and probabilities.
uvaSegTrain(patches, k, convControl, standardize = TRUE, patches2)uvaSegTrain(patches, k, convControl, standardize = TRUE, patches2)
patches |
input patch matrix, see |
k |
number of embedding layers |
convControl |
optional named list with control parameters ( see code )
|
standardize |
boolean controlling whether patches are standardized |
patches2 |
input target patch matrix, see |
model is output
Avants BB
## Not run: library(ANTsR) img <- ri( 1 ) %>% resampleImage( c(4,4) ) %>% iMath( "Normalize" ) mask = randomMask( getMask( img ), 50 ) r = c( 3, 3 ) patch = getNeighborhoodInMask( img, mask, r, boundary.condition = "NA" ) uvaSegModel = uvaSegTrain( patch, 6 ) ## End(Not run)## Not run: library(ANTsR) img <- ri( 1 ) %>% resampleImage( c(4,4) ) %>% iMath( "Normalize" ) mask = randomMask( getMask( img ), 50 ) r = c( 3, 3 ) patch = getNeighborhoodInMask( img, mask, r, boundary.condition = "NA" ) uvaSegModel = uvaSegTrain( patch, 6 ) ## End(Not run)
Original generative adverserial network from the paper:
https://arxiv.org/abs/1406.2661
and ported from the Keras (python) implementation:
https://github.com/eriklindernoren/Keras-GAN/blob/master/gan/gan.py
$initialize instantiates a new class and builds the
generator and discriminator.
$buildGeneratorbuild generator.
$buildGeneratorbuild discriminator.
Tustison NJ
## Not run: library( keras ) library( ANTsRNet ) keras::backend()$clear_session() # Let's use the mnist data set. mnist <- dataset_mnist() numberOfTrainingData <- length( mnist$train$y ) inputImageSize <- c( dim( mnist$train$x[1,,] ), 1 ) x <- array( data = mnist$train$x / 255, dim = c( numberOfTrainingData, inputImageSize ) ) y <- mnist$train$y numberOfClusters <- length( unique( mnist$train$y ) ) # Instantiate the DCEC model ganModel <- VanillaGanModel$new( inputImageSize = inputImageSize, latentDimension = 100 ) ganModel$train( x, numberOfEpochs = 100 ) ## End(Not run)## Not run: library( keras ) library( ANTsRNet ) keras::backend()$clear_session() # Let's use the mnist data set. mnist <- dataset_mnist() numberOfTrainingData <- length( mnist$train$y ) inputImageSize <- c( dim( mnist$train$x[1,,] ), 1 ) x <- array( data = mnist$train$x / 255, dim = c( numberOfTrainingData, inputImageSize ) ) y <- mnist$train$y numberOfClusters <- length( unique( mnist$train$y ) ) # Instantiate the DCEC model ganModel <- VanillaGanModel$new( inputImageSize = inputImageSize, latentDimension = 100 ) ganModel$train( x, numberOfEpochs = 100 ) ## End(Not run)
Wasserstein generative adverserial network from the paper:
https://arxiv.org/abs/1701.07875
and ported from the Keras (python) implementation:
https://github.com/eriklindernoren/Keras-GAN/blob/master/wgan/wgan.py
$initialize instantiates a new class and builds the
generator and critic.
$buildGeneratorbuild generator.
$buildGeneratorbuild critic.
Tustison NJ
library( keras ) library( ANTsRNet ) keras::backend()$clear_session() # Let's use the mnist data set. mnist <- dataset_mnist() numberOfTrainingData <- length( mnist$train$y ) inputImageSize <- c( dim( mnist$train$x[1,,] ), 1 ) x <- array( data = mnist$train$x / 255, dim = c( numberOfTrainingData, inputImageSize ) ) y <- mnist$train$y numberOfClusters <- length( unique( mnist$train$y ) ) # Instantiate the WGAN model ganModel <- WassersteinGanModel$new( inputImageSize = inputImageSize, latentDimension = 100 ) ganModel$train( x, numberOfEpochs = 5 )library( keras ) library( ANTsRNet ) keras::backend()$clear_session() # Let's use the mnist data set. mnist <- dataset_mnist() numberOfTrainingData <- length( mnist$train$y ) inputImageSize <- c( dim( mnist$train$x[1,,] ), 1 ) x <- array( data = mnist$train$x / 255, dim = c( numberOfTrainingData, inputImageSize ) ) y <- mnist$train$y numberOfClusters <- length( unique( mnist$train$y ) ) # Instantiate the WGAN model ganModel <- WassersteinGanModel$new( inputImageSize = inputImageSize, latentDimension = 100 ) ganModel$train( x, numberOfEpochs = 5 )
ported from this implementation:
weighted_categorical_crossentropy(y_true, y_pred, weights)weighted_categorical_crossentropy(y_true, y_pred, weights)
y_true |
True labels (Tensor) |
y_pred |
Predictions (Tensor of the same shape as |
weights |
weights for each class |
https://gist.github.com/wassname/ce364fddfc8a025bfab4348cf5de852d
function value
Tustison NJ
library( ANTsRNet ) library( keras ) model <- createUnetModel2D( c( 64, 64, 1 ), numberOfOutputs = 2 ) model %>% compile( loss = weighted_categorical_crossentropy( weights = c( 1, 1 ) ), optimizer = optimizer_adam( lr = 0.0001 ), metrics = "accuracy" )library( ANTsRNet ) library( keras ) model <- createUnetModel2D( c( 64, 64, 1 ), numberOfOutputs = 2 ) model %>% compile( loss = weighted_categorical_crossentropy( weights = c( 1, 1 ) ), optimizer = optimizer_adam( lr = 0.0001 ), metrics = "accuracy" )
Perform in-painting for whole-head MRI
wholeHeadInpainting( image, roiMask, modality = "t1", mode = "axial", verbose = FALSE )wholeHeadInpainting( image, roiMask, modality = "t1", mode = "axial", verbose = FALSE )
image |
input 3-D MR image. |
roiMask |
binary mask image |
modality |
Modality image type. Options include: "t1": T1-weighted MRI, "flair": FLAIR MRI. |
mode |
Options include: "sagittal": sagittal view network, "coronal": coronal view network, "axial": axial view network, "average": average of all canonical views, "meg": morphological erosion, greedy, iterative. |
verbose |
print progress. |
inpainted image
Tustison NJ
Perform white matter hyperintensity probabilistic segmentation using deep learning
wmhSegmentation( flair, t1, whiteMatterMask = NULL, useCombinedModel = TRUE, predictionBatchSize = 16, patchStrideLength = 32, doPreprocessing = TRUE, verbose = FALSE )wmhSegmentation( flair, t1, whiteMatterMask = NULL, useCombinedModel = TRUE, predictionBatchSize = 16, patchStrideLength = 32, doPreprocessing = TRUE, verbose = FALSE )
flair |
input 3-D FLAIR brain image. |
t1 |
input 3-D T1-weighted brain image (assumed to be aligned to the flair). |
whiteMatterMask |
input white matter mask for patch extraction. If None, calculated using deepAtropos (labels 3 and 4). |
useCombinedModel |
Original or combined. |
predictionBatchSize |
Control memory usage for prediction. More consequential for GPU-usage. |
patchStrideLength |
3-D vector or int. Dictates the stride length for accumulating predicting patches. |
doPreprocessing |
perform n4 bias correction, intensity truncation, brain extraction. |
verbose |
print progress. |
probabilistic image.
Tustison NJ
## Not run: library( ANTsRNet ) library( keras ) t1 <- antsImageRead( "t1.nii.gz" ) flair <- antsImageRead( "flair.nii.gz" ) results <- wmhSegmentation( t1, flair ) ## End(Not run)## Not run: library( ANTsRNet ) library( keras ) t1 <- antsImageRead( "t1.nii.gz" ) flair <- antsImageRead( "flair.nii.gz" ) results <- wmhSegmentation( t1, flair ) ## End(Not run)