weka.dl4j

Class Utils

java.lang.Object

weka.dl4j.Utils

public class Utils
extends java.lang.Object

Utility routines for the Dl4jMlpClassifier.

Constructor Summary

Constructors

Constructor and Description
Utils()

Method Summary

Modifier and Type	Method and Description
static org.nd4j.linalg.api.ndarray.INDArray	appendClasses(org.nd4j.linalg.api.ndarray.INDArray result, weka.core.Instances input) Appends the input Instances classes to the INDArray.
static double	computeScore(org.deeplearning4j.nn.graph.ComputationGraph model, org.nd4j.linalg.dataset.api.iterator.DataSetIterator iter) Compute the model score on a given iterator.
static weka.core.Instances	convertToInstances(org.nd4j.linalg.api.ndarray.INDArray result, weka.core.Instances input, java.util.Map<java.lang.String,java.lang.Long> attributesPerLayer) Converts the newly transformed instances to an Instances object.
static weka.core.Attribute	copyNominalAttribute(weka.core.Attribute oldAttribute) Copies the attribute name and values of a given nominal attribute.
static org.deeplearning4j.nn.conf.inputs.InputType.InputTypeConvolutional	decodeCNNShape(int[] shape) Decode a CNN Input shape.
static org.deeplearning4j.nn.conf.inputs.InputType.InputTypeConvolutional	decodeCNNShape(long[] shape) Decode a CNN Input shape.
static java.lang.String	defaultFileLocation() The default location for a file parameter.
static java.lang.String	getAttributeName(java.util.Map<java.lang.String,java.lang.Long> attributesPerLayer, int i) Helper function for getting the new layer name when using the Dl4jMlpFilter The attributes are named after the layer they originated from, so this function counts throught the attributes per layer, comparing it with the given index to determine a) which layer the activation came from and b) which number activation this is.
static <T> T	getFieldValue(java.lang.Object obj, java.lang.String fieldName) Access private field of a given object.
static org.nd4j.linalg.dataset.DataSet	getNext(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iter) Fix for issue with JVM crashing https://github.com/eclipse/deeplearning4j/issues/8976#issuecomment-639946904 It is recommended to use this helper function in WekaDeeplearning4j rather than using iter.next() directly.
static org.nd4j.linalg.dataset.DataSet	getNext(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iter, int num) Fix for issue with JVM crashing https://github.com/eclipse/deeplearning4j/issues/8976#issuecomment-639946904 It is recommended to use this helper function in WekaDeeplearning4j rather than using iter.next() directly.
static org.nd4j.linalg.dataset.DataSet	instancesToConvDataSet(weka.core.Instances insts, int height, int width, int channels) Converts a set of training instances to a DataSet prepared for the convolution operation using the height, width and number of channels.
static org.nd4j.linalg.dataset.DataSet	instancesToDataSet(weka.core.Instances insts) Converts a set of training instances to a DataSet.
static <T> T	invokeMethod(java.lang.Object obj, java.lang.String methodName, java.lang.Object... args) Invoke a method on a given object.
static boolean	isChannelsLast(org.nd4j.linalg.api.ndarray.INDArray activations) Shape will either be something like [1, 56, 56, 128] or [1, 128, 56, 56] If it's the former then return true.
static weka.core.Instances	ndArrayToInstances(org.nd4j.linalg.api.ndarray.INDArray ndArray, weka.core.Instances inputFormat, java.util.Map<java.lang.String,java.lang.Long> attributesPerLayer) Convert an arbitrary NDArray to Weka instances.
static boolean	needsReshaping(org.nd4j.linalg.api.ndarray.INDArray activationAtLayer) Determines if the activations need reshaping.
static boolean	notDefaultFileLocation(java.io.File file) Checks if the given file isn't the default file location (i.e., the user has selected a file).
static boolean	pathExists(java.lang.String path) Checks whether the path exists - a little tidier than the code it wraps.
static float	poolNDArray(org.nd4j.linalg.api.ndarray.INDArray array, PoolingType poolingType) Applies the pooling function to the given feature map.
static org.nd4j.linalg.api.ndarray.INDArray	reshapeActivations(org.nd4j.linalg.api.ndarray.INDArray activationAtLayer, PoolingType poolingType) Reshape the activations, either by pooling or simply multiplying the extra dimensions together.
static void	runWithLocalClassloader(java.lang.Class clz, VoidCallable block) Run some code-block using the local class loader from a given class.
static <T> void	setFieldValue(java.lang.Object obj, java.lang.String fieldName, T value) Set private field of a given object.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Utils

public Utils()

Method Detail

instancesToDataSet

public static org.nd4j.linalg.dataset.DataSet instancesToDataSet(weka.core.Instances insts)

Converts a set of training instances to a DataSet. Assumes that the instances have been suitably preprocessed - i.e. missing values replaced and nominals converted to binary/numeric. Also assumes that the class index has been set

Parameters:

insts - the instances to convert

Returns:

a DataSet

instancesToConvDataSet

public static org.nd4j.linalg.dataset.DataSet instancesToConvDataSet(weka.core.Instances insts,
                                                                     int height,
                                                                     int width,
                                                                     int channels)

Converts a set of training instances to a DataSet prepared for the convolution operation using the height, width and number of channels.

Parameters:

height - image height

width - image width

channels - number of image channels

insts - the instances to convert

Returns:

a DataSet

computeScore

public static double computeScore(org.deeplearning4j.nn.graph.ComputationGraph model,
                                  org.nd4j.linalg.dataset.api.iterator.DataSetIterator iter)

Compute the model score on a given iterator.

Parameters:

model - Model

iter - Iterator

Returns:

Model score on iterator data

copyNominalAttribute

public static weka.core.Attribute copyNominalAttribute(weka.core.Attribute oldAttribute)

Copies the attribute name and values of a given nominal attribute.

Parameters:

oldAttribute - attribute to copy

Returns:

duplicated nominal attribute

getAttributeName

public static java.lang.String getAttributeName(java.util.Map<java.lang.String,java.lang.Long> attributesPerLayer,
                                                int i)

Helper function for getting the new layer name when using the Dl4jMlpFilter The attributes are named after the layer they originated from, so this function counts throught the attributes per layer, comparing it with the given index to determine a) which layer the activation came from and b) which number activation this is.

Parameters:

attributesPerLayer - Hashmap defining how many attributes there are per layer

i - Index of layer we're currently looking at.

Returns:

Attribute name for transformed attribute.

ndArrayToInstances

public static weka.core.Instances ndArrayToInstances(org.nd4j.linalg.api.ndarray.INDArray ndArray,
                                                     weka.core.Instances inputFormat,
                                                     java.util.Map<java.lang.String,java.lang.Long> attributesPerLayer)
                                              throws weka.core.WekaException

Convert an arbitrary NDArray to Weka instances.

Parameters:

ndArray - Input array

inputFormat - Format to use for the instances

attributesPerLayer - Hashmap of layer names and how many attributes there are per layer

Returns:

Instances object

Throws:

weka.core.WekaException - Invalid input

getFieldValue

public static <T> T getFieldValue(java.lang.Object obj,
                                  java.lang.String fieldName)

Access private field of a given object.

Type Parameters:

T - Return type

Parameters:

obj - Object to be accessed

fieldName - Field name

Returns:

Value of field with name fieldName

setFieldValue

public static <T> void setFieldValue(java.lang.Object obj,
                                     java.lang.String fieldName,
                                     T value)

Set private field of a given object.

Type Parameters:

T - Field type

Parameters:

obj - Object to be accessed

fieldName - Field name

value - Field value to be set

invokeMethod

public static <T> T invokeMethod(java.lang.Object obj,
                                 java.lang.String methodName,
                                 java.lang.Object... args)

Invoke a method on a given object.

Type Parameters:

T - Return type

Parameters:

obj - Object to be referenced

methodName - Method name which is to be invoked

args - Method arguments

Returns:

Method return value

runWithLocalClassloader

public static void runWithLocalClassloader(java.lang.Class clz,
                                           VoidCallable block)

Run some code-block using the local class loader from a given class.

Parameters:

clz - Class to use the classloader from

block - Code block to run

needsReshaping

public static boolean needsReshaping(org.nd4j.linalg.api.ndarray.INDArray activationAtLayer)

Determines if the activations need reshaping.

Parameters:

activationAtLayer - Activations in question

Returns:

true if the activations need reshaping (too high dimensionality)

poolNDArray

public static float poolNDArray(org.nd4j.linalg.api.ndarray.INDArray array,
                                PoolingType poolingType)

Applies the pooling function to the given feature map.

Parameters:

array - feature map to pool

poolingType - pooling function to apply

Returns:

pooled value

isChannelsLast

public static boolean isChannelsLast(org.nd4j.linalg.api.ndarray.INDArray activations)

Shape will either be something like [1, 56, 56, 128] or [1, 128, 56, 56] If it's the former then return true.

Parameters:

activations -

Returns:

true if the activations are in channels-last format

reshapeActivations

public static org.nd4j.linalg.api.ndarray.INDArray reshapeActivations(org.nd4j.linalg.api.ndarray.INDArray activationAtLayer,
                                                                      PoolingType poolingType)

Reshape the activations, either by pooling or simply multiplying the extra dimensions together.

Parameters:

activationAtLayer - 4d activations e.g., [batch_size, 512, 64, 64]

poolingType - Pooling type to use to lower the dimensionality

Returns:

2D activations

appendClasses

public static org.nd4j.linalg.api.ndarray.INDArray appendClasses(org.nd4j.linalg.api.ndarray.INDArray result,
                                                                 weka.core.Instances input)

Appends the input Instances classes to the INDArray.

Parameters:

result - activations

input - original Instances

Returns:

activations with class value appended

convertToInstances

public static weka.core.Instances convertToInstances(org.nd4j.linalg.api.ndarray.INDArray result,
                                                     weka.core.Instances input,
                                                     java.util.Map<java.lang.String,java.lang.Long> attributesPerLayer)
                                              throws java.lang.Exception

Converts the newly transformed instances to an Instances object.

Parameters:

result - activations generated from feature layers

input - original input Instances

attributesPerLayer - Hashmap stating the feature layers and how many attributes each has

Returns:

Instances

Throws:

java.lang.Exception - File exception

getNext

public static org.nd4j.linalg.dataset.DataSet getNext(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iter)

Fix for issue with JVM crashing https://github.com/eclipse/deeplearning4j/issues/8976#issuecomment-639946904 It is recommended to use this helper function in WekaDeeplearning4j rather than using iter.next() directly.

Parameters:

iter - DatasetIterator to get images from

Returns:

Next DataSet

getNext

public static org.nd4j.linalg.dataset.DataSet getNext(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iter,
                                                      int num)

Fix for issue with JVM crashing https://github.com/eclipse/deeplearning4j/issues/8976#issuecomment-639946904 It is recommended to use this helper function in WekaDeeplearning4j rather than using iter.next() directly.

Parameters:

iter - DatasetIterator to get images from

num - Batch size to get

Returns:

Next DataSet

pathExists

public static boolean pathExists(java.lang.String path)

Checks whether the path exists - a little tidier than the code it wraps.

Parameters:

path - Path to check

Returns:

True if the path exists, false otherwise

notDefaultFileLocation

public static boolean notDefaultFileLocation(java.io.File file)

Checks if the given file isn't the default file location (i.e., the user has selected a file).

Parameters:

file - File to check

Returns:

true if the user has selected a file to load the model from

defaultFileLocation

public static java.lang.String defaultFileLocation()

The default location for a file parameter.

Returns:

Default file path

decodeCNNShape

public static org.deeplearning4j.nn.conf.inputs.InputType.InputTypeConvolutional decodeCNNShape(int[] shape)

Decode a CNN Input shape.

Parameters:

shape - Shape to decode

Returns:

Parsed input shape

decodeCNNShape

public static org.deeplearning4j.nn.conf.inputs.InputType.InputTypeConvolutional decodeCNNShape(long[] shape)

Decode a CNN Input shape.

Parameters:

shape - Shape to decode

Returns:

Parsed input shape