weka.dl4j.iterators.dataset.sequence.text.rnn

Class RnnTextEmbeddingDataSetIterator

java.lang.Object

weka.dl4j.iterators.dataset.sequence.text.rnn.RnnTextEmbeddingDataSetIterator

All Implemented Interfaces:

java.io.Serializable, java.util.Iterator<org.nd4j.linalg.dataset.DataSet>, org.nd4j.linalg.dataset.api.iterator.DataSetIterator

public class RnnTextEmbeddingDataSetIterator
extends java.lang.Object
implements org.nd4j.linalg.dataset.api.iterator.DataSetIterator, java.io.Serializable

A DataSetIterator implementation that reads text documents from an arff file and translates each document to a sequence of wordvectors, given a wordvector model.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
RnnTextEmbeddingDataSetIterator(weka.core.Instances data, org.deeplearning4j.models.embeddings.wordvectors.WordVectors wordVectors, TokenizerFactory tokenFact, TokenPreProcess tpp, weka.core.stopwords.AbstractStopwords stopWords, org.deeplearning4j.iterator.LabeledSentenceProvider sentenceProvider, int batchSize, int truncateLength) Constructor with necessary objects to create RNN features.

Method Summary

Modifier and Type	Method and Description
boolean	asyncSupported()
int	batch()
java.util.List<java.lang.String>	getLabels()
org.nd4j.linalg.dataset.api.DataSetPreProcessor	getPreProcessor()
boolean	hasNext()
int	inputColumns()
org.nd4j.linalg.dataset.DataSet	next()
org.nd4j.linalg.dataset.DataSet	next(int num)
int	numExamples()
void	remove()
void	reset()
boolean	resetSupported()
void	setPreProcessor(org.nd4j.linalg.dataset.api.DataSetPreProcessor preProcessor)
int	totalOutcomes()

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining

Constructor Detail

RnnTextEmbeddingDataSetIterator

public RnnTextEmbeddingDataSetIterator(weka.core.Instances data,
                                       org.deeplearning4j.models.embeddings.wordvectors.WordVectors wordVectors,
                                       TokenizerFactory tokenFact,
                                       TokenPreProcess tpp,
                                       weka.core.stopwords.AbstractStopwords stopWords,
                                       org.deeplearning4j.iterator.LabeledSentenceProvider sentenceProvider,
                                       int batchSize,
                                       int truncateLength)

Constructor with necessary objects to create RNN features.

Parameters:

data - Instances with documents and labels

wordVectors - WordVectors object

tokenFact - Tokenizer factory

tpp - Token pre processor

stopWords - Stop word object

batchSize - Size of each minibatch for training

truncateLength - If reviews exceed

Method Detail

next

public org.nd4j.linalg.dataset.DataSet next(int num)

Specified by:

next in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

inputColumns

public int inputColumns()

Specified by:

inputColumns in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

totalOutcomes

public int totalOutcomes()

Specified by:

totalOutcomes in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

reset

public void reset()

Specified by:

reset in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

resetSupported

public boolean resetSupported()

Specified by:

resetSupported in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

asyncSupported

public boolean asyncSupported()

Specified by:

asyncSupported in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

batch

public int batch()

Specified by:

batch in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

numExamples

public int numExamples()

getLabels

public java.util.List<java.lang.String> getLabels()

Specified by:

getLabels in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

hasNext

public boolean hasNext()

Specified by:

hasNext in interface java.util.Iterator<org.nd4j.linalg.dataset.DataSet>

next

public org.nd4j.linalg.dataset.DataSet next()

Specified by:

next in interface java.util.Iterator<org.nd4j.linalg.dataset.DataSet>

remove

public void remove()

Specified by:

remove in interface java.util.Iterator<org.nd4j.linalg.dataset.DataSet>

getPreProcessor

public org.nd4j.linalg.dataset.api.DataSetPreProcessor getPreProcessor()

Specified by:

getPreProcessor in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

setPreProcessor

public void setPreProcessor(org.nd4j.linalg.dataset.api.DataSetPreProcessor preProcessor)

Specified by:

setPreProcessor in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator