weka.filters.unsupervised.attribute

Class Dl4jStringToWord2Vec

java.lang.Object

weka.filters.Filter

weka.filters.SimpleFilter

weka.filters.SimpleBatchFilter

weka.filters.unsupervised.attribute.Dl4jStringToWordEmbeddings

weka.filters.unsupervised.attribute.Dl4jStringToWord2Vec

All Implemented Interfaces:

java.io.Serializable, weka.core.CapabilitiesHandler, weka.core.CapabilitiesIgnorer, weka.core.CommandlineRunnable, weka.core.OptionHandler, weka.core.RevisionHandler

public class Dl4jStringToWord2Vec
extends Dl4jStringToWordEmbeddings

An attribute filter that calculates word embeddings on a String attribute using the Word2vec implementation provided by DeepLearning4j. BibTeX:

 @@Article{Word2Vec,
  Title                    = {Efficient estimation of word representations in vector space.},
  Author                   = {Mikolov, Tomas and Chen, Kai and Corrado, Greg and Dean, Jeffrey},
  Journal                  = {arXiv preprint arXiv:1301.3781},
  Year                     = {2013}
 }



 

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
Dl4jStringToWord2Vec()

Method Summary

Modifier and Type	Method and Description
int	getBatchSize()
double	getLearningRate()
double	getMinLearningRate()
double	getNegative()
java.lang.String[]	getOptions()
double	getSampling()
weka.core.TechnicalInformation	getTechnicalInformation() Returns an instance of a TechnicalInformation object, containing detailed information about the technical background of this class, e.g., paper reference or book this class is based on.
java.lang.String	globalInfo() Returns a string describing this filter.
boolean	isAllowParallelTokenization()
boolean	isEnableScavenger()
boolean	isUseAdaGrad()
boolean	isUseHierarchicSoftmax()
java.util.Enumeration<weka.core.Option>	listOptions()
void	setAllowParallelTokenization(boolean m_allowParallelTokenization)
void	setBatchSize(int m_batchSize)
void	setEnableScavenger(boolean m_enableScavenger)
void	setLearningRate(double m_learningRate)
void	setMinLearningRate(double m_minLearningRate)
void	setNegative(double m_negative)
void	setOptions(java.lang.String[] options) Parses the options for this object.
void	setSampling(double m_sampling)
void	setUseAdaGrad(boolean m_useAdaGrad)
void	setUseHierarchicSoftmax(boolean m_useHierarchicSoftmax)

Methods inherited from class weka.filters.unsupervised.attribute.Dl4jStringToWordEmbeddings

getAction, getCapabilities, getConcat_words, getConcatWords, getEmbedding_prefix, getEpochs, getIterations, getLayerSize, getMinWordFrequency, getPreProcessor, getSeed, getStopWordsHandler, getTextIndex, getTokenizerFactory, getWindowSize, getWorkers, setAction, setConcat_words, setConcatWords, setEmbedding_prefix, setEpochs, setIterations, setLayerSize, setMinWordFrequency, setPreProcessor, setSeed, setStopWordsHandler, setTextIndex, setTokenizerFactory, setWindowSize, setWorkers

Methods inherited from class weka.filters.SimpleBatchFilter

allowAccessToFullInputFormat, batchFinished, input, input

Methods inherited from class weka.filters.SimpleFilter

setInputFormat

Methods inherited from class weka.filters.Filter

batchFilterFile, debugTipText, doNotCheckCapabilitiesTipText, filterFile, getCapabilities, getCopyOfInputFormat, getDebug, getDoNotCheckCapabilities, getOutputFormat, getRevision, isFirstBatchDone, isNewBatch, isOutputFormatDefined, main, makeCopies, makeCopy, mayRemoveInstanceAfterFirstBatchDone, numPendingOutput, output, outputPeek, postExecution, preExecution, run, runFilter, setDebug, setDoNotCheckCapabilities, toString, useFilter, wekaStaticWrapper

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface weka.core.OptionHandler

makeCopy

Constructor Detail

Dl4jStringToWord2Vec

public Dl4jStringToWord2Vec()

Method Detail

globalInfo

public java.lang.String globalInfo()

Returns a string describing this filter.

Specified by:

globalInfo in class weka.filters.SimpleFilter

Returns:

a description of the filter suitable for displaying in the explorer/experimenter gui

getTechnicalInformation

public weka.core.TechnicalInformation getTechnicalInformation()

Returns an instance of a TechnicalInformation object, containing detailed information about the technical background of this class, e.g., paper reference or book this class is based on.

Returns:

the technical information about this class

listOptions

public java.util.Enumeration<weka.core.Option> listOptions()

Specified by:

listOptions in interface weka.core.OptionHandler

Overrides:

listOptions in class Dl4jStringToWordEmbeddings

getOptions

public java.lang.String[] getOptions()

Specified by:

getOptions in interface weka.core.OptionHandler

Overrides:

getOptions in class Dl4jStringToWordEmbeddings

setOptions

public void setOptions(java.lang.String[] options)
                throws java.lang.Exception

Parses the options for this object.

Specified by:

setOptions in interface weka.core.OptionHandler

Overrides:

setOptions in class Dl4jStringToWordEmbeddings

Parameters:

options - the options to use

Throws:

java.lang.Exception - if setting of options fails

getBatchSize

@OptionMetadata(displayName="batchSize",
                description="The mini-batch size (default = 512).",
                commandLineParamName="batchSize",
                commandLineParamSynopsis="-batchSize <int>",
                displayOrder=15)
public int getBatchSize()

setBatchSize

public void setBatchSize(int m_batchSize)

getLearningRate

@OptionMetadata(displayName="learningRate",
                description="The learning rate (default = 0.025).",
                commandLineParamName="learningRate",
                commandLineParamSynopsis="-learningRate <double>",
                displayOrder=16)
public double getLearningRate()

setLearningRate

public void setLearningRate(double m_learningRate)

getMinLearningRate

@OptionMetadata(displayName="minLearningRate",
                description="This method defines minimal learning rate value for training (default = 1.0E-4).",
                commandLineParamName="minLearningRate",
                commandLineParamSynopsis="-minLearningRate <double>",
                displayOrder=17)
public double getMinLearningRate()

setMinLearningRate

public void setMinLearningRate(double m_minLearningRate)

isUseAdaGrad

@OptionMetadata(displayName="useAdaGrad",
                description="This method defines whether adaptive gradients should be used or not (default = false).",
                commandLineParamName="useAdaGrad",
                commandLineParamSynopsis="-useAdaGrad",
                commandLineParamIsFlag=true,
                displayOrder=18)
public boolean isUseAdaGrad()

setUseAdaGrad

public void setUseAdaGrad(boolean m_useAdaGrad)

getNegative

@OptionMetadata(displayName="negative",
                description="The negative sampling value for skip-gram algorithm (default = 0.0).",
                commandLineParamName="negative",
                commandLineParamSynopsis="-negative <double>",
                displayOrder=19)
public double getNegative()

setNegative

public void setNegative(double m_negative)

isUseHierarchicSoftmax

@OptionMetadata(displayName="useHierarchicSoftmax",
                description="Enable/disable hierarchic softmax (default = true).",
                commandLineParamName="useHierarchicSoftmax",
                commandLineParamSynopsis="-useHierarchicSoftmax",
                commandLineParamIsFlag=true,
                displayOrder=20)
public boolean isUseHierarchicSoftmax()

setUseHierarchicSoftmax

public void setUseHierarchicSoftmax(boolean m_useHierarchicSoftmax)

getSampling

@OptionMetadata(displayName="sampling",
                description="The sub-sampling threshold (default = 0.0).",
                commandLineParamName="sampling",
                commandLineParamSynopsis="-sampling <double>",
                displayOrder=21)
public double getSampling()

setSampling

public void setSampling(double m_sampling)

isAllowParallelTokenization

@OptionMetadata(displayName="allowParallelTokenization",
                description="Enables/disables parallel tokenization (default = true).",
                commandLineParamName="allowParallelTokenization",
                commandLineParamSynopsis="-allowParallelTokenization",
                commandLineParamIsFlag=true,
                displayOrder=22)
public boolean isAllowParallelTokenization()

setAllowParallelTokenization

public void setAllowParallelTokenization(boolean m_allowParallelTokenization)

isEnableScavenger

@OptionMetadata(displayName="enableScavenger",
                description="Enables/disables periodical vocab truncation during construction (default = false).",
                commandLineParamName="enableScavenger",
                commandLineParamSynopsis="-enableScavenger",
                commandLineParamIsFlag=true,
                displayOrder=23)
public boolean isEnableScavenger()

setEnableScavenger

public void setEnableScavenger(boolean m_enableScavenger)