org.apache.sysds.runtime.transform.tokenize

## Class TokenizerPostPosition

- java.lang.Object
- - org.apache.sysds.runtime.transform.tokenize.TokenizerPostPosition

- All Implemented Interfaces:
    Serializable, TokenizerPost

      public class TokenizerPostPosition
      extends Object
      implements TokenizerPost

  See Also:
    Serialized Form

- - ### Constructor Summary

    Constructors
    Constructor and Description
    `TokenizerPostPosition(org.apache.wink.json4j.JSONObject params, int numIdCols, int maxTokens, boolean wideFormat)`

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `appendTokensLong(List<Object> keys, List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.Token> tokenList, FrameBlock out)`
    `void` | `appendTokensWide(List<Object> keys, List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.Token> tokenList, FrameBlock out)`
    `long` | `getNumCols()`
    `long` | `getNumRows(long inRows)`
    `Types.ValueType[]` | `getOutSchema()`
    `FrameBlock` | `tokenizePost(List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.DocumentToTokens> tl, FrameBlock out)`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### TokenizerPostPosition

          public TokenizerPostPosition(org.apache.wink.json4j.JSONObject params,
                                       int numIdCols,
                                       int maxTokens,
                                       boolean wideFormat)

  - ### Method Detail

    - #### tokenizePost

          public FrameBlock tokenizePost(List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.DocumentToTokens> tl,
                                         FrameBlock out)

      Specified by:
        `tokenizePost` in interface `TokenizerPost`

    - #### appendTokensLong

          public void appendTokensLong(List<Object> keys,
                                       List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.Token> tokenList,
                                       FrameBlock out)

    - #### appendTokensWide

          public void appendTokensWide(List<Object> keys,
                                       List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.Token> tokenList,
                                       FrameBlock out)

    - #### getOutSchema

          public Types.ValueType[] getOutSchema()

      Specified by:
        `getOutSchema` in interface `TokenizerPost`

    - #### getNumRows

          public long getNumRows(long inRows)

      Specified by:
        `getNumRows` in interface `TokenizerPost`

    - #### getNumCols

          public long getNumCols()

      Specified by:
        `getNumCols` in interface `TokenizerPost`

Copyright © 2021 The Apache Software Foundation. All rights reserved.