Package org.apache.sysds.runtime.transform.tokenize

## Class TokenizerPostPosition

- java.lang.Object
- - org.apache.sysds.runtime.transform.tokenize.TokenizerPostPosition

- All Implemented Interfaces:
    `Serializable`, `TokenizerPost`

      public class TokenizerPostPosition
      extends Object
      implements TokenizerPost

  See Also:
    Serialized Form

- - ### Constructor Summary

    Constructors
    Constructor | Description
    --- | ---
    `TokenizerPostPosition​(org.apache.wink.json4j.JSONObject params, int numIdCols, int maxTokens, boolean wideFormat)` |

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method | Description
    --- | --- | ---
    `void` | `appendTokensLong​(List<Object> keys, List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.Token> tokenList, FrameBlock out)` |
    `void` | `appendTokensWide​(List<Object> keys, List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.Token> tokenList, FrameBlock out)` |
    `long` | `getNumCols()` |
    `long` | `getNumRows​(long inRows)` |
    `Types.ValueType[]` | `getOutSchema()` |
    `FrameBlock` | `tokenizePost​(List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.DocumentToTokens> tl, FrameBlock out)` |

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### TokenizerPostPosition

          public TokenizerPostPosition​(org.apache.wink.json4j.JSONObject params,
                                       int numIdCols,
                                       int maxTokens,
                                       boolean wideFormat)

  - ### Method Detail

    - #### tokenizePost

          public FrameBlock tokenizePost​(List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.DocumentToTokens> tl,
                                         FrameBlock out)

      Specified by:
        `tokenizePost` in interface `TokenizerPost`

    - #### appendTokensLong

          public void appendTokensLong​(List<Object> keys,
                                       List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.Token> tokenList,
                                       FrameBlock out)

    - #### appendTokensWide

          public void appendTokensWide​(List<Object> keys,
                                       List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.Token> tokenList,
                                       FrameBlock out)

    - #### getOutSchema

          public Types.ValueType[] getOutSchema()

      Specified by:
        `getOutSchema` in interface `TokenizerPost`

    - #### getNumRows

          public long getNumRows​(long inRows)

      Specified by:
        `getNumRows` in interface `TokenizerPost`

    - #### getNumCols

          public long getNumCols()

      Specified by:
        `getNumCols` in interface `TokenizerPost`