org.apache.storm.scheduler.resource.strategies.scheduling.sorter

## Class NodeSorterHostProximity

- java.lang.Object
- - org.apache.storm.scheduler.resource.strategies.scheduling.sorter.NodeSorterHostProximity

- All Implemented Interfaces:
    INodeSorter

      public class NodeSorterHostProximity
      extends Object
      implements INodeSorter

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class and Description
    --- | ---
    `static interface` | `NodeSorterHostProximity.ExistingScheduleFunc` interface for calculating the number of existing executors scheduled on a object (rack or node).

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `protected Cluster` | `cluster`
    `protected List<String>` | `favoredNodeIds`
    `protected List<String>` | `greyListedSupervisorIds`
    `protected BaseResourceAwareStrategy.NodeSortType` | `nodeSortType`
    `protected TopologyDetails` | `topologyDetails`
    `protected List<String>` | `unFavoredNodeIds`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `NodeSorterHostProximity(Cluster cluster, TopologyDetails topologyDetails)`
    `NodeSorterHostProximity(Cluster cluster, TopologyDetails topologyDetails, BaseResourceAwareStrategy.NodeSortType nodeSortType)` Initialize for the default implementation node sorting.

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `Map<String,Set<String>>` | `getRackIdToHosts()`
    `Map<String,AtomicInteger>` | `getScheduledExecCntByRackId()`
    `Iterable<ObjectResourcesItem>` | `getSortedRacks()` Racks are sorted by two criteria.
    `List<RasNode>` | `hostnameToNodes(String hostname)` hostname to Ids.
    `protected List<String>` | `makeHostToNodeIds(List<String> hosts)`
    `void` | `prepare(ExecutorDetails exec)` Prepare for node sorting.
    `Iterable<String>` | `sortAllNodes()`
    `protected Iterable<ObjectResourcesItem>` | `sortObjectResources(ObjectResourcesSummary resourcesSummary, ExecutorDetails exec, NodeSorterHostProximity.ExistingScheduleFunc existingScheduleFunc)` Scheduling uses `sortAllNodes()` which eventually calls this method whose behavior can be altered by setting `nodeSortType`.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### nodeSortType

          protected final BaseResourceAwareStrategy.NodeSortType nodeSortType

    - #### cluster

          protected Cluster cluster

    - #### topologyDetails

          protected TopologyDetails topologyDetails

    - #### greyListedSupervisorIds

          protected List<String> greyListedSupervisorIds

    - #### favoredNodeIds

          protected List<String> favoredNodeIds

    - #### unFavoredNodeIds

          protected List<String> unFavoredNodeIds

  - ### Constructor Detail

    - #### NodeSorterHostProximity

          public NodeSorterHostProximity(Cluster cluster,
                                         TopologyDetails topologyDetails)

    - #### NodeSorterHostProximity

          public NodeSorterHostProximity(Cluster cluster,
                                         TopologyDetails topologyDetails,
                                         BaseResourceAwareStrategy.NodeSortType nodeSortType)

      Initialize for the default implementation node sorting.

    - `BaseResourceAwareStrategy.NodeSortType#GENERIC_RAS` sorting implemented in `sortObjectResourcesGeneric(ObjectResourcesSummary, ExecutorDetails, NodeSorterHostProximity.ExistingScheduleFunc)`
    - `BaseResourceAwareStrategy.NodeSortType#DEFAULT_RAS` sorting implemented in `sortObjectResourcesDefault(ObjectResourcesSummary, NodeSorterHostProximity.ExistingScheduleFunc)`
    - `BaseResourceAwareStrategy.NodeSortType#COMMON` sorting implemented in `sortObjectResourcesCommon(ObjectResourcesSummary, ExecutorDetails, NodeSorterHostProximity.ExistingScheduleFunc)`

Parameters:
  `cluster` - for which nodes will be sorted.
  `topologyDetails` - the topology to sort for.
  `nodeSortType` - type of sorting to be applied to object resource collection `BaseResourceAwareStrategy.NodeSortType`.

- ### Method Detail

  - #### getRackIdToHosts

        public Map<String,Set<String>> getRackIdToHosts()

  - #### prepare

        public void prepare(ExecutorDetails exec)

    Description copied from interface: `INodeSorter`

    Prepare for node sorting. This method must be called before `INodeSorter.getSortedRacks()` and `INodeSorter.sortAllNodes()`.

    Specified by:
      `prepare` in interface `INodeSorter`
    Parameters:
      `exec` - optional, may be null.

  - #### sortObjectResources

        protected Iterable<ObjectResourcesItem> sortObjectResources(ObjectResourcesSummary resourcesSummary,
                                                                    ExecutorDetails exec,
                                                                    NodeSorterHostProximity.ExistingScheduleFunc existingScheduleFunc)

    Scheduling uses `sortAllNodes()` which eventually calls this method whose behavior can be altered by setting `nodeSortType`.

    Parameters:
      `resourcesSummary` - contains all individual `ObjectResourcesItem` as well as cumulative stats
      `exec` - executor for which the sorting is done
      `existingScheduleFunc` - a function to get existing executors already scheduled on this object
    Returns:
      an `Iterable` of sorted `ObjectResourcesItem`

  - #### makeHostToNodeIds

        protected List<String> makeHostToNodeIds(List<String> hosts)

  - #### sortAllNodes

        public Iterable<String> sortAllNodes()

    Specified by:
      `sortAllNodes` in interface `INodeSorter`

  - #### getScheduledExecCntByRackId

        public Map<String,AtomicInteger> getScheduledExecCntByRackId()

  - #### getSortedRacks

        public Iterable<ObjectResourcesItem> getSortedRacks()

    Racks are sorted by two criteria.

    1) the number executors of the topology that needs to be scheduled is already on the rack in descending order. The reasoning to sort based on criterion 1 is so we schedule the rest of a topology on the same rack as the existing executors of the topology.

    2) the subordinate/subservient resource availability percentage of a rack in descending order We calculate the resource availability percentage by dividing the resource availability on the rack by the resource availability of the entire cluster By doing this calculation, racks that have exhausted or little of one of the resources mentioned above will be ranked after racks that have more balanced resource availability. So we will be less likely to pick a rack that have a lot of one resource but a low amount of another.

    Specified by:
      `getSortedRacks` in interface `INodeSorter`
    Returns:
      an iterable of sorted racks

  - #### hostnameToNodes

        public List<RasNode> hostnameToNodes(String hostname)

    hostname to Ids.

    Parameters:
      `hostname` - the hostname.
    Returns:
      the ids n that node.

Copyright © 2021 The Apache Software Foundation. All rights reserved.