Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10889 [Umbrella] Queue Creation in Capacity Scheduler - Tech debts
  3. YARN-10646

TestCapacitySchedulerWeightMode test descriptor comments doesn't reflect the correct scenario

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersConvert to IssueMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      There is a mixup in TestCapacitySchedulerWeightMode in the configuration creator method comments and the test case descriptor comments. See the following code:

      /*
         * Queue structure:
         *                      root (*)
         *                  _______________________
         *                 /                       \
         *               a x(=100%), y(50%)   b y(=50%), z(=100%)
         *               ________________             ______________
         *              /                           /              \
         *             a1 ([x,y]: w=100)    b1(no)          b2([y,z]: w=100)
         *
         * Parent uses weight, child uses percentage
         */
        public static Configuration getCSConfWithLabelsParentUsePctChildUseWeight(
            Configuration config) {
      

      While inside the method all the queues (including the second level ones) are configured with capacity, only some labels are configured with weights:

          conf.setLabeledQueueWeight(CapacitySchedulerConfiguration.ROOT, "x", 100);
          conf.setLabeledQueueWeight(CapacitySchedulerConfiguration.ROOT, "y", 100);
          conf.setLabeledQueueWeight(CapacitySchedulerConfiguration.ROOT, "z", 100);
      
      ...
      
          conf.setQueues(A, new String[] { "a1" });
          conf.setCapacityByLabel(A1, RMNodeLabelsManager.NO_LABEL, 100);
          conf.setMaximumCapacity(A1, 100);
          conf.setAccessibleNodeLabels(A1, toSet("x", "y"));
          conf.setDefaultNodeLabelExpression(A1, "x");
          conf.setCapacityByLabel(A1, "x", 100);
          conf.setCapacityByLabel(A1, "y", 100);
      
          conf.setQueues(B, new String[] { "b1", "b2" });
          conf.setCapacityByLabel(B1, RMNodeLabelsManager.NO_LABEL, 50);
          conf.setMaximumCapacity(B1, 50);
          conf.setAccessibleNodeLabels(B1, RMNodeLabelsManager.EMPTY_STRING_SET);
      
          conf.setCapacityByLabel(B2, RMNodeLabelsManager.NO_LABEL, 50);
          conf.setMaximumCapacity(B2, 50);
          conf.setAccessibleNodeLabels(B2, toSet("y", "z"));
          conf.setCapacityByLabel(B2, "y", 100);
          conf.setCapacityByLabel(B2, "z", 100);
      

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            bteke Benjamin Teke Assign to me
            bteke Benjamin Teke
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - Not Specified
              Not Specified
              Remaining:
              Remaining Estimate - 0h
              0h
              Logged:
              Time Spent - 1h 10m
              1h 10m

              Slack

                Issue deployment