--- /dev/null
+CKRM Basics
+-------------
+A brief review of CKRM concepts and terminology will help make installation
+and testing easier. For more details, please visit http://ckrm.sf.net.
+
+Currently there are two class types, taskclass and socketclass for grouping,
+regulating and monitoring tasks and sockets respectively.
+
+To avoid repeating instructions for each classtype, this document assumes a
+task to be the kernel object being grouped. By and large, one can replace task
+with socket and taskclass with socketclass.
+
+RCFS depicts a CKRM class as a directory. Hierarchy of classes can be
+created in which children of a class share resources allotted to
+the parent. Tasks can be classified to any class which is at any level.
+There is no correlation between parent-child relationship of tasks and
+the parent-child relationship of classes they belong to.
+
+Without a Classification Engine, class is inherited by a task. A privileged
+user can reassigned a task to a class as described below, after which all
+the child tasks under that task will be assigned to that class, unless the
+user reassigns any of them.
+
+A Classification Engine, if one exists, will be used by CKRM to
+classify a task to a class. The Rule based classification engine uses some
+of the attributes of the task to classify a task. When a CE is present
+class is not inherited by a task.
+
+Characteristics of a class can be accessed/changed through the following magic
+files under the directory representing the class:
+
+shares: allows to change the shares of different resources managed by the
+ class
+stats: allows to see the statistics associated with each resources managed
+ by the class
+target: allows to assign a task to a class. If a CE is present, assigning
+ a task to a class through this interface will prevent CE from
+ reassigning the task to any class during reclassification.
+members: allows to see which tasks has been assigned to a class
+config: allow to view and modify configuration information of different
+ resources in a class.
+
+Resource allocations for a class is controlled by the parameters:
+
+guarantee: specifies how much of a resource is guranteed to a class. A
+ special value DONT_CARE(-2) mean that there is no specific
+ guarantee of a resource is specified, this class may not get
+ any resource if the system is runing short of resources
+limit: specifies the maximum amount of resource that is allowed to be
+ allocated by a class. A special value DONT_CARE(-2) mean that
+ there is no specific limit is specified, this class can get all
+ the resources available.
+total_guarantee: total guarantee that is allowed among the children of this
+ class. In other words, the sum of "guarantee"s of all children
+ of this class cannot exit this number.
+max_limit: Maximum "limit" allowed for any of this class's children. In
+ other words, "limit" of any children of this class cannot exceed
+ this value.
+
+None of this parameters are absolute or have any units associated with
+them. These are just numbers(that are relative to its parents') that are
+used to calculate the absolute number of resource available for a specific
+class.
+
+Note: The root class has an absolute number of resource units associated with it.
+
--- /dev/null
+Usage of CKRM without a classification engine
+-----------------------------------------------
+
+1. Create a class
+
+ # mkdir /rcfs/taskclass/c1
+ creates a taskclass named c1 , while
+ # mkdir /rcfs/socket_class/s1
+ creates a socketclass named s1
+
+The newly created class directory is automatically populated by magic files
+shares, stats, members, target and config.
+
+2. View default shares
+
+ # cat /rcfs/taskclass/c1/shares
+
+ "guarantee=-2,limit=-2,total_guarantee=100,max_limit=100" is the default
+ value set for resources that have controllers registered with CKRM.
+
+3. change shares of a <class>
+
+ One or more of the following fields can/must be specified
+ res=<res_name> #mandatory
+ guarantee=<number>
+ limit=<number>
+ total_guarantee=<number>
+ max_limit=<number>
+ e.g.
+ # echo "res=numtasks,limit=20" > /rcfs/taskclass/c1
+
+ If any of these parameters are not specified, the current value will be
+ retained.
+
+4. Reclassify a task (listening socket)
+
+ write the pid of the process to the destination class' target file
+ # echo 1004 > /rcfs/taskclass/c1/target
+
+ write the "<ipaddress>\<port>" string to the destination class' target file
+ # echo "0.0.0.0\32770" > /rcfs/taskclass/c1/target
+
+5. Get a list of tasks (sockets) assigned to a taskclass (socketclass)
+
+ # cat /rcfs/taskclass/c1/members
+ lists pids of tasks belonging to c1
+
+ # cat /rcfs/socket_class/s1/members
+ lists the ipaddress\port of all listening sockets in s1
+
+6. Get the statictics of different resources of a class
+
+ # cat /rcfs/tasksclass/c1/stats
+ shows c1's statistics for each resource with a registered resource
+ controller.
+
+ # cat /rcfs/socket_class/s1/stats
+ show's s1's stats for the listenaq controller.
+
+7. View the configuration values of the resources associated with a class
+
+ # cat /rcfs/taskclass/c1/config
+ shows per-controller config values for c1.
+
+8. Change the configuration values of resources associated with a class
+ Configuration values are different for different resources. the comman
+ field "res=<resname>" must always be specified.
+
+ # echo "res=numtasks,parameter=value" > /rcfs/taskclass/c1/config
+ to change (without any effect), the value associated with <parameter>.
+
+
--- /dev/null
+CKRM CPU Scheduling
+===================
+
+Overview
+--------
+
+In CKRM, cpu scheduling is based on a two level scheduling decision.
+Every time a new task is to be selected, the scheduler first determines
+which class to run next and then schedules the next task in selected
+task.
+
+The scheduling within a class is performed using the default Linux
+O(1) scheduler.
+
+The class scheduler also follows the O(1) principle and works as
+follows:
+
+Each class maintains a local runqueue per cpu aka <struct
+ckrm_runqueue> or short lrq. The existing O(1) scheduler is used to
+schedule within an <lrq>.
+
+Weights are assigned to each lrq that mirror the effectives shares of
+that class. Every time a task executes, its weighted cycles are
+charged against its class. Thus classes progress in time called
+cummulative virtual time (CVT). In essence the class with the smallest
+CVT is selected next. Provisions are made to keep interactivity and
+avoid starvation by longer sleeping classes.
+
+Load balancing across an SMP is performed by balancing the load of
+each class across CPUs such that they produce equal load and thus
+on the whole system maintain their share.
+
+Due to the fact that CKRM uses a class hierarchy, cycles that are unused
+by a class are redistributed to among busy siblings.
+Enabling the CKRM CPU scheduler
+-------------------------------
+
+The scheduler is integrated into the linux scheduler and therefore
+can not be loaded dynamically like other CKRM schedulers
+
+However it can be selected at boot time or dynamically at run time.
+
+The boot options "ckrmcpu" OR "nockrmcpu" enable / disable the CKRM
+cpu scheduler at boot time. Currently by default the scheduler is
+disabled.
+
+# cat /rcfs/taskclass/config
+
+"res=cpu,mode=enabled" indicates that the CKRM cpu scheduler is
+enabled
+
+"res=cpu,mode=disabled" indicates that the CKRM cpu scheduler is
+disabled
+
+The strings can also be used to dynamically change the scheduling modus
+at runtime. For example, to dynamically activate the scheduler.
+
+# echo "res=cpu,mode=enabled" > /rcfs/taskclass/config
+
+# cat /rcfs/taskclass/*/stats
+
+The cpu portion of the scheduler is shown
+
+ "cpu-usage(2,10,60)= 290 340 510"
+
+The 3 numbers represent the load for the 2 second, 10 second
+and 60 seconds. The base = 1000.
+Hence the system has 29.0%, 33.5% and 49.8% respectively
+
+For debugging purposes additional information can be printed out but
+that format should not be relied upon.
+
+Use `echo "res=cpu,usage_detail=3" for the highest detail on usage.
+Please consult the source code for the specifics.
+
+Assigning shares
+----------------
+
+Follows the general approach described under ckrm_basics.
+
+# echo "res=cpu,guarantee=val" > shares
+
+sets the minimum guarantee of a class.
+
+
+
--- /dev/null
+CRBCE
+----------
+
+crbce is a superset of rbce. In addition to providing automatic
+classification, the crbce module
+- monitors per-process delay data that is collected by the delay
+accounting patch
+- collects data on significant kernel events where reclassification
+could occur e.g. fork/exec/setuid/setgid etc., and
+- uses relayfs to supply both these datapoints to userspace
+
+To illustrate the utility of the data gathered by crbce, we provide a
+userspace daemon called crbcedmn that prints the header info received
+from the records sent by the crbce module.
+
+0. Ensure that a CKRM-enabled kernel with following options configured
+ has been compiled. At a minimum, core, rcfs, atleast one classtype,
+ delay-accounting patch and relayfs. For testing, it is recommended
+ all classtypes and resource controllers be compiled as modules.
+
+1. Ensure that the Makefile's BUILD_CRBCE=1 and KDIR points to the
+ kernel of step 1 and call make.
+ This also builds the userspace daemon, crbcedmn.
+
+2..9 Same as rbce installation and testing instructions,
+ except replacing rbce.ko with crbce.ko
+
+10. Read the pseudo daemon help file
+ # ./crbcedmn -h
+
+11. Run the crbcedmn to display all records being processed
+ # ./crbcedmn
+
--- /dev/null
+Kernel installation
+------------------------------
+
+<kernver> = version of mainline Linux kernel
+<ckrmver> = version of CKRM
+
+Note: It is expected that CKRM versions will change fairly rapidly. Hence once
+a CKRM version has been released for some <kernver>, it will only be made
+available for future <kernver>'s until the next CKRM version is released.
+
+1. Patch
+
+ Apply ckrm/kernel/<kernver>/ckrm-<ckrmversion>.patch to a mainline kernel
+ tree with version <kernver>.
+
+ If CRBCE will be used, additionally apply the following patches, in order:
+ delayacctg-<ckrmversion>.patch
+ relayfs-<ckrmversion>.patch
+
+
+2. Configure
+
+Select appropriate configuration options:
+
+a. for taskclasses
+
+ General Setup-->Class Based Kernel Resource Management
+
+ [*] Class Based Kernel Resource Management
+ <M> Resource Class File System (User API)
+ [*] Class Manager for Task Groups
+ <M> Number of Tasks Resource Manager
+
+b. To test socket_classes and multiple accept queue controller
+
+ General Setup-->Class Based Kernel Resource Management
+ [*] Class Based Kernel Resource Management
+ <M> Resource Class File System (User API)
+ [*] Class Manager for socket groups
+ <M> Multiple Accept Queues Resource Manager
+
+ Device Drivers-->Networking Support-->Networking options-->
+ [*] Network packet filtering (replaces ipchains)
+ [*] IP: TCP Multiple accept queues support
+
+c. To test CRBCE later (requires 2a.)
+
+ File Systems-->Pseudo filesystems-->
+ <M> Relayfs filesystem support
+ (enable all sub fields)
+
+ General Setup-->
+ [*] Enable delay accounting
+
+
+3. Build, boot into kernel
+
+4. Enable rcfs
+
+ # insmod <patchedtree>/fs/rcfs/rcfs.ko
+ # mount -t rcfs rcfs /rcfs
+
+ This will create the directories /rcfs/taskclass and
+ /rcfs/socketclass which are the "roots" of subtrees for creating
+ taskclasses and socketclasses respectively.
+
+5. Load numtasks and listenaq controllers
+
+ # insmod <patchedtree>/kernel/ckrm/ckrm_tasks.ko
+ # insmod <patchedtree>/kernel/ckrm/ckrm_listenaq.ko
--- /dev/null
+Rule-based Classification Engine (RBCE)
+-------------------------------------------
+
+The ckrm/rbce directory contains the sources for two classification engines
+called rbce and crbce. Both are optional, built as kernel modules and share much
+of their codebase. Only one classification engine (CE) can be loaded at a time
+in CKRM.
+
+
+With RBCE, user can specify rules for how tasks are classified to a
+class. Rules are specified by one or more attribute-value pairs and
+an associated class. The tasks that match all the attr-value pairs
+will get classified to the class attached with the rule.
+
+The file rbce_info under /rcfs/ce directory details the functionality
+of different files available under the directory and also details
+about attributes that can are used to define rules.
+
+order: When multiple rules are defined the rules are executed
+ according to the order of a rule. Order can be specified
+ while defining a rule. If order is not specified, the
+ highest order will be assigned to the rule(i.e, the new
+ rule will be executed after all the previously defined
+ evaluate false). So, order of rules is important as that
+ will decide, which class a task will get assigned to. For
+ example, if we have the two following rules: r1:
+ uid=1004,order=10,class=/rcfs/taskclass/c1 r2:
+ uid=1004,cmd=grep,order=20,class=/rcfs/taskclass/c2 then,
+ the task "grep" executed by user 1004 will always be
+ assigned to class /rcfs/taskclass/c1, as rule r1 will be
+ executed before r2 and the task successfully matched the
+ rule's attr-value pairs. Rule r2 will never be consulted
+ for the command. Note: The order in which the rules are
+ displayed(by ls) has no correlation with the order of the
+ rule.
+
+dependency: Rules can be defined to be depend on another rule. i.e a
+ rule can be dependent on one rule and has its own
+ additional attr-value pairs. the dependent rule will
+ evaluate true only if all the attr-value pairs of both
+ rules are satisfied. ex: r1: gid=502,class=/rcfs/taskclass
+ r2: depend=r1,cmd=grep,class=rcfstaskclass/c1 r2 is a
+ dependent rule that depends on r1, a task will be assigned
+ to /rcfs/taskclass/c1 if its gid is 502 and the executable
+ command name is "grep". If a task's gid is 502 but the
+ command name is _not_ "grep" then it will be assigned to
+ /rcfs/taskclass
+
+ Note: The order of dependent rule must be _lesser_ than the
+ rule it depends on, so that it is evaluated _before the
+ base rule is evaluated. Otherwise the base rule will
+ evaluate true and the task will be assigned to the class of
+ that rule without the dependent rule ever getting
+ evaluated. In the example above, order of r2 must be lesser
+ than order of r1.
+
+app_tag: a task can be attached with a tag(ascii string), that becomes
+ an attribute of that task and rules can be defined with the
+ tag value.
+
+state: states are at two levels in RBCE. The entire RBCE can be
+ enabled or disabled which writing 1 or 0 to the file
+ rbce_state under /rcfs/ce. Disabling RBCE, would mean that
+ the rules defined in RBCE will not be utilized for
+ classifying a task to a class. A specific rule can be
+ enabled/disabled by changing the state of that rule. Once
+ it is disabled, the rule will not be evaluated.
--- /dev/null
+Usage of CKRM with RBCE
+--------------------------
+
+0. Ensure that a CKRM-enabled kernel with following options configured
+ has been compiled. At a minimum, core, rcfs and atleast one
+ classtype. For testing, it is recommended all classtypes and
+ resource controllers be compiled as modules.
+
+1. Change ckrm/rbce/Makefile's KDIR to point to this compiled kernel's source
+ tree and call make
+
+2. Load rbce module.
+ # insmod ckrm/rbce/rbce.ko
+ Note that /rcfs has to be mounted before this.
+ Note: this command should populate the directory /rcfs/ce with files
+ rbce_reclassify, rbce_tag, rbce_info, rbce_state and a directory
+ rules.
+
+ Note2: If these are not created automatically, just create them by
+ using the commands touch and mkdir.(bug that needs to be fixed)
+
+3. Defining a rule
+ Rules are defined by creating(by writing) to a file under the
+ /rcfs/ce/rules directory by concatinating multiple attribute value
+ pairs.
+
+ Note that the classes must be defined before defining rules that
+ uses the classes. eg: the command # echo
+ "uid=1004,class=/rcfs/taskclass/c1" > /rcfs/ce/rules/r1 will define
+ a rule r1 that classifies all tasks belong to user id 1004 to class
+ /rcfs/taskclass/c1
+
+4. Viewing a rule
+ read the corresponding file.
+ to read rule r1, issue the command:
+ # cat /rcfs/ce/rules/r1
+
+5. Changing a rule
+
+ Changing a rule is done the same way as defining a rule, the new
+ rule will include the old set of attr-value pairs slapped with new
+ attr-value pairs. eg: if the current r2 is
+ uid=1004,depend=r1,class=/rcfs/taskclass/c1
+ (r1 as defined in step 3)
+
+ the command:
+ # echo gid=502 > /rcfs/ce/rules/r1
+ will change the rule to
+ r1: uid=1004,gid=502,depend=r1,class=/rcfs/taskclass/c1
+
+ the command:
+ # echo uid=1005 > /rcfs/ce/rules/r1
+ will change the rule to
+ r1: uid=1005,class=/rcfs/taskclass/c1
+
+ the command:
+ # echo class=/rcfs/taskclass/c2 > /rcfs/ce/rules/r1
+ will change the rule to
+ r1: uid=1004,depend=r1,class=/rcfs/taskclass/c2
+
+ the command:
+ # echo depend=r4 > /rcfs/ce/rules/r1
+ will change the rule to
+ r1: uid=1004,depend=r4,class=/rcfs/taskclass/c2
+
+ the command:
+ # echo +depend=r4 > /rcfs/ce/rules/r1
+ will change the rule to
+ r1: uid=1004,depend=r1,depend=r4,class=/rcfs/taskclass/c2
+
+ the command:
+ # echo -depend=r1 > /rcfs/ce/rules/r1
+ will change the rule to
+ r1: uid=1004,class=/rcfs/taskclass/c2
+
+6. Checking the state of RBCE
+ State(enabled/disabled) of RBCE can be checked by reading the file
+ /rcfs/ce/rbce_state, it will show 1(enabled) or 0(disabled).
+ By default, RBCE is enabled(1).
+ ex: # cat /rcfs/ce/rbce_state
+
+7. Changing the state of RBCE
+ State of RBCE can be changed by writing 1(enable) or 0(disable).
+ ex: # echo 1 > cat /rcfs/ce/rbce_state
+
+8. Checking the state of a rule
+ State of a rule is displayed in the rule. Rule can be viewed by
+ reading the rule file. ex: # cat /rcfs/ce/rules/r1
+
+9. Changing the state of a rule
+
+ State of a rule can be changed by writing "state=1"(enable) or
+ "state=0"(disable) to the corresponding rule file. By defeault, the
+ rule is enabled when defined. ex: to disable an existing rule r1,
+ issue the command
+ # echo "state=0" > /rcfs/ce/rules/r1
+
+
git://git.onelab.eu / linux-2.6.git / commitdiff