3 Last updated: Sep 21, 2004
9 CKRM's I/O scheduler is developed as a delta over a modified version of
10 the Complete Fair Queuing scheduler (CFQ) that implements I/O priorities.
11 The latter's original posting can be found at:
12 http://www.ussg.iu.edu/hypermail/linux/kernel/0311.1/0019.html
14 Please note that this is not the CFQ version currently in the linus kernel
15 (2.6.8.1 at time of writing) which provides equal, not prioritized,
16 bandwidth allocation amongst processes. Since the CFQ in the kernel is likely
17 to eventually move towards I/O priority implementation, CKRM has not renamed
18 the underlying I/O scheduler and simply replaces drivers/block/cfq-iosched.c
19 with the modified version.
24 1. Configure "Disk I/O Resource Controller" under CKRM (see
25 Documentation/ckrm/installation)
27 2. After booting into the new kernel, load ckrm-io
30 3. Verify that reading /rcfs/taskclass/shares displays values for the
31 I/O controller (res=cki).
33 4. Mount sysfs for monitoring bandwidth received (temporary solution till
34 a userlevel tool is developed)
35 # mount -t sysfs none /sys
41 For brevity, we assume we are in the /rcfs/taskclass directory for all the
44 Initially, the systemwide default class gets 100% of the I/O bandwidth.
48 <display from other controllers, snipped>
50 20 unused/default ioprio
52 The first value is the share of a class, as a parent. The second is the share
53 of its default subclass. Initially the two are equal. As named subclasses get
54 created and assigned shares, the default subclass' share (which equals the
55 "unused" portion of the parent's allocation) dwindles.
58 CFQ assigns one of 20 I/O priorities to all I/O requests. Each priority level
59 gets a fixed proportion of the total bandwidth in increments of 5%. e.g.
61 ioprio=2 gets 10%.....
62 all the way through ioprio=19 getting 95%
64 ioprio=0 gets bandwidth only if no other priority level submits I/O i.e. it can
66 ioprio=20 is considered realtime I/O and always gets priority.
68 CKRM's I/O scheduler distributes these 20 priority levels amongst the hierarchy
69 of classes according to the relative share of each class. Thus, root starts out
70 with the total allocation of 20 initially. As children get created and shares
71 assigned to them, root's allocation reduces. At any time, the sum of absolute
72 share values of all classes equals 20.
81 Its initial share is zero. The parent's share values will be unchanged. Note
82 that even classes with zero share get unused bandwidth under CFQ.
84 Setting a new class share
85 -------------------------
87 $ echo "res=cki,guarantee=20" > /rcfs/taskclass/a/shares
88 Set cki shares to 20 -1 -1 -1
92 res=cki,guarantee=20,limit=100,total_guarantee=100,max_limit=100
94 The limit and max_limit fields can be ignored as they are not implemented.
95 The absolute share of a is 20% of parent's absolute total (20) and can be seen
101 4 unused/default ioprio
103 Since a gets 4, parent's default's share diminishes accordingly. Thus
109 16 unused/default ioprio
115 Each priority level's request service rate can be viewed through sysfs (mounted
116 during installation). To view the servicing of priority 4's requests,
118 $ while : ; echo /sys/block/<device>/queue/iosched/p4 ; sleep 1 ; done
119 rq (10,15) sec (20,30) q (40,50)
121 <data above updated in a loop>
124 rq = cumulative I/O requests received (10) and serviced (15)
125 sec = cumulative sectors requested (20) and served (30)
126 q = cumulative number of times the queue was created(40)/destroyed (50)
128 The rate at which requests or sectors are serviced should differ for different
129 priority levels. The difference in received and serviced values indicates queue
130 depth - with insufficient depth, differentiation between I/O priority levels
131 will not be observed.
133 The rate of q creation is not significant for CKRM.
139 CFQ's I/O differentiation is still being worked upon so its better to choose
140 widely separated share values to observe differences in delivered I/O
143 CFQ, and consequently CKRM, does not provide limits yet. So it is not possible
144 to completely limit an I/O hog process by putting it in a class with a low I/O
145 share. Only if the competing classes maintain sufficient queue depth (i.e a
146 high I/O issue rate) will they get preferential treatment. However, they may
147 still see latency degradation due to seeks caused by servicing of the low
150 When limits are implemented, this behaviour will be rectified.
152 Please post questions on the CKRM I/O scheduler on ckrm-tech@lists.sf.net.