Kevin Webb (kcwebb@cs.ucsd.edu) 2009 11 03 --DRL manual-- DRL employs two principal abstractions, limiters and identities. The primary abstraction provided by DRL is an identity, which represents a global rate limit and the set of traffic whose enforcement is being coordinated by that limit. At any given node, an identity structure consists of a rate limit, a flow accounting table, a list of neighboring nodes, and other accounting information. Our implementation currently supports two types of identities. "Machine" identities limit the outgoing rate of all traffic leaving a machine, regardless of the traffic's sender. "Set" identities limit the outgoing rate of some subset of the traffic leaving a machine. Sets can contain other sets as well as leaves (which correspond to slivers in PlanetLab). A limiter is an entity which contains (and schedules) identities, attributes packets to identities, and sends and receives messages to other limiters on behalf of identities. Typically, there will be only one limiter per node, and it will be responsible for one or more identities. -Implementation and configuration- ulogd_DRL is a plugin for ulogd, the Linux userspace packet logging daemon. Using DRL requires that two configuration files be configured appropriately. The first file is ulogd's configuration file. Ulogd has a number of configuration options that are not relevant to DRL, and an explanation of those can be found in the ulogd documentation. The DRL sections of the ulogd config file are as follows: The ulogd_DRL.so plugin must be loaded. This is accomplished with a line such as: plugin="/usr/lib/ulogd/ulogd_DRL.so" where /usr/lib/ulogd/ is the path of your ulogd plugin directory. After the "plugin" line, the following parameters must be present: [DRL] nodelimit=0 policy=FPS estintms=500 drl_logfile="/root/pl1-log" drl_loglevel 2 drl_configfile="/root/config.xml" nodelimit specifies a static limit on the amount of network traffic that can be sent by the node as a whole (megabits/sec). NOTE: Set this to 0 for unlimited. policy specifies the allocation and enforcement policy. Valid options are GRD and FPS. estintms is the estimate interval. This specifies the time interval at which the limiter will schedule actions for its identities. Lower values give better responsiveness but incur higher overhead. NOTE: this is in milliseconds. Generally, 500ms works well for FPS. GRD needs a faster interval, generally 50-100ms. drl_logfile specifies where the drl logfile should be written. DRL will reopen the logfile when it receives SIGHUP. drl_loglevel specifies the verbosity of logging. 1 - Debug, 2 - Info, 3 - Critical. You probably know if you want Debug... Info logs detailed traffic rates and protocol state information for producing graphs. Critical only logs errors. drl_configfile specifies the location of the second, DRL-only configuration file that is used to configure identities (discussed below). Optionally, the ulogd.conf file can contain: create_htb Whether or not ulogd should rebuild the htb hierarchy (when using FPS). 0 will not rebuild it, any other value will. Set this to zero when the hierarchy is managed externally (for instance, by PlanetLab's node manager). When this is set to 0, identities require htb_node and htb_parent properties in the drl xml config. enforce_on Whether or not to turn enforcement on by default at startup. 0 for off, anything else for on. If unspecified, it defaults to 1. Enforcement can be turned on/off at runtime with SIGUSR1. bind_addr Can be used to optionally specify which address DRL should use when binding its inter-limiter communication socket. netem_loss netem_delay netem_slice These specifiy an artifical packet loss rate (%) or delay (ms) and the slice xid or the string ALL if it should apply to all slices. These are useful for experimentation. The second file (whose location is determined by drl_configfile) is an XML file containing a series of DRL identity specifications. Sending the ulogd process SIGUSR2 will cause it to re-read this XML config file. DRL supports two types of identities. 1) machine identities: A machine identity is responsible for limiting all traffic that leaves a machine, regardless of the traffic's sliver of origin. 2) set identities: A set identity is responsible for limiting the traffic from a set of slivers or other set identities. The following is an example DRL configuration file: 137.110.222.242 137.110.222.243 137.110.222.240 1f9 137.110.222.245 1fa 20 This file creates one machine identity and two set identities. The resulting hierarchy would look like this, where 1f9 and 1fa are sliver identifiers: 11 | 21 / \ 20 1fa | 1f9 With each identity specifier, the following fields must be defined: id is a globally unique identifier for the identity. limit is the identity's rate limit (in kilobits per second). commfabric specifies the way in which the identity communicates with its peers. Valid options are MESH and GOSSIP. If GOSSIP is select, the branch field must be present and positive. branch specifies the number of peers to which a message should be sent during each interval. Note that this field is ignored when commfabric is MESH. accounting specifies the packet accounting mechanism. Options are STANDARD, SAMPLEHOLD, or SIMPLE (GRD only). Assuming n is the number of flows: Standard is the most accurate - it tracks all currently active flows and keeps O(n) state. Sample&hold uses O(1) state and probabilistically samples packets to achieve good accuracy. Simple only tracks aggregate values and uses very little state. Simple is perfect for GRD, but useless for FPS. ewma determines the extent to which rate changes are smoothed using rate history information. 0.1 is generally a good value. loop_intervals specifies the number of estimate intervals (defined in the ulogd config file) to wait between executions of the estimate, allocate, enforce loop For example, if the estimate interval is 500ms and an identity sets intervals to 2, the identity will be scheduled for updates once every second. Assumes 1 if omited. comm_intervals is similar to loop_intervals, but it schedules communication with peers. Assumes 1 if omitted. htb_node & htb_parent specify which htb_node and htb_parent an identity should modify when the ulogd process is instructed not to rebuild the htb hierarchy (create_htb=0 in ulogd.conf). Optionally, a set identity can be specified as independent, in which case it will not be limited by any machine identities on the node. Assumes 0 if omitted. Each identity must also have one or more peers. Peers are listed within tags inside the identity specifier. In addition to peers, set identities must also have at least one or tag. tags refer to slice ids for slices that are available at the local node. tags refer to the globally unique id of another set identity. A "real" example: 137.110.222.241 137.110.222.242 137.110.222.243 137.110.222.125 137.110.222.126 137.110.222.127 207 This creates one identity numbered 11 with a 10mbps limit. It uses a gossip communication protocol to disseminate updates with a branching factor of two. It uses a standard flow table with an ewma smoothing value of 0.1. The identity has six peers, and it limits only the traffic associated with the slide whose xid is 207.