Turned enforcement on by default at startup. Added a ulogd.conf config option to...

diff --git a/drl/Manual.txt b/drl/Manual.txt
index bf25060..ba2e259 100644 (file)
--- a/drl/Manual.txt
+++ b/drl/Manual.txt
@@ -1,15 +1,20 @@
+Kevin Webb (kcwebb@cs.ucsd.edu)
+2009 11 03
+

 --DRL manual--
 
 DRL employs two principal abstractions, limiters and identities.
 
 --DRL manual--
 
 DRL employs two principal abstractions, limiters and identities.
 

-An identity is a group of nodes that are cooperating to enforce a rate limit.
-At any given node, an identity structure consists of a rate limit, a flow
-accounting table, a list of neighboring nodes, and other accounting
-information.  Our implementation currently supports two types of identities.
-"Machine" identities limit the outgoing rate of all traffic leaving a machine,
-regardless of the traffic's sender.  "Set" identities limit the outgoing rate
-of some subset of the traffic leaving a machine.  Sets can contain other sets
-as well as leaves (which correspond to slivers in PlanetLab).
+The primary abstraction provided by DRL is an identity, which represents a
+global rate limit and the set of traffic whose enforcement is being coordinated
+by that limit.  At any given node, an identity structure consists of a rate
+limit, a flow accounting table, a list of neighboring nodes, and other
+accounting information.  Our implementation currently supports two types of
+identities.  "Machine" identities limit the outgoing rate of all traffic
+leaving a machine, regardless of the traffic's sender.  "Set" identities limit
+the outgoing rate of some subset of the traffic leaving a machine.  Sets can
+contain other sets as well as leaves (which correspond to slivers in
+PlanetLab).

 
 A limiter is an entity which contains (and schedules) identities, attributes
 packets to identities, and sends and receives messages to other limiters on
 
 A limiter is an entity which contains (and schedules) identities, attributes
 packets to identities, and sends and receives messages to other limiters on


@@ -42,43 +47,64 @@ drl_loglevel 2
 drl_configfile="/root/config.xml"
 
 nodelimit specifies a static limit on the amount of network traffic that can be
 drl_configfile="/root/config.xml"
 
 nodelimit specifies a static limit on the amount of network traffic that can be

-sent by the node (megabits/sec).  NOTE: Set this to 0 for unlimited.
+sent by the node as a whole (megabits/sec).  NOTE: Set this to 0 for unlimited.

 
 

-policy specifies the enforcement policy.  Valid options are GRD and FPS.  GRD
-is currently broken, so use FPS for now.
+policy specifies the allocation and enforcement policy.  Valid options are GRD
+and FPS.

 
 estintms is the estimate interval.  This specifies the time interval at which
 
 estintms is the estimate interval.  This specifies the time interval at which

-DRL can schedule identity updates.  Lower values give better responsiveness but
-incur higher overhead.  NOTE: this is in milliseconds.
+the limiter will schedule actions for its identities.  Lower values give better
+responsiveness but incur higher overhead.  NOTE: this is in milliseconds.
+Generally, 500ms works well for FPS.  GRD needs a faster interval, generally
+50-100ms.

 
 drl_logfile specifies where the drl logfile should be written.
 
 
 drl_logfile specifies where the drl logfile should be written.
 

-drl_loglevel specifies the verbosity of logging. 1 - Debug, 2 - Warn,
-3 - Critical
+drl_loglevel specifies the verbosity of logging. 1 - Debug, 2 - Info, 3 -
+Critical.  You probably know if you want Debug...  Info logs detailed traffic
+rates and protocol state information for producing graphs.  Critical only logs
+errors.

 
 drl_configfile specifies the location of the second, DRL-only configuration
 
 drl_configfile specifies the location of the second, DRL-only configuration

-file.
+file that is used to configure identities (discussed below).
+
+Optionally, the ulogd.conf file can contain:
+
+enforce_on <int>
+Whether or not to turn enforcement on by default at startup.  0 for off,
+anything else for on.  If unspecified, it defaults to 1.  Enforcement can be
+turned on/off at runtime with SIGUSR1.
+
+netem_loss <int>
+netem_delay <int>
+netem_slice <int|ALL>
+
+These specifiy an artifical packet loss rate (%) or delay (ms) and the slice
+xid or the string ALL if it should apply to all slices.  These are useful for
+experimentation.

 
 

-The second file (whose location is determined by drl_configfile) is an XML file
-containing a series of DRL identity specifications.  DRL supports two types of
-identities.  1) machine identities: A machine identity is responsible for
-limiting all traffic that leaves a machine, regardless of the traffic's sliver
-of origin.  2) set identities: A set identity is responsible for limiting the
-traffic from a set of slivers or other set identities.
+
+The second file (whose location is determined by drl_configfile) is an XML
+file containing a series of DRL identity specifications. Sending the ulogd
+process SIGHUP will cause it to re-read this XML config file.  DRL supports
+two types of identities.  1) machine identities: A machine identity is
+responsible for limiting all traffic that leaves a machine, regardless of the
+traffic's sliver of origin.  2) set identities: A set identity is responsible
+for limiting the traffic from a set of slivers or other set identities.

 
 The following is an example DRL configuration file:
 
 <?xml version="1.0" encoding="UTF-8"?>
 <drl>
 
 The following is an example DRL configuration file:
 
 <?xml version="1.0" encoding="UTF-8"?>
 <drl>

-    <machine id="11" limit="10" commfabric="MESH" branch="0" accounting="STANDARD" ewma="0.1" intervals="2">
+    <machine id="11" limit="10000" commfabric="MESH" branch="0" accounting="STANDARD" ewma="0.1">

         <peer>137.110.222.242</peer>
         <peer>137.110.222.243</peer>
     </machine>
         <peer>137.110.222.242</peer>
         <peer>137.110.222.243</peer>
     </machine>

-    <set id="20" limit="15" commfabric="GOSSIP" branch="1" accounting="STANDARD" ewma="0.1" intervals="3">
+    <set id="20" limit="15000" commfabric="GOSSIP" branch="1" accounting="SAMPLEHOLD" ewma="0.1">

         <peer>137.110.222.240</peer>
         <xid>1f9</xid>
     </set>
         <peer>137.110.222.240</peer>
         <xid>1f9</xid>
     </set>

-    <set id="21" limit="25" commfabric="MESH" branch="0" accounting="STANDARD" ewma="0.1" intervals="1">
+    <set id="21" limit="25000" commfabric="MESH" branch="0" accounting="STANDARD" ewma="0.1" loop_intervals="2" comm_intervals="4" independent="1">

         <peer>137.110.222.245</peer>
         <xid>1fa</xid>
         <guid>20</guid>
         <peer>137.110.222.245</peer>
         <xid>1fa</xid>
         <guid>20</guid>


@@ -86,7 +112,7 @@ The following is an example DRL configuration file:
 </drl>
 
 This file creates one machine identity and two set identities.  The resulting
 </drl>
 
 This file creates one machine identity and two set identities.  The resulting

-hierarchy would look like this, where 1f9 and 1fa are slivers:
+hierarchy would look like this, where 1f9 and 1fa are sliver identifiers:

 
         11
         |
 
         11
         |


@@ -100,29 +126,59 @@ With each identity specifier, the following fields must be defined:
 
 id is a globally unique identifier for the identity.
 
 
 id is a globally unique identifier for the identity.
 

-limit is the identity's rate limit (in megabits per second).
+limit is the identity's rate limit (in kilobits per second).

 
 commfabric specifies the way in which the identity communicates with its peers.
 Valid options are MESH and GOSSIP.  If GOSSIP is select, the branch field must
 be present and positive.
 
 branch specifies the number of peers to which a message should be sent during
 
 commfabric specifies the way in which the identity communicates with its peers.
 Valid options are MESH and GOSSIP.  If GOSSIP is select, the branch field must
 be present and positive.
 
 branch specifies the number of peers to which a message should be sent during

-each estimate interval.  Note that this field is ignored when commfabric is
-MESH.
+each interval.  Note that this field is ignored when commfabric is MESH.

 
 

-accounting specifies the packet accounting mechanism.  Just leave this as
-STANDARD for now, or bad things might happen. :)
+accounting specifies the packet accounting mechanism.  Options are STANDARD,
+SAMPLEHOLD, or SIMPLE (GRD only).  Assuming n is the number of flows: Standard
+is the most accurate - it tracks all currently active flows and keeps O(n)
+state.  Sample&hold uses O(1) state and probabilistically samples packets to
+achieve good accuracy.  Simple only tracks aggregate values and uses very
+little state.  Simple is perfect for GRD, but useless for FPS.

 
 ewma determines the extent to which rate changes are smoothed using rate
 history information.  0.1 is generally a good value.
 
 
 ewma determines the extent to which rate changes are smoothed using rate
 history information.  0.1 is generally a good value.
 

-intervals specifies the number of estimate intervals (defined in the ulogd
-config file) to wait between updates.  For example, if the estimate interval is
-500ms and an identity sets intervals to 2, the identity will be scheduled for
-updates once every second.
+loop_intervals specifies the number of estimate intervals (defined in the ulogd
+config file) to wait between executions of the estimate, allocate, enforce loop
+For example, if the estimate interval is 500ms and an identity sets intervals
+to 2, the identity will be scheduled for updates once every second.  Assumes 1
+if omited.
+
+comm_intervals is similar to loop_intervals, but it schedules communication
+with peers.  Assumes 1 if omitted.
+
+Optionally, a set identity can be specified as independent, in which case it
+will not be limited by any machine identities on the node.  Assumes 0 if
+omitted.

 
 Each identity must also have one or more peers.  Peers are listed within <peer>
 tags inside the identity specifier.  In addition to peers, set identities must
 also have at least one <xid> or <guid> tag.  <xid> tags refer to slice ids for
 slices that are available at the local node.  <guid> tags refer to the globally
 unique id of another set identity.
 
 Each identity must also have one or more peers.  Peers are listed within <peer>
 tags inside the identity specifier.  In addition to peers, set identities must
 also have at least one <xid> or <guid> tag.  <xid> tags refer to slice ids for
 slices that are available at the local node.  <guid> tags refer to the globally
 unique id of another set identity.

+
+
+
+A "real" example:
+
+<?xml version="1.0" encoding="UTF-8"?>
+<drl>
+    <set id="11" limit="10240" commfabric="GOSSIP" branch="2" accounting="STANDARD" ewma="0.1">
+        <peer>137.110.222.241</peer>
+        <peer>137.110.222.242</peer>
+        <peer>137.110.222.243</peer>
+        <peer>137.110.222.125</peer>
+        <peer>137.110.222.126</peer>
+        <peer>137.110.222.127</peer>
+        <xid>207</xid>
+    </set>
+</drl>
+
+This creates one identity numbered 11 with a 10mbps limit.  It uses a gossip communication protocol to disseminate updates with a branching factor of two.  It uses a standard flow table with an ewma smoothing value of 0.1.  The identity has six peers, and it limits only the traffic associated with the slide whose xid is 207.

diff --git a/drl/ulogd_DRL.c b/drl/ulogd_DRL.c
index 83e5f13..f9d6d56 100644 (file)
--- a/drl/ulogd_DRL.c
+++ b/drl/ulogd_DRL.c
@@ -124,8 +124,16 @@
  * Add the config options for DRL. 
  */
 
  * Add the config options for DRL. 
  */
 

-static config_entry_t partition = {
+static config_entry_t enforce_on = {

     .next = NULL,
     .next = NULL,

+    .key = "enforce_on",
+    .type = CONFIG_TYPE_INT,
+    .options = CONFIG_OPT_NONE,
+    .u = { .value = 1 },
+};
+
+static config_entry_t partition = {
+    .next = &enforce_on,

     .key = "partition_set",
     .type = CONFIG_TYPE_INT,
     .options = CONFIG_OPT_NONE,
     .key = "partition_set",
     .type = CONFIG_TYPE_INT,
     .options = CONFIG_OPT_NONE,


@@ -1782,6 +1790,13 @@ static void _drl_reg_op(void)
         fprintf(stderr, "An error has occured starting ulogd_DRL.  Refer to your logfile (%s) for additional information.\n", drl_logfile.u.string);
         exit(EXIT_FAILURE);
     }
         fprintf(stderr, "An error has occured starting ulogd_DRL.  Refer to your logfile (%s) for additional information.\n", drl_logfile.u.string);
         exit(EXIT_FAILURE);
     }

+
+    if (enforce_on.u.value) {
+        pthread_rwlock_wrlock(&limiter.limiter_lock);
+        do_enforcement = 1;
+        printlog(LOG_CRITICAL, "--Switching enforcement on.--\n");
+        pthread_rwlock_unlock(&limiter.limiter_lock);
+    }

 }
 
 void _init(void)
 }
 
 void _init(void)