ofproto-dpif-governor: Improve performance when most flows get set up.

ofproto-dpif-governor: Improve performance when most flows get set up.

The "flow setup governor" was introduced to avoid the cost of setting up
short flows when there are many of them.  It works very well for short
flows, in fact.  However, when the bulk of flows are short, but still long
enough to be set up by the governor, we end up with the worst of both
worlds: OVS processes the first 5 packets of every flow "by hand" and then
it still has to set up a flow.

This commit refines the flow setup governor so that, when most of the flows
that go through it actually get set up, it in turn starts setting up most
flows at the first packet.  When it does this, it continues to sample a
small fraction of the flows in the governor's usual manner, so that if the
behavior changes it can react to it.

This increases netperf TCP_CRR transactions per second by about 25% in my
test setup, without affecting "ovs-benchmark rate" performance.

(I found that to get relatively stable performance for TCP_CRR, regardless
of whether Open vSwitch or any kind of bridging was involved, I had to pin
the netperf processes on each side of the link to a single core.  I found
that my NIC's interrupts were already pinned.  Thanks to Luca Giraudo
<lgiraudo@nicira.com> for these hints.)

Bug #12080.
Reported-by: Gurucharan Shetty <gshetty@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>