Use memcached for rate-limiting the API calls.
authorS.Çağlar Onur <caglar@cs.princeton.edu>
Tue, 9 Nov 2010 20:40:55 +0000 (15:40 -0500)
committerS.Çağlar Onur <caglar@cs.princeton.edu>
Tue, 9 Nov 2010 20:40:55 +0000 (15:40 -0500)
commitcdcd6351525acffb778d17658838e05aa4577534
tree63c766ce48eca9cb47bfbf932e8ecfdd21f22dd2
parentb2577abd4dc87458a35788708d4a69341398ee09
Use memcached for rate-limiting the API calls.

   On every request we work out the keys for the past five minutes and use get_multi to retrieve them.  If the sum of those counters exceeds the maximum allowed for that time period, we block the request. Ex;

        ratelimit_128.112.139.115_201011091532 = 1
        ratelimit_128.112.139.115_201011091533 = 14
        ratelimit_128.112.139.115_201011091534 = 11

[...]
[{'hostname': 'aspect.cs.princeton.edu'}, {'hostname': 'func.cs.princeton.edu'}]
8
[{'hostname': 'aspect.cs.princeton.edu'}, {'hostname': 'func.cs.princeton.edu'}]
9
Traceback (most recent call last):
  File "test.py", line 16, in <module>
    local_nodes = plc.GetNodes(auth, {}, ["hostname"])
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/xmlrpclib.py", line 1253, in request
    return self._parse_response(h.getfile(), sock)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/xmlrpclib.py", line 1392, in _parse_response
    return u.close()
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/xmlrpclib.py", line 838, in close
    raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 108: 'Permission denied: Maximum allowed number of API calls exceeded'>
aspects/__init__.py
aspects/ratelimitaspects.py [new file with mode: 0644]