So I have this proxy for the Project Gutenberg. It routs most of the pages through it from the original P. G. site, but to the pages that allow downloading the texts it adds the SONY Reader option. When user clicks on it, it fetches the text file from P. G. site, transcodes it, and returns it to the user.
One important thing that I had to solve was throttling the request stream to the original site so that I don't trigger their abuse logic.
In a more general terms, it can be formulated thus: A service can process N requests per second. Design a system that ensures that there's never more than N requests sent in any given one-second interval.
Note that the imprecise solution is beyond straightforward: have a counter and clear it every second. However, if N-1 requests are sent at time 0.75 and another N-1 requests at time 1.25, this simplified system would violate the requirement, because in one-second interval starting [0.5, 1.5) there will be 2N-2 requests.
Have a solution? Post it here. Interested in a solution? Check the comments; I will post mine in a couple of days here.