Limit request rate
Name
Limit request rate
Summary
An upper limit is enforced on the amount requests from entities that are processed by the system, e.g. a maximum number of requests per timeframe or a limit on requests concerning specific (highly demanded) resources. On one hand, it can be prevented that a small group of entities hinders thenprocessing of requests sent by other entities, e.g. by overwhelming the system with a flood of (bogus) requests. On the other hand, the chance of successful brute force attacks relying on many attempts can be decreased. For example, if an attacker is sufficiently limited in the number of allowed authentication requests it decreases its chance of guessing a correct credential.
Parent
None
Problem
Solution
The goal of this solution is to limit the number of requests any entity can make to the system within a given timeframe. To this end for each incoming request the request history of the originating entity is verified against an appropriate request rate policy. Any new request that would violate the policy is blocked and the entity is informed of this.
Roles
- Entity (: Entity) sends multiple requests to the system
- Enforcer (: Enforcement point) ensures that each incoming request is verified against the appropriate request rate policy
- Limiter (: Decision point) decides whether a request should be forwarded to the system or not
- History provider (: Information point) keeps track of the previous, relevant requests from entities
- Policy provider (: Information point) manages the rate limiting policies
- System (: Entity) responsible for processing allowed requests
Actions
- action (: Action) the action requested by the
Entity(Optional) - verify (: Action) whether the request from the identified entity is allowed or not
- get_history (: Action) retrieve the relevant previous requests for a given entity
- update (: Action) update the request history for the
Entitywith the most recent request and decision - get_policy (: Action) retrieve the relevant rate limit policy
Data elements
- data (: Data) submitted to the system by an entity, possibly as part of an action request
- id (: Data) identifies the requesting
Entity. TheEnforcercan derive this identifier from the provided data, e.g. username of session identifier, or retrieve the necessary information elsewhere, e.g. obtain the originating IP address from the underlying network protocol.- Tags: Identifier, Untampered
- history (: Data) an overview of an entity's previous requests. For example, a list containing the
nmost recent requests, the decision theLimitermade for them, and a timestamp of when they where received, or this may simply be a counter indicating the total number of requests made by theEntityover a certain period of time.- Tags: Secret, Untampered
- policy (: Data) a set of rules governing how many requests an entity is allowed to make, e.g. a certain maximum within a given timeframe
- Tags: Untampered
Behaviour
- The
Entitysends its request to the system. - The
Enforcerintercepts the request, determines an identifier of the entity where it originated and sends a verification request with the necessary information to theLimiter. - The
Limiterrequests the applicable policy from thePolicy provider. - The
Policy providerdetermines the appropriate policy (part) for the given id and sends it as a reply to theLimiter. - The
Limiterrequests an overview of the relevant request history for the requestingEntity. - The
Request historyreplies to theLimiterwith the requested history. - The
Limiterverifies whether the identifiedEntityis allowed to perform the request based on the received policy and history. It sends the decision to theRequest providerso it can be taken into consideration for future requests from theEntity. - The
Limiteralso sends its decision back to theEnforcer. - If the request of the
Entityis allowed by theLimiter, theEnforcerforwards it the to theSystemfor further processing. Otherwise, theEnforcerdiscards the request and may send an error message to theEntityinforming it of this. - The
Systemmay reply to theSubject, for example, to provide the result of the action or to confirm the request was successfully received. This reply does not need to pass through theEnforcer. (Optional)
Role considerations
Entity identification (Enforcer)
In order to determine whether an Entity is allowed to make the request at hand it is necessary to identify that Entity such that the appropriate history, if any, can be taken into account. The manner in which an entity is identified depends on several factors. Unauthenticated entities can only be identified using information available publicly or via the underlying communication protocol, e.g. its IP address or MAC address. In such case care should be taken that such information may be shared by multiple entities, e.g. if several entities are connected to the same network with a limited number of outward IP addresses. Furthermore, some public information, such as IP and MAC addresses, can be (easily) spoofed by an attacker, making such identifiers less reliable.
If an Entity is authenticated its confirmed principal should be used to determine whether the Entity is allowed to request further actions. The principal can be combined with the above information to provide additional security. For example, a single Entity interacting from multiple devices simultaneously may be suspicious depending on the system context, similarly when multiple entities interact from a single device.
History information (History provider)
The exact history kept by the History provider depends on what is needed to enforce the appropriate policies. This may range from a simple counter that is increased with each new requests and periodically decreased, to more complex bookkeeping that considers, for instance, also the type of action requested and properties of the provided arguments.
Care should be taken not to keep more history than needed to verify requests against a policy. This includes removing outdated history entries that are no longer needed to verify new requests, e.g. because the entries are older than the timeframe considered in the policy. Furthermore, it should be avoided to keep data in the history that is not used for verifying requests. For instance, if the policy only concerns the amount of requests over a certain time period it is not necessary to keep track of the type of requests. For example, if an attacker gains access to the history it could use information such as which users where allowed to perform privileged actions to conduct more targeted attacks against these users.
Resource usage (Enforcer, Limiter, History provider, Policy provider)
Since all incoming requests are to be verified the involved entities should be able to handle this. While designing the software, one should consider that the entities involved in verifying incoming requests likely have to process more requests than the System that actually processes allowed requests.
Log verification failures
In order to be able to assess possible attacks one can log denied requests beyond the time relevant for the history. This can aid system administrators to determine whether or not an attack was attempted and, if necessary, inform them on what additional measures can be taken.
Data considerations
id
The Limiter uses the received identifier to retrieve the correct history and policy. Consequently, any attacker that can tamper with this identifier might succeed in getting requests allowed that should be blocked or vice versa. If the identifier is transmitted over an uncontrolled channel additional measures to protect it from tampering are warranted.
History
The Entity's request history is crucial information used by the Limiter to reach a decision on whether a new request is allowed. Consequently, it should be ensured that no unauthorised entity can modify this information while in transit or kept by the History provider. Any kept request history should only be modified by the History provider itself based on updates from the Limiter, the appropriate policy rules (e.g. periodically remove old entries that are no longer relevant), and in response to policy changes. Furthermore, the request history may contain sensitive information, e.g. usernames or IP addresses could be considered personally identifiable information (PII). In such a case appropriate measures should be taken to ensure the security of this information when transmitted and stored.
Policy definition
The rate at which entities should be allowed to submit requests to the system depends heavily on the system and context at hand, e.g. system's functionality, available (hardware) resources, expected system load, etc. At least a policy should be defined that states the maximum number of requests an Entity is allowed to make within an appropriate timeframe.1
Furthermore, different policies may apply to different types of requests. For example, one might have a stricter policy for authentication requests compared to other requests.
Implementations
None
Xet Storage Details
- Size:
- 11.1 kB
- Xet hash:
- 907915e43c27181d3adb3f6a634abf5d5bb457b61bae3fbf5f9e36345194c441
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.