GraphQL - Rate Limiting
Limits: the Total Nodes Limit and Rate Limits
If a web service would allow anyone to use the API as much as they want at any time, it would shoot itself in the foot. Web services need to protect themselves from excessive usage, whether intended or unintended, to maintain service availability. That’s why limits and Service Quotas are applied.
In the previous section you already discovered a first limit: when you query a Connection you need to supply a first or last argument and the values of first and last must be within 1-100.
But due to the architecture of GraphQL you can query through the graph of nodes and get more nodes than the limitation of 100 nodes at the first level.
Think about this example: there is a many to many relationship between requests and configuration items. With GraphQL you could ask for the first 100 requests and next for the 100 first configuration items linked to each of these requests , and for each of these configuration items you could ask for the first 100 contracts that are linked to the CI. Let’s give it a try!
Exercise:
Create a GraphQL query to get the first 100 requests. For each of these requests, get the first 100 configuration items linked to it. And for each of the configuration items get the first 100 contracts that are linked to these configuration items. How many nodes could be (theoretically) included in the data?
You could theoretically retrieve 100 * 100 * 100 = 1,000,000 nodes with this query. However, you will receive the following error message: message": "Individual calls cannot request more than 500,000 total nodes."
The 4me GraphQL API does not allow individual calls to request more than 500,000 total nodes. This limit is checked before the service executes the query (to check the limit after executing the query would not be the right strategy, at that point the damage is already done ….). It’s up to you to write GraphQL queries within the Total Nodes Limit. When you reduce one of the first arguments in the previous query to 10, you will be able to execute (don’t forget to add read access to the record types Configuration Item and Contract in the personal access token).
Limits: the Rate Limits
Preventing an integration from requesting too many nodes at a time is not enough to maintain service availability and performance: by posting many requests in a short period of time an integration could go around the Total Nodes Limitation and become a threath for the excellent service availability and performance the 4me service normally provides.
That’s why there exist 2 Rate Limits you need to consider.
The first rate limit is called the Request Rate Limit. It sets the maximum number of GraphQL requests you can perform in a certain period of time. Whenever you exceed this limit, you will receive a 429 Too Many Requests HTTP status code. In the response body’s JSON object you get the retryAfter value which represents the number of seconds you need to wait before you retry your operation.
Example of a request rate limit response:
Status: 429 Too Many Requests
Content-Type: application/json; charset=utf-8
Retry-After: 300
{
"documentationUrl":"https://developer.4me.com/graphql/#service-quotas",
"message": "Too Many Requests",
"retryAfter": 300
}
Because one GraphQL query can lead to a significant load on the web service, there is also a Query Cost Limit. Every integration user (identified by his personal access token) gets a number of available credit points per hour. Each time the integration user performs a query some credit points are consumed.
4me GraphQL provides the RateLimit Object that gives all the information an integration needs to stay in control of the rate limits:
limit: The maximum number of points the client is permitted to consume in a 60-minutes window.cost: The cost for the current call.remaining: The number of points remaining in the current rate limit window.resetAt: The time at which the current rate limit window resets (format: UNIX time or Epoch time).
Exercise:
Redo the GraphQL query of the previous exercise but make sure to stay under the Total Nodes Limit (for example get the first 100 requests. For each of these requests, get the first 100 configuration items linked to it. And for each of the configuration items get the first 10 (!) contracts that are linked to these configuration items). Add the rateLimit field to the query and check the cost of this request
This is the GraphQL query with a Selection Set for the rateLimit field.
In the response you should find now some info on your rate limit credits:
You can find all the details on how the rate limits are calculated here.
The rate limits may change at any time. What's important is that your integration checks if any rate limit has been hit and takes an appropriate action (retry after ...).