IpRateLimitMiddleware(Github: AspNetCoreRateLimit ) is a stream-limiting middleware for ASPNETCore that controls how often clients invoke APIs. If clients frequently access the server, it can limit its frequency and reduce the pressure of accessing the server.Or if a crawler is crawling critical data, you can limit the number of calls per day for an API or some IP to limit the speed at which it crawls.
Of course, I'm actually trying to solve another problem.The WebApi we write sometimes has some APIs, and we only want other internal applications to call it, such as HelthCheck for WebApi. We want only our Middleground to call regularly to get information, but the front end is not.This allows us to place the internal IP address in the IpWhitelist configuration item and limit the number of specific API calls to zero so that only the addresses in the whitelist can access the corresponding endpoints, as shown below.
"GeneralRules": [ { "Endpoint": "get:/api/healthstatus", "Period": "1s", "Limit": 0 }]
IPRateLimitMiddleWare detailed usage instructions, translated from Github(AspNetCoreRateLimit)
establish
NuGet installation:
Install-Package AspNetCoreRateLimit
Startup.cs code:
public void ConfigureServices(IServiceCollection services) { // needed to load configuration from appsettings.json services.AddOptions(); // needed to store rate limit counters and ip rules services.AddMemoryCache(); //load general configuration from appsettings.json services.Configure<IpRateLimitOptions>(Configuration.GetSection("IpRateLimiting")); //load ip rules from appsettings.json services.Configure<IpRateLimitPolicies>(Configuration.GetSection("IpRateLimitPolicies")); // inject counter and rules stores services.AddSingleton<IIpPolicyStore, MemoryCacheIpPolicyStore>(); services.AddSingleton<IRateLimitCounterStore, MemoryCacheRateLimitCounterStore>(); // Add framework services. services.AddMvc(); // https://github.com/aspnet/Hosting/issues/793 // the IHttpContextAccessor service is not registered by default. // the clientId/clientIp resolvers use it. services.AddSingleton<IHttpContextAccessor, HttpContextAccessor>(); // configuration (resolvers, counter key builders) services.AddSingleton<IRateLimitConfiguration, RateLimitConfiguration>(); } public void Configure(IApplicationBuilder app, IHostingEnvironment env) { app.UseIpRateLimiting(); app.UseMvc(); }
IPRateLimiting should be registered before other middleware, otherwise API request calculation may be incorrect.
If you load-balance your application, you need to use IDistributedCacheRedis or SQLServer so that all kestrel instances have the same rate-limiting storage.Instead of using MemoryCache, you should inject distributed storage like this:
// inject counter and rules distributed cache stores services.AddSingleton<IIpPolicyStore, DistributedCacheIpPolicyStore>(); services.AddSingleton<IRateLimitCounterStore,DistributedCacheRateLimitCounterStore>();
Configuration and general rules appsettings.json:
"IpRateLimiting": { "EnableEndpointRateLimiting": false, "StackBlockedRequests": false, "RealIpHeader": "X-Real-IP", "ClientIdHeader": "X-ClientId", "HttpStatusCode": 429, "IpWhitelist": [ "127.0.0.1", "::1/10", "192.168.0.0/24" ], "EndpointWhitelist": [ "get:/api/license", "*:/api/status" ], "ClientWhitelist": [ "dev-id-1", "dev-id-2" ], "GeneralRules": [ { "Endpoint": "*", "Period": "1s", "Limit": 2 }, { "Endpoint": "*", "Period": "15m", "Limit": 100 }, { "Endpoint": "*", "Period": "12h", "Limit": 1000 }, { "Endpoint": "*", "Period": "7d", "Limit": 10000 } ] }
If EnableEndpointRateLimiting is set to false, restrictions will be applied globally and only rules with endpoints will be applied*.For example, if you set a limit of five calls per second, any HTTP calls to any endpoint will count towards that limit.
If EnableEndpointRateLimiting is set to true, the restriction applies to each endpoint, such as {HTTP_Verb}{PATH}.For example, if you set a limit of five calls per second for the *:/api/values client, you can call GET/api/values five times per second, but you can also call PUT/api/values five times.
If StackBlockedRequests is set to false, the rejected API call is not added to the call count.For example, if the client makes three requests per second and you set a limit of one call per second, other limits, such as the counters per minute or per day, will only record the first call, which is a successful API call.If you want rejected API calls to be counted against other time displays (minutes, hours, etc.), you must set StackBlockedRequests to true.
When using RealIpHeader, your Kestrel server is backed by a reverse proxy, if your proxy server uses a different header and then extracts the client IP X-Real-IP to use this option to set it.
The ClientIdHeader is used to extract the client ID from the whitelist.If a client ID exists in this header and matches the value specified in ClientWhitelist, no rate limit is applied.
Override the general rules of specific IP appsettings.json:
"IpRateLimitPolicies": { "IpRules": [ { "Ip": "84.247.85.224", "Rules": [ { "Endpoint": "*", "Period": "1s", "Limit": 10 }, { "Endpoint": "*", "Period": "15m", "Limit": 200 } ] }, { "Ip": "192.168.3.22/25", "Rules": [ { "Endpoint": "*", "Period": "1s", "Limit": 5 }, { "Endpoint": "*", "Period": "15m", "Limit": 150 }, { "Endpoint": "*", "Period": "12h", "Limit": 500 } ] } ] }
IP fields support IP v4 and v6 values and ranges such as "192.168.0.0/24", "fe80::/10" or "192.168.0.0-192.168.0.255".
If you define static rate policies in the appsettings.json configuration file, you need to seed them at application startup:
public static async Task Main(string[] args) { IWebHost webHost = CreateWebHostBuilder(args).Build(); using (var scope = webHost.Services.CreateScope()) { // get the IpPolicyStore instance var ipPolicyStore = scope.ServiceProvider.GetRequiredService<IIpPolicyStore>(); // seed IP data from appsettings await ipPolicyStore.SeedAsync(); } await webHost.RunAsync(); }
Define rate limit rules
Rules consist of endpoints, periods, and constraints.
The endpoint format is {HTTP_Verb}:{PATH}, and you can use the asterix symbol to locate any HTTP predicate.
The period format is {INT}{PERIOD_TYPE}. You can use one of the following period types: s, m, h, d.
The restriction format is {LONG}.
Example:
Limit the rate of all endpoints to two calls per second:
{ "Endpoint": "*", "Period": "1s", "Limit": 2 }
If, in the same IP, you make three GET calls to the api/value in the same second, the last call will be blocked.But if you also call the PUT api / value within the same second, the request will pass because it is a different endpoint.When endpoint rate limits are enabled, each call is {HTTP_Verb}{PATH} based on rate limits.
Use any HTTP verb to restrict calls, /api/values make five calls every 15 minutes:
{ "Endpoint": "*:/api/values", "Period": "15m", "Limit": 5 }
Rate Limit GET Calls/api/values 5 calls per hour:
{ "Endpoint": "get:/api/values", "Period": "1h", "Limit": 5 }
If, within the same IP, you make six GET calls to the api/value within an hour, the last call will be blocked.But if you also call GET api / values / 1 within the same hour, the request will pass because it is a different endpoint.
behavior
When a client makes an HTTP call, IpRateLimitMiddleware: RateLimitMiddleware <IpRateLimitProcessor>performs the following actions:
- Extract IP, Client ID, HTTP predicate and URL from the request object. If you want to implement your own extraction logic, you can override the IpRateLimitMiddleware.ResolveIdentity method or implement a custom IP/Client Parser:
public class CustomRateLimitConfiguration : RateLimitConfiguration { protected override void RegisterResolvers() { base.RegisterResolvers(); ClientResolvers.Add(new ClientQueryStringResolveContributor(HttpContextAccessor, ClientRateLimitOptions.ClientIdHeader)); } }
Search the whitelist for IP, client ID, and URL, and do nothing if anything matches.
Search for matches in IP rules, all applicable rules are grouped by period, and the most restrictive rule used for each period.
Search in a matching general rule, which is also used if the matching general rule has a defined period that does not exist in an IP rule.
For each match rule, the rate limit counter increments, and if the counter value is greater than the rule limit, the request is blocked.
If the request is blocked, the client receives the following text response:
Status Code: 429 Retry-After: 58 Content: API calls quota exceeded! maximum admitted 2 per 1m.
You can customize the response to HttpStatusCode and QuotaExceededMessage by changing these options, and if you want to achieve your own response, you can rewrite IpRateLimitMiddleware.ReturnQuotaExceededResponse.The etry-After header value is expressed in seconds.
If the request is not rate constrained, the maximum time defined in the matching rule is used to compose X-Rate-Limit headers, which are injected into the response:
X-Rate-Limit-Limit: the rate limit period (eg. 1m, 12h, 1d) X-Rate-Limit-Remaining: number of request remaining X-Rate-Limit-Reset: UTC date time (ISO 8601) when the limits resets
Clients can parse things like X-Rate-Limit-Reset:
DateTime resetDate = DateTime.ParseExact(resetHeader, "o", DateTimeFormatInfo.InvariantInfo);
You can disable the X-Rate-Limit and Retry-After headers true by setting the DisableRateLimitHeaders option in appsettings.json.
By default, Microsoft.Extensions.Logging.ILogger is logged using Block Requests, and IpRateLimitMiddleware.LogBlockedRequest can be overridden if you want to implement your own logging.When a request is made for a rate limit, the default logger sends the following information:
info: AspNetCoreRateLimit.IpRateLimitMiddleware[0] Request get:/api/values from IP 84.247.85.224 has been blocked, quota 2/1m exceeded by 3. Blocked by rule *:/api/value, TraceIdentifier 0HKTLISQQVV9D.
Update rate limit at run time
At application startup, the defined IP rate limit rules load appsettings.json into the cache by any MemoryCacheClientPolicyStore or DistributedCacheIpPolicyStore based on the type of cache provider you use.You can access the Ip policy store within the controller and modify the IP rules as follows:
public class IpRateLimitController : Controller { private readonly IpRateLimitOptions _options; private readonly IIpPolicyStore _ipPolicyStore; public IpRateLimitController(IOptions<IpRateLimitOptions> optionsAccessor, IIpPolicyStore ipPolicyStore) { _options = optionsAccessor.Value; _ipPolicyStore = ipPolicyStore; } [HttpGet] public IpRateLimitPolicies Get() { return _ipPolicyStore.Get(_options.IpPolicyPrefix); } [HttpPost] public void Post() { var pol = _ipPolicyStore.Get(_options.IpPolicyPrefix); pol.IpRules.Add(new IpRateLimitPolicy { Ip = "8.8.4.4", Rules = new List<RateLimitRule>(new RateLimitRule[] { new RateLimitRule { Endpoint = "*:/api/testupdate", Limit = 100, Period = "1d" } }) }); _ipPolicyStore.Set(_options.IpPolicyPrefix, pol); } }
This allows you to store IP rate limits in a database and push each application to the cache after it starts.