Skip to main content

Overload Protection

Storage Capacity Protection​

Rill Flow checks the storage capacity when submitting the execution process.When submitting execution processes, Rill Flow checks for storage capacity. If the storage exceeds the set limit, Rill Flow will reject the submission and return error code 100 with the message:

dag runtime storage usage limit

The maximum threshold for storage capacity can be configured in the application.properties file with the following property:

weibo.flow.runtime.redis.default.storage.max.usage=90

The range for this setting is between 0 to 100, with the default being 90, indicating the maximum permitted percentage of used Redis capacity. Setting it to 0 or 100 means no storage capacity check will be performed.Set to 0 or 100 to exclude storage capacity checks.

Circuit Breaker​

When Rill Flow calls an executor to perform tasks, the executor can return circuit breaking information to prevent repeated retries from overloading resources.

If the executor needs to circuit break a resource after dispatching a task, it can return the following error message:

{
  "error_detail": {
    "retry_interval_seconds": 10
  }
}

Here, retry_interval_seconds is the duration (in seconds) of the circuit break. If set to 0, no circuit break will occur.

Upon receiving this circuit break information from the executor, Rill Flow will temporarily stop calling the same resourceName for the specified duration.

Rate Limiting​

Rill Flow supports the refusal of execution process submissions under resource circuit breaking conditions, thus enabling traffic control and protection of overloaded resources.

Four Modes of Rate Limiting Strategies​

Rill Flow supports four modes of rate limiting strategies:

  1. Short Board Mode (short_board): Triggers rate limiting for the execution process if any of the dependent resources are circuit-broken. This is the default strategy in Rill Flow.Circuit Breaking
  2. Long Board Mode (long_board): Rate limiting occurs only if all the resources depended on by the execution process are circuit-broken.
  3. Key Resource Mode (key_resource): Rate limiting is triggered only when all specified key resources are circuit-broken. Additional configuration of key_resources is required to specify these key resources.Additional configuration of key_resources is required to specify key resources.
  4. Skip Mode (skip): No rate limiting is performed.

Rate Limiting Configuration Structure​

Rill Flow supports custom rate limiting strategies through configuration or parameter passing. The rate limiting configuration uses a json structure, as shown in the example below:Limit flow configuration is in json format, below：

{
    "check_type": "key_resource",
    "key_resources": [
        "http://127.0.0.1:8080/sample/start.json",
        "http://127.0.0.1:8080/sample/end.json"
    ]
}

In this example, the check_type specifies the rate limiting strategy as Key Resource Mode (key_resource), and key_resources defines what the key resources are.

Configuration Methods​

Rill Flow supports the following three methods for configuring rate limiting strategies, listed in order of decreasing priority:

  1. Parameters passed during the creation of the execution process;
  2. Configuration through the properties file;
  3. Rill Flow's default configuration.

1) Via Parameter Passing​

Passing through parameters allows flexibility to set limits for each task execution.Rate limiting strategies for each execution task can be flexibly set by passing the rate limiting configuration as a json string in the resource_check parameter (which needs to be UrlEncoded). For example:e.g.：

curl -XPOST 'http://127.0.0.1:8080/flow/submit.json?descriptor_id=demoFlowTest:demoTest&resource_check=%7B%22check_type%22%3A%22long_board%22%7D'
    -H'Content-Type:application/json'
    -d '{"left":5,"right":5}'

In this example, the value of the resource_check parameter %7B%22check_type%22%3A%22long_board%22%7D is the UrlEncoded result of {"check_type":"long_board"}, indicating the use of the long board mode for rate limiting.

2. Through Properties Configuration​

Rate limiting strategies can be set via the java properties file (in the application.properties file of the project), allowing different strategies for different workflow business IDs (businessId). For example:e.g.：

weibo.flow.runtime.resource.check.id.to.config={'demoFlowTest':'{"check_type":"long_board"}'}

This configuration is a map structure, where the key is the workflow business ID and the value is a String type json configuration of the rate limiting structure.

3. Rill Flow Default Configuration​

If neither the resource_check parameter is passed nor a properties configuration for the specific workflow business ID is set, Rill Flow will use the default configuration, which is the short board mode (short_board).