Priority
As of Spider version 1.6.11, a new functionality has been added that workload items can be processed based on priority.
WorkloadIds are created for requests created by the Spider. Previously, each WorkloadId had the same priority. In this new release, priorities are now set to the following items:
- ProcessSteps Runtime
- Scheduled requests
- Import tasks
- Normal requests
Processing sequence
The Getwork functionality will process workloads based on this sequence:
- Step 1: First search for high priority workloads with a value between 99 - 21
- Step 2: If step 1 is not the case, it verifies whether there are workloads with the planner's priority, value 20.
- Step 3: If step 2 is not the case, it searches for low priority workloads with a value between 19 - 1.
ProcessSteps Priority
For a processStep, the runtime can now be set (can even vary per day). All new requests added to the Spider will get the new Priority.
Tip
The Priority setting in the ProcessStep will override all other priority settings Only IF the Priority value exceeds 10 (import task) or 20 (scheduled request).
Standaard prioriteiten
This table shows the default priorities that have been set. Note the Priority for a normal request. We will show some examples.
| Prio | Description |
|---|---|
| 20 | Scheduled Request |
| 10 | Import Task |
| 1 | Normal request, where the ProcessStep priority is set to 1 |
Tip
SLA: If a normal request (Priority 1) has exceeded the SLA processing time, it will be processed earlier only if there are no more requests with a higher prio than 1.
More information about the Priority settings in ProcessSteps can be found at: Runtime ProcessSteps
Er worden enkele voorbeelden gegeven om de prioriteitsinstellingen uit te leggen.
Example 1
Some examples what the prio of the workloadId's will be. The Priority of the processStep can determine the final Priority. If the Priority of the ProcessStep is lower than the default value of a Scheduled or Import task then the value of the latter applies. Otherwise the Priority of the ProcessStep overwrites the default value.
| ProcessStep | Priority | Normal request | Scheduled task | Import task |
|---|---|---|---|---|
| Step_A | 1 | >1? = 1 | > 20? = 20 | >10? = 10 |
| Step_B | 1 | >1? = 1 | > 20? = 20 | >10? = 10 |
| Step_C | 1 | >1? = 1 | > 20? = 20 | >10? = 10 |
Example 2
| ProcessStep | Priority | Normal request | Scheduled task | Import task |
|---|---|---|---|---|
| Step_A | 18 | >1? = 18 | > 20? = 20 | >10? = 18 |
| Step_B | 35 | >1? = 35 | > 20? = 35 | >10? = 35 |
| Step_C | 58 | >1? = 58 | > 20? = 58 | >10? = 58 |
As soon as new requests come in with a higher priority, the Workload is automatically updated and so the robot automatically picks up work with the highest priority.
Example 3
In the example below, we see that Step_A has a low priority and Step_B and Step_C have a higher one. However, in order to process Step_B and Step_C, Step_A is required. In this case, Step_A will then be processed first. So a higher prio does not always mean earlier processing
| ProcessStep | Priority | Normal request | Scheduled task | Import task |
|---|---|---|---|---|
| Step_A | 1 | 1 | 20 | 10 |
| Step_B | 30 | 30 | 30 | 30 |
| Step_C | 58 | 58 | 58 | 58 |
Example 4
Now there are a number of requests in the Spider ready to be picked up. If there are multiple requests with the same Priority, the one entered first in the Spider will be picked up.