1. Project Introduction
Very dry and practical! Distributed task scheduling and distributed computing framework
Second, the implementation of functions
Trigger type: In the task orchestration system, the trigger type refers to the way in which the task is triggered. Common types of triggers include:
Manual triggering: The user manually clicks the execute button or enters a command from the command line to trigger the execution of the task.
Scheduled trigger: Automatically triggers the execution of a task at a specified time point or interval based on a preset time rule. For example, it is performed once a day in the early hours of the morning, every hour, and so on.
Conditional triggering: When certain conditions are met, the task is automatically executed. Conditions can be changes in the state of the system, triggering of external events, updates to data, etc.
Task orchestration: Task orchestration refers to the organization of multiple tasks in a specific logic and order to form a complete workflow. In task orchestration, you can define dependencies, execution order, and concurrency between tasks.
Task orchestration systems typically provide a graphical interface or programming interface for creating and managing task orchestrations. Through task orchestration, complex business logic and process control can be realized, and the efficiency and reliability of task execution can be improved.
Task sharding: Task sharding is to split a large task into multiple small tasks for parallel execution. Task sharding can improve the execution speed and resource utilization of tasks, while reducing the risk of individual task execution.
Task sharding usually needs to consider the dependencies between tasks and the order of execution to ensure that the sharded tasks can be executed correctly in parallel, and that they can be merged or processed in the next step after all shards are executed.
Stop & Resume:
Stopping and resuming a task means that during the execution of a task, the execution of the task is suspended, the current execution state is saved, and then the execution of the task is resumed from the saved state when it is started again.
Stopping a task can be a manual action or triggered by a condition. When a task is stopped, the system saves information such as the execution environment and running logs of the task for subsequent task resumption.
Save execution snapshot:
Saving an execution snapshot allows you to save the state and data of a task during execution so that you can restore it to the state and continue the execution of the task when needed.
Execution snapshots usually include information such as the input and output data of the task, running logs, and execution progress. Saving execution snapshots improves the reliability and flexibility of task execution while reducing the cost of re-executing tasks.
Failure retry: Failure retry is an automatic retry operation after a task fails to be executed to try to resolve the problem of task execution failure and continue the task execution.
Failure retries are usually performed automatically based on the preset retry policy and number of retries. The retry policy can be configured based on the nature of the task and the execution environment to achieve the best retry effect.
Background management: Background management refers to the creation, configuration, monitoring, and management of tasks through the management interface or interface of the task orchestration system.
Background management can include creating, editing, and deleting task orchestrations, configuring trigger types and trigger conditions, monitoring and scheduling task execution status, and viewing execution logs and reports.
Monitoring and alarming: Monitoring and alarming refers to real-time monitoring of key indicators and anomalies during task execution, and sending alarm notifications when problems occur.
Monitoring metrics can include task execution progress, running time, and resource usage. Anomalies can be task execution failures, execution timeouts, insufficient resources, and so on.
View execution logs:
Viewing execution logs allows you to view the log information generated during task execution through the management interface or interface of the task orchestration system.
The execution log records the execution details, input and output data, running status, and error messages of the task. You can view the execution logs to understand the execution status of the task, troubleshoot problems, and diagnose faults.
3. Technology selection
html+css+js
mysqlredis
nacoszookeeper
5. Source code address
Private message reply: 89