Enterprise AI Analysis

This section presents RL-MOTS, a scheduling system designed for heterogeneous cloud-edge situations. Figure 1 illustrates the comprehensive architecture. The growing complexity and dynamic characteristics of these environments require a decision-making system capable of optimizing numerous conflicting objectives concurrently. Consequently, RL-MOTS is designed to simultaneously improve energy efficiency, cost-effectiveness, and job priority, which are essential and frequently conflicting elements in extensive distributed systems. To attain these objectives, the system employs a priority-sensitive queue management strategy that utilizes adaptive queue structures alongside virtual waiting time matrices. These matrices change dynamically according to system status and task characteristics, allowing the scheduling agent to make context-sensitive judgments on task execution.

This mechanism guarantees that tasks are executed according to their computational needs, urgency levels, and resource availability, thereby improving responsiveness and equity. Task scheduling in cloud-edge infrastructures involves the immediate assignment of incoming tasks to a variety of computing units, comprising resource-limited edge nodes and high-capacity cloud virtual machines. This process must account for various dynamic factors, including fluctuating workloads, diverse resource capabilities, and changing network conditions. A significant challenge arises from the decentralized architecture of cloud-edge ecosystems, complicating global coordination and the consistent delivery of performance. Furthermore, the scheduling method must guarantee compliance with rigorous QoS requirements by minimizing execution latency and preventing deadline breaches. These requirements underscore the need for a learning-oriented adaptive policy that evolves over time, responds effectively to environmental changes, and facilitates scalable decision-making. Figure 2 illustrates the scheduling architecture based on Deep Q-Network (DQN). The state extractor initially processes the task queue and monitoring data, producing a proposed state-reward tensor that includes layers for waiting time, energy, and cost. This tensor is input into a deep Q-network (DQN), where the reward feedback from the performance evaluator is also aligned with the tensored objectives. The DQN generates Q values that inform the action selector and facilitate task allocation to edge or cloud resources.

The scheduling process is regulated by a DQN algorithm intended to acquire an optimal policy via ongoing interaction with the cloud-edge environment. The methodology utilizes a value iteration approach, wherein the Q-function Q (St, At) is successively updated to estimate the predicted discounted reward. The approach commences with arbitrary network weights and a replay memory to retain experience tuples (St, At, Rt, St+1). At each iteration, the agent perceives the state St, chooses an action At employing a € -greedy strategy, and performs the action, calculating the reward Rt according to the multi-objective function41. The Q-value is revised utilizing the Bellman equation: Q (St, At) ← Q (St, At) + a [Rt + γ maxQ (St+1, At+1) - Q(St, At) 4)] (9) where a denotes the learning rate and γ represents the discount factor. The procedure integrates preemption management through the virtual FIFO queue and converges on a policy that equilibrates waiting time, energy, and cost targets while adhering to QoS restrictions. This methodology offers a resilient structure for scalable and adaptable job scheduling in dynamic cloud-edge environments. Algorithm 1 introduces RL-MOTS to address the intricate issue of task scheduling in hybrid cloud edge computing systems, necessitating the simultaneous optimization of many objectives: performance, energy efficiency, and cost-effectiveness. In contrast to conventional heuristic-based scheduling methods, which frequently struggle to adjust to fluctuating workloads and resource limitations, RL-MOTS use reinforcement learning using a DQN to dynamically assign jobs to edge nodes or cloud virtual machines (VMs). This novel use of reinforcement learning facilitates real-time adaptive decision-making and addresses the constraints of static optimization techniques.

The selection of DQN as the primary learning algorithm was predicated on the discrete characteristics of the scheduling activities, in which value-based approaches excel. To enhance the proposed methodology, PPO and A3C are examined as supplementary baselines in the findings section, with RL-MOTS consistently demonstrating superior performance. Furthermore, the out-of-policy nature of DQN with replay memory improved data efficiency under non-stationary and bursty inputs, and the objective network mechanism stabilized training—features that have been found essential for online scheduling. Thus, PPO and A3C are reported as actor-critic baselines, and it is discussed where MARL is preferred when decentralization is needed.

To assess the efficacy of the proposed RL-MOTS system, simulations were performed utilizing CloudSim 3.0.3, emulating a hybrid cloud-edge scenario. The simulation comprises three categories of virtual machines with varying configurations Table 2, edge nodes with minimal computing resources, provisions for virtual machine resilience, and the capability to dynamically allocate and deallocate virtual machines according to workload requirements. The values enumerated under cost/second in Table 2 denote Cr (R) and are utilized in calculating the overall cost of job execution during our tests. The hyperparameters of the DQN scheduler were meticulously optimized to guarantee a fair and reproducible assessment. Subsequent to the initial grid search, the ensuing setup was implemented: Learning rate a = 1 × 10-3, discount factor γ = 0.95, and replay buffer size of 10,000 transitions. The mini-batch size was established at 64, and updates to the target network occurred every 100 steps. The ε-greedy exploration approach commenced at ε = 1.0 and underwent linear decay to ε = 0.1 over 20,000 steps, subsequently maintaining a constant value. Tasks are generated following a Poisson distribution with a rate of λ = 3. Every task is defined by its computational duration, input/output size, deadline, cost, and preemptive status. Tasks are categorized into three kinds Table 3. Deadlines are determined by normal distributions centered on the average execution time for each class, hence assuring realistic SLA limits.

The suggested methodology is evaluated against FCFS (First-in-First-served), min-min scheduling, priority-based scheduling, multi-objective function (MOF), and non-dominated sorting with thresholding algorithms. Each baseline signifies a unique scheduling philosophy, from static heuristics to cost-performance trade-offs. We evaluated the performance using the metrics of total time from the arrival of the first task to the completion of the last task, average waiting time, total energy consumption, operational cost, deadline violation rate, number of active virtual machines over time, and convergence of Q value during training to evaluate the dynamics of reinforcement learning. The makespan, defined as the total duration from the arrival of the initial task to the completion of the final work, serves as a fundamental metric of scheduling efficacy. Figure 5 illustrates that RL-MOTS consistently attains reduced makespan values as job volumes increase, in comparison to conventional baseline approaches. This illustrates the framework's capacity to dynamically assign jobs to suitable resources, thereby reducing idle periods and enhancing throughput. Energy usage and expenditure are crucial in extensive dispersed systems. Figure 6 demonstrates that RL-MOTS markedly decreases energy use, achieving up to 28% lower usage relative to heuristic-based approaches. This is ascribed to the DQN agent's capacity to prioritize energy-efficient nodes in real time. Also, Figure 7 illustrates the whole operational expenditure, whereby RL-MOTS exhibits enhanced cost optimization proficiency, realizing roughly 20% savings. This cost-effectiveness arises from dynamic virtual machine selection and diminished deadline infringements that frequently result in penalties. Adhering to SLAs is essential. As illustrated in Figure 8, RL-MOTS demonstrates a significantly reduced deadline violation rate across all task volumes. This signifies improved responsiveness and reliability, especially under high-load scenarios. Furthermore, Figure 9 illustrates the model's elasticity by monitoring the quantity of active VMs over time. RL-MOTS dynamically modifies VM utilization in reaction to variable workloads, demonstrating effective resource allocation and deallocation. Figure 10 depicts the convergence of average Q-values throughout numerous training episodes to evaluate the learning efficiency of the DQN agent. The consistent convergence trend affirms the stability and efficacy of the learning process, substantiating the agent's capacity to develop an ideal scheduling policy. The experimental results collectively demonstrate that RL-MOTS significantly enhances performance, energy efficiency, cost reduction, SLA compliance, and learning stability relative to traditional scheduling methods. These benefits render it a formidable contender for practical implementation in dynamic cloud-edge settings. These improvements can be directly ascribed to the architecture of RL-MOTS. The dynamic weight update technique allows the scheduler to prioritize energy efficiency during high use, resulting in a reduction of total energy consumption by up to 28% compared to baseline measurements. The anticipatory queue management technique prioritizes urgent or cost-critical tasks, reducing waiting times and thereby decreasing overall operational expenses by around 20%. Furthermore, the application of the state-reward tensor representation equips the DQN with a systematic perspective on performance, energy, and cost objectives concurrently, enabling the policy to leverage trade-offs more efficiently.