A Survey of Multipath TCP Scheduling Schemes: Open Challenges and Potential Enablers

Today, mobile devices like smartphones are supported with various wireless radio interfaces including cellular (3G/4G/LTE) and Wi-Fi (IEEE 802.11) [42]. The legacy devices can only communicate with only one interface. The Transmission Control Protocol, or TCP, has a limitation inability to change connection settings without breaking the connection.In this paper, we explain how multi-path TCP (MPTCP) protocol has been proposed to solve TCP single-interface limitation and provides a huge improvement on application performance by using multiple paths transparently (auto path changing). We discuss the last mile, which is the final networking segment which carried all network traffic. Indeed, the available bandwidth in last-mile link can be effectively harms the network throughput as it limits the amount of transmitted data. We found that the quality of the last mile networks significantly determines the reliability and quality of the carrying network. We believe MPTCP can provide a convenient solution for the last mile problem. We provide a holistic view of the challenges and potential enablers in details.


Introduction
Today, mobile devices smartphones and tablets are supported with multi-wireless communication interfaces such as cellular (3G/4G/LTE) and Wi-Fi (IEEE 802.11). The transmission control protocol (TCP) is the standard communication schema in current computer network systems. TCP protocol was originally designed for a single nodeto-node and once a connection is established, it's elements cannot be changed without breaking the connection. Such elements are: sender IP address and port, and receiver IP address and port. However, TCP does not benefit from the multi-interface capability and mobility of mobile devices and thus limits the performance multiple routes usage in case of non-stable network conditions. A potential solution is the adoption of Multi-path TCP (MPTCP) to improve application performance by using multiple paths transparently (auto path changing). Pokhrel et al. [39] found that wireless channel errors and buffer overflows have negative impact on achieving fair throughput over TCP links. MPTCP is an extension of the TCP which allows mobile devices to to use various network interfaces simultaneously. Figure 1 shows atypical MPTCP architecture. The core idea idea of MPTCP is to split a single byte stream to multiple byte streams and transfer them over multiple disjoint network paths, so called subflows. The MPTCP is prominent to enhance communication network throughput and achieve robustness [13]. The quality of a subflow is determined by its connection path including signal converge, loss rate, queue delay, performance of involving links and connectivity of access points [23,4]. To manage the distribution of data packet over heterogeneous subflows, a packet scheduler is employed to select the optimized scheduling plan based on various objectives such as reducing transmission delay, reducing communication cost and increasing network throughput, as shown in Figure 2 . The scheduling process implies determining the amount of data packets to be distributed onto the subflows, which has a significant impact on the performance of MPTCP. The core idea of MPTCP scheduling optimization is to shift all data traffic onto the least-congested path. This called multipath congestion control procedure citewischik2011design. For a flow connection, MPTCP should provide throughput as minimum of SPTCP (single-path TCP) on the high-quality paths and no more capacity on any path or collection of paths in SPTCP. In context of IoT applications, MPTCP scheduling have challenges to improve performance such as scalability, reliability and latency [34].

Transport Protocol Limitations under Heterogeneity
Heterogeneous networks, such as in IoV systems, bring challenges for achieving high performance MPTCP scheduling. In 4G/LTE networks, the connection path uses large buffers with long fluctuating in transmission, with high delays and packet loss rates. On the other hand, in case of WiFi path, its short delays and higher packet loss rates. The network performance is affected by various parameters such as buffer sizes at data receivers, queueing, number of flows sharing the connection and others. In IoV networks, which rely on WiFi networks, the wireless medium is shared by connections causing issues like repeated handoffs and network collisions. In general, IoV applications share the requirements of throughput-intensive (low end-to-end delay) and high demanded for reliable data delivery. Thus, wide range of MPTCP protocols and techniques focus on designing TCP congestion control schemes to maximize network throughput. Figure 1: Multi-path TCP architecture [2] MPTCP has certain challenges to achieve high throughput packet delivery. Firstly, the Head-of-line blocking, or shortly HOL-blocking, which is a networking state performancelimiting phenomenon that takes place when a sequence of packets is blocked by the first packet [45]. In HOL-blocking state, packets scheduled on the faster subflows arrive at destination buffer and then wait fo the arrival of the packets scheduled on the slower path. This cause two main issues, high-congestion at the receiver buffer and out-of-order(OFO) packet delivery. The out-of-order(OFO) issue also can be occurred with high packet loss and drop rate subflows. One solution for OFO is to maximize the waiting queue at the receive side [28]. However, employing large buffers (queues) can cause bufferbloat issue where packets remain enqueued for a long time, particularly with under congestion paths. These issues are drastically decrease the network performance (in term of throughput), expressly for delay-sensitive applications in IoT and IoV application contexts [1]. To conclude, to achieve a satisfied level of MPTCP performance, both congestion control policy and packet scheduler should be convenient to align the requirements of the networking system environment. According to Raiciu et al. [44], the core concept of MPTCP is to schedule packets to reduce the traffic on high congested paths through controlling the congestion window. This can be obtained by tuning the congestion window size to achieve high throughput packet distribution over multiple paths (packet scheduling). The last mile is the final networking segment which carried all network traffic. The available bandwidth in last-mile link can be effectively harms the network throughput as it limits the amount of transmitted data. Another issue is the single point of failure which might cause whole network failure. Thus, MPTCP can be convenient to resolve last-mile link issues and achieve the desirable network performance QoS. Moreover, Pokhrel et al. [38] observed throughput unfairness of MPTCP with regular in last-mile WiFi networks.
Congestion control is an algorithm to adapt the congestion window (CWND) within Figure 2: Multipath scenario should consider the path heterogeneity [49] subflows to control the data transfer rate at each subflow [27]. A congestion control should satisfy three rules. The first rule is" Improving Throughput", which implies that each multi-path flow performance should be at least as a single path flow performance. The second rule "Do not Harm", in which, a multi-path flow uses an equal capacity of the shared resources. The last rule is "Balance Congestion" which implies that a multipath flow should transmit traffic like the most congested paths. There are two types of MPTCP congestion control algorithms, uncoupled and coupled. In the uncoupled congestion control, the simplest form, such as in EWTCP [17], each subflow is treated like an independent TCP connection. This congestion control algorithm does not satisfy fairness rule of MPTCP (rules 1 and 2) [51]. MPTCP cannot apply the standard TCP control scheme without unfair to normal TCP flows [6]. To resolve the unfairness issue, all congestion windows of all subflows are coupled. This allows an adaptive process to maintain the congestion window (CWND) of each subflow. Fully Coupled [16] and LIA [20] are examples of MPTCP coupled congestion controls. Barré et al. [6] provide a structured design framework on how to implement the MPTCP in the Linux kernel in order provide performance analysis to demonstrate that coupled congestion control is fairer than the standard one in TCP. Pokhrel et la. [37] shows that the coupled MPTCP LIA greatly suffers from the competition with TCP because of the reordering delay at the receiver. The next section provide a review of MPTCP scheduling techniques.

MPTCP: Packet scheduling
MultiPath TCP (MPTCP) is a modified version of TCP protocol which provides guidelines to distribute data packets over several subflows to maximize resource utilization and throughput. Generally, a successful MPTCP scheduler should be able to: provides reliable scheduling while meeting network heterogeneity (such as delay, capacity and loss rate), 2) balance a variety of QoS constraints (such as maximizing throughput, reducing RTT and minimizing latency) and 3) adaptive to network dynamicity (real time network conditions). For example, the high fluctuation of cellular and Wi-Fi networks performance [47]. Currently, the MPTCP Linux kernel [44] (LIA) is the best-known open source implementation. The coupled congestion control adapts the congestion window of each subflow by considering an end-to-end connection and different properties (e.g., fairness, friendliness, responsiveness, and congestion balance). In comparison to standard TCP (using only Wi-Fi), results show a throughput increases by 50% to 100% under various Wi-Fi coverage. Khalili et al. [21] reported that (LIA) is not Pareto-optimal and violates the fairness goal of MPTCP (achieving both fairness and responsiveness). They developed an improved version of LIA, called opportunistic linked-increases algorithm (OLIA), a window-based congestion-control mechanism which couples the additive increases and uses unmodified TCP behavior in the case of a loss. They concluded that it is impossible to increase the throughput of one subflow without harming other subflows throughput or even increasing the congestion cost. Peng et al. [29] proposed a fluid model called, Balanced Adaptive LIA (BALIA) presented comprehensive analysis for designing MPTCP and proposed balanced linked adaptation (Balia) as a generalization of MPTCP congestion control algorithms. Result show a good balance in terms of TCP-friendliness and responsiveness. Pokhrel et al. [37] proposed an analytical algorithm to improve the coupled congestion-based approach, in term of goodput, which is a network throughput measurement at the application-level (only concern about useful data transmission). The proposed approach works by by controlling the reordering delay at the receiver side, with consideration to packet loss rate and data transmission delay. Results show an improved goodput under constraint of low variation data rate.
The literature shows alternative strategies for MPTCP scheduling, apart from congestion window control based. MinRTT [22] is based on Round-Trip Time (RTT), which is the elapsed time measured by the sender between sending a data packet with a particular sequence number and receiving an acknowledgment packet that covers that sequence number [43]. It starts by filling the congestion window (CWND) of the subflow with the lowest RTT before advancing to other subflows with higher RTTs. MinRTT suffers from HoL-blocking in heterogeneous networks as it mainly concerns about utilizing Congestion Window (CWND) and does not provide a pre-estimation of number of packets over available paths. Ferlin et al. proposed BLEST (BLocking ESTimation) [12] which distributes packets based on the minimum risk of HoL-blocking. It leaves some subflows under-used, which can inflates the download time for larger-size files. The resulting scheduler allows an increase by 12% in application goodput with bulk traffic while reducing unnecessary retransmissions by 80% as compared to default MPTCP and other schedulers.
Lim et al. [24] proposed ECF (Earliest Completion First) examine whether the default MPTCP path scheduler can provide applications the ideal aggregate bandwidth, i.e., the sum of available bandwidths of every paths. Experimental results show that heterogeneous paths cause underutilization of the fast path, resulting in undesirable behaviors for a real-time streaming application quality. The ECF consistently utilizes all available paths more efficiently than other approaches under path heterogeneity, particularly for streaming applications. Guo et al. [14] reported that high buffer size causes HoL-blocking with large data files. They proposed DEMS (DEcoupled Multipath Scheduler) which aims to reduce the data transfer time through an adaptive data redundant transmission (for small data chunks) over multi-paths. Adarsh et al. [2] argue that the main limitation of the default MPTCP scheduler is only the consideration of the RTT. However, this is not the prefect scheduler in case of heterogeneous subflows; matrices like performance and loss rate should be also considered. Pokhrel et al. [31] proposed a novel throughput-based MPTCP algorithm to schedule data packets in timevarying heterogeneous wireless paths. The algorithm targets Internet of Vehicle (IoV) systems, delay-sensitive networks, and applied a jointly technique of load balancing and forward error correction (FEC) for performing coupled congestion control MPTCP. The proposed technique outperforms traditional congestion control model through intelligent congestion control coupling with load balancing IoV networks.
Pokhrel and Garg [35] observed that optimization based MPTCP techniques will not be sufficient for high dynamic networks in terms of model complexity and realtime scheduling accuracy. The proposed learning-based scheduling can overcome these challenges through enabling the source to learn the best way to control itself from its own experience.

Learning-based schedulers
Heterogeneous paths, such as in cellular networks, are highly variable and unpredictable channels in terms of self-inflicted queuing delays and loss rate. This motivates the research towards adaptive scheduling techniques which consider the variability in network conditions while optimizing objectives like reducing delay and loss rate and increasing throughput. Learning-based schedulers are proposed to achieve these objectives through applying machine learning and deep learning techniques to understand network behaviour and generate efficient scheduling policies accordingly. Pokhrel and Singh [41] designed a novel analytical model to evaluate the performance of the coordinated TCP (C-TCP) flows for IoT applications over WiFi networks using a federated learning (FL) technique.
Chung et al. [10] proposed a novel path management scheme called MPTCP-ML which controls path usage based on machine learning mechanism. The objective is to periodically evaluate the quality of active paths at real-time based on patterns collected from heuristics. various quality metrics included such as throughput, RTT, signal quality and data rate. The random forests model was used to extract these patterns. Results show high prediction accuracy in mobile environment. Beig et al. [7] throughput-based learning algorithm to find the best signal quality rate in respect to interface type and transmitted file size. Three techniques examined: LTE, WiFi and MPTCP fro mobile devices. The performance evaluation shows MPTCP throughput improvement by 10% compared SPTCP over LTE and WiFi. However, the model does not consider other parameters like loss rate and buffer size. Pokhrel and Mandjes [37] developed a comprehensive technique to evaluate the performance of durable (mostly permanent) MPTCP flows over heterogeneous network (WiFi and Cellular network). They considered various network features like retransmission limit and the buffer sizes. They proposed a learning algorithm to exploit the route heterogeneity as well as study the influence of heterogeneity variation on over all MPTCP performance. The coexistence of heterogeneous networks can be achieved by MPTCP through dynamic subflows management (to add and drop) according to according to certain QoS requirements in a best-effort manner. Such applications are in-city railway [25] and coordinating the convoy of drones [36]. Chiariotti et al. [9] worked on delay-sensitive MPTCP systems under user-defined constraint. They proposed a QoSbased MPTCP protocol, latency-controlled end-to-end aggregation protocol, LEAP for short. The LEAP schedules/distributes packets over multiple parallel links based on user-defined QoS constraints.
Zhang et al. [52] proposed ReLeS,a Reinforcement Learning based Scheduler for MPTCP, which applies deep reinforcement learning (DRL) techniques to learn a neural network (NN) to find the best MPTCP distribution policy. The employed reward function is complex as it determines multi QoS features. For performance evaluation, they consider the following metrics: application goodput, application delay, number out-of-order bytes at receiver (indication of HOL-blocking, and download time. ReLeS significantly outperforms the state-of-the-art schedulers in terms of adaptive learning and real-time scheduling. The main concern is the complexity of the optimization function which may increase the optimization time in complex scheduling scenarios. Wu et al. [49] proposed Peekaboo, a novel learning-based multipath QUIC (MPQUIC) scheduler in that keeps monitoring the impact caused by the current dynamicity level of each path and selects the most suitable scheduling strategy accordingly. According to the reward function, paths with highest throughput are selected. The proposed model is a mixed strategy of deterministic and online adaptive models. To reduce learning complexity, Peekaboo only confiders available paths, i.e., have enough CWND. Peekaboo consistently offers superior or similar performance to the best state of the art schedulers, with the performance improvements of Peekaboo reaching by up to 31.2% in emulated networks and up to 36.3% in real network scenarios.
Pokhrel and Williamson [42] applied game theory to define a utility function for cooperative MPTCP. The idea is to model MPTCP subflows contests as a game with common coupled constraints, such that, if a subflow fails to achieve the constraints common to other subflows (such as throughout), all other subflows will be affected. Corresponding issues like out-of-order packet delivery of packet lost are common for cooperative MPTCP subflows. Network emulation experiments show higher throughput and responsiveness in compare to baseline techniques for handling heterogeneous network paths. Huang et al. [18] proposed a distributed Deep Reinforcement Learning (DRL)based congestion control algorithm in MPTCP. The technique can effectively avoid the HoL-blocking with BLEST scheduler under complex network condition and improve the throughput by 11% compared to the previous MPTCP congestion control algorithms. The C-TCP applies a mixed congestion control strategy to maintain loss and delay in congestion windows [48]. Pokhrel and Garg [35] designed a deep Q-learning based framework for scheduling traffic in IoT applications using deep Qnetwork [26] which is a common benchmark to build an evaluate applications in deep reinforcement domain. A deep Q-network is designed to achieve human-like intelligence through learning policies directly from high-dimensional data using end-to-end reinforcement learning. Abbasloo et al. [1] proposed DeepCC which leverages advanced deep reinforcement learning (DRL) techniques to let machines automatically learn how to steer throughput-oriented TCP algorithms toward achieving applications' desired delays in a highly dynamic network such as the cellular network. DeepCC fundamentally differs from current learningbased schemes as it attempts to use learning-based techniques to help and boost the performance of the existing TCP schemes instead of replacing them. DeepCC's aim to keep the average delay of packets below applications' desired Targets while keeping the throughput high. Even though MPTCP offers reliable and high-performance connection, but, privacy still a major concern when sending data over multi network interfaces for continuous connectivity. Pokhrel and Choi [30] proposed a blockchain-based federated learning (BFL) design to achieve high communication/data privacy and reduce delay for distributed on-vehicle machine learning processing and data exchange. Analytical results show model efficiency to reduce system delay by utilizing communication an adaptive arrival rate blocking based on current network conditions. The work extended to decentralized fashion to reduce the implications of centralized control on the blockchain mechanism [32]. Later, the authors proposed a federated learning framework to preserve IoV privacy and improve the IoV communication performance [33]. To meet the challenges of IoT Big Data, in terms of variety and volume, the learning process of TCP protocols can be accelerated [15].

Concluding remarks and future directions
IoT embraces the emergence of new forms of network applications and technologies, which moves towards more complex structures and high resource demanded in terms of computing and networking. The enormous increase of IoT devices leads to new generation workload model, called IoT Big Data [8,3]. IoT Big Data commonly refers to the "3Vs" model, that is, the increase of volume, velocity, and variety of data [11,5]. However, the current MPTCP techniques provide inflexible transmission strategies to meet the performance requirements of IoT Big Data of high responsiveness and high reliability as they do not consider the huge traffic of data in mostly high speed [50]. For example, Pokhrel et al. [40] studied the application of MPTCP on Industrial IoT, so called IIoT, domain. IIoT is an extension of IoT employed in industry for high operational efficiency and intelligent monitoring and tracking applications, and predictive and preventive maintenance [46]. Even though the literature presents promising MPTCP techniques to handle IoT networks, such as IoV. But features like data size and speed at large scale are not well-studied. Moreover, challenges of heterogeneous networks, like mobile networks, including network reliability and timeliness need to be further investigated to consider in context of IoT devices mobility and low-energy contexts [19].
In this work, we will investigate the MPTCP scheduling in IoT Big data and how jointly optimization of maximizing network goodput and reducing delay. Moreover, we will study the development of DRL technique with federated learning to benefit from sharing experience from different network conditions. Finally, the proposed model will be evaluated in comparison with state of the art MPTCP schedulers.