Privacy Cost Optimization of Smart Meters Using URLLC and Demand Side Energy Trading

In this article, we consider ultra-reliable low-latency communication (URLLC) for efficient energy trading over a smart grid (SG) network using home-based smart meters (SM). We develop a cost-friendly privacy preservation framework based on existing demand-side energy management by employing random bidirectional energy trading among customers. Customers in our design can be either producers or consumers and mostly both (‘prosumers’). Our aim is to develop a decentralized optimization framework that not only reduces energy costs, but also improves privacy preservation and energy trading ability directly from the customer’s end. One of the vital costs for energy consumers is the supply charge. Our method can minimize it by orchestrating energy trading among customers in a decentralized adaptive fashion. To predict the energy demand by optimizing between privacy and cost, we employ an extension of the follow the regularized leader (FTRL) algorithm. We perform a theoretical analysis to demonstrate the convergence of the FTRL, the benefits of URLLC for the SG network, and the cost-effective privacy preservation ability of the proposed model. In addition to enabling energy trading efficiently, our extensive simulation results demonstrate that our proposed framework outperforms the state-of-the-art methods in terms of the cost-friendly privacy of SMs.


I. INTRODUCTION
T HE energy demand is highly fluctuating and ensuing surge due to modernization.We are aware of the prediction that energy demand will increase 47% globally by 2030. 1 However, energy generation and scheduling are highly challenging in meeting such ensuing and dynamic demands.Therefore, to reduce energy generation loss and make power system scheduling more efficient, the traditional grid is rapidly replaced by the smart grid (SG).Engineers and architects have now developed energy-positive houses that produce energy while consuming little energy themselves.
There are advantages of SG for energy consumers, producers, and energy companies (ECs).The producers and consumers The authors are with the IoT Research Lab, School of Information Technology, Deakin University, Geelong, VIC 3220, Australia (e-mail: m.hossain@deakin.edu.au;shiva.pokhrel@deakin.edu.au;jinho.choi@deakin.edu.au).
Digital Object Identifier 10.1109/TSC.2023.3310939 1 https://tinyurl.com/mr6343af(refer to as prosumers) can reduce energy costs by using SG, whereas the EC can reduce the loss in energy scheduling and increase the power system's efficiency.This involves communication, the delay and reliability of which are the bottleneck challenge in existing grid networks [1], [2].SGs improve the reliance on resilience and sustainability of the grid.It also increases efficiency compared to its earlier centralized nature by integrating distributed and renewable energy sources.There are estimated 528.4 GWs of distributed energy generation and integration with SG by 2026. 2 With these rapid increases in the addition of distributed and renewable energy sources, the intermittency of these resources, and the different timing constraints introduced by the EC [3], [4] (such as the power-response scheme, line differential protection, etc.), we need ultra-reliable low latency communication (URLLC) for real-time operation [5].In addition, using renewable energy sources in large amounts will cause more voltage and frequency fluctuation.Thus, URLLC is essential to make the system dynamics stable.
In Table I, we summarize the communication latency and reliability requirements for different SG operations.Most of the operations listed in Table I require advanced communication technology such as URLLC.In addition, other operations listed in Table I can also improve its reliability and reduce latency by using URLLC.Due to the advantages and requirements of some grid operations, we consider URLLC technology to communicate between the edge server (ES) and the energy control unit (ECU) to perform energy trading between neighborhood houses.

A. Data Networking in Smart Grid (SG)
One of the core elements of SG data networks is a smart meter (SM) at the customer end.By using SMs over Internet-of-Things (IoT), we are transforming the existing unidirectional power system into a composite of power and information network.In general, the SG network consists of a Home Area Network (HAN), Neighborhood Area Network (NAN), and Wide Area Network (WAN).HANs are the end-user-level SG network where SMs are installed.Demand-side energy mechanisms, including demand response programs and integration of small-scale distributed energy resources, are to be performed in HAN.NANs are responsible for different critical SG operations such as dynamic price control (by grid distribution automation) [6], supervisory control and data acquisition (SCADA), millisecond-level load control, detection of a power failure (by intelligent distribution), charging piles of electric vehicles (EVs), video surveillance, and feeder automation.The majority of critical protection, control, and security services are to be performed by the NAN.As a result, URLLC should fully support it [7].Furthermore, energy trading over URLLC reduces the trading loss considerably.
WAN needs to handle communication among consumers to generate, transmit and distribute power for energy trading (e.g., using Multipath TCP [8]).It enables phasor measurement units (PMUs), protection systems, monitoring, and control of the vast area.Some SG WAN applications have stringent requirements, such as comprehensive area monitoring, control, and protection.For this, wireless cellular technologies cannot support the required latency and reliability without URLLC.

B. Privacy and Communication Challenges
As noted, URLLC requires connectivity guarantees and > 99.999% of the times URLLC data networking meets the quality of wired communication systems.Furthermore, the latency of URLLC should be ≤ 10 ms [9]; 10 ms is the maximum latency value, where in most cases it is expected to be below 1 ms.Therefore, URLLC has the potential to overcome the challenge of optimal allocation, prediction and economic dispatch of available energy among customers [10], [11].Using the compute power deployed of the controller, embedded system, IoT, and intelligence at the edge (SMs) of the SG network, we can considerably reduce the amount of data that needs to be processed centrally.In addition, it can curtail unnecessary network data traffic, such as from the central server to the end node.Furthermore, enabling the URLLC network can ameliorate communication bottlenecks for more sophisticated schemes, such as trading consensus and negotiation.URLLC can be implemented without change in network fabric, and such technology can manage resources in a reconfigurable and heavily interconnected SG network.Researchers have started to implement URLLC technology in the existing SG network [12], [13].They have shown the importance of URLLC between edge devices, such as SMs, in time-critical operations and studied the potential consequences when communication reliability and availability are not met.
The use of different types of edge devices for the advanced interconnection of SGs and the conversion of the traditional power grid to the information network have risked data privacy [14].SGs generate confidential and private data through thousands of interconnected sensors, distributed energy sources, and intelligent electronic devices.The data are used to detect sudden power fluctuations, disturbance or fault analysis, state estimation, forecasting, load modeling, and cyber/physical abnormalities [15], [16].Nevertheless, by analyzing the high-resolution SM data, it is possible to identify the usage of the appliance in the house, the number of people living in the house, their daily routine, sleeping habits, whether a house is empty or not [17].This sensitive personal information can be used by different companies, such as advertising and insurance, to gain benefits.To overcome the problem, it is essential to preserve the privacy of SMs.Data manipulation methods [18] can be used to preserve the privacy of SMs.However, there are issues with data tampering methods such as inaccurate state estimation, misleading control signal, and complex billing [19].Furthermore, due to the small length of the URLLC code, it is not feasible to include data manipulation methods (such as encryption) for privacy and security concerns.To overcome these issues, we use methods based on demand energy management [20], [21] to preserve SM privacy.The energy storage device (ESD), such as a rechargeable battery (RB), has constraints such as limited capacity and maximum charge-discharge rate.Due to the constraints of an ESD, existing demand-side energy management-based privacy preservation methods cannot perform flexibly.To this end, we extend the Follow the Regularized Leader (FTRL) method [22] to provide a new framework to design and analyze online algorithms in a versatile fashion.The structural properties of FTRL are suitable for addressing the nonstationary dynamics of SG and can be successfully implemented in highly dynamic trading domains with inherent adaptivity.

C. Contributions and Novelties
We introduce a novel energy trading mechanism using URLLC between groups of neighboring prosumers.With the proposed trading mechanism, the ECU at the prosumer end can buy/sell energy at an optimal cost, preserve better privacy, and reduce energy transmission loss.
The main contributions of this paper are r We enhance the cost-friendly privacy preservation ability of SMs using demand-side energy management by predicting the energy demand in a time span (TS).
r The proposed framework has been extensively evaluated with data sets and simulations to ensure i) accurate billing, ii) correct state estimation, and iii) reliable low delay control signals over the SG network.In particular, we employ a random access scheme for energy trading, which dynamically assigns a shared medium to a set of consumers participating in the trading, each with relatively high data traffic.SMs sense the medium to reduce collisions, and no Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
new trading communication is initiated when the shared medium is busy.This is motivated by the celebrated "listen before talk approach" in the field of wireless communication.
Two key novelties of this paper are r We develop a novel mechanism to predict the energy de- mand in a time span (TS) for a house by enhancing FTRL that flexibly handles the SG dynamics under the energy trading setup with guaranteed convergence conditions.
r A novel energy trading mechanism is developed between neighborhood houses using URLLC to minimize power transmission loss and improve the cost-friendly privacy of SMs.

D. Roadmap
The rest of the sections of this paper are organized as follows.Section II, contains the literature review on cost-friendly privacy preservation methods of SMs.The methodologies used in the proposed approach are included in Section III.Section IV includes the system model of the proposed method.Section V details the mechanism of our proposed approach.Theoretical analysis of achieving better privacy and cost savings as well as the benefits of using URLLC, has been included in Section VI.Section VII discusses the numerical results obtained and analyzes the better performance of the proposed approach compared to other existing approaches.Finally, Section VIII concludes the article.

II. GAP IN LITERATURE
Of particular importance to this work are the demand-side energy management-based methods for the privacy of SMs.They have been well studied [21], [23], [24], [25] in the literature.For example, [23] applied a backward water filling algorithm (WF) to preserve the cost-effective privacy of SMs.The mechanism considered a target output load (which is the average energy demand of the house) to optimize the privacy and cost of energy consumers.However, the method cannot preserve privacy when the variation in the energy demand is very high.To preserve the privacy of data networking, [24] considered a dynamic programming framework to develop a control algorithm using Lyapunov optimization (Lyap), which improved performance in privacy, but cost savings is relatively poor.
The work in [25] modeled the cost-friendly differential (CDP) privacy of SMs.However, their probability distribution suffers, as the charge-discharge of an RB depends only on the energy demand without considering the state of charge (SoC) of an RB.In addition, no noise is generated in the extreme SoC of an RB as it captures the approximate differential privacy of SMs.The work in [21] used a heuristic algorithm empowered by artificial fish swarm optimization (AFSO).However, AFSO is not adaptive to the change in energy demand in a house, and energy trading is beyond their scope.Another advanced averaging algorithm designed by [26] is not adaptive and suffers from similar problems for trading.Another recent work, [27], considered optimizing the cost-effective privacy of electric vehicles (EVs), applying reinforcement learning to optimize between privacy and cost of SMs.They adopted federated learning for privacy (FLP).
Although [27] improved adaptivity, it did not consider the aspect of energy trading to improve cost-friendly privacy.Considering their cost-privacy aspects, the study on energy trading among neighboring houses is a nontrivial challenge in the field and has been poorly understood in the literature.

A. FTRL Algorithm
The formulation of FTRL [22] is suitable for addressing nonstationary applications and can be successfully implemented in highly dynamic domains with inherent adaptivity. 3To efficiently predict the energy demand and enable trading, we extend FTRL in a game setting of the prosumers as players, playing repeatedly for T iterations.The player j where j ∈ {1, 2} determines his mixed strategy profile π t j = ∇(A j ) based on previous observations for every iteration t ∈ T .After that, each player observes their new feedback.At the end of iteration t, according to π t j each player j chooses an action a t j .In the end, each player observes the utility u j (a t 1 , a t 2 ).FTRL [29] is a widely used algorithm in repeated game settings.For a player j, the FTRL method is defined with a regularization function ψ j : ∇(A j ) → R that is continuously differentiable in ∇(A j ) and strictly convex.For FTRL, every player j determines their strategy π t j at iteration t using the following equation where z t j (a j ) = t−1 s=1 q π s j (a j ) and learning rate η > 0. In addition, we apply the KL (Kullback-Leibler) divergence, also known as the Bregman divergence, for entropy regularization, ψ(s) = j s j ln s j , which is denoted as KL(s, s ) = j s j ln s j s j .

B. Two-Player Game
We consider the preservation of privacy and the cost-saving ability of energy consumers and producers.Both producers and consumers can prefer privacy preservation over cost-saving for trading, or vice versa.Thus, it can be considered a two-player zero sum game.A suitable solution concept for privacy and cost optimization of the two-player zero sum game is the Nash equilibrium [30].In particular, we consider the equilibrium where ∀π 1 ∈ ∇(A 1 ) and ∀π 2 ∈ ∇(A 2 ) ensure the following condition where π j is the player strategy profile j, The expected utility for strategy π of a player j is ūπ j and conditional expected utility for action a j ∈ A j is q π j (a j ).The notation −j is used to represent the opponent of the player.The approximation of the equilibrium can be represented by -Nash with the following inequality The exploitability of a given strategy profile π can be expressed as Observe that exploit(π) is a metric for a two-player zero-sum game which quantifies the closeness of π to the intended equilibrium π * (see [31] for details).Therefore, for the games considered with two players, the sum of the KL divergences on the strategies can be estimated as

IV. SYSTEM MODEL
The energy trading system model of the demand-side privacy preservation method is shown in Fig. 1.Through distribution lines, the SG power is directly connected to the SM.The power is supplied to the appliances in the house to meet the customer's demand.The energy control unit (ECU) controls the energy supplied to the appliances from the SG as well as from the energy storage device (ESD).The ECU can charge the ESD by supplying energy from the SG.It can also take energy from the ESD to supply the appliances in the homes.The ESD can be a rechargeable battery (RB) or a capacitor that can store energy.The ECU strategically charges and discharges the ESD to mask the real energy consumption of the house.The energy company can observe the SM data for billing purposes.Therefore, the generation of energy using renewable energy resources (RERs) and the sale to the neighborhood is only recorded in the ECU of the respective house.An SM only records the energy sold back to the main grid by a house.We consider RERs such as solar panels/windmills available in every house.The energy generated by the RER is either stored in ESD or sold directly back to the neighborhood houses/main grid.The ECU strategically controls the energy to sell back from the RER to the neighborhood houses/main grid to mask the real energy produced by the RER.Each ECU sends information about current energy demand, surplus energy to sell, and unit price of selling energy to the Edge server (ES).
For energy trade optimization, we consider an orthogonal frequency division multiple access based cellular network [32]  that supports multiple access to the ES.In Fig. 2 each station based on the Edge contains an Edge computing device or Edge Server (ES).The base station is responsible for transmission, and ES is responsible for computing.We assume that the transfer time between the ES and the base station is negligible, as they are in the same location.A single task can be performed either on the ES or locally without further segmentation.It is also assumed that within a certain time frame, each user randomly generates a task and the task generation follows the Poisson distribution [33].We assume that the ECU has sufficient computing power to perform the necessary task.Thus, most of the computation is performed locally.
The ES can only observe the requested energy demand by the ECU.It has no idea about the masking of real energy consumption by charging-discharging of ESD.The ECU sends the energy demand or sale information as well as the bid price.
In this way, the ECU can exchange energy between them at a reasonable price to preserve the better cost-friendly privacy of SMs.In addition, ECU can also allocate energy appliances to preserve better cost-friendly privacy of SMs.In what follows, we describe our privacy, cost, and channel model used for our proposed approach.

A. Privacy Model
Let's consider the demand of a house is To preserve the privacy of a house, the output of an SM should be masked in such a way that by observing the output no one can infer the real energy consumption of a house.By implementing demand-side energy management using an ESD, it is possible to mask the real energy consumption of an energy consumer.One way to do this is to make the output reach a certain value (such as an average value) for any energy demand in the home [23].For privacy, an ESD charges and discharges in such a way that the difference between output (y i ) and the average energy consumption (E) is reduced.However, due to the constraints of an ESD, the formula in [23] cannot perform well, and the value of E is determined by considering the energy demand both for the off-peak and peak period.To improve privacy preservation ability by using limited capacity ESD, we can consider average energy for off-peak and peak periods.In this way, the privacy of energy prosumers can be improved with limited capacity ESD.For the period off-peak i = 1, 2, . ..., N o and for the peak period The average demand for a house during the off-peak and peak periods is E o and E p respectively, where and The value of x i is collected from the historical data of a house.In this case, to preserve privacy, the EMU aims to minimize

B. Cost Model
The unit price of energy in the time interval (TS) i is p i where p i = {p 1 , p 2 , . ..., p N }.To reduce the cost, the ESD charges more during the off-period as the unit price of energy is low.On the other hand, the ESD discharges more during the peak period when the unit price is high.Also, due to energy trading between a group of energy consumers, they can buy energy at a lower price and sell energy at a higher price.The objective of this model is to preserve efficient privacy and reduce energy costs as much as possible.

C. URLLC Model
For energy trading, we need a real-time operation to reduce energy loss.Thus, we use URLLC to transmit data between ES and ECU.The ECU needs a URLLC terminal, as it requires high reliability and low latency services for the transmission of information from the ES to the ECU and the interaction between the ECUs of different houses.For practical implementation, we consider imperfect estimation of channel state information.For the URLLC terminal, the Doppler shift has a great effect on small-scale fading.We can formulate the power gain for the URLLC terminal as [34] g where L is the path loss constant, the distance between the ES and the ECU of a house is represented by D ut , a denotes the path loss exponent, h ut and λ satisfies the distribution of CN (0, 1).Using Jake's statistical model [35], the channel correlation is described by the coefficient θ, where 0 < θ < 1.Here, ut and U denote the URLLC terminal index and the set of URLLC terminals, respectively.It requires short data packets for URLLC.In addition, the transmission is not also free from error.As a result, to formulate the achievable rate for a given probability of error, we cannot apply the Shannon capacity formula.Thus, we consider the transmission error rate and the short packet length.With a finite block length b ut , the achievable rate r ut (bit/s) is [36] where ∀ut ∈ U, B ut is the allocated bandwidth, for URLLC terminal, the signal-to-noise ration (SNR) is S ut = [(P ut g p ut )/(B ut σ 2 )], P ut is the transmit power, the Gaussian noise power spectral density is denoted by σ 2 , the finite block error is e ut , the inverse of the Gaussian Q function is denoted by Q −1 (.).

D. Optimization Formulation
The aim of the proposed model is to preserve efficient costfriendly privacy so that min where Here r ut is the data rate for the wireless communication, β ranging from 0 to 1.For β = 0, prosumers prefer saving cost over privacy, and for β = 1, prosumers prefer privacy over cost-saving.E m is the capacity of an ESD and the maximum charge-discharge rate of the ESD is E c and E d , respectively.To preserve cost-friendly privacy, the ECU strategically generates noise n i at TS i by charging-discharging the ESD where E c ≥ n i ≥ E d .By using only ESD the SM output at TS i is V. PROPOSED FRAMEWORK Due to the constraints of ESD, existing methods based on energy management on the demand side cannot preserve better cost-friendly privacy.However, if it is possible to predict the energy demand of a house in each TS, the ECU would be able to manipulate the state-of-charge (SoC) of ESD for that TS to preserve better cost-friendly privacy of SMs.In addition, correct prediction of the amount of energy generated by renewable energy can also help improve the cost-friendly privacy preservation ability of a model.We consider using enhanced FTRL to improve the prediction ability of the ECU to predict the energy demand of a house.In addition, we also consider the energy trade with lower prices between a group of houses to further improve the cost-friendly privacy preservation ability of our proposed model.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

A. Privacy Preservation
To preserve privacy, the ECU of a house tries to make the SM output in the off-peak and peak period equal to E o and E p , respectively.As the prediction of the energy demand of a house is based on considering privacy preservation and cost savings, the ECU can control the SoC of the ESD to make the output energy demand equal to the target output load.Let the real energy demand of a house at TS i is x i .The ECU predicts a value of energy demands x p i .To predict the energy demand of a TS, the ECU considers (7), where y i = x p i .After predicting the energy demand of a TS i, the ECU changes the SoC of the ESD so that the output load for the off-peak and peak period is near or equal to E o and E p , respectively.For example, in the case of an off-peak period the predicted energy demand and the real energy demand of a house at TS i are x po i and x i , respectively.The target demand is E o .Thus, the ECU controls the energy supply to the house so that where n i is the charge/discharge rate of the ESD at TS i, n rer i is the energy supply capacity of the RER at TS i, and n sb i is the energy sell or buy from the neighborhood.The ECU first tries to use the energy from the RER, then the neighborhood, and lastly from the ESD.The energy from the RER and the neighborhood is of an intermittent nature and is low cost.As a result, the ECU first tries to utilize the energy from them.In addition, based on the prediction of the energy demand of TS i + 1 the ECU tries to modify the SoC of the ESD in TS i so that it becomes easier for the ECU to make the output near or equal to E o in TS i + 1.For TS i, if ESD, RER and the neighborhood have sufficient energy to supply (or ESD can store and the neighborhood can buy sufficient energy) to make it equal to E o then otherwise In the case of the peak period, the target output load is E p and the predicted energy demand is x pp i .To preserve the privacy of energy prosumers at TS i, the ECU controls the energy supply as well as the SoC of the ESD so that Finally, if ESD, RER, and neighborhood have sufficient energy to supply (or consume) to make the output load equal to E p then otherwise, the output y i is calculated using (11).

B. Cost-Reduction
To minimize the cost, the ECU always attempts to fully charge the ESD at the end of the off-peak period and fully discharge at the end of the peak period.For this, the ECU uses less energy from the ESD of TS i o where i o = N o − 2E m E c .The ECU performs this to fully charge the ESD at the end of the off-peak period to reduce the average energy cost.The ECU also uses the energy stored in the RER to reduce costs.Finally, if the ECU can buy the energy available in the neighborhood at a lower price than the energy from the primary grid, the ECU chooses the neighborhood.
In TS i o the SoC of ESD is E op i .Therefore, the energy required to fully charge the ESD at the end of the off-peak period is E m − E op i .There are two scenarios to consider to fully charge the ESD.Let us consider that the prediction of energy demand is not fully accurate.In this case, to fully charge the ESD by the end of the off-peak period, the ESD needs to charge every TS at least by for this, the ECU charges the ESD in every TS by E f where E f ≤ E c .Now, for E op i = 0, we get from ( 14) as As The ECU charges the ESD by E f or less to fully charge the ESD by the end of the off-peak period.On the other hand, when we train the model with enhanced FTRL, the prediction of the energy demand of the houses is more accurate (after several iterations).In this case, we calculate the predicted energy required to supply the demand and fully charge the ESD.The predicted energy demand of the house at TS i o is where The ECU supplies x o T to fully charge the ESD by the end of the off-peak period.
To reduce the cost of energy consumption, the ECU fully discharges the ESD at the end of the peak period.The ECU uses more energy from the ESD at TS i p to fully discharge the ESD by the end of the peak period, where i p = N − 2E m E d .In this case, in TS i p , the ECU prioritizes the sale of the energy generated by the RER.In the event of a less accurate prediction, the ECU calculates the energy required to empty the ESD.Let's consider that the SoC of the ESD at TS i p is E p i .To remove the ESD, it must be discharged by E p i − E o at the end of the peak period.In case of less accurate prediction, the ESD discharges in every TS after i p by where E e ≤ E d .In training with enhanced FTRL, after several iterations, the prediction of energy demand is more accurate.In this case, the predicted energy demand needs to be fully discharged and the ESD is where x pp t = N i=i p x p i .

C. Energy Trading in Neighborhood
Excessive energy storage is costly (battery capacities are finite), and latency is critical to minimize energy loss and keep the SG system stable.Therefore, we consider URLLC for energy trading.As discussed, the ECU calculates the surplus energy (E s i ) generated by the ESD as well as the RER (recall Fig. 1).The surplus energy for a TS can be estimated as Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
where q − equal to q if q ≤ 0 and 0 otherwise.A prosumer's energy demand could be much less at a certain period.During the same period, the energy demand of some commercial buildings could be higher.For example, during office hours, the energy demand of residential houses is deficient as the house residents are either in the office or some other places based on their daily activities.However, the demand for commercial buildings is very high during the same period.Thus, the surplus energy generated by residential houses can be traded with commercial buildings.
The reverse situation could also be considered in the case of after hours.
The buying price of energy is very high.On the other hand, the price to sell energy back to the grid is small compared to the price to buy [37].Thus, the ECU first tries to sell energy to the consumers nearby at a price that is higher than the selling price but lower than the current buying price of energy.If the selling and buying price of energy with the primary grid at TS i is p s and p i , respectively, then the trading price of energy between prosumers is where ∀i p c i > p s as p i > p s .The surplus energy needed to sell back to the main grid can be calculated on the basis of the privacy priority of the consumers.
1) High Priority for Privacy: In this case, the consumers always make the output equal to E o or E p .If a customer's demand is very low at TS i, still buys E o or E p in the off-peak or peak period, respectively.In this case, the surplus energy to sell back to the neighborhood or SG during the off-peak period is in case of peak period, the ECU uses E p instead of E o in (21).However, in this case, the extra energy bought from the primary grid at a price p i needs to sell back to the neighborhood at p c i or to the primary grid at p s .Thus, there is an extra cost involved to preserve privacy.
2) Low Priority for Privacy: In this case, the prosumers only sell energy back when the generated energy is higher than the energy demand in a TS.Thus, the surplus energy to sell back to the neighborhood or the primary grid at TS i is For trading energy, the ECU sends information about (p c i , E s i ) to the ES from all houses in a group, as shown in Fig. 2. Energy information of a house indicates that the house has surplus energy to sell to the neighborhood.On the other hand, E − i indicates that the house is interested in buying energy from the neighborhood and E i refers to not being interested in the trade of energy or the house having sufficient energy to support cost-friendly privacy.The ES collects all the information and sends it back to the houses of E + i to the houses of E − i or vice versa.Fig. 3 shows the energy trading mechanism between neighborhood houses.The left side of Fig. 3 shows the initial state i and E + i , respectively, at a unit price of p c i .The data communications for trading follow slotted carrier sense multiple access among active houses over a shared medium to avoid collisions: no new trading is initiated in the slot when the shared medium is busy.After contending in the shared medium, ES sends H 2 , H 3 , H 5 , and H 7 under E i as the demand of these houses for trading energy is fulfilled. 4ext, the ES again looks for the lowest energy difference between the rest of the houses selling and buying energy.It is observable that the difference between H After completing the trade of energy between neighborhood houses, if any house still needs energy, the ECU of that house purchases the energy directly from the primary grid.On the other Algorithm 1: Energy trading mechanism between houses.
Input: The surplus energy of that house (or houses) is/are sold back to the primary grid 14: end if 15: end for hand, if some houses cannot sell energy to the other houses, the ECU of that house sells energy directly to the primary grid.In this way, the trading of energy between houses in a TS is performed.The detail of the energy trading mechanism between neighborhood houses is described in Algorithm 1.

D. Predicting Energy Demand With FTRL
The enhanced FTRL (EFTRL) uses mutation for the perturbation of the probabilities of actions.It is equivalent to replicatormutation dynamic (RMD) [38] when induced by the entropy regularizer.Using an entropy regularizer, the FTRL trajectory converges at an exponentially fast rate to an approximate Nash equilibrium.In our method, we use enhanced FTRL to predict the energy demand of a customer.For this, we update the strategy profile π t j of the player j.The strategy profile π t j can be defined as where, the learning rate η > 0, the parameter μ (0 > μ > 1) ensures the trajectory of learning dynamics to reach an approximate equilibrium, and the reference strategy c j ∈ ∇ • (A j ).The strategy profile π t j is updated using (23).In this case, it converges to a Algorithm 2: Cost-friendly privacy of SMs by using FTRL.
Input: Real energy consumption data {x 1 , x 2 , .., x 3 } Output: y i 1: Calculate E o and E p using ( 3) and ( 4) respectively.2: for i = 1 : N do 3: Use the strategic profile π t j in (23) of enhanced FTRL to predict x p i by considering (7) for privacy cost optimization 4: ECU updates the ESD SoC so that x p i → E o and x p i → E p for the off-peak and peak period, respectively.

5:
For cost-saving, ECU charges the ESD more after TS i o in the off-peak period using ( 14) or ( 16) determined by the prediction accuracy.6: For the peak period, based on the prediction precision, the ESD is discharged more by using ( 17) or ( 18) after TS i p 7: For enhanced cost-friendly privacy of SMs, energy trade is performed based on Algorithm 1 8: Finally, calculate the output y i by using ( 10) or (11) 9: end for stationary point that is different from the Nash equilibrium of the original game.The stationary point should be the original game's 2π Nash equilibrium.However, a stationary point is not a Nash equilibrium, unless c j is a Nash equilibrium.Thus, a technique of adapting the reference strategy is used to converge to Nash equilibrium for the privacy preservation and cost-saving of SMs.For this, the probability of π t j is copied to c j in each iteration until M ≤ T (M for the updated frequency, and T is the total number of iterations), which is similar to the technique in [39].The prediction of demand f i (π t i , x i ) where x p i = N i=1 π t i x i .In this way, the strategy is updated to predict the energy demand x p i in TS i.
Based on the explanation of the above formulations, it is now apparent that the prediction and update processes are continuously evolving alternatively.The EFTRL prediction model becomes more accurate with the increase in the volume of mode samples collected and learned.The details of the privacy preservation mechanism of our proposed model are in Algorithm 2.

VI. THEORETICAL ANALYSIS
In this section, we theoretically analyze the ability of our proposed approach to preserve the cost-friendly privacy of SMs.For this, we consider privacy, cost-saving, modeling energy trading, the benefits of using URLLC, and the convergence of enhanced FTRL.

A. Privacy Preservation
Using the definition of mutual information (MI), MI between two sets X = {x 1 , x 2 , . .., x N } and Y = {y 1 , y 2 , . ...., y N } can be computed as Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
In (24), H(X) is a constant.Thus, minimizing I(X; Y ) is equivalent to maximizing H(X|Y Since it is not possible to directly access the conditional probability density function p(X|Y ), we alternate to maximize the lower bound or where D KL indicates KL divergence and q(X|Y ) can have any distribution with a known probability density function.Without loss of generality, it can be assumed that q(X|Y ) obeys the Gaussian distribution N (X; Y, σ 2 I).Thus, we can write where s is a constant.Therefore, the lower bound can be optimized by following the difference between the input and output, that is, with the use of energy from ESD (n i ), RER (n rer i ), and trading between houses (n sb i ), the ECU in the proposed framework always seeks to attain E o during the off-peak period and reach E p during the peak period.Thus, the input and output of an SM are often always different, indicating that the output y i is not purely deterministic and doesn't fully depend on the input x i .In the case of only using RB and RER the ability of MI is determined by n i and n rer i .In our proposed method, the ability of MI is determined by n i , n rer i , and n sb i where n sb i does not have any constraints (limited capacity, maximum change discharge rate) like n i and n rer i .As a result, for our proposed approach I(X; Y ) ≥ I(X; Y p ) where Y p = f (x i , n i , n rer i , n sb i ).Thus, the minimization of I(X; Y ) intrinsically decreases the upper bound of I(X; Y p ).Therefore, the privacy preservation ability of our proposed approach is much better compared to other existing demand-side energy management-based privacy preservation approaches of SMs.

B. Cost-Saving
For the cost-saving ability of a model, it is sufficient to show that In our proposed approach, the ESD charges fully by the end of the off-peak period and discharges fully by the end of the peak period.Besides, we also trade our storage energy between houses under an ES.The price of energy during the off-peak and peak period is p o and p p , respectively, where p o < p p .Energy consumption of a house during the off-peak period is x T o = N o i=1 x i .Our proposed approach aims to make the output y i during the off-peak period equal to E o (for the peak period E p ).Thus, we can write Besides, the ESD gets fully charged during the off-peak period.So, the total energy consumption during the off-peak period is as a result, the total price of energy for the off-period is In case of peak period, the total price of energy of a house is therefore, the total price of energy in the case of the proposed approach is p T o + p T p .In normal cases, the total price of energy is This indicates that the total energy price of the proposed method is lower than the regular energy price.Thus, our proposed approach can save the cost of energy during the preservation of the privacy of energy consumers.
In addition, considering the energy trade between consumers, the ECU of a house can sell energy to the neighborhood at a better price than the selling price back to the primary grid.Depending on the situation, only a tiny amount of energy must be sold back to the primary grid.Thus, energy prosumers can also get the monetary benefits of selling energy at a better price.

C. Modeling Energy Trading
We consider a slotted energy trading process of duration τ .Let Z be a random variable that represents the time between two tradings (successful or failure), and S T denotes the total number of successful energy tradings after T seconds.Then the aggregated trading rate is given by where a and p are the probability of trading attempts and failures, and E[Z] is the average time between two consecutive tradings given by E[Z] = P 0 τ + P req T req + P res T res , (32) where P 0 is the probability that a slot is idle, P req is the probability that a slot is a successful transmission of a trading request, and P res is the probability that a slot is a successful transmission of the trading response.Given the probabilities of request or response failure due to channel impairments are p e , the probabilities P 0 , P req , and P res can be computed as P 0 = (1 − a) n , P req = a(1 − (1 − a) n (1 − p e )), P res = a(1 − P 0 − P req )(1 − p e ).Under this setup, the values of p given a can be obtained as where n is the number of houses.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

D. With and Without URLLC
Energy trading between neighborhood houses requires realtime data transfer (or operation) between houses and the ES.Therefore, it is important to use URLLC for our proposed model.The houses can not store surplus energy when the ESD is full.Thus, the ECU of a house sells energy back to the neighborhood houses or the main grid.
To formulate the loss of energy (L e ) due to latency, we consider a case scenario for trading energy between neighborhood houses.In our approach, for energy trading, the ECU of all houses sends information to the ES (the distance between houses and the ES is d H T ).After that, ES sends the information about houses of E − i to houses of E + i .In this case, it is the distance of each house to ES except the houses not interested in trading energy which is Here E in are the houses that already participated in energy trading but need more rounds of trading to fully complete energy trading.As a result, it is going to participate in the next round to complete the energy trade.After completing the first round of energy trading, ES has to look for the rest of the houses to identify the lowest energy difference between selling and buying houses.Thus, the total distance for communication between ES and ECU is d With each round of energy trade E i increases, and thus d H T − d (E i −E in ) decreases.The total latency for the data transfer to complete trading can be expressed as From (33), it is obvious that with the increase in the number of houses (H T ) using RER and participating in the sale of energy, the total latency increases, which also increases the loss of energy and reduces the stability of the system.The energy loss for using different communication systems with different latency can be calculated using the following equation (34) where L e is the energy loss in kW/h, L c is the latency of a wireless communication system given by ( 31) H T is the total number of houses under an ES, H E i is the number of houses that do not need to participate in selling/buying energy or fulfilled the required energy trading, H E in is the number of houses that already participated in energy trading but need to participate again to complete the energy trading fully, and P rer is the energy generation capacity of the RER.Consider that the latency with URLLC is L u c and the latency without URLLC (of other communication techniques) is 34), it is obvious that L e ∝ L c .Now, L u c < L ot c .So, the power loss in SG with URLLC technology is the lowest compared to other existing wireless communication technologies.The benefit of using URLLC technology compared to the existing wireless communication technologies has been demonstrated in Fig. 4.

E. Convergence Conditions
With the definition of exploitability (using [40,Lemma 11.6 Along the lines of [38,Lemma 3.5], in case of replicatormutator dynamics (RMD), for a stationary point π μ it satisfy that ∀j ∈ {1, 2} and a j ∈ A j , we can get q π µ j (a j ) − ūπ µ j ≤ μ.As a result, (max πj ∈∇(A j ) ū πj ,π µ −j j ) can be bounded using the definition of zero-sum game ( 2 j=1 ūπ µ j = 0) as Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
VII. NUMERICAL RESULTS ANALYSIS Fig. 5 shows the sample dataset used for the numerical analysis of our proposed approach.We use real SM data (House 1, 2, and 3) which was collected using BS EN62053 − 21 : 2003 SM, and the data were recorded from East Midlands, U.K. [41].In addition, we also use another popular dataset (House 4) which is the REDD dataset [42].To predict the energy demand of a TS considering privacy preservation and cost reduction, we develop a framework based on FTRL.To demonstrate the convergence conditions of the proposed framework, we use random utility games with action sizes A 1 = A 2 = 15 and 20.The utility matrix is generated uniformly at random [0; 1] for a random utility game.We then average the result for 100 TS for each game.We apply the actual price tariffs [43] and the sales price is used from [37].
For our experiments, the buying price of energy is p i = 31 c / kWh and the selling price is p s = 11 c / kWh, and the exchange price of energy is p c i = p i +p s 2 = 21 c/kWh.The learning rates To calculate the privacy preservation ability of our model, we use MI as defined in [44].For calculating the cost reduction ability, the cost saving (CS) of a house is defined as In (40), E s i is the amount of energy sold back from a house to the neighborhood or the primary grid and E b is the amount of energy a house buys from the neighborhood or the primary grid in a TS.The convergence of enhanced FTRL (EFTRL) to predict the energy demand of a house in a TS is shown in Fig. 6.It is evident from Fig. 6 that for A 1 = A 2 = 15 and 20, EFTRL converges faster compared to other methods such as FTRL or optimistic FTRL (OFTRL) [45].Thus, the convergence ability of EFTRL is better compared to other methods.As a result, we use EFTRL to predict the energy demand of a house.
In the case of the off-line method, the ECU has knowledge of the energy demand of a house in every TS.By predicting energy consumption (considering cost-friendly privacy) using EFTRL and trading energy between houses, it is possible to enhance a house's cost-friendly privacy preservation ability near the offline privacy preservation method.Thus, we compare our proposed approach with the offline approach with respect to privacy and cost savings.
The comparison of the proposed approach with the offline method for the energy demand of House 1 is shown in Fig. 7.We can easily observe that the cost-friendly privacy preservation ability of the proposed approach and the offline method are almost the same.As for both approaches, the variation of energy demand is very low compared to the original energy demand of House 1.Therefore, by analyzing the energy demand output of House 1, adversaries or ECs cannot identify the actual usage of the appliances or activities inside House 1.The better performance is due to the trade of energy between neighborhood houses and the better prediction accuracy of the EFTRL algorithm.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.Figs.8-10 compare the cost-effective privacy preservation ability of the offline method and the proposed method for the energy demand data set of House 2, House 3 and House 4, respectively.It is evident that the cost-effective privacy preservation ability of the proposed approach is almost similar to the offline approach.In the case of both the offline approach and the proposed approach, we can see that the SM output is very smooth and has only 2 to 3 levels.Therefore, by analyzing this output, an adversary cannot identify the activities performed inside the house.As a result, the proposed approach preserves the privacy of energy prosumers.Due to the constraints of ESD, the output of SMs is not always a constant value.It can be overcome by using a higher capacity ESD.
Next, we compare the privacy preservation ability of the proposed approach with other existing and offline approaches.The lower the value of MI, the better the privacy preservation ability of a method.As shown in Fig. 11, the MI of our proposed approach is nearly equal to that of the offline method and very   low compared to the other existing SM privacy preservation methods.As a result, our method preserves better privacy compared to existing approaches.
As shown in Fig. 12 , the cost-saving ability of the proposed approach and the cost-saving ability of offline methods is almost equal.Cost-saving capacity also depends on the nature of the energy consumption data.However, as shown in Fig. 12, the cost-saving ability of our proposed approach is higher compared to other state-of-the-art approaches.Thus, our method has the ability to save higher costs compared to other existing cost-friendly privacy preservation approaches.
To demonstrate the benefits of using URLLC over other communication technologies, we calculate the energy trading rate S T and the average time between two consecutive trades E[Z] for different wireless communication technologies and compare it with URLLC technology.Fig. 13 compares the  aggregated trading rate with the increase in the number of houses considering different wireless communication technologies for different probability of trading attempts (a).With URLLC, the energy trading rate is always higher than for other technologies.As a result, the energy loss from URLLC is much lower than that of other wireless communication technologies.
Finally, Fig. 14 compares the average trading time with the increase in the number of houses for different wireless communication technologies considering different probabilities of trading attempts (a).It is evident that the average energy trading time is meager with URLLC compared to other technologies for any value of Due to the lower energy trading time in URLLC, it is very important to use URLLC technology to reduce the loss of renewable energy sources and improve the stability of the SG system.

VIII. CONCLUSION
We developed a framework for distributed energy trading between neighborhood houses with a demand-side privacy preservation approach using URLLC.While existing demand-side privacy preservation methods suffer from communication bottlenecks and poor cost-friendly privacy preservation ability due to the constraints in ESD, it was shown that the developed framework allows the performance to be distributed with online optimization of the energy trading.Thus, our method was able to predict the energy demand by extending FTRL to modify the SoC of the ESD to make the output load equal to the target output load during the off-peak and peak period, which employs regularization to improve the stability of the dynamical system.Theoretical analysis has shown that our proposed model has provable convergence guarantees and can preserve the cost-friendly privacy of energy trading with SMs.The numerical results demonstrated that the proposed framework outperforms existing approaches in terms of privacy preservation and cost savings for prosumers.

Manuscript received 6
January 2023; revised 27 August 2023; accepted 29 August 2023.Date of publication 4 September 2023; date of current version 13 December 2023.This work was supported by the Australian Government through the Australian Research Council's Discovery Projects funding scheme under Grant DP200100391.Recommended for acceptance by E. Damiani.(Corresponding author: Shiva Raj Pokhrel.)

Fig. 1 .
Fig. 1.System model for the cost-friendly privacy of the proposed method.

Fig. 2 .
Fig. 2. System model for the wireless communication between ECU of houses and ES.
For every base station b l where B = {b 1 , b 2 , . .., b m } for l = {0, 1, . .., m} there is a coverage area with radius r b and r b ∈ B. The coverage status s km of a user c k where C = {c 1 , c 2 , . .., c p } for k = {0, 1, . .., p} with the base station b m is available using the relative position relationship between them.Here, s km = 1 when user c k is covered by the based station b m otherwise s km = 0.In our proposed model, each user is covered by at least one base station.As a result, s km has the following constraint.
4 and H 6 or |H 4 | − |H 6 | = 1 is lower compared to the difference between H 8 and H 6 or |H 8 | − |H 6 | = 3.Thus, ES performs energy trading between H 4 and H 6 .As the sales demand for H 4 is fulfilled, ES sends it to E i .However, because the energy trading demand is not completed, H 6 still remains below E − i , and its value is updated from H 6 (−4) to H 6 (−1).Next, ES again finds the lowest difference between the remaining houses selling or buying energy.It is clear that the difference between H 8 and H 6 or |H 8 | − |H 6 | = 0 is the lowest.Thus, ES makes an energy trade between H 8 and H 6 and sends them to E i .In this way, the energy trade is performed.

Fig. 4 .
Fig. 4. Importance of using URLLC model for the energy trading or costfriendly privacy of SMs.

Fig. 5 .
Fig. 5. Energy consumption of different houses in a period of one day.

Fig. 7 .
Fig. 7. Comparison of the output of house 1 for the offline method and the proposed method.

Fig. 8 .
Fig. 8.Comparison of the output of house 2 for the offline and proposed methods.

Fig. 9 .
Fig. 9. Comparison of the output of house 3 for the offline and proposed methods.

Fig. 10 .
Fig. 10.Comparison of the output of house 4 for the offline and proposed methods.

Fig. 11 .
Fig. 11.Comparison of privacy preservation ability of the proposed method with other methods.

Fig. 12 .
Fig. 12.Comparison of cost saving ability of the proposed method with other methods.

Fig. 13 .
Fig. 13.Comparison of energy trading rate by using the URLLC and other communication technologies with the increase in the number of houses.

Fig. 14 .
Fig. 14.Comparison of average energy trading time by using the URLLC and other communication technologies with the increase in the number of houses.