A New Framework of Evolutionary Multi-Objective Algorithms with an Unbounded External Archive

This paper proposes a new framework for the design of evolutionary multi-objective optimization (EMO) algorithms. The main characteristic feature of the proposed framework is that the optimization result of an EMO algorithm is not the ﬁnal population but a subset of the examined solutions during its execution. As a post-processing procedure, a pre-speciﬁed number of solutions are selected from an unbounded external archive where all the examined solutions are stored. In the proposed framework, the ﬁnal population does not have to be a good solution set. The point of the algorithm design is to examine a wide variety of solutions over the entire Pareto front and to select well-distributed solutions from the archive. In this paper, ﬁrst we explain diﬃculties in the design of EMO algorithms in the existing two frameworks: non-elitist and elitist. Next, we propose the new framework of EMO algorithms. Then we demonstrate advantages of the proposed framework over the existing ones through computational experiments. Finally we suggest some interesting and promising future research topics.


INTRODUCTION
Evolutionary multi-objective optimization (EMO) has been an active research area in the last three decades [1].An m-objective minimization problem is written as follows: where f i (x) is the ith objective function to be minimized (i = 1, 2, ..., m), x is an n-dimensional decision vector (x = (x 1 , x 2 , ..., x n ) T ), and S is the feasible region of x.This problem has an n-dimensional decision space and an m-dimensional objective space.A solution x in the n-dimensional decision space is mapped by the objective functions to a point f(x) = (f 1 (x), f 2 (x), ..., f m (x)) T in the m-dimensional objective space.Objective functions are usually conflicting with each other.That is, any single solution cannot simultaneously optimize all objective functions.Good solutions for some objective functions are not good for others.In multi-objective optimization, solutions are compared using the Pareto dominance relation defined as follows: A solution a is referred to as being dominated by another solution b when the following two relations hold: f i (b)  f i (a) for i = 1, 2, ..., m and f j (b) < f j (a) for at least one j. 1 Shenzhen Key Laboratory of Computational Intelligence, University Key Laboratory of Evolving Intelligent Systems of Guangdong Province, Department of Computer Science and Engineering, Southern University of Science and Technology, email: hisao@sustech.edu.cn,panglm@sustech.edu.cn,kshang@foxmail.com(Corresponding Author: Hisao Ishibuchi) Using this relation, some basic concepts of optimality are defined for multi-objective optimization as follows: Definition 1 (Pareto Optimal Solution): If a solution a is not dominated by any other feasible solutions in S of the multiobjective problem, a is a Pareto optimal solution.
Definition 2 (Pareto Optimal Solution Set): The Pareto optimal solution set of the multi-objective problem is the set of all Pareto optimal solutions.Definition 3 (Pareto Front): The Pareto front of the multiobjective problem is the set of points f(x) in the objective space corresponding to all Pareto optimal solutions.That is, the Pareto front is the projection of the Pareto optimal solution set to the objective space by f(x).
When m = 2 (i.e., two-objective problem), the Pareto front is the tradeoff curve between the two objective functions in the twodimensional objective space.When m = 3 (i.e., three-objective problem), the Pareto front is the tradeoff surface among the three objective functions in the three-dimensional objective space.
Various EMO algorithms have been proposed to approximate the entire Pareto front of the multi-objective problem using a finite number of solutions (e.g., 100 solutions).One clear advantage of using population-based search (including EMO algorithms) over other approaches is that a set of solutions can be obtained by a single run.Moreover, we can obtain an arbitrary number of solutions since the population size is a user-defined parameter in population-based search.The final population is presented to human users as the result of multi-objective optimization.In the EMO community, it is assumed that the selection of a single final solution is performed by a decision maker.Thus, the task of EMO algorithms is to present a pre-specified number of well-distributed solutions over the entire Pareto front to the decision maker.
EMO algorithms are usually classified into two categories based on their generation update mechanisms: non-elitist and elitist [1].In the non-elitist framework, the current population is entirely replaced by a newly generated offspring population as shown in Fig. 1.Even if a good solution is included in the current population, it cannot survive to the next population.This framework is inefficient since any good solutions cannot be used to generate offspring solutions over multiple generations.
Moreover, any good solutions generated in the middle of evolution are not included in the final population unless they are generated again at the end of evolution.The main difficulty of the non-elitist framework is that the final offspring population is used as the final solution set.Thus, all solutions in the final offspring population should be good and well-distributed over the entire Pareto front.
In early 1990s, some well-known non-elitist algorithms were proposed such as MOGA [2] (Multi-Objective Genetic Algorithm), NSGA [3] (Non-dominated Sorting Genetic Algorithm), and NPGA [4] (Niched-Pareto Genetic Algorithm).Then, in late 1990s, Zitzler and Thiele [5] demonstrated that their elitist algorithm called SPEA (Strength Pareto Evolutionary Algorithm) clearly outperformed non-elitist algorithms.In the elitist framework, good solutions can survive to the next generation as shown in Fig. 2. They can be used to generate offspring solutions over multiple generations.Since the next population is selected from the current and offspring populations, the offspring population does not have to be a good solution set.It may be enough to include a few better solutions than the current ones.The elitist framework is clearly more efficient than the non-elitist framework.Since the proposal of SPEA [5], a number of elitist EMO algorithms have been proposed in the literature, which will be explained later in Section 2. Whereas the elitist framework is more efficient than the nonelitist framework, it still has a difficulty related to the selection of the next population from the current and offspring populations.In single-objective optimization, if a solution x is better than another solution y in one generation, x is always better than y.However, when x and y are non-dominated in multi-objective optimization, the following situation can happen: x has a higher fitness than y in some generations, and y has a higher fitness in other generations.Let us consider an elitist EMO algorithm with the population size 5 for a two-objective minimization problem with a linear Pareto front shown by the red line in Fig. 3 (a).We assume that the algorithm needs to choose five solutions from six solutions A-F as the next population.Of course, the true Pareto front is unknown.In Fig. 3 (a), almost all elitist EMO algorithms delete solution E. This is because all solutions are non-dominated and E is closely located to its two neighbors D and F. Fig. 3 (b) shows an ideal set of five solutions to approximate the entire Pareto front.Solution E is one of the five ideal solutions in Fig. 3 (b) whereas it looks the worst in Fig. 3 (a).As shown in this simple example, solution evaluation in multi-objective optimization is very difficult.Good solutions can be deleted in the generation update phase even in the elitist framework.Some elitist algorithms such as SPEA [4] have a bounded external archive to store good solutions separately from the current population.In those algorithms, the external archive is used as the final result of multi-objective optimization.However, the same difficulty as in Fig. 3 exists in the archive update phase since the archive size is bounded (e.g., 100 solutions).
To overcome this difficulty, we propose a new framework for the design of EMO algorithms.The proposed framework uses an unbounded external archive where all the examined solutions are stored.The final solution set is selected from the archive as shown in Fig. 4. The use of an unbounded external archive is not a new idea as we will explain later in Section 3. The new idea in this paper is to proposed the framework with an unbounded external archive for the design of new EMO algorithms.The proposed framework increases the flexibility of EMO algorithms since the final population does not have to be a good solution set (i.e., since it is not the final solution set).The proposed framework also improves the performance of existing algorithms since the final solution set is selected from all the examined solutions.The remainder of this paper is organized as follows.In Section 2, we explain some well-known and frequently-used elitist EMO algorithms.We also explain why it is difficult for those algorithms to find well-distributed solutions over the entire Pareto front.In Section 3, we demonstrate that the proposed framework clearly improves the performance of the elitist EMO algorithms explained in Section 2. In Section 4, we suggest some interesting and promising future research directions in the proposed framework.Finally, we conclude this paper in Section 5.

ELITIST EMO ALGORITHMS
In the last two decades, various elitist EMO algorithms have been developed since the proposal of SPEA [5].Those EMO algorithms are often categorized into three classes based on their fitness evaluation mechanisms.One is Pareto dominance-based algorithms such as SPEA [5] and NSGA-II [6] (the second version of NSGA).In this class, the Pareto dominance relation among solutions is used as the primary fitness evaluation criterion together with a diversity measure as a secondary criterion.
Another class is indicator-based algorithms such as SMS-EMOA [7] (S-Metric Selection Evolutionary Multi-objective Optimization Algorithm) and HypE [8] (Hypervolume Estimation algorithm).In this class, a multi-objective problem is handled as a single-objective problem to optimize a performance indicator which is used to evaluate a solution set (i.e., population).Fitness evaluation of each solution is based on its individual contribution to the performance indicator.
The other class is decomposition-based algorithms where a multi-objective problem is decomposed into a number of singleobjective problems.Each single-objective problem has the same scalarizing function with a different weight vector.A wide variety of solutions are searched for by solving those single-objective problems in a cooperative manner.A representative of this class is MOEA/D [9] (Multi-Objective Evolutionary Algorithm based on Decomposition).Recently, a number of EMO algorithms in this class have been proposed such as NSGA-III [10] (the third version of NSGA) and MOEA/DD [11] (Multi-Objective Evolutionary Algorithm based on Dominance and Decomposition).
Whereas the above-mentioned elitist algorithms outperform non-elitist algorithms, they also have their own disadvantages.In Pareto dominance-based algorithms, a diversity-related secondary criterion is used to choose the next population when the number of non-dominated solutions in the current and offspring populations is larger than the population size (since the Pareto dominance-based primary criterion gives the same fitness to all the non-dominated solutions).However, it is very difficult to appropriately specify the secondary criterion in a high-dimensional objective space.For example, NSGA-II uses the projection of solutions to each axis of the objective space to calculate the crowding distance.However, the importance of each solution in a high-dimensional objective space cannot be precisely measured by such a projection-based distance calculation.Fig. 5 shows experimental results by NSGA-II on the three-objective DTLZ2 [12] and Minus-DTLZ2 [13] test problems.The DTLZ2 test problem has been frequently used to evaluate recently-proposed EMO algorithms in the literature (e.g., [7]- [11]).The Minus-DTLZ2 test problem is a modified version of DTLZ2 where a minus sign "" is assigned to all objective functions to invert the shape of the Pareto front.Our computational experiments are performed in the standard setting which has been commonly used in many other studies (e.g., [10], [11], [13]).The population size and the maximum number of generations are specified as 92 and 250, respectively.NSGA-II is applied to each test problem 21 times.A typical single run with the median evaluation is selected from the 21 runs.The hypervolume [14] of the final population is calculated to evaluate each run.In hypervolume calculation, the reference point (1.1, 1.1, 1.1) T is used in the normalized objective space with the ideal point (0, 0, 0) T and the nadir point (1, 1, 1) T .Fig. 5 shows the final population of the selected single run with the median hypervolume value for each test problem.We can observe that the obtained solutions are not well-distributed over the entire Pareto front of each test problem.
Another difficulty of Pareto dominance-based algorithms is their poor scalability to many-objective problems [15], [16].When a multi-objective problem has many objectives, almost all solutions in the current population become non-dominated with each other in early generations.In this situation, the Pareto dominance relation cannot generate strong selection pressure to push the population towards the Pareto front since it gives the same fitness value to almost all solutions.Thus the fitness of each solution is evaluated only by a diversity measure, which simply increases the diversity of solutions with almost no progress towards the Pareto front.For example, it was demonstrated through computational experiments in [17] that NSGA-II was outperformed by a random search-based method on DTLZ2 with 10, 15 and 20 objectives.
In indicator-based algorithms such as SMS-EMOA [7] and HypE [8], the hypervolume indicator [14] has been frequently used.This is because no other unary indicator with the strictly Pareto compliant property is known [18].The hypervolume indicator can generate a strong selection pressure towards the Pareto front even in the case of many objectives.At the same time, the diversity of solutions can be increased by the hypervolume indicator.It was reported in Wagner et al. [19] that SMS-EMOA outperformed NSGA-II (and some other EMO algorithms) in their applications to frequently-used test problems DTLZ1 [12] and DTLZ2 [12] with five and six objectives.One difficulty of indicator-based algorithms is their large computation load when they are applied to many-objective problems.This difficulty has been tackled from two directions: One is to increase the speed of exact hypervolume calculation [20], [21], and the other is to improve the accuracy of approximate hypervolume calculation [22], [23].
Another difficulty of hypervolume-based algorithms such as SMS-EMOA and HypE is that uniformly distributed solutions are not obtained for non-linear Pareto fronts.This is because a set of uniformly distributed solutions is not optimal for hypervolume maximization [24], [25].In the same manner as Fig. 5, we perform computational experiments using SMS-EMOA.Experimental results are shown in Fig. 6.It is known that the obtained solution sets by SMS-EMOA depend on the reference point specification for hypervolume calculation [26], [27].Especially when the Pareto front is inverted triangular, totally different solution sets can be obtained depending on the reference point specification.In Fig. 6, the same specification mechanism as in the original SMS-EMOA paper [7] is used (i.e., the reference point is the worst value of each objective in the current population plus 1.0).We can observe that uniformly distributed solutions are not obtained in Fig. 6.In Fig. 6 (a), solutions are biased along the sides and around the center of the Pareto front.Similar distributions of solutions are always obtained independent of the location of the reference point for DTLZ2 unless the reference point is too close to the Pareto front [25]- [27].In Fig. 6 (b), no solutions are obtained on the sides of the Pareto front of Minus-DTLZ2.When the reference point is far away from the Pareto front, many solutions are close to the sides of the Pareto front of Minus-DTLZ2 [25]- [27].However, in this case, no solutions are obtained around the center of the Pareto front.This issue will be further discussed in Section 4.
As shown in Fig. 6, uniformly distributed solutions cannot be obtained by hypervolume-based algorithms for non-linear Pareto fronts.This is because a uniform distribution of solutions is not optimal for hypervolume maximization.Some modifications of the hypervolume indicator (e.g., cone-based calculation [28]) have been proposed to search for a set of uniformly distributed solutions.
In decomposition-based algorithms such as MOEA/D [9], a set of uniformly distributed weight vectors is generated using the Das and Dennis method [29].More specifically, all weight vectors w = (w 1 , w 2 , ..., w m ) T satisfying the following conditions are generated: where H is an integer parameter which defines the number of weight vectors.The weight vectors are uniformly located on the region specified by the following relations: w 1 + w 2 + ... + w m = 1 and 0  w i  1 for i = 1, 2, ..., m.The basic idea of decompositionbased algorithms is to find well-distributed solutions by singleobjective optimization of a scalarizing function with each weight vector.The point is to perform single-objective optimization for all weight vectors in a cooperative manner, which leads to efficient population-based search for multi-objective optimization.Since a single solution is assigned to each weight vector (i.e., since each single-objective problem has a single solution), the population size is the same as the number of the weight vectors.We perform computational experiments using MOEA/D on the three-objective DTLZ2 and Minus-DTLZ2 test problems in the same manner as in Figs. 5 and 6.As in the original MOEA/D paper [9], the PBI (Penalty-based Boundary Intersection) function with the penalty parameter  = 5 is used as a scalarizing function.The integer parameter H is specified as H = 12 (i.e., the population size is 91).The total number of generations is 250 as in Figs. 5 and 6.Experimental results are shown in Fig. 7.The obtained solution set in Fig. 7 (a) on DTLZ2 is much better than those in Fig. 5 (a) and Fig. 6 (a) with respect to the uniformity of solutions.However, well-distributed solutions are not obtained in Fig. 7 (b) on Minus-DTLZ2 where many solutions are around the sides of the Pareto front (but no solutions on the three corners).Since the distribution of the weight vectors in (3)-( 4) is triangular, good results are not obtained for test problems with inverted triangular Pareto fronts such as Minus-DTLZ2 in Fig. 7 (b).
As shown in Fig. 7, the performance of decomposition-based algorithms strongly depends on the shape of the Pareto front [13].When the weight vector distribution in (3)-( 4) is consistent with the Pareto front shape as in Fig. 7 (a), well-distributed solutions are obtained.However, when the weight vector distribution and the Pareto front shape are inconsistent as in Fig. 7 (b), solutions are biased towards some regions of the Pareto front.Currently, weight vector adaptation is a hot research topic to obtain well-distributed solutions for various shapes of Pareto fronts [30]- [32].However, complicated mechanisms are needed to adjust the weight vectors (i.e., weight vector adjustment is not easy).

PROPOSED EMO FRAMEWORK
As we have already explained, it is not easy for EMO algorithms to obtain a well-distributed solution set over the entire Pareto front, especially for nonlinear inverted triangular Pareto fronts.One clear reason for this difficulty is that the final population is used as the final result of multi-objective optimization.As explained in Section 1 using Fig. 3, good solutions may be obtained in the middle of evolution.However, they are not necessarily included in the final population.As a remedy of this difficulty, we propose a new EMO framework with an unbounded external archive where all the examined solutions during the execution of an EMO algorithm are stored.As a post-processing procedure, a pre-specified number of solutions are selected from the archive.The selected solutions are used as the final result of multi-objective optimization.In this manner, we can choose good solutions which are not included in the final population.The use of an unbounded external archive is not a new idea.For example, MOGLS (Multi-Objective Genetic Local Search [33]) proposed for multi-objective scheduling in late 1990s has an unbounded external archive where all non-dominated solutions are stored.Since multi-objective scheduling problems in [33] have a small number of Pareto optimal solutions, many non-dominated solutions are not obtained even when all non-dominated solutions are stored in the unbounded external archive.Thus, all the nondominated solutions among the examined ones are used as the final result of multi-objective optimization without worrying about the archive size.However, this framework has not been used in the mainstream of the EMO community where the focus has always been placed on continuous multi-objective optimization.In general, a continuous multi-objective optimization problem has an infinitely large number of Pareto optimal solutions.This is the main reason why an unbounded external archive has not been used in the mainstream of the EMO community.
In some studies [34]- [36], an unbounded external archive was used to evaluate existing EMO algorithms.In Bringmann et al. [34], it was demonstrated that the performance of EMO algorithms was improved by solution selection where a pre-specified number of solutions were selected from stored non-dominated solutions.Their computational experiments were performed only for two-objective problems due to high computational complexity of indicator-based solution selection.In Ishibuchi et al. [35], an unbounded external archive was used for fair comparison of different EMO algorithms with different population size.By selecting the same number of non-dominated solutions from the archive as the final result of each algorithm, different EMO algorithms with different population size were fairly compared.In Tanabe et al. [36], EMO algorithms were compared in two performance comparison scenarios: One is based on the final population and the other is based on the selected solutions from the archive.
Whereas an unbounded external archive was used in [34]- [36] for continuous multi-objective optimization, its purpose was for performance evaluation/comparison of existing EMO algorithms.In this paper, we propose the use of an unbounded external archive for the design of new EMO algorithms.That is, we propose a new framework for EMO algorithm design.
In an EMO algorithm with the proposed framework, all the examined solutions are stored in an unbounded external archive.The final result of multi-objective optimization is a pre-specified number of solutions selected from the archive.Solution selection is a post-processing procedure after the execution of the EMO algorithm.In this section, we demonstrate how this framework can improve the performance of existing EMO algorithms using the experimental results in Section 2.
Our experiments are performed using the selected run of each algorithm for each test problem in Section 2. In the selected run, all the examined solutions are stored in an unbounded external archive.After the execution of the selected run, all non-dominated solutions are selected from the stored solutions.Then, a pre-specified number of non-dominated solutions are selected using a distancebased solution selection method of Singh et al. [37].In order to compare the selected solution sets with the results in Section 2, the number of solutions to be selected is specified as the population size in each EMO algorithm.The solution selection method in [37] chooses one extreme non-dominated solution as the first solution, which has the best objective value for a randomly selected one objective function.The non-dominated solution with the largest distance from the first solution is selected as the second solution.The third solution is the non-dominated solution with the largest distance from the selected solution set including the first two solutions.In this manner, a pre-specified number of solutions are selected.This solution selection is similar to the method of Tanabe et al. [36] where m extreme non-dominated solutions are first selected for an m-objective problem.Each extreme solution has the best objective value for one of the m-objective functions. In

New Algorithm Design
In this paper, we proposed the use of the new framework for the design of EMO algorithms.The main future research topic is the design of new EMO algorithms using the proposed framework.In the proposed framework, the final population does not have to be a good solution set.Thus, we have much higher flexibility in the design of EMO algorithms than the case of the existing two frameworks: non-elitist and elitist.For demonstrating the high flexibility of algorithm design in the proposed framework, we show two examples in this subsection.One is the use of a dynamically changing reference point for hypervolume calculation in SMS-EMOA.Ishibuchi et al. [38] proposed an idea of gradually changing a reference point r = (r, r, ..., r) T in the normalized objective space from r = 10 at the initial population to r = 1 + 1/H at the final population (where H is the integer parameter in MOEA/D and can be calculated from the population size).The normalization of the objective space is performed in each generation so that the estimated ideal and nadir points by non-dominated solutions are (0, 0, ..., 0) T and (1, 1, ..., 1) T , respectively.The specification of r as r = 1 + 1/H is to obtain a good final population.In the proposed framework, we do not have to worry about the distribution of solutions in the final population.The point of the algorithm design in the proposed framework is to examine a wide variety of solutions over the entire Pareto front during the execution of an EMO algorithm.As an example, we apply SMS-EMOA to the three-objective Minus-DTLZ2 problem in the same manner as in Fig.In Fig. 11 (a), many solutions are around the center of the Pareto front.However, well-distributed solutions are selected in Fig. 11 (b).This example clearly shows that the final population does not have to be a good solution set.As explained in Section 2, it is difficult to obtain a well-distributed solution set as the final population of SMS-EMOA for Minus-DTLZ2.For comparison, we also show experimental results of SMS-EMOA with two different settings of a dynamically changing reference point.In one setting, the reference point r = (r, r, r) T is changed from r = 10 to r = 2.In the other setting, it is changed from r = 1 to r = 10.Experimental results are shown in Fig. 12 and Fig. 13, respectively.The final population in Fig. 12 (a) is better than that in Fig. 11 (a) since r = 2 is a better setting than r = 1 as the final location of the reference point.However, the uniformity of the obtained solutions in Fig. 12 (a) is not very good if compared with the selected solutions in Fig. 12 (b).In Fig. 13 (a), many solutions are close to the sides of the Pareto front.This is because r = 10 is too large as the final location of the reference point (i.e., the reference point is too far away from the Pareto front).Even in this case, a good solution set is obtained in Fig. 13 (b) by the solution selection from all the examined solutions.This is because many solutions around the center of the Pareto front were examined during the execution of SMS-EMOA whereas they are not included in the final population in Fig. 13     The idea of a dynamically changing reference point can also be used in decomposition-based algorithms.Most decompositionbased algorithms need to specify the origin of weight vectors in the objective space (which is called the reference point in the original MOEA/D paper [9]).In MOEA/D, the estimated nadir point z Nad is used as the origin of weight vectors.We examine a variant of MOEA/D with the PBI function ( = 5) where the origin of weight vectors is gradually changed from the standard specification (i.e., z Nad ) in MOEA/D at the initial population to the standard specification minus 10 (i.e., z Nad  (10, 10, 10) T ) at the final population.Since the objective space is not normalized in the original MOEA/D algorithm, we do not used the normalized objective space for computational experiments using MOEA/D.Experimental results are shown in Fig. 14.Since the origin of weight vectors is far away from the Pareto front at the final population, only a single solution is obtained around the center of the Pareto front in Fig. 14     It should be noted here that all computational experiments are performed in the same manner as in Section 2 and Section 3.That is, each algorithm is applied to the test problem 21 times.A single run with the median hypervolume value (calculated for the final population) is selected from the 21 runs as a typical run.Then, the final population and the selected solutions are compared for the selected run in each figure.

Maintenance of External Archive
In our computational experiments in this paper, all the examined solutions are simply stored in an unbounded external archive.After the execution of each EMO algorithm, non-dominated solutions are selected from the archive.Then, a pre-specified number of nondominated solutions are selected as the final solution set.Since the final solution set is always selected from non-dominated solutions, the archive can be maintained periodically by removing dominated solutions.However, the archive maintenance at every generation is not likely to be a good idea since it leads to a huge computational overhead.Analysis of optimal archive maintenance (e.g., optimal archive maintenance frequency) is an interesting future research topic.Development of an efficient method to select non-dominated solutions from all the examined solutions is also an important future research issue since non-dominated sorting has usually been discussed for a relatively small solution set for generation update and bounded external archive maintenance in the literature.
In the proposed framework, we assume that an unbounded external archive is used simply to store all the examined solutions.That is, we do not assume any utilization of the stored solutions.However, it is possible to utilize the stored solutions to evaluate current solutions and/or to generate new solutions.For example, the stored solutions can be used as a kind of reference points for evaluating the fitness of each solution in the current population.The distribution of the stored solutions can be used to navigate population-based multi-objective search towards less explored regions in the decision or objective space.Some stored solutions close to those less-explored regions can be used as parents to increase the diversity of offspring solutions.

Solution Selection
In our computational experiments in this paper, we used the distance-based solution selection method in [37].Whereas it is efficient (i.e., it is fast even for a large solution set), it is not necessarily the best solution selection method with respect to the quality of selected solutions.As in Bringmann et al. [34], we can use a performance indicator for solution selection.A wide variety of performance indicators have been proposed in the literature [39].
Most performance indicators can be used for solution selection.This is because they have been proposed to evaluate a solution set.
In principle, we can use any performance indicator as an objective function to search for the best solution set with respect to the used indicator.For example, by using the IGD (inverted generational distance) indicator [40], we may be able to obtain more uniformly distributed solutions than distance-based solution selection.This is because IGD minimization leads to higher uniformity of solutions than the use of other performance indicators as objective functions such as hypervolume maximization [26], [41].Proposal of new solution selection methods is a promising future research direction.This is because solution selection from a large solution set in a high-dimensional objective space has not been discussed in many studies in the literature.
A related important future research topic is to discuss the following fundamental question: "What is a good solution set?"In this paper, we implicitly assume that a well-distributed solution set over the entire Pareto front is a good solution set.However, the answer to this question depends on the request from the decision maker in a multi-objective problem at hand.If the decision maker wants to understand the shape of the entire Pareto front, a welldistributed solution set over the entire Pareto front may be a good solution set.A large number of non-dominated solutions may be needed in this case.However, if the decision maker wants to choose a single final solution after quickly examining only a small number of promising or representative solutions, it is unclear whether a well-distributed solution set over the entire Pareto front including boundary solutions is a good solution set or not.This is because solutions on the boundary of the Pareto front usually show imbalanced tradeoff where some objectives are very good but others are very bad.

CONCLUSION
In this paper, we proposed a new framework for the design of EMO algorithms.In the proposed framework, all the examined solutions are stored in an unbounded external archive.The final solution set is selected from the archive.The proposed framework has at least two advantages over the existing non-elitist and elitist frameworks.One is its high performance.In principle, the selected solution set is better than or at least the same as the final population since good solutions are selected from all the examined solutions (if a good solution selection method is used).Another advantage is its high flexibility.Since the final population does not have to be a good solution set, we have a large variety of possible choices in the design of EMO algorithms as demonstrated in Subsection 4.1.One disadvantage is the computation overhead of post-processing for solution selection.This computation overhead can be negligible in real-world applications where solution evaluation is expensive (e.g., a few minutes is needed to evaluate a single solution).Another disadvantage is an additional memory requirement for storing all the examined solutions.This may be also negligible except for the case of very large-scale multi-objective problems with millions of decision variables.Moreover, the total number of examined solutions is usually limited in real-world applications since solution evaluation is expensive.The main future research topics are the design of new high-performance EMO algorithms in the proposed framework and their performance evaluation for multi-objective and many-objective problems.Archive maintenance and solution selection are also promising future research topics.
(a) Six solutions.(b) Ideal set of five solutions.

Figure 3 .
Figure 3. Difficulty of selecting five good solutions from six solutions.

Figure 4 .
Figure 4. Generation update in the proposed framework.

Figs. 8 -
10, the selected solution sets are shown.It is clear from the comparison between Figs. 8-10 and Figs.5-7 that the solution selection from all the examined solutions improves the uniformity of solutions in the final solution sets (i.e., the final results of multi-objective optimization).These experimental results support the usefulness of the proposed framework.
6 (b) by changing the reference point r = (r, r, ..., r) T in the normalized objective space from r = 10 at the initial population to r = 1 at the final population.The large initial value and the small final value of r are to focus the search around the boundary and the center of the Pareto front, respectively.Experimental results are shown in Fig. 11.

Figure 11 .
Figure 11.Experimental results by SMS-EMOA with a dynamically changing reference point from r = 10 to r = 1. (a).

Figure 12 .
Figure 12.Experimental results by SMS-EMOA with a dynamically changing reference point from r = 10 to r = 2.

Figure 13 .
Figure 13.Experimental results by SMS-EMOA with a dynamically changing reference point from r = 1 to r = 10.
(a).However, well-distributed solutions are obtained in Fig.14 (b) by the solution selection from the examined solution.For comparison, we perform computational experiments by changing the reference point in the opposite way from z Nad  (10, 10, 10) T to z Nad .Experimental results are shown in Fig.15.Whereas the final population in Fig.15 (a) is totally different from that in Fig.14 (a), similar solution sets are selected in Fig.14 (b) and Fig.15 (b).This observation suggests that the point of algorithm design in the proposed framework is not to find a good final population but to examine a wide variety of solutions over the entire Pareto front during the execution of the algorithm.

Figure 14 .
Figure 14.Experimental results by MOEA/D with a dynamically changing reference point from z Nad to z Nad  (10, 10, 10) T .

Figure 15 .
Figure 15.Experimental results by MOEA/D with a dynamically changing reference point from z Nad  (10, 10, 10) T to z Nad .