Multivariable case-based reason adaptation based on multiple-output support vector regression with similarity-related weight for parametric mechanical design

Abstract

Using historical cases’ solutions to obtain feasible solution for new problem is fundamentally to successfully applying case-based reason technique in parametric mechanical design. As a well-known intelligent algorithm, the formulation of support vector regression has been taken for case-based reason adaptation, but the standard support vector regression can only be used as a univariate adaptation method because of its single-output structure, which would result in the ignorance of the possible interrelations among solution outputs. To handle the complicated case adaptation task with large number of problem inputs and solution outputs more efficiently, this study investigates the possibility of multivariable case-based reason adaptation with multiple output by applying multiple-output support vector regression. Furthermore, inspired by the fact that training sample which contains two closer cases can provide more useful information than others, this study adds the similarity-related weight into multiple-output support vector regression and gives high weights to the information provided by such useful training sample during multi-dimensional regression estimation. The superiority of proposed multiple-output support vector regression with similarity-related weight is validated by the actual design example and quantitative comparisons with other adaptation methods. The comparative results indicate that multiple-output support vector regression with similarity-related weight achieves the best performance for large-quantity case-based reason adaptation because of its higher accuracy and relatively lower cost.

Introduction

Case adaptation for parametric mechanical design

Parametric mechanical design^1,2 is the key technology in realizing rapid development of mechanical product, but the massive parameters should be determined by designers in the process of complex machine design. Hence, case-based reasoning (CBR) methodology is proposed to figure out the solution parameter values of new mechanical product by referring the solution parameter values of existing cases in case base.^3–6 So far, CBR has been employed successfully to design many mechanical products, such as micro-electromechanical device, ⁷ test turntable, ⁸ low power transformer, ⁹ extrusion die, ¹⁰ bearing device, ¹¹ gear reducer ¹² and welding fixture. ¹³ To obtain the feasible solution for new problem in CBR, finding a similar solution from existing case and further adapting this old solution are two main steps. Comparing with case retrieval, case adaptation is a more challenging issue, because in most situations, the old solution cannot be applied directly to the new problem.

Early CBR systems used in parametric mechanical design only focus on retrieving cases, and these systems commonly selected the solution of most similar case as the only candidate to be modified to satisfy the new problem.^14–17 There are two main problems in classical case adaptation under 1-nearest neighbour (1-NN) principle: first, the adjustment of old solution values heavily depends on human subjective judgement, ¹⁸ and second, the solutions from other relative similar cases are ignored. Hence, the automatic case adaptation under K-nearest neighbour (KNN) principle, that is, using K (K > 1) similar cases to generate more accurate solution for new problem, is the greatest challenge for CBR researches.^19,20

Early studies of CBR adaptation in the principle of KNN were based on the manual definition of adaptation rules, and engineers have to spend much effort on the acquisition of adaptation rules before adaptation process. ²¹ Thus, given a specific knowledge-light case adaptation method for CBR has significant practical implication. Statistical adaptation has been first proposed in the research of case adaptation since 1990s, such as the equal mean, ²² median, ²³ weighted mean ²⁴ and multivariate regression analysis (MRA). ²⁵ The major shortcoming of statistical case adaptation is its relative low adaptation accuracy. Our previous studies^26,27 have tried to extract various implicit knowledge hidden in similar case data to improve the performance of case adaptation based on weighted mean. Another way to overcome the shortcoming of statistical case adaptation is to perform case adaptation based on intelligent machine learning techniques, such as neural networks (NNs),^28–30 decision tree ¹⁹ and hybrid method.^31,32 These techniques have been introduced into CBR adaptation researches, which explore the utilization of inductive learning to acquire the differences between cases and their solutions, and apply the acquired knowledge to implement automatic case adaptation. ³³ However, these enumerated intelligent case adaptation methods perform poorly when retrieved cases have a large number of attributes.

Support vector regression–based case adaptation

Recently, some scholars have paid attention to another intelligent algorithm, that is, support vector machine (SVM), because SVM shows its better performance in solving classification and regression problems. The formulation of SVM for solving regression problem is also called as support vector regression (SVR). Compared to other machine learning methods, SVR could generate a global optimal solution because of its optimum network structure and has better generalization performance for retrieved cases with many attributes.^34,35 Hence, some studies have employed SVR to automatically perform CBR adaptation with various degrees of success.^36–38

Despite the SVR has been employed preliminarily in CBR adaptation, the applications of SVR in CBR adaptation also expose a potential problem, namely, the standard formulation of SVR can only be used as a univariate modelling technique for CBR adaptation due to its inherent single-output structure. ³⁹ Consequently, on one hand, SVR-based adaptation studies highlight the superiority of SVR for CBR adaptation. On the other hand, these studies have to construct different SVR-adaptation model for each solution feature separately, because of the inherent single-output structure of SVR, which would result in the ignorance of the possible interrelations among output solutions. If the parametric design case in case base has N solution values, then in traditional SVR-based adaptation, we have to build total N univariate adaptation engines for N solution features. To deal with the interrelations among solution outputs in CBR adaptation, one way is rule-based method, namely, recognizing and using these interrelations according to the predefined rules, and the other way is model-based method, that is, building multiple-output model which contains these relationships. Considering the complexity of interrelations among output solutions, the second way is a kind of feasible one. Hence, how to build a multivariable adaptation engine is a potential CBR adaptation research issue.

In other research areas, to generalize the SVR from regression estimation and function approximation to multi-dimensional problems, Pérez-Cruz et al. ⁴⁰ first proposed a multi-dimensional SVR (multiple-output support vector regression (MSVR)) that uses a cost function with a hyper-spherical intensive zone, capable of obtaining better predictions than using an SVR independently for each dimension. Subsequently, the new SVR algorithm has become a viable tool for solving the multiple-input, multiple-output regression problem, such as nonlinear channel estimation, ⁴¹ biophysical parameter evaluation, ⁴² converter gas tank level prediction, ⁴³ dynamic load identification, ⁴⁴ stock price index forecasting ⁴⁵ and multiple-step-ahead time series prediction. ⁴⁶ Although past studies have applied MSVR in various fields, as far as we know, there have been very few, if any, efforts to examine the feasibility of MSVR for CBR adaptation. So, we think it is worth to investigate the utilization of MSVR to conduct the adaptation engine, and apply MSVR-based adaptation method in actual mechanical design.

Motivation and originality of this research

As mentioned in section ‘Support vector regression–based case adaptation’, an alternative way to improve the performance of SVR-based adaptation is to utilize the MSVR. So, we have proposed an MSVR-based adaptation approach in our previous study. ⁴⁷ We adopt the differential heuristic^36–38,48 to build the training sample of MSVR. By using differential heuristic, the training sample consists of the differences between extracted case and retrieved case, rather than a single case in case base. As mentioned in section ‘Support vector regression–based case adaptation’, if the design case in case base has M problem values and N solution values, then the training sample of MSVR has M + N-dimensional input vector (including M problem differences between extracted case and retrieved case plus N solution values of retrieved case) and N-dimensional output vector (N solution values of extracted case). The details of generation of training sample will be presented in section ‘Training sample generation’. However, like the conventional SVR-based adaptation, the empirical error of each training sample in MSVR is equally penalized, which means every sample affects the generalization ability equally. ³⁷ However, actually, different training sample has different effect for MSVR modelization. That is, training sample which contains two closer cases is more useful for MSVR. Inspired by this, we attend to give higher similarity-related weight on the information provided by such useful sample in MSVR, and the new proposed method is named as MSVR with similarity-related weight (MSVR-SW). In MSVR-SW, the error penalty parameter of MSVR is given the similarity-related weight calculated between extracted and retrieved cases to show the impact of training sample. The detailed descriptions of MSVR-SW and corresponding practical application are introduced in the next sections. The breakdown of this research is divided into five sections. Section ‘Methodologies’ gives the specification of MSVR-SW, including the generations of training sample and similarity-related weight for MSVR-SW, and the construction of MSVR-SW-based adaptation engine. Section ‘Example’ gives an example to describe the process of MSVR-SW-based adaptation. A comparative experiment is carried out in section ‘Comparison’. Section ‘Conclusion’ concludes the article.

Methodologies

Training sample generation

Before building the MSVR-SW-based CBR adaptation engine, a set of training sample needs to be produced beforehand. As mentioned in section ‘Support vector regression–based case adaptation’, if the design case in case base has M problem values and N solution values, then the training sample of MSVR has M + N-dimensional input vector and N-dimensional output vector. Main et al. ⁴⁹ argued that if only the closest cases are used to train the adaptation engine, it could cripple the learning of adaptation knowledge. To guarantee the sufficient data for adaptation engine construction, we use the leave-one-out approach to generate training samples. Suppose that the number of design cases in case base is H, each time one case is picked randomly as the extracted case, and K cases similar to extracted case are retrieved from case base, then total H × K training samples could be produced, enough quantity of data for MSVR-SW-based adaptation engine training. Let $C_k$ be the kth case in case base with (M + N)-dimensional vector, and it can be expressed as

$$\begin{array}{l}{C}_k=\{{\mathbf{p}}_k,{\mathbf{s}}_k\}=\{(p_{k1},p_{k2},\dots ,p_{ki},\dots ,p_{kM}),\\ (s_{k1},s_{k2},\dots ,s_{kj},\dots ,s_{kN})\}\end{array}$$ (1)

where ${\mathbf{p}}_k$ and ${\mathbf{s}}_k$ are the problem and solution feature-value vectors of $C_k$ . $p_{\mathrm{ki}}$ and $s_{\mathrm{kj}}$ denote the ith and jth problem value and solution value of ${\mathbf{p}}_k$ and ${\mathbf{s}}_k$ . Similarly, let $C_0$ be the extracted case, then the (M + N)-dimensional vector of $C_0$ can be expressed in the following way

$$C_0=\{{\mathbf{p}}_0,{\mathbf{s}}_0\}=\{(p_{01},p_{02},\dots ,p_{0i},\dots ,p_{0M}),(s_{01},s_{02},\dots ,s_{0j},\dots ,s_{0N})\}$$ (2)

where ${\mathbf{p}}_0$ is the new problem value vector, and ${\mathbf{s}}_0$ is the corresponding adapted solution value vector. Then, the training sample which embeds the training data of $C_k$ and $C_0$ can be expressed as

$$D_{k0}=\{({\mathbf{p}}_k-{\mathbf{p}}_0)\oplus {\mathbf{s}}_k,{\mathbf{s}}_0\}=\{(p_{k1}-p_{01},p_{k2}-p_{02},\dots ,p_{\mathrm{kM}}-p_{0M},{\mathbf{s}}_k),{\mathbf{s}}_0\}$$ (3)

where $D_{k0}$ is the training sample originated from $C_k$ and $C_0$ .

Similarity-related weight calculation

The key issue of similarity-based weight calculation is the similarity measurement (SM) of problem feature-values between extracted and retrieved cases. Referring to related SM studies,^2,19,26,50 this article adopts multi-algorithm-oriented hybrid SM strategy to amplify the advantages of individual SM techniques and minimize their limitations. The hybrid SM strategy integrates four SM metrics, that is, Euclidian distance, Manhattan distance, Gaussian function and grey coefficient. The distance (Euclidian distance and Manhattan distance)-based SM metrics are in fact particular cases of the Minkowski measurement, expressed as follows

$$\begin{array}{l}{e}_{0k}=1-\mathrm{dis}\left(C_0,C_k\right)=1-\mathrm{dis}\left({\mathbf{p}}_0,{\mathbf{p}}_k\right)\\ =1-{\left({\displaystyle \sum _{i=1}^M{\left|\frac{p_{0i}-p_{ki}}{\max \left(p_{0i},p_{ki}\right)}\right|}^{\delta }}\right)}^{1/\delta }\end{array}$$ (4)

where $e_{0k}$ is the similarity value between $C_0$ and $C_k$ . If $\delta =1$ , distance formula $\mathrm{dis}(C_0,C_k)$ will become the Manhattan distance, $\delta =2$ the Euclidian one and $\delta =\infty$ the Chebychev distance.

The third SM metric is SM based on Gaussian transformation function, ⁵⁰ which transfers the differences between $C_0$ and $C_k$ into transformation indicator $\mathrm{gau}(C_0,C_k)$ . $\mathrm{gau}(C_0,C_k)$ represents the similarity between $C_0$ and $C_k$ , and it can be expressed as follows

$$\begin{array}{c}\multicolumn{1}{c}{e_{0k}=\mathrm{gau}(C_0,C_k)=\mathrm{gau}({\mathbf{p}}_0,{\mathbf{p}}_k)=\sum _{i=1}^M}\\ \multicolumn{1}{c}{\exp [-{\left(\frac{(p_{0i}-p_{\mathrm{ki}})/max(p_{0i},p_{\mathrm{ki}})}{\sqrt{2}\times {\sigma }_i}\right)}^2]}\end{array}$$ (5)

where ${\sigma }_i$ is the flexure point for ith problem feature.

The fourth SM method comes from the grey coefficient degree between $C_0$ and $C_k$ . We suppose $[{\overline{\mathbf{p}}}_k]$ is a problem set of all retrieved cases except for $C_k$ and ${\mathbf{p}}_0\in [{\overline{\mathbf{p}}}_k]$ . $\mathrm{grey}(C_0,C_k)$ denotes the corresponding grey coefficient degree, and $\underset{i}{inf}|[{\overline{\mathbf{p}}}_k]-p_{\mathrm{ki}}|$ and $\underset{i}{sup}|[{\overline{\mathbf{p}}}_k]-p_{\mathrm{ki}}|$ represent the minimum and maximum distance of ${\mathbf{p}}_k$ and $[{\overline{\mathbf{p}}}_k]$ . Then, the SM metric based on grey coefficient is expressed as follows

$$\begin{array}{l}{e}_{0k}=\mathrm{grey}\left(C_0,C_k\right)=\mathrm{grey}\left({\mathbf{p}}_0,{\mathbf{p}}_k\right)\\ =\sqrt{{\displaystyle \sum _{i=1}^M{\left(\frac{\underset{i}{\inf }\left|\left[{\overline{\mathbf{p}}}_k\right]-p_{ki}\right|+\left(\underset{i}{\sup }\left|\left[{\overline{\mathbf{p}}}_k\right]-p_{ki}\right|\right)/2}{\mathrm{dis}\left(p_{0i},p_{ki}\right)+\left(\underset{i}{\sup }\left|\left[{\overline{\mathbf{p}}}_k\right]-p_{ki}\right|\right)/2}\right)}^2}}\end{array}$$ (6)

After the similarity between $C_0$ and $C_k$ is calculated by individual SM metric, a linear mean method is adopted to determine the similarity-related weight of training sample $D_{k0}$ , and the formula is expressed as

$$w_{0k}=\frac{\frac{(e_{0k}^M+e_{0k}^E+e_{0k}^G+e_{0k}^R)}{4}}{\sum _{k=1}^K\left(\frac{(e_{0k}^M+e_{0k}^E+e_{0k}^G+e_{0k}^R)}{4}\right)}$$ (7)

where $w_{0k}$ is the weight value of $D_{k0}$ , and $e_{0k}^M$ , $e_{0k}^E$ , $e_{0k}^G$ and $e_{0k}^R$ are the similarity values calculated from four SM metrics based on Euclidian distance, Manhattan distance, Gaussian function and grey coefficient, respectively.

MSVR-SW-based adaptation

Framework

The framework of multivariable CBR adaptation based on MSVR-SW is described in Figure 1. Sections ‘Training sample generation’ and ‘Similarity-related weight calculation’ have described the process of training sample and similarity-related weight generation. Training samples are used to conduct the MSVR model. Different from other MSVR researches, the proposed MSVR-SW introduces the $w_{0k}$ into MSVR model to support that the training sample which consists of two similar cases (extracted and retrieved cases) is more significant that other ones containing less similar cases during regression estimation. When MSVR-SW adaptation engine faces new problem feature-value vector ${\mathbf{p}\prime }_0$ , the adaptation engine first selects the most similar case $C{\prime }_k$ for ${\mathbf{p}\prime }_0$ from case base, and transfer the ${\mathbf{p}\prime }_0$ and $C{\prime }_k$ into new input vector $\mathbf{x}\prime$ , and then it employs trained MSVR-SW models to output solution feature-value vector ${\mathbf{s}\prime }_0$ . A brief introduction of MSVR-SW is given in the next session.

Figure 1.

Framework of multivariable CBR adaptation based on MSVR-SW.

MSVR for case adaptation

Pérez-Cruz et al. ⁴⁰ appointed out that ‘the key idea of MSVR is extending Vapnikε-insensitive loss function to multi-dimensional output situation, i.e. a hyper-spherical insensitive zone, which handles all the outputs together’. Some studies^41–46 have proved that MSVR can improve generalization performance of decision model especially when only scarce samples are available. In the absence of adequate complex mechanical design cases, a common phenomenon in most industry companies, MSVR is an ideal option. Inspired by that, this study employs MSVR to CBR adaptation in mechanical design, and inventively introduces similarity-related weight into MSVR model. Given a set of training sample $\{{\mathbf{x}}_i,{\mathbf{y}}_i{\}}_{i=1}^{H\times K}$ for training; according to Figure 1, the multivariable CBR adaptation based on MSVR-SW is regarded as building the mapping between the differences of problem values plus old solution values ${\mathbf{x}}_i\in R^{M+N}$ and new solution values ${\mathbf{y}}_i\in R^N$ . The MSVR-SW solves this issue by finding the regressor ${\mathbf{w}}^j$ and $b^j$ ( $j=1,2,\dots ,N$ ) for every output that minimizes

$$\underset{\mathbf{w},b}{min}\mathrm{Lp}=\frac{1}{2}\sum _{j=1}^N{\Vert {\mathbf{w}}^j\Vert }^2+\gamma \sum _{i=1}^{H\times K}{w}_{i}L\left(u_i\right)$$ (8)

where $u_i=\Vert {\mathbf{e}}_i\Vert =\sqrt{({\mathbf{e}}_i^T{\mathbf{e}}_i)},{\mathbf{e}}_i^T={\mathbf{y}}_i^T-\varphi \left({\mathbf{x}}_i\right)\mathbf{W}-{\mathbf{b}}^T,\mathbf{W}=[{\mathbf{w}}^1,\dots ,{\mathbf{w}}^N],\mathbf{b}=[b^1,\dots ,b^N]$ . $\varphi (·)$ is the mapping function from primal space to feature space, and $\gamma$ is a hyper parameter which determines the trade-off between the regularization and the error reduction term. $w_i$ is the similarity-related weight for the ith training sample. $L(u)$ is a quadratic $\epsilon$ -insensitive cost function defined as

$$L\left(u\right)=\{\begin{array}{c}0\hfill \\ \multicolumn{1}{c}{u^2+2u\epsilon +{\epsilon }^2}\end{array}\begin{array}{c}u<\epsilon \hfill \\ \multicolumn{1}{c}{u\ge \epsilon }\end{array}$$ (9)

In equation (9) when $\epsilon$ is nonzero, it will take into account all outputs to construct each individual regressor and will obtain more robust adaptation results, then yield a single support vector set for all dimensions. As equation (8) cannot be solved straight forwardly, an iterative method, named IRWLS (iteratively reweighted least squares), is utilized to obtain a desired solution. ⁴¹ By introducing a first-order Taylor expansion of cost function $L(u)$ , the objective of equation (8) will be approximated by the following equation

$$\underset{\mathbf{w},b}{min}\mathrm{Lp}\prime =\frac{1}{2}\sum _{j=1}^N{\Vert {\mathbf{w}}^j\Vert }^2+\frac{1}{2}\sum _{i=1}^{H\times K}{w}_i{a}_i{u}_i^2+\mathrm{CT}$$ (10)

where

$$a_i=\{\begin{array}{c}0\hfill \\ \multicolumn{1}{c}{2\gamma (u_i^l-\epsilon )/u_i^l}\end{array}\begin{array}{c}{u}_i^l<\epsilon \hfill \\ \multicolumn{1}{c}{u_i^l\ge \epsilon }\end{array}$$ (11)

and CT is a constant term which does not depend on $\mathbf{W}$ and $\mathbf{b}$ , and the superscript l is the lth iteration.

To optimize equation (10), an IRWLS procedure is constructed which linearly searched the next step solution along the descending direction based on the previous solution. According to the representer theorem, ⁵¹ the best solution to minimization of equation (10) can be expressed as ${\mathbf{w}}^j=\sum _i\varphi ({\mathbf{x}}_i){\beta }^j={\mathrm{\Phi }}^T{\mathit{\beta }}^j$ , where $\beta$ is a parameter to reflect the linear combination of the training samples in the feature space. The IRWLS of MSVR can be summarized in the following steps ⁴¹

• Step 1. Set $l=0$ , ${\mathit{\beta }}^l=0$ and ${\mathbf{b}}^l=0$ . Calculate $u_i^l$ and $a_i$ .

• Step 2. Compute the solution ${\mathit{\beta }}^{\mathbf{s}}$ and ${\mathbf{b}}^s$ according to the next equation

  $$\left[\begin{array}{cc}\mathbf{K}+{\mathbf{D}}_a^{-1}\hfill & 1\hfill \\ \multicolumn{1}{c}{{\mathbf{a}}^T\mathbf{K}}& 1^T\mathbf{a}\hfill \end{array}\right]\left[\begin{array}{c}{\mathit{\beta }}^j\hfill \\ \multicolumn{1}{c}{{\mathbf{b}}^j}\end{array}\right]=\left[\begin{array}{c}{\mathbf{y}}^j\hfill \\ \multicolumn{1}{c}{{\mathbf{a}}^T{\mathbf{y}}^j}\end{array}\right],j=1,2,\dots ,N$$ (12)

  where $\mathbf{a}={[a_1,\dots ,a_H]}^T$ , $({\mathbf{D}}_a{)}_{\mathrm{ij}}=a_i\delta (i-j)$ , $1$ is a column vector of H ones and $\mathbf{K}$ is the kernel matrix. The line search algorithm can be readily expressed in terms of ${\mathit{\beta }}^j$ . Here, the radial basis function (RBF) is selected as kernel function. Define the corresponding descending direction as

  $${\mathbf{P}}^l=\left[\begin{array}{c}{\mathbf{w}}^s-{\mathbf{w}}^l\hfill \\ \multicolumn{1}{c}{{({\mathbf{b}}^s-{\mathbf{b}}^l)}^T}\end{array}\right]$$ (13)

• Step 3. Use a backtracking algorithm to compute ${\mathit{\beta }}^{l+1}$ and ${\mathbf{b}}^{l+1}$ , and further obtain $u_i^{l+1}$ and $a_i$ . Go back to step 2 until convergence.

The proof of convergence of the above algorithm is given by Sánchez-Fernández et al. ⁴¹ Because $u_i^l$ and $a_i$ are calculated using every dimension of $\mathbf{y}$ , each individual regressor contains the information of all outputs, which improves the regression performance of MSVR.

Example

Training sample and weight generation

Referring to our previous study, ³⁷ this article also uses the power transformer design with S1, S2, S3 and S4 series as a practical example to implement the MSVR-SW-based adaptation. Assume that each power transformer design case contains four problem features and four solution features, that is, $M=4$ and $N=4$ , and let K be 4 for the convenience of computing. We utilize hybrid SM strategy as mentioned in section ‘Similarity-related weight calculation’ to retrieve K most similar cases, before adaptation operation. We use case (S1-80) as an example to describe the process of training sample generation. When case (S1-80) is considered as the extracted case, four cases with relatively higher similarity values can be retrieved, as listed in Table 1. Then, the corresponding training sample and their weights are expressed as

$$\begin{array}{c}{D}_{01}=\{\mathbf{x},\mathbf{y}\}=\{(p_{11}-p_{01},p_{12}-p_{02},p_{13}-p_{03},p_{14}-p_{04},s_{11},s_{12},s_{13},s_{14}),(s_{01},s_{02},s_{03},s_{04})\}\hfill \\ =\{(20,0,0,0,370,30,2.2,31.81),(370,30,2.2,33.81)\}\hfill \\ w_{01}=\frac{0.9375}{(0.9375+0.935+0.8943+0.8594)}=0.2585\hfill \end{array}$$ (14)

$$\begin{array}{c}{D}_{02}=\{\mathbf{x},\mathbf{y}\}=\{(p_{21}-p_{01},p_{22}-p_{02},p_{23}-p_{03},p_{24}-p_{04},s_{21},s_{22},s_{23},s_{24}),(s_{01},s_{02},s_{03},s_{04})\}\hfill \\ =\{(-17,0,0,0,370,30,2.5,33.28),(370,30,2.2,33.81)\}\hfill \\ w_{02}=\frac{0.935}{(0.9375+0.935+0.8943+0.8594)}=0.2578\hfill \end{array}$$ (15)

$$\begin{array}{c}{D}_{03}=\{\mathbf{x},\mathbf{y}\}=\{(p_{31}-p_{01},p_{32}-p_{02},p_{33}-p_{03},p_{34}-p_{04},s_{31},s_{32},s_{33},s_{34}),(s_{01},s_{02},s_{03},s_{04})\}\hfill \\ =\{(-30,-0.3,0,0,370,30,2.5,33.51),(370,30,2.5,33.81)\}\hfill \\ w_{03}=\frac{0.8943}{(0.9375+0.935+0.8943+0.8594)}=0.2466\hfill \end{array}$$ (16)

$$\begin{array}{c}{D}_{04}=\{\mathbf{x},\mathbf{y}\}=\{(p_{41}-p_{01},p_{42}-p_{02},p_{43}-p_{03},p_{44}-p_{04},s_{41},s_{42},s_{43},s_{44}),(s_{01},s_{02},s_{03},s_{04})\}\hfill \\ =\{(45,0,0,0,415,30,2.5,37.28),(370,30,2.5,33.81)\}\hfill \\ w_{04}=\frac{0.8594}{(0.9375+0.935+0.8943+0.8594)}=0.237\hfill \end{array}$$ (17)

Table 1.

S1-80 and its four similar cases, that is, S1-100, S1-63, S1-50 and S1-125.

Problem/solution	Attribute	Extracted case	Retrieved case
		S1-80 (C0)	S1-100 (C1)	S1-63 (C2)	S1-50 (C3)	S1-125 (C4)
Problem feature	Rated capacity (p1)	80 kVA	100 kVA	63 kVA	50 kVA	125 kVA
Problem feature	Primary voltage (p2)	6.3 kV	6.3 kV	6 kV	6 kV	6.3 kV
Problem feature	Secondary voltage (p3)	0.4 kV	0.4 kV	0.4 kV	0.4 kV	0.4 kV
Problem feature	Connection symbol (p4)	Yyno	Yyno	Yyno	Yyno	Yyno
Solution feature	Armature diameter (s1)	370 mm	370 mm	370 mm	370 mm	415 mm
Solution feature	Insulation radius (s2)	30 mm2	30 mm2	30 mm2	30 mm2	30 mm2
Solution feature	Coil radial thickness (s3)	2.2 mm	2.2 mm	2.5 mm	2.5 mm	2.2 mm
Solution feature	Wire cross-section (s4)	33.81 mm2	31.81 mm2	33.28 mm2	33.51 mm2	37.28 mm2
Similarity for S1-80		1.0000	0.9375	0.935	0.8943	0.8594

Adaptation engine construction

The parameter selection plays a crucial role on the adaptation engine construction. RBF $\exp (-\lambda ||x_i-x_j|{|}^2)$ is adopted as the kernel function of MSVR. So, there are three parameters to be optimized in MSVR, that is, $\gamma$ , $\lambda$ and $\epsilon$ . Following our previous studies,^19,43,48 the performance of SVR is insensitive to $\epsilon$ and the reasonable value of $\epsilon$ is 0.01, while $\gamma$ and $\lambda$ are not known beforehand whose values are the best for one problem. At present, some existing parameter search approaches are employed in SVR, for example, cross-validation via parallel grid search, ⁵² genetic algorithm, ⁵³ simulated annealing, ⁵⁴ artificial immune algorithm, ⁵⁵ particle swarm optimization^43,46 and firefly algorithm. ⁴⁵ Han et al. ⁴³ pointed out that cross-validation expensed a long period of time when facing a high real-time demand. However, for the median-sized problems such as mechanical product design with limited number of cases, cross-validation might be the most reliable way for model parameter selection, and grid search is applicable to optimize the parameters with one or two dimensionalities. ⁵⁰ Thus, this study prefers a grid search on $(\gamma ,\lambda )$ using five-fold cross-validation.

Meanwhile, Lee ⁵⁶ suggested that an exponentially increasing sequence is a feasible method to obtain optimal parameters. Hence, the search spaces of $\gamma$ and $\lambda$ are defined as an exponentially growing space: ${\log }_2\gamma =-5,-3,\dots ,13,15$ and ${\log }_2\lambda =-15,-13,\dots ,1,3$ . In order to prevent the over-fitting problem in $(\gamma ,\lambda )$ optimization, the grid search was performed respectively in training set containing 60 experiment cases and test set containing 30 experiment cases to find the optimal solution. Five options with highest cross-validation rates in training set and test set are shown in Figure 2, from which we can find out that the best solutions of $(\gamma ,\lambda )$ for MSVR-SW is $(2^7,2^{-5})$ . After the construction of adaptation engine based on MSVR-SW, we can input a new design problem in the form of object–attribute–value, and subsequently obtain the inductive adaptation results.

Figure 2.

Parameter optimization results of MSVR-SW.

Comparison

Data description

We select 300 historical power transformer design cases from UCI Repository of ML Databases ⁵⁷ as the raw data. The reasons for choosing such data are well-expressed parametrical case and moderate computational complexity. So, it is a representative data set for CBR adaptation researches.^{19,26,27,37,47} Furthermore, we would like to compare the performances between different SVRs with different number of inputs and outputs. Referring to Hu et al., ²⁷ we collect a total of 16 problem features and 10 solution features and build three datasets for this comparison according to different problem and solution features. Table 2 describes the selection of problem and solution features in three datasets. For dataset I, there are four problem features and four solution features. For dataset II, there are 10 problem features and 6 solution features, and all problem and solution features (sixteen problem features and ten solution features) are selected in dataset III. Besides, we intend to carry out adaptation processes using various K values, namely, the comparative adaptation methods are performed under 3-NN, 5-NN, 7-NN, 9-NN, 11-NN and 13-NN principles. The objective of this experiment is to find out the influence of the number of retrieved similar cases for adaptation result and to obtain the reasonable K value for CBR adaptation in parametric mechanical design, because higher K value may increase the adaptation accuracy, but it also means requiring large amount of computational time and the reduction in calculative efficiency.

Table 2.

Three datasets with different problem and solution features. ²⁷

	Problem feature	Dataset				Solution feature	Dataset
		I	II	III			I	II	III
P 1	Rated capacity (kVA)	•	•	•	S 1	Insulation radius (mm)	•	•	•
P 2	No-load loss (kW)	•	•	•	S 2	Armature diameter (mm)	•	•	•
P 3	Load loss (kW)	•	•	•	S 3	Wire cross-section (mm)	•	•	•
P 4	Connection symbol	•	•	•	S 4	Coil radial thickness (mm)	•	•	•
P 5	Total weight (kg)			•	S 5	Number of primary winding turns		•	•
P 6	Primary winding voltage (kV)		•	•	S 6	Number of secondary winding turns		•	•
P 7	Primary winding current (A)		•	•	S 7	Primary winding wire diameter (mm)			•
P 8	Secondary winding voltage (kV)		•	•	S 8	Secondary winding wire diameter (mm)			•
P 9	Secondary winding current (A)		•	•	S 9	Mean path of bobbin (mm)			•
P 10	Current frequency (Hz)			•	S 10	Core material grade			•
P 11	Temperature rise (°)			•
P 12	No-load current (A)		•	•
P 13	Load current (A)		•	•
P 14	Rated high voltage (kV)			•
P 15	Rated low voltage (kV)			•
P 16	Tapping range of rated voltage (%)			•

To investigate the superiority of MSVR-SW in CBR adaptation, traditional adaptation methods are selected in this comparison. As mentioned in section ‘Case adaptation for parametric mechanical design’, the knowledge-light case adaptation falls into two categories: statistical adaptation and intelligent adaptation. One of the most used tools in intelligent adaptation is NNs. Therefore, two typical statistical methods, that is, MRA and median, and one NN, that is, back-propagation neural network (BPNN), are employed. In addition, classical SVRs, that is, standard SVR and MSVR, are also used as the comparative methods. So, there are a total of six examined methods, that is, MSVR-SW, MSVR, standard SVR, BPNN, MRA and median. Among them, MSVR-SW, MSVR-SW, MSVR and SVR are carried out using LIBSVM tool. ⁵⁸ Meanwhile, BPNN and MRA are implemented using WEAK library ³⁷ in this comparison.

Validation techniques

This comparison uses the hold-out methodology. In each dataset, two-thirds of the power transformer design cases are used as estimation samples, while reminders constitute the hold-out samples. Comparative adaptation methods were employed and trained on the estimation samples and produced the adaptations for the entire hold-out samples. The adaptation results were then compared to the hold-out sample set to evaluate the out-of-sample performance of each method. A five-fold cross-validation was used in the training phase to avoid over-fitting. To assess the adaptation abilities of the different methods, we compared the out-of-sample adaptations using two different approaches because it is generally impossible to specify an evaluation criterion that is universally acceptable. ⁴⁵ First, we examined the adaptation accuracies of all comparative methods by calculating the mean absolute percentage error (MAPE), ⁴⁶ as MAPE, rather than mean absolute error, can reflect the mean adaptation performance of all solution components. ³⁷ The definition of MAPE is shown as follows

$$\mathrm{MAPE}=\frac{1}{G}\sum _{i=1}^G\left(\frac{1}{N}\sum _{j=1}^N\left(\frac{|s_{\mathrm{ij}}-{s\prime }_j|}{s_{\mathrm{ij}}}\right)\right)$$ (18)

where $s_{\mathrm{ij}}$ is the jth solution feature-value of ith case from hold-out sample, and $s{\prime }_j$ is the corresponding adapted value. G is the number of cases in hold-out sample and G = 100 in this experiment. We repeated the previous evaluation process 50 times, yielding 50 adaptation accuracies for each method, and the performances of the examined methods under each K-NN principle are judged in terms of the mean of the MAPE of the 50 replications for hold-out samples. Second, the analysis of variance (ANOVA) test was employed in this experiment to determine whether statistically significant differences exist among the comparative methods in out-of-sample adaptation. Moreover, we also used Tukey’s honestly significant difference (HSD) test to compare all pairwise significant differences between any two methods, as Xiong and colleagues^45,46 have employed Tukey’s HSD test to evaluate the performances of MSVR in multiple-input, multiple-output time series forecasting. Note that Tukey’s HSD test is a post hoc test, meaning that Tukey’s HSD test should not be performed unless the results of the ANOVA test are positive. ⁴⁵ The procedure of this comparative experiment is shown in Figure 3.

Figure 3.

Process of comparative experiment.

Implementation of methodologies

Once the experimental dataset is set, the selected optimization method is employed for parameter space searching. Similar to MSVR-SW, the kernel functions of MSVR and SVR applied in this comparison experiment were RBF. Hence, the three hyper-parameters, namely, $\epsilon$ , $\gamma$ and $\lambda$ , should be determined. Concerning the selection of parameters by themselves in the optimization approaches, such as artificial immune algorithm, ⁵⁵ particle swarm optimization,^43,46 firefly algorithm ⁴⁵ and glowworm swarm optimization, ⁵⁹ it is yet another challenging selection task. For the sake of simplicity, five-fold cross-validation via parallel grid search ⁵² is mentioned in section ‘Adaptation engine construction’, which was employed in this experiment to produce the optimal parameters. Note that, the grid size will affect the final parameters, and this article makes a trade-off between computational precision and cost to set the grid size. We initially select ${\log }_2\gamma =-5,-3,\dots ,13,15$ and ${\log }_2\lambda =-15,-13,\dots ,1,3$ to compose the grid search space, by referring to Lee. ⁵⁶ Table 3 summarizes the final parameters of SVR, MSVR and MSVR-SW in datasets I, II and III. For BPNN, the corresponding adaptation engine includes 2M + 1 input nodes and one output node, like SVR-based adaptation engine. Referring to Qi et al., ³⁷ in BPNN model, we set 25 hidden nodes, 0.1 and 0.6, respectively, for stopping criterion, learning rate and the momentum term. Sigmoid transfer function is used as the hidden and output nodes of BPNN, and learning epochs per one training example is set as 2000. Besides machine learning techniques, one of the statistical approaches for case adaptation is the median of the solutions of K similar cases, where K > 2. It is a measure of central tendency and a more robust statistic than mean method when the number of cases increases. ⁶⁰ Another statistical method in this experiment is MRA, and it derives a linear combination of independent variables to model the relations between problem values and solution values, which is expressed as

$$\hat{s}=(w_1{p}_1+w_2{p}_2+\cdots +w_N{p}_N)|\theta$$ (19)

where p and w are the problem value of existing case and related weight, and $\theta$ is the confidence coefficient.

Table 3.

Final parameters of SVR, MSVR and MSVR-SW in datasets I, II and III.

Parameters		SVR	MSVR	MSVR-SW
Kernel function		Radial basis function	Radial basis function	Radial basis function
$\epsilon$		0.01	0.01	0.01
Dataset I	$\gamma$	25	27	29
	$\lambda$	2−3	2−5	2−7
Dataset II	$\gamma$	27	29	211
	$\lambda$	2−5	2−7	2−7
Dataset III	$\gamma$	211	211	211
	$\lambda$	2−5	2−3	2−7

SVR: support vector regression; MSVR: multiple-output support vector regression; MSVR-SW: MSVR with similarity-related weight.

Results and discussion

Result

This section focuses on the out-of-sample adaptation abilities of MSVR-SW, MSVR, SVR, BPNN, MRA and median in terms of statistical accuracy and computational cost. Table 4 lists the mean MAPE values of examined methods on 50 times hold-out operations in three datasets to reflect the general adaptation performances, from which we can also abstract the change of accuracy of each method with increasing K values as shown in Figure 4. To figure out the statistical differences of adaptation performance among MSVR-SW, MSVR, SVR, BPNN, MRA and median methods for each K-NN principle and dataset, we also launched the ANOVA and Tukey’s HSD tests, and the calculation results are described in Tables 5 and 6. In Table 6, we rank the six methods from 1 (the best) to 6 (the worst). Besides adaptation accuracy, the computational load is another important and critical assessment criterion from a practical viewpoint, as case adaptation under K-NN principle may be used for assistant decision-making purposes in real-world design activities, and the low construction cost is a real advantage for the underlying approach. Therefore, it is reasonable to compare the examined methods for their computational costs. The required times of the six comparative methods in three datasets with different K values are presented in Figure 5.

Table 4.

MAPE values of six comparative methods with various K values in three datasets.

K	MSVR-SW	MSVR	SVR	BPNN	MRA	Median
Dataset I
3	0.3434	0.3268	0.3238	0.4140	0.4474	0.4422
5	0.3126	0.3166	0.3086	0.3576	0.3878	0.4436
7	0.2858	0.2950	0.2866	0.3476	0.3810	0.4248
9	0.2374	0.2674	0.2676	0.2956	0.3754	0.4022
11	0.2426	0.2582	0.2850	0.3136	0.3672	0.4074
13	0.2476	0.2632	0.3072	0.3178	0.3648	0.4474
Dataset II
3	0.3138	0.3064	0.3332	0.3568	0.4288	0.4258
5	0.2858	0.2768	0.2836	0.3374	0.3974	0.4136
7	0.2528	0.2442	0.2602	0.2944	0.3842	0.4038
9	0.2170	0.2272	0.2536	0.2764	0.3528	0.3742
11	0.2238	0.2354	0.2742	0.2772	0.3458	0.3772
13	0.2458	0.2522	0.2936	0.2838	0.3568	0.3536
Dataset III
3	0.2528	0.2442	0.2938	0.3148	0.3716	0.4068
5	0.2408	0.2232	0.2727	0.2706	0.3432	0.3934
7	0.1732	0.1764	0.2616	0.2516	0.3206	0.3658
9	0.1542	0.1584	0.2634	0.2314	0.2902	0.3432
11	0.1432	0.1788	0.2598	0.2232	0.2758	0.3456
13	0.1744	0.1958	0.2744	0.2572	0.2746	0.3208

MAPE: mean absolute percentage error; MSVR-SW: MSVR with similarity-related weight; MSVR: multiple-output support vector regression; SVR: support vector regression; BPNN: back-propagation neural network; MRA: multivariate regression analysis.

Figure 4.

Trends of adaptation accuracies of six comparative methods in (a) dataset I, (b) dataset II and (c) dataset III.

Table 5.

ANOVA test results for hold-out sample.

K	ANOVA test
	Dataset I		Dataset II		Dataset III
	Statistics F	p-Value	Statistics F	p-Value	Statistics F	p-Value
3	23.452	0.000*	21.456	0.000*	26.357	0.000*
5	22.465	0.000*	20.435	0.001*	27.345	0.000*
7	17.467	0.001*	18.456	0.000*	18.356	0.001*
9	8.563	0.000*	11.387	0.001*	10.346	0.000*
11	13.235	0.001*	12.456	0.000*	16.223	0.000*
13	25.678	0.000*	24.465	0.000*	30.567	0.000*

ANOVA: analysis of variance.

*

The mean difference among MSVR-SW, MSVR, SVR, BPNN, MRA and median is significant at the 0.05 level.

Table 6.

Comparison results with ranked comparative methods for hold-out sample.

K	Ranks of methods
	1		2		3		4		5		6
Dataset I
3	SVR	<	MSVR	<*	MSVR-SW	<*	BPNN	<*	Median	<	MRA
5	SVR	<*	MSVR-SW	<*	MSVR	<*	BPNN	<*	MRA	<*	Median
7	MSVR-SW	<*	SVR	<*	MSVR	<*	BPNN	<*	MRA	<*	Median
9	MSVR-SW	<*	MSVR	<	SVR	<*	BPNN	<*	MRA	<*	Median
11	MSVR-SW	<	MSVR	<*	SVR	<*	BPNN	<*	MRA	<*	Median
13	MSVR-SW	<	MSVR	<*	SVR	<	BPNN	<*	MRA	<*	Median
Dataset II
3	MSVR	<*	MSVR-SW	<	SVR	<*	BPNN	<*	Median	<	MRA
5	MSVR	<*	SVR	<	MSVR-SW	<*	BPNN	<*	MRA	<*	Median
7	MSVR	<*	MSVR-SW	<*	SVR	<*	BPNN	<*	MRA	<*	Median
9	MSVR-SW	<*	MSVR	<*	SVR	<*	BPNN	<*	MRA	<*	Median
11	MSVR-SW	<	MSVR	<*	SVR	<	BPNN	<*	MRA	<*	Median
13	MSVR-SW	<*	MSVR	<*	BPNN	<	SVR	<*	Median	<	MRA
Dataset III
3	MSVR	<*	MSVR-SW	<*	SVR	<*	BPNN	<*	MRA	<*	Median
5	MSVR	<*	MSVR-SW	<*	SVR	<	BPNN	<*	MRA	<*	Median
7	MSVR-SW	<	MSVR	<*	BPNN	<	SVR	<*	MRA	<*	Median
9	MSVR-SW	<*	MSVR	<*	BPNN	<*	SVR	<*	MRA	<*	Median
11	MSVR-SW	<*	MSVR	<*	BPNN	<*	SVR	<*	MRA	<*	Median
13	MSVR-SW	<*	MSVR	<*	BPNN	<*	SVR	<	MRA	<*	Median

SVR: support vector regression; MSVR: multiple-output support vector regression; MSVR-SW: MSVR with similarity-related weight; BPNN: back-propagation neural network; MRA: multivariate regression analysis.

*

The mean difference among MSVR-SW, MSVR, SVR, BPNN, MRA and median is significant at the 0.05 level.

Figure 5.

Required time of six adaptation methods.

Comparison of adaptation accuracy

When the comparative results of adaptation accuracy achieved by six methods are presented in Table 4, we can deduce the following observations. Overall, by increasing the number of problem and solution features, the performances of CBR adaptation based on machine learning techniques (i.e. MSVR-SW, MSVR, SVR and BPNN) also increase, because of more abundant data to be trained. Meanwhile, the adaptation methods based on SVRs (MSVR-SW, MSVR and SVR) outperform classical adaptation methods based on BPNN and statistic methods in across the three datasets and six K-NN principles, except for SVR and BPNN in dataset III. Considering the comparisons among MSVR-SW, MSVR and SVR, MSVR-SW is the best-performing method which ranks first in almost adaptation situations with different features and K values, followed by MSVR and SVR. SVR displays its superiority in dataset I with smaller K values; however, it is outperformed by MSVR-SW, MSVR and BPNN in datasets II and III. It is conceivable that the reason for the inferiority of the standard SVR-based adaptation in datasets II and III is that SVR ignores the possible mutual dependencies among solution values of design cases. Hence, MSVR-SW and MSVR are better than SVR in CBR adaptation with large number of problem and solution features and higher K values. It proves that multivariable CBR adaptation is an effective method to handle the complicated case adaptation task.

Figure 4 concludes the changes of adaptation accuracies of adaptation methods with different K values in three datasets. In all datasets, along with the increasing K, the performances of MSVR-SW, MSVR and SVR could increase. From Figure 4, we can also find out that the performances of MSVR-SW, MSVR and SVR tend to be stable and even decrease when the value of K exceeds 11 in all datasets. It means that the greater the K value, the more the training samples to be trained and the higher the probability of disturbed data existing in training samples. Thus, large K (> 11) may not improve the accuracy of adaptation method significantly. On the premise of making comprehensive consideration for the adaptation accuracy and computational cost, we prefer to selecting K = 9 or 11 as the favourable value for adaptation method in regular CBR adaptation operation.

Comparison of adaptation difference

To compare the adaptation differences that exist among the six methods, we performed the ANOVA procedure in this experiment. All of the ANOVA results listed in Table 5 are significant at the 0.05 level, which means that there are significant differences among MSVR-SW, MSVR, SVR, BPNN, MRA and median. To further identify the significant difference between any two comparative methods, we also used Tukey’s HSD test to compare all pairwise adaptation differences at the 0.05 level. Table 6 summarizes the results of Tukey’s HSD test, from which we can find that when MSVR-SW with K = 3, 5, 7 and 9 and MSVR with K = 11 and 13 in datasets are treated as the testing target, the mean difference between MSVR-SW and MSVR is significant at the 0.05 level (with the exception of the K = 11 and 13 in dataset I, K = 11 for MSVR in dataset II and K = 7 for MSVR-SW in dataset III), indicating that the MSVR-SW (K = 7, 9, 11 and 13) and MSVR (K = 3 and 5), respectively, perform the best in adaptation process under corresponding K-NN principles. Second, SVR yields better results than BPNN in dataset I, and the performance measures for SVR and BPNN are mixed in datasets II and III. BPNN outperforms SVR in dataset III when K values increase (with the exception of the K = 3 and 5).

Comparison of computation cost

It is important to note that the computational costs of six adaptation methods are different. From a practical viewpoint, the computational cost is an important and critical issue. Thus, the computational load of each method under K-NN principle was also compared in this study. Figure 5 shows the required time for case adaptation on the hold-out sample for a single replicate, from which we can deduce that the required time of all comparative methods could naturally increase when the number of inputs, outputs and retrieved cases increase, except for the required time of MRA and median with little change. Overall, comparing the machine learning methods (i.e. MSVR-SW, MSVR, SVR and BPNN) with the statistical methods, the statistical methods are less expensive.

SVR and BPNN, which use problem and solution feature-values directly and output only one adapted solution values, are computationally much more expensive than multivariable CBR adaptation methods (MSVR-SW and MSVR), and the differences of computational costs are enlarging with increasing number of inputs and outputs. When comparing MSVR-SW with MSVR on computational cost index, the MSVR-SW is the winner, because MSVR-SW adopts differential strategy to generate the training samples from K retrieved cases, and the number of input of MSVR-SW is smaller than MSVR’s.

Summation

According to the results of empirical comparisons, we can find that adaptation method using different number of inputs, outputs and retrieved cases could produce different adaptation results. In general, multivariable CBR adaptation with multiple output is more effective than univariate CBR adaptation with single output to handle the complicated case adaptation task from the views of the adaptation accuracy and computational cost. Furthermore, MSVR-SW improves the performance of MSVR-based adaptation by giving different weights to different training samples. Experiment results show that MSVR-SW is the best method among six comparative methods (MSVR-SW, MSVR, SVR, BPNN, MRA and median) across three datasets and six K-NNs, because of its higher adaptation accuracy and relative lower required time, and the reasonable K value can further increase its accuracy.

Conclusion

In this article, because of its inherent single-output structure, standard SVR is incapable of handling the possible interrelations among solution outputs in CBR adaptation under K-NN principle, which suffers from either low adaptation accuracy or expensive computational cost. Building a multiple-output model which contains these complex interrelations is a feasible alternative to SVR. As mentioned above, MSVR is an approach structured as multiple-input, multiple-output model, where the output value is not a scalar quantity but a vector of values. One of the advantages of MSVR for CBR adaptation is preserving the implicit stochastic dependency hidden in the adapted solutions. Therefore, in this article, we put forward a new multivariable case adaptation method by creatively employing MSVR model in the field of parametric mechanical design. Another innovation point of this article is considering the different contributions of different training samples for MSVR construction, and following the fact that training sample which contains two closer cases can provide more useful information than others. By taking the advantages of different SM algorithms, this article puts forward a similarity-related weight generation method, and gives more weights to the useful training samples. MSVR-SW model for multivariable CBR adaptation is built up by training these weighted training samples. Comparing with classical SVR-based adaptation, the new proposed MSVR-SW can not only output a vector of adapted solution values for new design problem but also improves the performance of multivariable CBR adaptation. Besides the theoretical descriptions, this article also gave an example to assess the feasibility of MSVR-SW in parametric mechanical design and carried out the quantitative and comprehensive comparisons between MSVR-SW and other methods in the three datasets to discuss the superiorities of MSVR-SW for case adaptation. According to the obtained results, the MSVR-SW is the promising techniques with high-quality adaptations and acceptable computational loads for multi-dimensional case adaptation.

In future, we intend to investigate the parameter optimization of MSVR-SW for CBR adaptation. For example, in this comparison experiment, we just performed the coarse grid search on $(\gamma ,\lambda )$ to find the initial optimized values, and a finer grid search with smaller grid size can be conducted on the neighbour of this initial values to identify better $(\gamma ,\lambda )$ , or we may try to use other more efficient optimization methods which have been successfully used in SVM. In addition, we think it is necessary to research the input feature selection approach in MSVR-SW, especially for the large number of design problem inputs, to further improve MSVR-SW’s adaptation qualities and reduce its computational loads. Another way to improve the efficiency of adaptation engine for large-scale design case is to improve the structure of MSVR. Ding and colleagues^59,61 introduced an excellent extension of SVM named as twin SVM (TWSVM), and TWSVM has been proven almost four times faster than standard SVM. Enlightened by that, we will try to apply TWSVM in CBR adaptation and build new multiple-output twin SVR model. Finally, in our previous study, ³⁷ we have put forward the concept the adaptability-related knowledge in standard SVR model, and we will try to incorporate this knowledge into MSVR model and study the feasibility of adaptability-involving MSVR for CBR adaptation.