Weighted linear loss twin support vector machine for large-scale classificationKnowledge-Based Systems

About

Authors
Yuan-Hai Shao, Wei-Jie Chen, Zhen Wang, Chun-Na Li, Nai-Yang Deng
Year
2015
DOI
10.1016/j.knosys.2014.10.011
Subject
Artificial Intelligence / Information Systems and Management / Software / Management Information Systems

Similar

Imbalanced data classification based on scaling kernel-based support vector machine

Authors:
Yong Zhang, Panpan Fu, Wenzhe Liu, Guolong Chen
2014

Bi-level stochastic gradient for large scale support vector machine

Authors:
Nicolas Couellan, Wenjuan Wang
2015

A Novel Twin Support Vector Machines with Manifold Regularization

Authors:
Xiao Peng Hua, Xian Feng Li
2014

Selecting marker genes for cancer classification using supervised weighted kernel clustering and the support vector machine

Authors:
Jooyong Shim, Insuk Sohn, Sujong Kim, Jae Won Lee, Paul E. Green, Changha Hwang
2009

Automated classification of protein crystallization images using support vector machines with scale-invariant texture and Gabor features

Authors:
Shen Pan, Gidon Shavit, Marta Penas-Centeno, Dong-Hui Xu, Linda Shapiro, Richard Ladner, Eve Riskin, Wim Hol, Deirdre Meldrum
2006

Text

m-N a

Keywords: e a tw ds t aliz time.  2014 Elsevier B.V. All rights reserved. nes (S reach 20,15,4

It should be mentioned that in the second way there is an interesting approach where two non-parallel hyperplanes are constructed, rather than constructing two parallel supporting hyperplanes in traditional SVMs. It goes back to generalized eigenvalue proximal support vector machine (GEPSVM) [16] which needs to solve generalized eigenvalue problems. Subsequently, the twin support vector machine (TWSVM) [11] is proposed. t vector m d linear lo

WSVM, the version of our WLTSVM constructs two non-parallel hype such that each hyperplane is proximal to one class and as possible from the other class. However, different from T in the linear version of our WLTSVM, a weighted linear loss function is introduced. The main cost of our WLTSVM is solving two systems of linear equations that are much simpler than that of

TWSVM, where two QPPs are needed to be solved. Besides, distinct from LSTSVM, our WLTSVM keeps the more reasonable constraint ‘‘the other class as far as possible from the hyperplane’’ in TWSVM.

In fact, theoretical analysis shows that our WLTSVM not only ⇑ Corresponding author. Tel./fax: +86 057187313551.

E-mail address: dengnaiyang@cau.edu.cn (N.-Y. Deng).

Knowledge-Based Systems 73 (2015) 276–288

Contents lists availab a .e lestablishing new model and finding simpler problem to replace the QPP, e.g. Proximal SVM [9] and Least Squares SVM [39,38], where the QPP is replaced by a linear system of equations since the squared loss function instead of the hinge one is introduced. also be weakened [18].

In this paper, we propose a twin-type suppor with weighted linear loss function, called weighte support vector machine (WLTSVM). Following Thttp://dx.doi.org/10.1016/j.knosys.2014.10.011 0950-7051/ 2014 Elsevier B.V. All rights reserved.achine ss twin linear rplanes far as

WSVM,training stage involves solving a quadratic programming problem (QPP) with rather high computational complexity Oðm3Þ, where m is the total size of training data points. This drawback restricts the application of SVM to large-scale problems. There are two ways to address this problem. One aims at solving the QPP in the traditional SVMs more efficiently, e.g. Chunking [42], SMO [24], SVMLight [12], LIBSVM [4], and LIBLINEAR [8]. The other one aims at in TWSVM, leading to very fast training speed since two QPPs are replaced by two systems of linear equations. However, it has been pointed out in [1] that LSTSVM relaxes the constraint ‘‘the other class as far as possible from the hyperplane’’ to ‘‘the other class has a distance from the hyperplane’’, which may result in the reduction of classification ability, and meanwhile, the characteristic of constructing two non-parallel hyperplanes in TWSVM mayPattern recognition

Support vector machines

Twin support vector machines

Large-scale classification

Weighted linear loss function 1. Introduction

Traditional support vector machi

SVC [3] and m-SVC [32], have already in supervised machine learning [VMs) [5,3,6] such as Ced many achievements 5,43]. However, their

Different from GEPSVM, TWSVM solves two small related QPPs.

Due to its strong generalization ability [13,33], TWSVM and its variants have been studied extensively [25,23,18,41,40,28,29,27].

Specifically, least squares type TWSVM (LSTSVM) [1] has been presented by using the squared loss function instead of the hinge oneout any extra external optimizers. The experimental results on several benchmark datasets indicate that, comparing to TWSVM, our WLTSVM has comparable classification accuracy but with less computationalWeighted linear loss twin support vector classification

Yuan-Hai Shao a, Wei-Jie Chen a, Zhen Wang b, Chun a Zhijiang College, Zhejiang University of Technology, Hangzhou 310024, PR China b School of Mathematical Sciences, Inner Mongolia University, Hohehot 010021, PR Chin cCollege of Science China Agricultural University, Beijing 100083, PR China a r t i c l e i n f o

Article history:

Received 7 February 2014

Received in revised form 8 September 2014

Accepted 11 October 2014

Available online 18 October 2014 a b s t r a c t

In this paper, we formulat called weighted linear loss loss, our WLTSVM only nee while, maintains the gener

Knowledge-B journal homepage: wwwachine for large-scale a Li a, Nai-Yang Deng c,⇑ twin-type support vector machine for large-scale classification problems, in support vector machine (WLTSVM). By introducing the weighted linear o solve simple linear equations with lower computational cost, and meanation ability. So, it is able to deal with large-scale problems efficiently withle at ScienceDirect sed Systems sevier .com/locate /knosys

SVM can be expressed as asedmin w;b;n1 ;n2 1 2 jjwjj2 þ C e>1 n1 þ e>2 n2   s:t: Awþ e1bP e1  n1; n1 P 0;  ðBwþ e2bÞP e2  n2; n2 P 0; ð3Þ where C > 0 is a parameter. Note that the minimization of the regularization term 12 kwk2 is equivalent to the maximization of the margin between two parallel supporting hyperplanes w>xþ b ¼ 1 and w>xþ b ¼ 1, and the structural risk minimization principle is implemented in this problem. Fig. 1(a) shows the geometric interpretation of this formulation for a toy example. After we obtain the optimal solution of (3), a new data point is classified as þ1 or 1 according to whether the decision function, Classi ¼ sgnðw>xþ bÞ, yields 1 or 1 respectively. 2.2. Least squares SVMmaintains the merits of the TWSVM but also has lower computational cost. Furthermore, the two systems of linear equations in our WLTSVM can be solved efficiently by using the well-known conjugate gradient algorithm, resulting in the ability to deal with large-scale datasets without any extra external optimizers. It should be pointed out that our WLTSVM including its linear version and nonlinear version have been extended to multiple classification problems. Comparing to TWSVM [11,34], LSTSVM [1],

NHSVM [35], GEPSVM [16], SVM, and LSSVM, the preliminary numerical experiments on several benchmark datasets show that our WLTSVM gains comparable generalization ability but with remarkable less training time.