Model-Tree Ensembles for noise-tolerant system identificationAdvanced Engineering Informatics


Darko Aleksovski, Juš Kocijan, Sašo Džeroski
Artificial Intelligence / Information Systems


Model Tree Ensembles for the Identification of Multiple-Output Systems

Darko Aleksovski, Jus Kocijan, Sasso Dzeroski

Modeling an ensemble of systems coupled by the Lennard-Jones interaction

O. I. Gorskii, Yu. P. Kuchugurnyi

Fault tolerant system performance modeling


A formal approach to fault tree synthesis for the analysis of distributed fault tolerant systems

Mark L. McKelvin, Gabriel Eirea, Claudio Pinello, Sri Kanajan, Alberto L. Sangiovanni-Vincentelli


s ,c na,


Received in revised form 9 June 2014

Accepted 26 July 2014

Available online xxxx


Decision tree ensemble

Fuzzified model tree

Nonlinear dynamic system identification discrete-time variant of this task is commonly reformulated as a regression problem. As tree ensembles have proven to be a successful predictive modeling approach, we investigate the use of tree ensembles for put) error.

We explore the use of tree ensemble methods for regression for modeling dynamic systems from measured data. We propose a r shortcom gression t

The sectio cludes by laying out the aims of this paper and giving an of the remainder of the paper. 1.1. Discrete-time modeling of dynamic systems

The task of discrete-time modeling of nonlinear dynamic systems using measured input–output data is to find difference (recurrence) equations using the input variable (u) and output and system variable (y). These equations describe the system at a q Handled by C.-H. Chen. ⇑ Corresponding author at: Department of Knowledge Technologies, Jozˇef Stefan

Institute, Jamova cesta 39, Ljubljana, Slovenia.

E-mail addresses: (D. Aleksovski), (J. Kocijan), (S. Dzˇeroski).

Advanced Engineering Informatics xxx (2014) xxx–xxx

Contents lists availab

Advanced Enginee journal homepage: www.and Gaussian process regression. While most approaches to solving this task try to minimize the one-step prediction error, the learned models are typically evaluated in terms of their simulation (outapproaches to solving this task and some of thei next discusses tree ensemble approaches for re plan to use for overcoming these deficiencies. 1474-0346/ 2014 Elsevier Ltd. All rights reserved.

Please cite this article in press as: D. Aleksovski et al., Model-Tree Ensembles for noise-tolerant system identification, Adv. Eng. Informat. (2014), It hat we n conoutlineIn this paper, we address the task of identification of nonlinear dynamic systems from measured input–output data. In particular, we address the discrete-time variant of this task, which can be transformed into a regression problem of predicting the next state/output of the system from states and inputs in the recent past. Different regression approaches have been used for solving this task, including neural networks, support vector machines domized attribute selection and fuzzified splits. The approach includes an optimization step of ensemble pruning, which is based on the simulation (output) error criterion. We evaluate the performance of our Model-Tree Ensembles, comparing them to existing state-of-the-art methods used for system identification, focusing on their performance on noisy identification data.

The remainder of this section first introduces the task of discrete-time modeling of dynamic systems. It then discusses existing1. Introductionsolving the regression problem. While different variants of tree ensembles have been proposed and used, they are mostly limited to using regression trees as base models. We introduce ensembles of fuzzified model trees with split attribute randomization and evaluate them for nonlinear dynamic system identification.

Models of dynamic systems which are built for control purposes are usually evaluated by a more stringent evaluation procedure using the output, i.e., simulation error. Taking this into account, we perform ensemble pruning to optimize the output error of the tree ensemble models. The proposed Model-Tree

Ensemble method is empirically evaluated by using input–output data disturbed by noise. It is compared to representative state-of-the-art approaches, on one synthetic dataset with artificially introduced noise and one real-world noisy data set. The evaluation shows that the method is suitable for modeling dynamic systems and produces models with comparable output error performance to the other approaches. Also, the method is resilient to noise, as its performance does not deteriorate even when up to 20% of noise is added.  2014 Elsevier Ltd. All rights reserved. novel approach for learning ensembles of model trees with ran-Article history:

Received 29 August 2013

This paper addresses the task of identification of nonlinear dynamic systems from measured data. TheModel-Tree Ensembles for noise-tolerant

Darko Aleksovski a,c,⇑, Juš Kocijan b,d, Sašo Dzˇeroski a aDepartment of Knowledge Technologies, Jozˇef Stefan Institute, Jamova cesta 39, Ljublja bDepartment of Systems and Control, Jozˇef Stefan Institute, Jamova cesta 39, Ljubljana, c Jozˇef Stefan International Postgraduate School, Jamova cesta 39, Ljubljana, Slovenia dUniversity of Nova Gorica, Vipavska 13, Nova Gorica, Slovenia a r t i c l e i n f o a b s t r a c tystem identificationq

Slovenia eniale at ScienceDirect ring Informatics elsevier .com/ locate/ae ihttp:// time instant k using past values of the input and output variables.

Through the external dynamics approach [1] the modeling problem is reformulated as a regression task. The value of the system variable(s) at time instant k; yðkÞ, needs to be predicted from the lagged values of the input and system variable(s), uðk 1Þ;uðk 2Þ; . . . ;uðk nÞ; yðk 1Þ; yðk 2Þ; . . . ; yðk nÞ using a static function approximator.

The evaluation of the performance of a dynamic system’s model is carried out according to the purpose of the model and often requires a stringent and purpose-specific evaluation. When evaluating a model using one-step-ahead prediction, as shown in

Machines [5]. both subproblems at once [1]. Out of the plethora of possible solu2 D. Aleksovski et al. / Advanced EngineeriFig. 1(a), the predicted values for the system variable are compared to the measured values. On the other hand, the procedure of simulation, illustrated in Fig. 1(b), introduces one substantial difference: the one-step-ahead model predictions are fed back to the model to produce predictions for the more distant future.