Документ взят из кэша поисковой машины. Адрес оригинального документа : http://acat02.sinp.msu.ru/presentations/shibzoukhov/acat2002_spnn.pdf
Дата изменения: Tue Jul 16 12:22:19 2002
Дата индексирования: Mon Oct 1 20:33:40 2012
Кодировка:
VIII International Workshop on Advanced Computing and Analysis Techniques in Physics Research ACAT '2002, June, 24­28, 2002, Moscow

CONSTRUCTIVE METHOD FOR SUPERVISED LEARNING WITH COMPLEXITY MINIMIZATION OF ARTIFICIAL NEURAL NETWORKS OF ALGEBRAIC -NEURONS Shibzoukhov Z.M. szport@fromru.com RESEARCH INSTITUTE OF APPLIED MATHEMATICS AND AUTOMATION, KBSC RAS, NALCHIK

1


ALGEBRAIC -NEURON xn x1
.....



i1

..... . . .

w1

............

i

N

.....

wN
out

y
Let K be a ring without of denominators of zero. An algebraic -neuron implement composition of a polylinear function sp(x1, . . . , xn) and a scalar function out(s): spn(x1, . . . , xn) = out sp(x1, . . . , xn) ,
N

sp(x1, . . . , xn) = +
k =1

wk
iik

xi ,

and x1, . . . , x output, coefficient, ik

n are inputs from K, y K be an

K be a bias, wk K be a weight {1, ..., n} be a multiindex (i = ).
2


ZERO-ORDERED SEQUENCES OF SPARSE VECTORS Let X = {xk } be a sequence of sparse vectors from Kn, I = {ik } be a sequence of multiindices. Definition. X is zero-ordered with respect to I if for any pair of j < k there exists t = t(j, k ) ik such that xj t = 0 and xkt = 0. We say X is zero-ordered if ik = {1, . . . , n} for all k . Let for any x K 1, if x = 0, x= 0, if x = 0 and x = (x1, . . . , xn) for any x Kn. Let X = {xk }. Lemma. Any sequence X Kn can be zero-ordered iff X not contain identical elements. For any zero-ordered sequence X we denote by i(xk ) a multiindex {i : xki = 0}. Lemma. For any zero-ordered sequence X there exists a sequence I (ik i(xk ) for all k ) such that X is zeroordered with respect to I.

3


TRIANGULAR SEQUENCES OF PRODUCTS Let's denote by p(x; i) a following product p(x; i) =
ii

xi .

For any I let's define a sequence of products P(I) = {p(x; ik )}. Definition. P(I) is triangular on X if the following conditions are true: 1) for any pair of j < k the equality p(xj ; ik ) = 0 is true; 2) for all k the inequality p(xk ; ik ) = 0 is true too. We further suppose that ik i(xk ) for all k . Lemma. P(I) is triangular on X iff X is zero-ordered with respect to I. Let X be zero-ordered with respect to I. We say that that multiindex ik is minimal with respect to zero-ordering of X if exclusion of any i ik make broken this property of X. Also we say that a product p(x; ik ) is minimal with respect of triangularity P(I) on X if exclusion of any factor xi from p(x; ik ) make broken triangularity of P(I). Definition. We say that sequence I is minimal with respect to zero-ordering of X if any multiindex ik I
4


is minimal with respect to zero-ordering. We also say that P(I) is minimal with respect to triangularity on X if any product p(x; ik ) is minimal. So we obtain a statement. Lemma. P(I) is minimal with respect to triangularity on X iff I is minimal with respect to zero-ordering of X.

5


RECURRENT SEQUENCES OF -NEURONS Let SP = {spk (x)} be a sequence of ­forms. Definition. SP be a recurrent sequence of ­ forms if for any k spk (x) = spk-1(x) + wk p(x; ik ), where wk K, ik be a mutiindex, sp0(x) be any ­form. Let's suppose that for any s K, 0 = p K, y D(out) there exists a solution w(s, p, y ) of equation out(s + pw) = y . Let 0, if out(sk ) = yk wk = w(sk , pk , yk ), if out(sk ) = yk , where sk = spk-1(xk ), pk = p(xk ; ik ). Let X be zero-ordered with respect to I. For any k an expression X[a, b] denotes a subsequence {xj : j = a, ..., b}. Under these suppositions the following statement is true. Proposition. For any k -th element of recurrent sequence SP the equivalence y (x) out(spk (x)) is true on X[1, k ].
6


CONSTRUCTIVE LEARNING OF SEQUENTIAL CHAIN OF LINKED -NEURONS xn x1
.....

1 2

.....

u1 u2

.....

...................................

L

.....

uL
x1
.....

x1
.....

xn NNp-1 u1 u
p-1

xn NNp-1 u1 NN
p

up-1
.....

up

Let's consider a following sequential chain SPN of linked -neurons: u1 = spn1(x), up = spnp(x, up-1),
7


where p = 2, . . . , L, L is length of the chain. For any p let's define a trasformation function SPNp(x): SPN1(x) = spn1(x), SPNp(x) = spnp(x; SPNp-1(x)). Let sp0(u) be a -form such that out(sp0(u)) u on D(out). Proposition. For any sequence 0 < N1 < · · · < NL = N there exists a chain SPN such that y (x) SPNp(x) on X[1, Np]. We learn -neurons spn1, . . . , spnL sequentially; p-th -neuron spnp learns using construction of the recurrent sequences of -forms spp0, spp1, . . ., where for any p > 1 we put spp,0(x; up-1) = sp0(up-1).

8


CONSTRUCTIVE LEARNING MULTY-LAYERED NETWORK WITH FIXED WIDTH OF -NEURONS Let's denote by spn(x) a tuple of -neurons: spn(x) = (spn1(x), . . . , spnm(x)). Let's consider a following multy-layered network SPNN of -neurons: u1 = spn1(x), up = spnp(x, up-1), where p = 2, . . . , L, L is length of the chain, up = (up1, . . . , upm). For any p let's define a trasformation function SPNNp(x):

SPNN1(x) = spn1(x), SPNNp(x) = spnp(x; SPNNp-1(x)). Let sp0(u) be a -form such that out(sp0(u)) u on D(out). Proposition. For any sequence 0 < N1 < · · · < NL = N there exists a multy-layered network SPNN such that y (x) SPNNp(x) on X[1, Np]. We learn layers sequentialy, -neurons spn1, . . . , sp sequentially; all -neurons of p-th tuple spnp learns using construction of the recurrent sequences
9


of -forms. For any p > 1 we for j -th -neuron spnpj we put sppj,0(x; up-1) = sp0j (up-1,j ).

10


CONSTRUCTIVE LEARNING OF SEQUENTIAL CHAIN OF LINKED -NEURONS WITH A FIXED DEPTH
x1
.....

xn

1 2

.....

y1 y2 yl

.....

................................... -1

l

.....

.....

yl yp-l+1

...................................

p-l+2

.....

yp-l+2 yp-1 yp

....................................................

p

.....

.......

......................................................................

yL-1
L ..... .......

y

L

11


Let's consider a following sequential chain SPN of linked -neurons: u1 = spn1(x), uq = spn1(x; u1, . . . , uq -1), up = spnp(x, up- , . . . , up-1), where q = 2, . . . , , p = + 1, . . . , L, L is length of the chain. For any p let's define a trasformation function SPNp(x): SPN1(x) = spn1(x), SPNq (x) = spnq (x; SPN1(x), . . . , SPNq -1(x)), SPNp(x) = spnp(x; SPNp- (x), . . . , SPNp-1(x)). Let sp0(u) be a -form such that out(sp0(u)) u on D(out). Proposition. For any sequence 0 < N1 < · · · < NL = N there exists a chain SPN such that y (x) SPNp(x) on X[1, Np]. We learn any -neuron spnp using construction of the recurrent sequences of -forms spp0, spp1, . . ., where for any p > 1 we put spp,0(x; up-1) = sp0(up-1).

12