On Popov-type stability criteria for neural networks

Daniela Danciu and Vladimir Răsvan^{∗}

Department of Automatic Control, University of Craiova, 13,A.I.Cuza Str., Craiova, RO-1100, Romania

                 e-mail: vrasvan@automation.ucv.ro

Abstract

This note presents some improvement of the stability criteria for continuous-time neural networks. It is taken into account that the nonlinear functions are bounded and slope restricted. This information allows application of some earlier results of Halanay and Rasvan (Int. J. Syst. Sci., 1991) on systems with slope restricted nonlinearities thus improving the results of Noldus et. al (Int. J. Syst. Sci., 1994). In this way a new frequency domain criterion for dichotomy and other qualitative behavior is obtained for a system with several equilibria.

AMS Classification: 93D30, 34D20, 34D25, 92B20 Keywords: several equilibria: qualitative behavior; frequency domain inequal-

ity.

This paper is in final form and no version of it will be submitted for publication elsewhere.

∗The final version of this paper has been elaborated during author’s stage as Associate Directeur de Recherche at HEUDIASYC(UMR CNRS 6599), Université de Technologie de Compiègne FRANCE

1 Introduction and problem statement.

State of the art

Neural networks are systems with several equilibrium states. It is exactly this fact (existence of several equilibria) that grants to the neural networks their computational and problem solving capabilities. We shall not insist more on these engineering facts connected with the point of view that a neural network is an associative memory.

The point of intersection of the (analogue) neural networks (modelled by differential equations) with the theory of dynamical systems and differential equations is Liapunov stability of the equilibria. According to the concise but meaningful description of Noldus et al[1],[2], when the neural network is used as a classification network, system’s equilibria constitute the ”prototype” vectors that characterize the different classes: the i-th class consists of those vectors x which, as an initial state for network’s dynamics, generate a trajectory converging to the i-th ”prototype” equilibrium state. When the network is used as an optimizer the equilibria represent optima.

It is stated in the cited paper that an essential operating condition for a neural network is that it must be nonoscillatory: each trajectory must converge to one of the equilibrium states. In fact the qualitative behavior of the neural networks as dynamical systems must be viewed within the framework of qualitative theory of systems with several equilibria. This theory starts from the paper of Moser[3] and has been developped in a comprehensive way by Yakubovich, Leonov and their co-workers[4],[5]. Interesting references in the field are also the papers of V.M.Popov[6],[7] and, in the context of integral and integro-differential equations, the publications of Corduneanu[8], Halanay[9], Nohel and Shea[10].

Some qualitative concepts are of interest:

1^{0} Dichotomy: all bounded solutions tend to the equilibrium set.

2^{0} Global asymptotics: all solutions tend to the equilibrium set.

3^{0} Gradient-like behavior: the set of equilibria is stable in the sense of Liapunov and any solution tends asymptotically to some equilibrium point.

It is dichotomy that signifies genuine nonoscillatory behavior: there may exist unbounded solutions but no oscillations are allowed. On the other hand it is the gradient-like behavior that represents the desirable behavior for neural networks. If the equilibria are isolated (and this is the case with the neural networks) then global asymptotics and gradient-like behavior are equivalent.

We shall consider here the problem of finding sufficient conditions for gradientlike behavior for the following model of neural networks[1],[2] :

ẋ = Ax − ∑_{1}^{m} b_{k}φ_{k}(c_{k}^{∗}x) − h,  (1)

where φ_{k}(σ) are differentiable, slope restricted and bounded. The boundedness condition is specific for sigmoidal (and other) nonlinearities of the neural networks.

It is a known fact that the main tool for studying the qualitative properties of the systems with several equilibria is the Liapunov function. The results of [1],[2] are based on a specifically designed Liapunov function whose coefficients are obtained by solving Lurie-type equations. Existence of solutions for such equations is ensured by a frequency domain inequality of Popov type but with a PI multiplier i.e. of the type 1 + β(iω)^{−1} instead of the usual PD multiplier 1 + β(iω) . The introduction of this multiplier has a long history that goes back to Yakubovich[11]; a long list of references is given in [12] but even this list is not complete : to mention only the papers of Noldus[13],[14].

The introduction of the PI multiplier in the multivariable case (with several nonlinear elements) requires some structure restrictions on the linear part. As shown in [12], a technical assumption allowing stability proof is that of static decoupling : c_{k}^{∗}A^{−1}b_{j} = 0 ∀k ≠ j . This assumption does not hold in the case of neural networks. On the other hand positivity of the Liapunov function is no longer necessary in the analysis of systems with several equilibria. In [1],[2] the background is given by the papers of La Salle [15],[16] with their ”generalized Liapunov function” (i.e. nonincreasing along the solutions but not necessarily of definite sign). Within this framework dichotomy follows almost immediately. Since the nonlinearities are bounded, boundedness of all solutions is obtained in a trivial way. This enables us to state that (1) has global asymptotics.

The fact that this stability criterion uses only slope information about the nonlinearities [12] does not allow to make use of the early results of Gelig [17] concerning systems with bounded nonlinearities. The conditions of Gelig upon the nonlinear functions, which are of the form

φ_{k}^{2}(σ) − \overline{φ}_{k}φ_{k}(σ)σ + \overline{φ}_{k}ϵσ < 0  (2)

do not seem of much help to handle the inequalities that may ensure gradient-like behavior for neural networks.

However, if we assume that the equilibria are isolated - a natural assumption for neural networks description - then global asymptotics will imply gradient-like behavior since any trajectory may not approach the stationary set otherwise than approaching some equilibrium point.

In what follows we shall take the approach of [12] but we shall not assume any longer that C^{∗}A^{−1}B is a diagonal matrix (here B is the matrix with b_{i} as columns and C is the matrix with c_{i} as columns).

2 Minimality, invariant set and equilibria

We shall assume the following:

i) detA ≠ 0 ;

ii) (A, B) is a controllable pair and (C^{∗}, A) is an observable pair i.e. (A, B, C) is any minimal realization of T (s) = C^{∗}(sI − a)^{−1}B , the matrix transfer function of the linear part of (1). Remark that the entries of T (s) are the transfer functions of various input/output channels γ_{kj}(s) = c_{k}^{∗}(sI − A)^{−1}b_{j}

iii) detC^{∗}A^{−1}B = detT (0) ≠ 0 ; this assumption ensures controllability of the pair

(A, B) = [ \begin{matrix}A & 0 \\ C^{∗} & 0 \end{matrix} ] , [ \begin{matrix}B \\ 0 \end{matrix} ]  (3)

provided ((A, B) is controllable;it will be useful in other development also.

Denoting

y = col(ξ_{1}, ..., ξ_{m}), ξ_{k} = c_{k}^{∗}x, f(y) = col(φ_{1}(ξ_{1}), ...φ_{m}(ξ_{m}))

then we have the following preliminary result:

Proposition 1 Let x(t) be a solution of (1). Then the pair (z(t), y(t)) defined by

\{ \begin{matrix}z(t) = Ax(t) − Bf(C^{∗}x(t)) − h \\ y(t) = C^{∗}x(t) \end{matrix}  (4)

is a solution of the system

\{ \begin{matrix}ż = Az − B(diag(φk^{′}(ξ_{k})))C^{∗}z \\ ẏ = C^{∗}z \end{matrix}  (5)

confined to the invariant set

y − C^{∗}A^{−1}z − (C^{∗}A^{−1}B)f(y) − C^{∗}A^{−1}h = 0  (6)

Conversely, if (z(t), y(t)) is a solution of (5) satisfying (6) then x(t) defined by

x(t) = A^{−1}(z(t) + Bf(y(t)) + h)  (7)

is a solution of (1)

The proof goes as in [12] and we shall omit it.

The equilibria of (1) are defined by

Ax − Bf(C^{∗}x) − h = 0  (8)

while the equilibria of (5) are defined by

Az − B(diag(φk^{′}(ξ_{k})))C^{∗}z = 0, C^{∗}z = 0  (9)

what gives z = 0 and y arbitrary. If system (5) is confined to the invariant set (6), its equilibria are of the form (0, \overline{y}) where \overline{y} satisfies

\overline{y} − (C^{∗}A^{−1}B)f(\overline{y}) − C^{∗}A^{−1}h = \overline{y} − T (0)f(\overline{y}) − C^{∗}A^{−1}h = 0  (10)

We may state the following result on equilibria, which is easy to prove.

Proposition 2 If \overline{x} is an equilibrium of (1) then (0, C^{∗}\overline{x}) is an equilibrium of (5) located on (6) i.e.satisfying (10) with \overline{y} = C^{∗}\overline{x} . Conversely, if (0, \overline{y}) is an equilibrium of (5) located on (6) i.e. satisfying (10), then \overline{x} defined by

\overline{x} = A^{−1}(Bf(\overline{y}) + h)  (11)

is an equilibrium of (1).

The proof is in fact straightforward and will be omitted. We shall assume, additionally, that these equilibria are isolated (a sufficient condition for this would be analyticity of the functions φ_{k}(σ)).

3 The Popov integral index and the frequency domain inequality.Main stability inequality.

To system (5) we shall associate the controlled linear system

\{ \begin{matrix}ż = Az + Bu(t) \\ ẏ = C^{∗}z \end{matrix}  (12)

together with the integral index

η(0, T) = ∑_{1}^{m} ∫_{0}^{T} [θ_{k}(μ_{k}(τ) + \underline{φ}_{k}c_{k}^{∗}z(τ))(μ_{k}(τ)/\overline{φ}_{k} + c_{k}^{∗}z(τ)) + q_{k}μ_{k}(τ)ξ_{k}(τ)]dτ

= ∫_{0}^{T} [u^{∗}(τ)Θ\overline{Φ}^{−1}u(τ) + \frac{1}{2}u^{∗}(τ)Θ(I + \underline{Φ}\overline{Φ}^{−1})C^{∗}z(τ)

+\frac{1}{2}z^{∗}(τ)C(I + \underline{Φ}\overline{Φ}^{−1})Θu(τ) + \frac{1}{2}u^{∗}(τ)Qy(τ) + \frac{1}{2}y^{∗}(τ)Qu(τ)

+z^{∗}(τ)CΘ\underline{Φ}C^{∗}z(τ)]dτ  (13)

where the diagonal matrices Θ, \underline{Φ}, \overline{Φ}, Q are defined by the (up to now) arbitrary constants θ_{k} ≥ 0, \underline{φ}_{k}, \overline{φ}_{k} > 0, q_{k} ≠ 0 :

Θ = diag(θ_{1}, ..., θ_{m}), \underline{Φ} = diag(\underline{φ}_{1}, ..., \underline{φ}_{m}) etc

Assume that

iv) The arbitrary constants are such that the following frequency domain inequality holds

Θ\overline{Φ}^{−1} + ℜ\{[Θ(I + \underline{Φ}\overline{Φ}^{−1}) + (iω)^{−1}Q]T (iω)\} + T^{∗}(−iω)Θ\underline{Φ}T (iω) ≥ 0 (14)

where ≥ 0 is understood in the sense of the quadratic forms.

This is exactly the frequency domain inequality for the Popov system (12)-(13). Since i) - iii) hold the system is controllable and we may use the Yakubovich-Kalman-Popov lemma in the controllable case. Along the line of [12] (but without the assumption that C^{∗}A^{−1}B is diagonal) we shall have existence of V = diag(γ_{1}, ..., γ_{m}), W, P such that

\{ \begin{matrix}V^{2} = Θ\overline{Φ}^{−1} \\ P B + W V = \frac{1}{2}C(I + \underline{Φ}\overline{Φ}^{−1})Θ \\ P A + A^{∗}P + W W^{∗} = CΘ\underline{Φ}C^{∗} − \frac{1}{2}CQ(C^{∗}A^{−1}B)^{−1}C^{∗}A^{−1}− \\ \frac{1}{2}(A^{∗})^{−1}C(B^{∗}(A^{∗})^{−1}C)^{−1}QC^{∗} \end{matrix}  (15)

In order to continue the proof along the line of [12] we shall consider that q_{k} are such that Q(C^{∗}A^{−1}B)^{−1} is symmetric. This condition - which had been considered in [12] as restrictive and replaced by the assumption that Q(C^{∗}A^{−1}B)^{−1} was diagonal - appears as necessary from the symmetry conditions of Yakubovich-Kalman-Popov lemma for system (12)-(13)(and also from the frequency domain inequality (14)); it is assumed as fulfilled in [1],[2](worth mentioning also that if \underline{Φ} = 0 then (14) is exactly the frequency domain inequality of [1],[2]).

If symmetry of Q(C^{∗}A^{−1}B)^{−1} holds then we may proceed as in [12] and find

η(0, T) = ∫_{0}^{T} |V u(τ) + W^{∗}z(τ)|^{2}dτ

+z^{∗}(T)P z(T) + y^{∗}(T)Q(C^{∗}A^{−1}B)^{−1}(C^{∗}A^{−1}z(T) − \frac{1}{2}y(T))

−z^{∗}(0)P z(0) − y^{∗}(0)Q(C^{∗}A^{−1}B)^{−1}(C^{∗}A^{−1}z(0) − \frac{1}{2}y(0)) (16)

where V, W, P are those of (15).

We may take then

u(t) = −(diag(φk^{′}(ξ_{k}(t))))C^{∗}z(t)  (17)

and proceed along the lines of [12] to obtain the following equality

∫_{0}^{T} | − V (diag(φk^{′}(ξ_{k}(τ))))C^{∗}z(τ) + W^{∗}z(τ)|^{2}dτ + z^{∗}(T)P z(T)

+\frac{1}{2}y^{∗}(T)Q(C^{∗}A^{−1}B)^{−1}(y(T) − 2C^{∗}A^{−1}h) − ∑_{1}^{m} q_{k} ∫_{\overline{ξ}_{k}}^{ξ_{k}(T)} φ_{k}(λ)dλ

= − ∑_{1}^{m} θ_{k} ∫_{0}^{T} (φk^{′}(ξ_{k}(τ)) − \underline{φ}_{k})(1 − φk^{′}(ξ_{k}(τ))/\overline{φ}_{k})(c_{k}^{∗}z(τ))^{2}dτ + z^{∗}(0)P z(0)

+\frac{1}{2}y^{∗}(0)Q(C^{∗}A^{−1}B)^{−1}(y(0) − 2C^{∗}A^{−1}h) − ∑_{1}^{m} q_{k} ∫_{\overline{ξ}_{k}}^{ξ_{k}(0)} φ_{k}(λ)dλ (18)

where \overline{ξ}_{k}, k = \overline{1, m}, are coordinates of some equilibrium point, more precisely, of \overline{y}, (0, \overline{y}) being the equilibrium.

Equality (18) is obtained by equating (16) with what is obtained from (13) with the choice of u(t) from (17); it is called main stability equality because it leads after some (tedious but straightforward) manipulation to a Liapunov function in the sense of La Salle.

4 The Liapunov function and its properties

The main stability equality (18) suggests the following candidate Liapunov function:

Ψ(z, y) = z^{∗}P z + \frac{1}{2}y^{∗}Q(C^{∗}A^{−1}B)^{−1}(y − 2C^{∗}A^{−1}h) − ∑_{1}^{m} q_{k} ∫_{\overline{ξ}_{k}}^{ξ_{k}} φ_{k}(λ)dλ

where \overline{ξ}_{k}, k = \overline{1, m}, are coordinates of some equilibrium point (0, \overline{y}) located on the invariant set hence satisfying

\overline{y} − (C^{∗}A^{−1}B)f(\overline{y}) − C^{∗}A^{−1}h = 0

Therefore

Ψ(z, y) = z^{∗}P z − \frac{1}{2}y^{∗}Q(C^{∗}A^{−1}B)^{−1}(y − \overline{y}) + \frac{1}{2}y^{∗}Q(C^{∗}A^{−1}B)^{−1}\overline{y} + y^{∗}Qf(\overline{y})

− ∑_{1}^{m} q_{k} ∫_{\overline{ξ}_{k}}^{ξ_{k}} φ_{k}(λ)dλ = z^{∗}P z − \frac{1}{2}(y − \overline{y})^{∗}Q(C^{∗}A^{−1}B)^{−1}(y − \overline{y})

+\frac{1}{2}\overline{y}^{∗}Q(C^{∗}A^{−1}B)^{−1}\overline{y} − ∑_{1}^{m} q_{k} ∫_{\overline{ξ}_{k}}^{ξ_{k}}(φ_{k}(λ) − φ_{k}(\overline{ξ}_{k}))dλ + ∑_{1}^{m} q_{k}φ_{k}(\overline{ξ}_{k})\overline{ξ}_{k}

and it is obvious that the Liapunov function is

V(z, y) = z^{∗}P z − \frac{1}{2}(y − \overline{y})^{∗}Q(C^{∗}A^{−1}B)^{−1}(y − \overline{y}) − ∫ y ; \overline{y}(f(v) − f(\overline{y}))^{∗}Qdv (19)

where the line integral is defined as

∫ y ; \overline{y}(f(v) − f(\overline{y}))^{∗}Qdv = ∑_{1}^{m} q_{k} ∫_{\overline{ξ}_{k}}^{ξ_{k}}(φ_{k}(λ) − φ_{k}(\overline{ξ}_{k}))dλ

Since equality (18) holds modulo any added constant we shall have

∫_{0}^{T} | − V (diag(φk^{′}(ξ_{k}(τ))))C^{∗}z(τ) + W^{∗}z(τ)|^{2}dτ + V(z(T), y(T)) =

− ∑_{1}^{m} θ_{k} ∫_{0}^{T} (φk^{′}(ξ_{k}(τ)) − \underline{φ}_{k})(1 − φk^{′}(ξ_{k}(τ))/\overline{φ}_{k})(c_{k}^{∗}z(τ))^{2}dτ + V(z(0), y(0)) (20)

Equality (20) shows that V(z(t), y(t)) is nonincreasing along the solutions of (5) that are confined to (6) hence along the solutions of (1).

As known (Lemma 2.3.1 from [4]) the system would be dichotomic if V(z(t), y(t)) would be constant only along those bounded solutions that are equilibria.

Assume that

\underline{φ}_{k} < φk^{′}(σ) < \overline{φ}_{k}, k = \overline{1, m}  (21)

It follows then from (20) that on the set where V(z(t), y(t)) is constant we have c_{k}z(t) ≡ 0 hence y(t) ≡ const and z(t) is a solution of ż = Az satisfying C^{∗}z(t) ≡ 0 ; from observability we deduce that z(t) ≡ 0 and this shows that V(z(t), y(t)) is constant on equilibria only. System (1) is thus dichotomic or non-oscillatory in the language of [1],[2]. We may thus state

Theorem 1 Consider system (1) under the assumptions i) − iii) of Section 2. If there exist the sets of parameters θ_{k} ≥ 0, \underline{φ}_{k}, \overline{φ}_{k} > 0, q_{k} ≠ 0, k = \overline{1, m}, such that (14) holds and Q(C^{∗}A^{−1}B)^{−1} is symmetric then system (1) is dichotomic for all slope restricted nonlinear functions satisfying (21). If additionally, all equilibria are isolated, then each bounded solution approaches an equilibrium point.

The last statement is easy to prove, while the simple reference to [15](see [1],[2]) is not enough. The argument is as follows: each bounded solution has a nonempty ω − limit set contained in the largest invariant set included in the set where V(z(t), y(t)) is constant. But this largest invariant set is composed of (isolated) equilibria only. It follows that the ω − limit set is of equilibria only and these equilibria are isolated. The ω − limit set being connected, it is in fact a single equilibrium point what proves the assertion.

5 The case of the bounded nonlinearities

We shall assume that (1) has a property of minimal stability i.e.it is internally stable for a linear function of the class: there exist the numbers φ̃_{i} ∈ (\underline{φ}_{i}, \overline{φ}_{i}) such that (A − ∑_{1}^{m} b_{i}φ̃_{i}c_{i}^{∗}) is a Hurwitz matrix.

Assume also that the following boundedness condition holds for the nonlinearities:

|φ_{k}(σ) − φ̃_{k}σ| ≤ m_{k} ≤ m  (22)

We may re-write (1) as follows:

ẋ = (A − ∑_{1}^{m} b_{i}φ̃_{i}c_{i}^{∗})x − ∑_{1}^{m} b_{i}(φ_{i}(c_{i}^{∗}x) − φ̃_{i}c_{i}^{∗}x) − h  (23)

Let U be the solution of the Liapunov equation

(A − ∑_{1}^{m} b_{i}φ̃_{i}c_{i}^{∗})^{∗}U + U(A − ∑_{1}^{m} b_{i}φ̃_{i}c_{i}^{∗}) = −I  (24)

and since (A−∑_{1}^{m} b_{i}φ̃_{i}c_{i}^{∗}) is a Hurwitz matrix, U > 0. Taking x^{∗}Ux as a Liapunov function for (23) and using (22) we obtain ultimate boundedness for the solutions of (23). Combining ultimate boundedness with boundedness of solutions in bounded sets of the state space, boundedness of all solutions of (1) is obtained. In fact we proved

Theorem 2 Consider system (1) under the assumptions of Theorem 1. Assume additionally that it is minimally stable, the nonlinear functions satisfy (22) and the equilibria are isolated. Then each solution of (1) approaches asymtotically an equilibrium state.

6 The case of the neural networks

As in [1],[2] we shall consider the case of the Hopfield-type classification networks described by

\frac{dv_{i}}{dt} = −\frac{1}{R_{i}C_{i}} v_{i} + \frac{1}{C_{i}} [∑_{1}^{n}(φ_{j}(v_{j}) − v_{i})/R_{ij} + I_{i}] i = \overline{1, n}  (25)

which is of the type (1) with

A = diag(−\frac{1}{C_{i}} (\frac{1}{R_{i}} + ∑_{j=1}^{n} \frac{1}{R_{ij}}))_{i=1}^{n}, f(v) = col(φ_{i}(v_{i}))_{i=1}^{n},

h = −col(I_{i}/C_{i})_{i=1}^{n}, C^{∗} = I, B = −ΓΛ, Γ = diag(1/C_{i})_{i=1}^{n}, Λ = (1/Rij)_{i,j=1}^{n}

Matrix Λ is the synaptic matrix of the neural network.

It is obvious that A is here a Hurwitz matrix since all physical parameters are positive; hence we may take φ̃_{i} = 0 . We have also

T (s) = I(sI − A)^{−1}B = −(sI − A)^{−1}ΓΛ

= −(diag((sC_{i} + 1/R_{i} + ∑_{j=1}^{n}(1/R_{ij}))^{−1})_{i=1}^{n})Λ

It is easy to check that for usual nonlinear functions of the neural networks - various sigmoidal functions - we have \underline{φ}_{k} = 0, 0 < \overline{φ}_{k} < +∞ .By choosing

θ_{k} = C_{k}, q_{k} = 1/R_{k} + ∑_{j=1}^{n}(1/R_{kj})  (26)

the frequency domain inequality (14) holds provided Λ = Λ^{∗} i.e.the synaptic matrix is symmetric. This condition, mentioned also in [1],[2] is quite known in the stability studies for neural networks and it is a normal design condition since the choice of the synaptic parameters is controlled by network adjustment in the process of ”learning”.

7 Some conclusions and open problems

The results of this paper allow an embedding of neural network stability analysis in the more general framework of qualitative theory of systems with several equilibrium points. The frequency domain criterion is easy to manipulate in applications but requires a symmetry assumption that is desirable to relax in order to obtain other stability criteria. This goal is achievable by a suitable choice of the information about nonlinearities. It is known that these functions are monotone and slope restricted. All criteria obtained for such functions(e.g.those of Yakubovich, Brockett and Willems) may show useful in this analysis. Moreover, the associated Liapunov functions may allow establishing new qualitative behavior in relaxed assumptions over the system. Remark that these Liapunov functions may be quite different in comparison with usual energy function of the neural networks.

References

[1] E.Noldus, R.Vingerhoeds and M.Loccufier, ”Stability of analogue neural classification networks”, Int.Journ.Systems.Sci. 25, no.1, pp.19-31, 1994.

[2] E.Noldus and M.Loccufier, ”An application of Liapunov’s method for the analysis of neural networks”, Journ.of Comp.and Appl.Math. 50, pp.425-32, 1995.

[3] J.Moser, ”On nonoscillating networks”, Quart.Appl.Math. 25, pp.1-9, 1967.

[4] A.Kh.Gelig, G.A.Leonov and V.A.Yakubovich, Stability of nonlinear systems with non-unique equilibrium state (in Russian), Moscow, Nauka Publ.House, 1978.

[5] G.A.Leonov, V.Reitmann and V.B.Smirnova, Pendulum-like feedback systems, Teubner Verlag, Leipzig, 1991.

[6] V.M.Popov, ”Monotonicity and Mutability”, Journ.of Differ.Equations 31, no.3, pp.337-358, March 1979.

[7] V.M.Popov, ”Monotone-Gradient Systems”, Journ.of Differ.Equations 41, no.2, pp.245-261, August 1981.

[8] C.Corduneanu, Integral Equations and Stability of Feedback Systems, Academic Press, N.Y., 1973.

[9] A.Halanay, ”On the asymptotic behavior of the solutions of an integrodifferential equation”, Journ.Math.Anal.Appl. 10, pp.319-324, 1965.

[10] J.A.Nohel and D.F.Shea, ”Frequency domain methods for Volterra equations”, Adv.in Mathematics 22, pp.278-304, 1976.

[11] V.A.Yakubovich, ”Frequency domain conditions for the absolute stability of nonlinear control systems”, in Proc.Inter-University Conference on Applied stability theory and analytical mechanics (in Russian), Kazan Aviation Inst.Publ.House, Kazan, 1962

[12] A.Halanay and Vl.Răsvan, ”Absolute stability of feedback systems with several differentiable non-linearities”, Int.Journ.Systems.Sci. 22, no.10, pp.1911-1927, 1991.

[13] E.Noldus, ”An absolute stability criterion”, Appl.Sci.Res. 21, p.218, 1969.

[14] E.Noldus, A.Galle and L.Josson, ”The computation of stability regions for systems with many singular points”, Int.Journ.Control 17, pp.641-652, 1973.

[15] J.P.La Salle, ”An invariance principle in the theory of stability”, in Differential Equations and Dynamical Systems (J.P.La Salle and J.K.Hale editors), Acad.Press, N.Y., pp.277-286, 1967.

[16] J.P.La Salle, ”Stability Theory for Ordinary Differential Equations”, Journ.of Differ.Equations 4, no.1, pp.57-65, January 1968.

[17] A.Kh.Gelig, ”Stability of controlled systems with bounded nonlinearities”,Autom. Remote Control 1968, pp.1724-1731.