Automatica 141 (2022) 110317

Contents lists available at ScienceDirect

Automatica
journal homepage: www.elsevier.com/locate/automatica

Brief paper

Generalized Nash equilibrium seeking algorithm design for distributed
constrained noncooperative games with second-order players✩
∗

Zhenhua Deng , Yangyang Liu, Tao Chen

∗

School of Automation, Central South University, Changsha, 410075, China

article

info

Article history:
Received 26 June 2021
Received in revised form 14 February 2022
Accepted 3 March 2022
Available online 23 April 2022
Keywords:
Second-order multi-agent systems
Distributed algorithms
Noncooperative games
Generalized Nash equilibrium
Inequality constraints

a b s t r a c t
In this paper, we study the noncooperative games of multi-agent systems. Different from the wellknown noncooperative games, our problem involves not only the coupling inequality constraints and
the local inequality constraints of decisions, but also the second-order dynamics of players. Due to the
second-order dynamics and the inequality constraints, existing generalized Nash equilibrium seeking
algorithms for noncooperative games cannot solve our problem. Besides, the second-order dynamics
together with the inequality constraints give rise to the difficulties in distributed algorithm design
and analysis. In order to seek the variational generalized Nash equilibrium of the games, we design a
distributed algorithm based on gradient descent, state feedback and projection operations. Moreover,
we analyze the asymptotic convergence of the algorithm via variational analysis and Lyapunov stability
theory. Finally, two examples verify the effectiveness of the algorithm.
© 2022 Elsevier Ltd. All rights reserved.

1. Introduction
Noncooperative games are the essential problems in various
fields, such as smart grids, communication networks, economic
markets and social networks (see Esmalifalak, Shi, Han, & Song,
2013; Ghaderi & Srikant, 2014; Lou, Hong, Xie, Shi, & Johansson,
2016; Yin, Shanbhag, & Mehta, 2011). In noncooperative games,
players compete with each other to selfishly minimize their private cost functions, which depend on the decisions of all players
(see Fang, Wen, Huang, Fu, & Hu, 2020; Frihauf, Krstić, & Başar,
2011; Gadjov & Pavel, 2019; Li, Li, & Ding, 2020). Consequently,
the players want to search the generalized Nash equilibrium
(GNE) of noncooperative games.
Recently, many efforts have been made to seek the GNE of
noncooperative games. For instance, for unconstrained noncooperative games, Ye and Hu (2017) proposed a leader-following
consensus-based algorithm. For noncooperative games with inequality constraints, Zeng, Chen, Liang, and Hong (2019) developed a primal–dual-based algorithm. For noncooperative games
with convex set constraints, De Persis and Grammatico (2019)
✩ This work was supported by the National Natural Science Foundation
of China under Grant 61803385 and the Hunan Provincial Natural Science
Foundation of China under Grant 2019JJ50754. The material in this paper was
not presented at any conference. This paper was recommended for publication in
revised form by Associate Editor Kostas Margellos under the direction of Editor
Christos G. Cassandras.
∗ Corresponding authors.
E-mail addresses: zhdeng@amss.ac.cn (Z. Deng), yyangliu@csu.edu.cn
(Y. Liu), ct2046@csu.edu.cn (T. Chen).
https://doi.org/10.1016/j.automatica.2022.110317
0005-1098/© 2022 Elsevier Ltd. All rights reserved.

designed a projection-based algorithm. For differentiable noncooperative games, Tatarenko, Shi, and Nedić (2021) provided a
gradient-based algorithm. For nonsmooth noncooperative games,
Deng (2021b) presented a differential inclusion-based algorithm.
For noncooperative games with unknown cost functions, Liu and
Krstić (2011) gave an extremum seeking-based algorithm. For
noncooperative games over directed graphs, Zhu, Yu, Wen, and
Chen (2021) exploited an average consensus-based algorithm.
For noncooperative games under persistent attacks, Wang, Sun,
Teel, and Liu (2020) presented a robust seeking algorithm. For
noncooperative games with external disturbances, Romano and
Pavel (2019) designed an internal model-based algorithm.
In plenty of engineering practices, there are multifarious constraints on the decisions of players, due to production capacity,
execution capacity and/or security consideration. For example,
in power systems, the output powers of generators are constrained by upper and lower bounds (see Mudumbai, Dasgupta,
& Cho, 2012). In wireless networks, the quality of service are
constrained by coupling network resource constraints (see Tan,
Zhu, Ge, & Xiong, 2015). Hence, the players may be subject to
local and coupling constraints in noncooperative games. On the
other hand, projection-based methods are often used to deal with
distributed constrained optimization problems and constrained
noncooperative games. For instance, Yang, Liu, and Wang (2017)
utilized projection methods to deal with distributed optimization
problems with bounded constraints, and Deng and Nian (2018)
employed projection operators to solve noncooperative games
with convex set constraints.
Cyber–physical systems develop rapidly in the past decades,
and appear commonly in widespread engineering applications,

Z. Deng, Y. Liu and T. Chen

Automatica 141 (2022) 110317

a vector. Define x ◦ y := col(x1 y1 , . . . , xq yq ) for x, y ∈ Rq . ∥x∥
denotes the standard Euclidean norm of vector x. For a matrix A,
∥A∥ is its spectral norm, ϑmin (A) is its smallest eigenvalue, and
AT is its transpose. In is a n × n identity matrix. ∇ J(·) denotes the
gradient of a function J(·). Define diag {x1 , x2 , . . . , xn } as a diagonal
matrix with diagonal elements x1 , x2 , . . . , xn .

such as transportation systems and power networks (see Kim &
Kumar, 2012; Zhang, Papachristodoulou, & Li, 2018). Under the
development of cyber–physical systems, increasing attention has
been paid to distributed algorithms united with the dynamics
of physical systems, for the purpose that physical systems are
used to accomplish distributed tasks autonomously. For example, Deng and Liang (2019), Huang, Zou, and Meng (2021) and
Zhang, Liang, Wang, and Ji (2020) developed distributed Nash
equilibrium algorithms for first-order and second-order nonlinear
multi-agent systems, respectively. Deng (2021a), Tran, Wang, Liu,
Xiao, and Lei (2019) and Yi, Yao, Yang, George, and Johansson
(2018) designed distributed optimization algorithms for secondorder linear multi-agent systems. It is noteworthy that many
physical systems, such as torque motors and gas jets (see Yang,
Zhang, & Zhang, 2011), have second-order dynamics. Nevertheless, to the best of our knowledge, there are no results about
noncooperative games of second-order systems with local inequality and coupling inequality constraints. Moreover, existing
Nash equilibrium seeking algorithms for noncooperative games,
such as De Persis and Grammatico (2019), Deng (2021b), Deng
and Liang (2019), Fang et al. (2020), Frihauf et al. (2011), Gadjov
and Pavel (2019), Huang et al. (2021), Li et al. (2020), Liu and
Krstić (2011), Romano and Pavel (2019), Tatarenko et al. (2021),
Wang et al. (2020), Ye and Hu (2017), Zhang et al. (2020), Zhu
et al. (2021) and Zeng et al. (2019), may be ineffective for the
problem. These observations motivate us to study the constrained
noncooperative games of second-order multi-agent systems.
This paper aims to investigate noncooperative games with
local inequality and coupling inequality constraints and design
a distributed algorithm to seek the variational GNE. The main
contributions of the paper are summarized as follows.

2. Preliminaries and formulation
This section presents some preliminaries, and then formulates
our problem.
2.1. Graph theory
Here we review some concepts about graph theory (see Godsil
& Royle, 2001). An undirected graph is denoted by[ G ]:= {V , E , A},
where V := {1, . . . , N }, E ⊆ V × V and A := aij N ×N are the
vertex set, the edge set and the adjacency matrix, respectively.
An edge of G is denoted by (i, j) ∈ E if vertexes i and j can send
information to each other, i.e., i and j are neighbors. Besides, there
is no self-connection in graph G , that is, (i, i) ∈
/ E for all i ∈ V . A
path of length l in G is given by a sequence of distinct vertexes
v1 , . . . , vl+1 , where (vk , vk+1 ) ∈ E for k = 1, . . . , l. If there exists
a path between every pair of vertexes, then the undirected graph
G is connected. Moreover, in the adjacency matrix A, the element
aij is the weighting of (i, j) with aij = aji > 0 if (i, j) ∈ E ,
and aij = 0, otherwise. The degree of vertex i is expressed as
∑N
degi =
j=1 aij . Let L = D − A be the Laplacian matrix of G , where
D = diag {deg1 , . . . degN }. Clearly, L1N = 0N . Denote ϑ1 , . . . ϑN as
the eigenvalues of L, where ϑi ≤ ϑj if i ≤ j. Furthermore, G is
connected if and only if ϑ2 > 0.
For a connected undirected graph G , we have the following
result (see Hong, Hu, and Gao (2006, Lemma 3)).

1. We study the constrained noncooperative games of secondorder multi-agent systems. This formulation extends the
well-known noncooperative games, such as Frihauf et al.
(2011) and Ye and Hu (2017), by adding the second-order
dynamics of players. Besides, existing noncooperative
games involving the dynamics of players usually do not
contain the local inequality constraints, while our problem
includes the local inequality constraints as well as coupling
inequality constraints.
2. We design a distributed algorithm to seek the variational
GNE of the constrained noncooperative games.
Compared with the algorithms in Deng (2021b), Deng and
Nian (2018), Gadjov and Pavel (2019), Li et al. (2020),
Tatarenko et al. (2021), Wang et al. (2020) and Zhu et al.
(2021), our algorithm does not have any requirements for
the initial decisions. Different from the algorithms in Zeng
et al. (2019) requiring every player knows the decisions
of all other players, the decisions of non-adjacent players
are estimated by an embedded learning strategy in our
algorithm. Furthermore, the convergence of the algorithm
is analyzed. Under the algorithm, all players asymptotically converge to the exact variational GNE, rather than a
neighborhood of the GNE in Frihauf et al. (2011).

Lemma 1. The matrix L + M is positive definite, where L is the
Laplacian matrix of a connected undirected graph G ; M is a diagonal
matrix with nonnegative elements and at least one of diagonal
elements is positive.
2.2. Variational analysis and projection
In this subsection, some definitions about variational analysis
and projection are introduced (see Rockafellar & Wets, 2004).
A function f : Rn → R is convex if
f (α x + (1 − α )y) ≤ α f (x) + (1 − α )f (y), ∀α ∈ [0, 1]
for any x, y ∈ Rn .
A map F : Rn → Rn is ω-strongly monotone (ω > 0) if
(x − y)T (F (x) − F (y)) ≥ ω∥x − y∥2
for any x, y ∈ Rn .
A map F : Rn → Rn is θ -Lipschitz (θ > 0) if

∥F (x) − F (y)∥ ≤ θ∥x − y∥

The rest of this paper is organized as follows. Section 2 introduces some basic knowledge and presents the noncooperative game. Section 3 proposes a distributed variational GNE
seeking algorithm, and analyzes its convergence. Section 4 gives
two examples to verify the algorithm. Section 5 provides the
conclusion.
Notations: × and ⊗ represent the Cartesian product and the
Kronecker product, respectively. Define (x)+ := max{x, 0}. Let R
be the set of real numbers and Rn be the n-dimensional Euclidean
space. The column vectors of n ones and zeros are denoted by 1n
and 0n , respectively. col(x1 , . . . , xn ) = [xT1 , . . . , xTn ]T with xi being

for any x, y ∈ Rn .
The projection of x on C is defined as
PC (x) = arg min ∥x − y∥
y∈C

where x ∈ Rn and C ⊂ Rn is a closed convex set.
Some lemmas about the projection are presented as follows.
Lemma 2 (See Yang, Liu, & Wang, 2018). Let C ⊂ Rn be a closed
convex set, and the following properties hold for the projection PC (x).
2

Z. Deng, Y. Liu and T. Chen

Automatica 141 (2022) 110317

(1) ⟨x − PC (x), PC (x) − y⟩ ≥ 0, ∀x ∈ Rn , ∀y ∈ C ;
(2) ∥PC (x) − PC (y)∥2 ≤ ⟨PC (x) − PC (y), x − y⟩ , ∀x, y ∈ Rn .

Remark 1. Assumption 1 implies that Slater’s condition is satisfied. Assumptions 1 and 2 indicate that there exists a unique
variational GNE for the constrained noncooperative game (1), due
to the uniqueness of the solution of VI (C ∩ Ω , F ) (see Facchinei
and Pang (2003, Theorem 2.3.3)).

Lemma 3 (See Deng, Nian, & Hu, 2019). A function V : Rn × Rn →
R is defined as
V (x, y) =

1
2

Lemma 4. Suppose Assumptions 1 and 2 hold. x∗ = (x∗i , x∗−i ) is a
p
variational GNE of the game (1) if and only if there exist λ∗ ∈ R+
q
i
and µ∗i ∈ R+ such that

(∥x − PC (y)∥2 − ∥x − PC (x)∥2 )

where x ∈ Rn , y ∈ Rn , and C ⊂ Rn is a closed convex set. Then the
following statements hold for the function V (x, y).

∇xi Ji (x∗i , x∗−i ) + ATi λ∗ + BTi µ∗i = 0n

(1) V (x, y) ≥ (1/2) ∥PC (x) − PC (y)∥2 ;
(2) V (x, y) is differentiable in x, and ∇x V (x, y) = PC (x) − PC (y).

N
∑

2.3. Problem formulation

λ∗ ◦ (

Ai xi ≤

N
∑

i=1

di }

N
∑

i=1

v̇i = ui

(3d)

µi ◦ (Bi xi − ci ) = 0qi .

(3e)

i=1

∗

(1)

di

i=1

A GNE of the game (1) is defined as follows (see Deng & Nian,
2018, Facchinei & Kanzow, 2010).
Definition 1. A strategy profile x∗ = (x∗i , x∗−i ) is said to be a GNE
of the game (1) if

Remark 2. Compared with the well-studied noncooperative
games like (Frihauf et al., 2011; Ye & Hu, 2017), the second-order
dynamics of players are involved in our formulation. Moreover, in
contrast to existing noncooperative games with the dynamics of
players, the local inequality constraints and coupling inequality
constraints are contained in the game (1). Without involving
the second-order dynamics and the inequality constraints, existing GNE seeking algorithms for noncooperative games may
not guarantee that the second-order player (4) converges to the
variational GNE of noncooperative game (1). Besides, the variational GNE must satisfy the local inequality constraints as well
as the coupling inequality constraints. However, the decisions
of players cannot be directly decided by their control inputs,
because of the second-order dynamics. Hence, it is difficult to
design a distributed GNE seeking algorithm for our problem.

Ji (xi , x∗−i ) ≤ Ji (xi , x∗−i ), ∀ xi : (xi , x∗−i ) ∈ C ∩ Ω , i ∈ V
∗

where C = C1 × · · · × CN .
Based on Definition 1, a GNE is a strategy profile on which
player i cannot decrease its cost function Ji (xi , x−i ) by changing
its own decision unilaterally.
Some standard assumptions are presented as follows.
Assumption 1. There exists xi such that
Bi xi < ci .

∑N

i=1 Ai xi

<

∑N

i=1 di and

Assumption 2. The cost function Ji (xi , x−i ) is differentiable in
x and is convex in xi ; The pseudo-gradient F (x) is ω-strongly
monotone and θ -Lipschitz in x, where
F (x) = col(∇x1 J1 (x1 , x−1 ), . . . , ∇xN JN (xN , x−N )).

(4)

where vi ∈ Rn is the derivative of xi , and ui ∈ Rn is the control
input. It is worth noting that the inequality constraints in (1) do
not act on the second-order multi-agent system (4), i.e., there are
no any constraints on the dynamics (4).
The objective of this paper is to design a distributed algorithm
for the second-order player (4) such that the outputs of all players
converge to the variational GNE of the game (1), which means
under the to-be-designed algorithm, the second-order player (4)
can seek the variational GNE of the game (1) autonomously.

Bi xi ≤ ci .

Assumption 3.

Bi x∗i − ci ≤ 0qi

N
∑

Player i ∈ V has the following second-order dynamics:
ẋi = vi

min Ji (xi , x−i )
Ai xi ≤

(3c)

Ai x∗i −

i=1

xi ∈Rn

N
∑

di ) = 0p

N
∑

Proof. According to Facchinei and Pang (2009, Proposition 12.3),
x∗ is a variational GNE of the game (1) if and only if x∗ , along
q
p
with suitable λ∗ ∈ R+ and µ∗i ∈ R+i , is a solution of the
variational inequality VI (C ∩ Ω , F ). Based on Ruszczyński (2006,
Theorem 3.34), the solution of VI (C ∩ Ω , F ) satisfies (3). Thus the
conclusion is yielded. □

where x = col(x1 , . . . , xN ), Ai ∈ Rp×n and di ∈ Rp . The objective
of player i is to minimize its own cost function Ji (xi , x−i ). That is,
player i ∈ V faces the following problem:

s.t .

(3b)

∗

where Bi ∈ Rqi ×n and ci ∈ Rqi . Besides, the decisions of all players
are coupled by the following inequality:
N
∑

di ≤ 0 p

i=1

Ci = {xi ∈ Rn |Bi xi ≤ ci }

(3a)

i=1

i=1

Consider a noncooperative game of N players over an undirected graph G . Player i ∈ V has a cost function Ji (xi , x−i ) :
RNn → R, where xi ∈ Rn is the decision of player i, x−i =
col(x1 , . . . , xi−1 , xi+1 , . . . , xN ). The decision of player i is constrained by the following inequality:

Ω = {x ∈ RNn |

Ai x∗i −

N
∑

Remark 3. Some distributed optimization problems (DOPs) also
involve the dynamics of agents, such as Yi et al. (2018) and Tran
et al. (2019), but our problem is different from these DOPs: (1)
In DOPs, all agents aim to minimize the global cost function of
networks that is the sum of the cost functions of all agents via
cooperating with their neighbors, while in noncooperative games,
every player only wants to minimize its own cost function by
competing with other players. (2) In DOPs, the cost functions

(2)

The undirected graph G is connected.

Under Assumption 2, the solutions of variational inequality
VI (C ∩ Ω , F ) are called the variational GNE of the noncooperative
game (1), and the set of variational GNE is a subset of the set of
GNE (see Facchinei & Kanzow, 2010 and references therein).
3

Z. Deng, Y. Liu and T. Chen

Automatica 141 (2022) 110317

from the algorithms in Deng (2021b), Deng and Nian (2018),
Gadjov and Pavel (2019), Li et al. (2020), Tatarenko et al. (2021),
Tran et al. (2019), Wang et al. (2020), Yi et al. (2018) and Zhu
et al. (2021).

of all agent are function of a common global solution, while
in noncooperative games, every player has it own decision and
the cost function of every player depends on the decisions of
all players; Due to the dependence of cost functions on the
decisions of all players in noncooperative games, the distributed
GNE seeking algorithm design and analysis are more difficult than
that of DOPs, since all players have to estimate the decisions of
other players to obtain pseudo-gradients. (3) In DOPs, all agents
converge to a common solution, while in noncooperative games,
all players converge to the GNE and the decisions of all players
may be different at the GNE. Additionally, most of DOPs involving
the dynamics of players do not consider local constraints, such
as Tran et al. (2019) and Yi et al. (2018). As stated in Remark 2,
the existence of constraints and the involvement of second-order
dynamics also bring the difficulty in the distributed algorithm
design, because the constraints are not easy to be satisfied by the
outputs of second-order systems and the variational GNE must
satisfy the constrained.

3.2. Convergence analysis
This subsection analyzes the convergence of the algorithm (5).
Combining (4) with (5), we can obtain the following system:
ẋ = v

v̇ = − F̂ (x, x̂) − A λ̃ − B µ̃ − k1 v
T

(6b)

µ̇ = − µ + µ̃ + Bv

(6c)

λ̇ = − λ + λ̃ + Ax − d + (L ⊗ Ip )w − (L ⊗ Ip )λ̃ + Av

(6d)

ẇ = − (L ⊗ Ip )λ̃
x̂˙ = − k2 (((L ⊗ IN ) ⊗ In )x̂ + (M ⊗ In )(x̂ − 1N ⊗ x))

(6e)
(6f)

where

3. Main results

x = col(x1 , . . . , xN )

λ = col(λ1 , . . . , λN )

This section presents a distributed GNE seeking algorithm, and
then analyzes its convergence.

λ̃ = col(λ̃1 , . . . , λ̃N )
v = col(v1 , . . . , vN )
µ = col(µ1 , . . . , µN )
µ̃ = col(µ̃1 , . . . , µ̃N )
w = col(w1 , . . . , wN )
d = col(d1 , . . . , dN )
x̂ = col(x̂1 , . . . , x̂N )
x̂i = col(x̂i1 , . . . , x̂iN )
A = diag {A1 , . . . , AN }
B = diag {B1 , . . . , BN }

3.1. Distributed variational GNE seeking algorithm
The distributed projection-based GNE seeking algorithm for
player i ∈ V is developed as follows:

⎧
ui = − ∇xi Ji (xi , x̂−i ) − ATi λ̃i − BTi µ̃i − k1 vi
⎪
⎪
⎪
⎪
⎪
µ̇i = − µi + µ̃i + Bi vi
⎪
⎪
⎪
⎪
N
⎪
∑
⎪
⎪
⎪
λ̇
aik (wi − wk )
i = − λi + λ̃i + Ai xi − di +
⎪
⎪
⎪
⎪
k=1
⎪
⎪
⎪
N
⎪
∑
⎨
−
aik (λ̃i − λ̃k ) + Ai vi
⎪
⎪
k=1
⎪
⎪
⎪
N
⎪
∑
⎪
⎪
⎪ẇi = −
aik (λ̃i − λ̃k )
⎪
⎪
⎪
⎪
k=1
⎪
⎪
⎪
N
⎪
∑
⎪
)
⎪
⎪ x̂˙ ij = − k2 (
⎪
aik (x̂ij − x̂kj ) + aij (x̂ij − xj )
⎩

(6a)
T

and F̂ (x, x̂) = col(∇x1 J1 (x1 , x̂−1 ), . . . , ∇xN JN (xN , x̂−N )). Obviously,
F̂ (x, 1N ⊗ x) = F (x).
Based on Lemma 4, we have the following result about the
equilibrium point of (6).

(5)

Theorem 1.
Under Assumptions 1–3, consider the constrained
noncooperative game (1) with second-order players’ dynamics (4).
x∗ is a variational GNE of the game (1) if and only if there exist
2
∗
Np
q
v∗ ∈ ∑
RNn , λ∗ ∈ R+ , µ∗ ∈ R+ , w∗ ∈ RNp , x̂ ∈ RN n with
N
∗
∗ ∗
∗
∗ ∗
q =
i=1 qi , such that (x , v , λ , µ , w , x̂ ) is an equilibrium
point of (6).

k=1

where µ̃i = (µi + Bi xi − ci )+ , λ̃i = (λi )+ , x̂−i = col(x̂i1 , . . . ,
x̂i(i−1) , x̂i(i+1) , . . . , x̂iN ), x̂ij ∈ Rn is the estimation of player i on xj of
player j, k1 > 2θω + ∥B∥2 + N + 2 with B = diag {B1 , . . . , BN }, k2 >
2

3θ 2 +ωθ 2 +ω
4ωϑmin (H)

with H = L ⊗ IN + M and M = diag {a11 , . . . , a1n , . . . ,
an1 , . . . , ann }.

Proof.
∗
(i) For an equilibrium point (x∗ , v∗ , λ∗ , µ∗ , w∗ , x̂ ) of (6), we
have

Remark 4. The algorithm (5) consists of five parts:

0Nn = v∗

1. ∇xi Ji (xi , x̂−i ) for seeking the variational GNE;
2. vi for stabilizing the second-order player (4);
3. λi and ωi for the satisfaction of the coupling constraint Ω ;
4. µi for the satisfaction of the local constraint Ci ;
5. x̂ij for estimating the decisions of the non-adjacent players.

(7a)
∗

∗

0Nn = − F̂ (x , x̂ ) − A λ̃ − B µ̃ − k1 v

(7b)

0q = − µ + µ̃ + Bv

(7c)

∗

∗

T

∗

∗

∗

∗

T

∗

∗

0Np = − λ + λ̃ + Ax − d + (L ⊗ Ip )w
∗

∗

∗

− (L ⊗ Ip )λ̃ + Av∗
0Np = − (L ⊗ Ip )λ̃

Remark 5. The algorithm (5) estimates the decisions of nonadjacent players via an embedded learning strategy, rather than
obtains these decisions directly, in contrast to the algorithms in
Zeng et al. (2019). Under the algorithm (5), player i shares the
decision xi , the estimation x̂ij , and the auxiliary variables wi and
λ̃i with its neighbors. Moreover, there is no requirement for the
initial decisions of players in the algorithm (5), which is different

(7d)

∗

(7e)
∗

∗

0N 2 n = − k2 (((L ⊗ IN ) ⊗ In )x̂ + (M ⊗ In )(x̂ − 1N ⊗ x )).
∗

(7f)

Because the undirected graph G is connected, we have ((L ⊗
IN ) ⊗ In )(1N ⊗ x∗ ) = 0N 2 n . Therefore, (7f) indicates that ((L ⊗ IN ) ⊗
∗
In + (M ⊗ In ))x̂ = ((L ⊗ IN ) ⊗ In + (M ⊗ In ))(1N ⊗ x∗ ). Then, by
the connectedness of the undirected graph G and the definition
4

Z. Deng, Y. Liu and T. Chen

Automatica 141 (2022) 110317

Theorem 1 shows that if (6) converges, the second-order
player (4) approaches the variational GNE of the constrained
game (1). Next, we analyze the convergence of (6).
Before giving the convergence analysis, the following lemma
about F̂ (x, x̂) is presented.

of M, it follows from Lemma 1 that (L ⊗ IN ) ⊗ In + (M ⊗ In )
∗
is positive definite. Hence, x̂ = 1N ⊗ x∗ , which implies that
∗ ∗
∗
F̂ (x , x̂ ) = F (x ), based on the definition of F̂ (x, x̂). Further, (7a)
and (7b) imply that
∗

F (x∗ ) + AT λ̃ + BT µ̃ = 0Nn .
∗

(8)

Lemma 5.

Because the undirected graph G is connected, i.e. L1N = 0N
and 1TN L = 0TN , (7e) yields that

λ̃∗i = λ̃∗j , ∀i, j ∈ {1, . . . , N }.

Proof. Define ŷ−i , ŷi and ŷ in the same way as x̂−i , x̂i and x̂.
Then, based on Assumption 2, ∥∇xi Ji (xi , x̂−i ) − ∇xi Ji (xi , ŷ−i )∥2 ≤
∥F (x̃i ) − F (ỹi )∥2 ≤ θ 2 ∥x̃i − ỹi ∥2 = θ 2 ∥x̂−i − ŷ−i ∥2 ≤ θ 2 ∥x̂i −
ŷi ∥2 , ∀i ∈ {1, . . . , N }, where x̃i = col(x̂i1 , . . . , xi , . . . , x̂iN ) and
ỹi = col(ŷi1 , . . . , xi , . . . , ŷiN ). Further, the∑following inequaliN
ties are obtained: ∥F̂ (x, x̂) − F̂ (x, ŷ)∥2 =
i=1 ∥∇xi Ji (xi , x̂−i ) −
∑N 2
2
2
2
θ
∥x̂
−
ŷ
∥
=
θ
∥x̂
−
ŷ ∥2 , which yield
∇xi Ji (xi , ŷ−i )∥ ≤
i
i
i=1
that F̂ (x, x̂) is θ -Lipschitz in x̂. □

(9)

With (7a), left multiplying (7d) by (1N ⊗ Ip )T , we have
N
∑

N
∑

λ∗i −

i=1

λ̃∗i =

N
∑

i=1

Ai x∗i −

i=1

N
∑

di .

(10)

i=1

It results from λ̃i = (λi )+ and λ̃∗i − λ∗i ≥ 0p that

λ̃∗i ≥ 0p , ∀i ∈ {1, . . . , N }
N
∑

Ai x∗i −

i=1

N
∑

With Lemma 5 and Theorem 1, the following result is obtained.

(11)

di ≤ 0p .

Theorem 2. Under Assumptions 1–3, the second-order player (4)
with the algorithm (5) asymptotically converges to the variational
GNE of the constrained noncooperative game (1).

(12)

i=1

Besides, if λ̃∗i = 0p , we have

λ̃i ◦ (
∗

N
∑

∗

Ai xi −

i=1

N
∑

di ) = 0 p

Proof. Without loss of generality, let n = 1 for simplicity.
Take the following candidate Lyapunov function for (6).

(13)

1
V = ∥x − x∗ + v − v∗ ∥2
2
1
∗
+ (∥λ − λ̃ ∥2 − ∥λ − λ̃∥2 )
2
1
1
+ ∥w − w∗ ∥2 + (k1 − 1)∥x − x∗ ∥2
2
2
1
1
∗ 2
+ ∥µ − µ ∥ + ∥x̂ − 1N ⊗ x∥2
2
2

i=1

∑N

∗
and if λ̃∗i > 0p , then λ∗i = λ̃∗i and
i=1 (Ai xi − di ) = 0p , namely,
(13) is also satisfied in this case.
By (7c) and the definition of µ̃, we have µ∗ = (µ∗ + Bx∗ − c)+ .
Obviously, µ∗ ≥ 0q and Bx∗ − c ≤ 0q . If µ∗ > 0q , then Bx∗ − c =
0q , which means that µ∗ ◦ (Bx∗ − c) = 0q . If µ∗ = 0q , we also
have µ∗ ◦ (Bx∗ − c) = 0q . Therefore, we have

Bi x∗i − ci ≤ 0qi

(14a)

µi ◦ (Bi xi − ci ) = 0qi .

(14b)

∗

∗

where k1 , a parameter of algorithm (5), satisfies k1 > 2θω +∥B∥2 +
N + 2 > 1.
As a result of Lemma 3 (1), we have
2

According to Lemma 4, (8), (9), (10), (11), (12), (13) and (14)
indicate that x∗ is a variational GNE of the game (1).
(ii) Based on Lemma 4, when x∗ is a variational GNE of the
p
constrained noncooperative game (1), there exist λ∗ ∈ R+ and
qi
∗
µi ∈ R+ such that the following equations hold:

∇xi Ji (x∗i , x∗−i ) + ATi λ∗ + BTi µ∗i = 0n

∑

Ai x∗i −

∑

i=1
N
∑

λ ◦(
∗

(15a)

di ≤ 0p

(15b)

i=1

∗

A i xi −

i=1

∗

1
1
V ≥ ∥x − x∗ + v − v∗ ∥2 + ∥w − w∗ ∥2
2
2
1
1
∗
+ ∥λ̃ − λ̃ ∥2 + (k1 − 1)∥x − x∗ ∥2
2
2
1
1
∗ 2
+ ∥µ − µ ∥ + ∥x̂ − 1N ⊗ x∥2 .
2
2
Obviously, V ≥ 0.
The derivative of V with respect to time t is

N

N

N
∑

∗

di ) = 0 p

∗

+ (1 − k1 )(v − v∗ ) − BT µ̃ + BT µ̃∗ + AT λ̃ )

(15c)

∗

∗

Bi xi − ci ≤ 0qi

(15d)

µ∗i ◦ (Bi x∗i − ci ) = 0qi .

(15e)

∗

(16)

V̇ = (x − x∗ + v − v∗ )T (F̂ (x∗ , x̂ ) − F̂ (x, x̂) − AT λ̃

i=1

∗

∗

∗

∗

∗

+ (λ̃ − λ̃ )T (−λ + λ∗ + λ̃ − λ̃ + Ax − Ax∗
∗

+ L w − L w∗ + L λ̃ − L λ̃ + Av − Av∗ )
− (w − w∗ )T L λ̃ + (k1 − 1)(x − x∗ )T (v − v∗ )

Let x̂ = 1N ⊗ x , λ̃ = 1N ⊗λ , λ = λ̃ + Ax − d, w = 1N ⊗w
with w ∈ Rp and v∗ = 0Nn . Hence, (7a), (7d), (7e) and (7f) are
satisfied.
Let µ̃∗i = (µ∗i + Bi x∗i − ci )+ . By (15d) and (15e), it is obvious that
if Bi x∗i − ci < 0qi , then µ∗i = 0qi . Further, µ̃∗i = 0qi , i.e., µ̃∗i = µ∗i .
If Bi x∗i − ci = 0qi , we also have µ̃∗i = µ∗i . Thus, by taking
µ∗ = col(µ∗1 , . . . , µ∗N ) and µ̃∗ = col(µ̃∗1 , . . . , µ̃∗N ), (7b) and (7c)
hold.
∗
Np
Accordingly, there exists (v∗ , λ∗ , µ∗ , w∗ , x̂ ) ∈ RNn × R+ ×
2
∗
q
∗
R+ × RNp × RN n such that (x∗ , v∗ , λ , µ∗ , w∗ , x̂ ) is an equilibrium
point of (6). □
∗

Under Assumption 2, F̂ (x, x̂) is θ -Lipschitz in x̂.

∗

+ (µ − µ∗ )T (µ̃ − µ) + (µ − µ∗ )T B(v − v∗ )
− k2 (x̂ − 1N ⊗ x)T H(x̂ − 1N ⊗ x)
− (x̂ − 1N ⊗ x)T (1N ⊗ ẋ)
∗

∗

∗

= (x − x∗ )T (F̂ (x∗ , x̂ ) − F̂ (x, x̂)) − (λ̃ − λ̃ )T L(λ̃ − λ̃ )
∗

∗

+ (λ̃ − λ̃ )T (−λ + λ∗ + λ̃ − λ̃ ) + (1 − k1 )∥v − v∗ ∥2
∗

+ (λ̃ − λ̃ )T L(w − w∗ ) − (w − w∗ )T L λ̃
∗

+ (v − v∗ )T (F̂ (x∗ , x̂ ) − F̂ (x, x̂))
+ (µ − µ∗ )T (µ̃ − µ) + (v − v∗ )T BT (µ − µ̃)
5

Z. Deng, Y. Liu and T. Chen

Automatica 141 (2022) 110317

− (x̂ − 1N ⊗ x)T (1N ⊗ ẋ) − (x − x∗ )T BT (µ̃ − µ̃∗ )
− k2 (x̂ − 1N ⊗ x)T H(x̂ − 1N ⊗ x)

− (k2 ϑmin (H) −
(17)

− ∥µ − µ̃∥2
− ∥µ − µ̃∥2
= − µ̃∗T (Bx∗ − c) + µ̃T (Bx∗ − c) − ∥µ − µ̃∥2
(18)

4. Simulations

It follows from Lemma 2(2) that
∗

∗

(λ̃ − λ̃ )T (−λ + λ∗ + λ̃ − λ̃ ) ≤ 0.

(19)

In this section, two simulation examples are given to illustrate
our results.

Based on the ω-strong monotonicity of F (x), Lemma 5,
F̂ (x, 1N ⊗ x) = F (x) and F̂ (x∗ , 1N ⊗ x∗ ) = F (x∗ ), we have

4.1. Example 1

∗

(x − x∗ )T (F̂ (x∗ , x̂ ) − F̂ (x, x̂))

In electricity markets, the competition among distributed energy resources can be described by noncooperative games (see
Hobbs & Pang, 2007). Here consider the noncooperative game
among six generation systems, whose communication topology
is an undirected ring graph. The generation system i ∈ V faces
the following constrained noncooperative game:

= −(x − x∗ )T (F̂ (x, x̂) − F̂ (x, 1N ⊗ x) + F (x) − F (x∗ ))
≤ −ω∥x − x ∥ + θ ∥x − x ∥∥x̂ − 1N ⊗ x∥
2
3θ 2
≤ − ω∥x − x∗ ∥2 +
∥x̂ − 1N ⊗ x∥2 .
3
4ω
∗ 2

∗

(20)

It follows from the connectedness of the undirect graph that
∗

(λ̃ − λ̃ )T L(λ̃ − λ̃ ) ≥ 0

min Ji (Pi , P−i )

(21)

∗

(λ̃ − λ̃ )T L(w − w∗ ) − (w − w∗ )T L λ̃ = 0.

Pi ∈R

(22)

s.t .

Lemma 1 implies that H is symmetric positive definite, and
then, we have

≤ − k2 ϑmin (H)∥x̂ − 1N ⊗ x∥2 .
By virtue of Young’s inequality, we have
∗

(v − v∗ )T (F̂ (x∗ , x̂ ) − F̂ (x, x̂))

≤ ∥v − v ∥∥F (x) − F (x∗ )∥
∗

4
− (x̂ − 1N ⊗ x)T (1N ⊗ ẋ)

≤

4
and

(24)

∥x̂ − 1N ⊗ x∥2 + N ∥v − v∗ ∥2

Pdi

(28)

i=1

≤ Pi ≤ Pimax

Tmi

Tmi

(29)

1
1
⎪
⎪
⎩Ẋei = − Xei + ui
Tei

(25)

Tei

where Xei is the valve opening of the ith generator; Kmi is the
gain of the ith machine’s turbine; Tmi is the time constant of the
ith machine’s turbine, in s; Tei is the time constant of the ith
machine’s speed governor, in s; and ui is the control input of the
ith generator.
System (29) can be rewritten as follows:

1

∥µ − µ̃∥2 .
(26)
4
With (17), (18), (19), (20), (21), (22), (23), (24), (25) and (26),
we have
1
3
V̇ ≤ − ω∥x − x∗ ∥2 − ∥µ − µ̃∥2
6
4
− (k1 −

N
∑

⎧
Kmi
1
⎪
⎪
Pi +
Xei
⎨ Ṗi = −

∥x̂ − 1N ⊗ x∥2 + ∥1N ⊗ ẋ − 1N ⊗ ẋ∗ ∥2

(v − v∗ )T BT (µ − µ̃) ≤ ∥B∥2 ∥v − v∗ ∥2 +

Pi =

where Pi is the output power of the generation system i; Pimax ,
Pimin are the upper and lower bounds of the output power Pi ,
respectively; Ji (Pi , P−i ) = ci (Pi ) − p(σ )Pi is the cost function of
the generation system i. Specifically, ci (Pi ) := αi + βi Pi + γi Pi2 is
the generation cost with αi , βi and γi being the characteristics of
the generation system
∑N i; p(σ ) := p0 − aN σ is the electricity price
with σ (P) := N1
i=1 Pi and p0 , a being constants; Pdi is the local
load demand.
When the mechanical and electromagnetic losses are neglected, i.e., the output electrical powers of generators are equal
to the input mechanical powers of turbines, the dynamics of the
ith turbine-generator system can be written as follows (see Deng
& Liang, 2019; Guo, Hill, & Wang, 2000).

(23)

+ ∥v − v∗ ∥∥F̂ (x, x̂) − F̂ (x, 1N ⊗ x)∥
≤ θ∥v − v∗ ∥∥x − x∗ ∥ + θ∥v − v∗ ∥∥x̂ − 1N ⊗ x∥
1
θ2
≤ ω∥x − x∗ ∥2 + (
+ 1)∥v − v∗ ∥2
2
2ω
θ2
+ ∥x̂ − 1N ⊗ x∥2

N
∑
i=1
Pimin

− k2 (x̂ − 1N ⊗ x)T H(x̂ − 1N ⊗ x)

4
1

(27)

4

Remark 6. Compared with the algorithms in Frihauf et al. (2011)
designed for unconstrained noncooperative games with twice
continuously differentiable cost functions, which converge to a
neighborhood of the GNE, the algorithm (5) converges to the
exact variational GNE.

≤ − (x − x∗ )T BT (µ̃ − µ̃∗ ) + (Bx∗ − c)T (µ̃ − µ̃∗ )

1

4

1

− )∥x̂ − 1N ⊗ x∥2 .

min

= − (x − x∗ )T BT (µ̃ − µ̃∗ ) − (µ̃ − µ∗ )T (µ − µ̃)

≤

θ2

Consequently, (6) is asymptotically stable, which implies that
under the algorithm (5), the second-order player (4) converges
to the variational GNE of the constrained noncooperative game
(1) asymptotically. □

− (x − x∗ )T BT (µ̃ − µ̃∗ ) − (µ − µ∗ )T (µ − µ̃)

∗

4ω

−

It is obvious from (27) that V̇ ≤ 0, because the algorithm pa2
2 +ωθ 2 +ω
rameters k1 , k2 satisfy k1 > 2θω +∥B∥2 + N + 2 and k2 > 34θ ωϑ
.
(H)

where L = L ⊗ Ip .
∗
By Lemma 2(1), we have (µ + Bx − c − µ̃)T (µ̃ − µ̃ ) ≥ 0, which
yields that

≤ − ∥µ − µ̃∥2 .

3θ 2

P̈i = −

θ2
− ∥B∥2 − N − 2)∥v − v∗ ∥2
2ω

Tmi + Tei
Tmi Tei

Ṗi −

1
Tmi Tei

Pi +

Kmi
Tmi Tei

ui .

Then, it follows from Theorem 2 that the following algorithm
can ensure the output power of system (29) converges to the
6

Z. Deng, Y. Liu and T. Chen

Automatica 141 (2022) 110317

Fig. 1. The evolutions of the output powers.
Fig. 2. The total output power.

variational GNE of game (28):

⎧
1
⎪
⎪
((Tmi + Tei )Ṗi − Tmi Tei (∇Pi Ji (Pi , P̂−i )
ui =
⎪
⎪
K
mi
⎪
⎪
⎪
⎪
+ ATi (λi )+ + BTi (µi + Bi Pi − ci )+ + k1 Ṗi ) + Pi )
⎪
⎪
⎪
⎪
⎪
µ̇i = − µi + (µi + Bi Pi − ci )+ + Bi Ṗi
⎪
⎪
⎪
⎪
N
⎪
⎪
∑
⎪
+
⎪
⎪
λ̇
=
−
λ
+
(
λ
)
+
A
P
−
d
+
aij (wi − wj )
i
i
i
i
i
i
⎪
⎪
⎪
⎪
j
=
1
⎨
N
∑
⎪
⎪
−
aij ((λi )+ − (λj )+ ) + Ai Ṗi
⎪
⎪
⎪
⎪
j
=
1
⎪
⎪
⎪
N
⎪
∑
⎪
⎪
⎪
⎪
ẇ
=
−
aij ((λi )+ − (λj )+ )
⎪
⎪ i
⎪
⎪
j=1
⎪
⎪
⎪
N
⎪
⎪
)
(∑
⎪
⎪
aik (P̂ij − P̂kj ) + aij (P̂ij − Pj )
⎪ P̂˙ ij = − k2
⎩

4.2. Example 2
In order to further verify the algorithm, another example with
complex cost functions is given. Consider a constrained noncooperative game of four players with a ring graph, where the
dynamics of players are ẍi = ui , i ∈ {1, 2, 3, 4}. The cost functions
of players are as follows:
J1 (x1 , x−1 ) =x21a + 3x21b + ∥x2 − [4 3]T ∥2 x1a

+ ∥x3 − [1 2]T ∥2 x1b

(30)

J2 (x2 , x−2 ) =∥x2 − [3 2]T ∥2 + ln(e−0.02x4a + e0.02x4b )x2a

+ (x3a − x3b )x2b
(x2a + 4)x23b
(x1a + 2)x23a
+
+ ∥ x3 ∥ 2
J3 (x3 , x−3 ) =
2
20x3a + 1
20x23b + 1
+ (x4a + 1)x3a + (x4b + 2)x3b
J4 (x4 , x−4 ) =ln(e−0.05x4a + e0.05x4b ) + (x1b − 3)2 x4b

k=1

where xi = [xia xib ]T ∈ R2 with i ∈ {1, . . . , 4}.
The parameters of inequality constraints are as follows:

where Ai = [1, −1]T , Bi = [1, −1]T , ci = [Pimax , −Pimin ]T and
di = [Pdi , −Pdi ]T .
The characteristics of the generation systems are chosen as
follows:

[

6
A1 =
4

(α1 , . . . , α6 ) = (5, 8, 6, 9, 7, 8)

[
A3 =

(β1 , . . . , β6 ) = (12, 10, 11, 11, 13, 14)
(γ1 , . . . , γ6 ) = (1.0, 0.5, 0.8, 0.7, 1.1, 0.6)

B1 =

(P1min , . . . , P6min ) = (31, 35, 24, 39, 28, 30).

[
B3 =

Besides, p0 = 80 and a = 0.1.
When t ∈ [0, 100] (s), di is randomly chosen in [30, 40] (MW),
and t ∈ [100, 200] (s), Pdi is randomly chosen in [50, 60] (MW).
The initial value Pi (0) is randomly chosen in [30, 50] (MW), and
the initial values of Ṗi (0), µi (0), λi (0), wi (0), P̂i (0) are all zeros. The
parameters of algorithm (30) are k1 = 12, k2 = 5.
Figs. 1 and 2 exhibit the simulation results. Particularly, Fig. 1
shows the evolutions of output powers, and Fig. 2 reflects the
total output power. It is obvious from Figs. 1 and 2 that the output
powers of six turbine-generator systems converge to the variational GNEs P ∗ = col(31.00, 47.28, 30.00, 39.00, 28.00, 36.93)
when t ∈ [0, 100] (s) and P ∗ = col(46.86, 63.00, 58.47, 56.00,
42.35, 52.00) when t ∈ [100, 200] (s), and the total output
power can match the total demand, even though the total load
is changed at time 100 s.

1
1

,

[

0
,
0
1

]

−1
]
0
,
1

T

,

]

4
A2 =
−1

]

−4

0
3

]

1

4

[

(P1max , . . . , P6max ) = (57, 63, 68, 56, 65, 52)

−2

[
A4 =

7
0

1
−3

]
−1
]

5

[
B2 =

1
0

0
1

[

−1

1
0

B4 =

0

0
−1

]T

0
1

c1 = [2 4] , c2 = [1 2]T , c3 = [0 1]T , c4 = [0 5 2 2]T ,
d1 = [−2 − 1]T , d2 = [4 − 10]T , d3 = [4 − 16]T , d4 = [−1 2]T .
The parameters of algorithm (5) are k1 = 9, k2 = 4.
Fig. 3 depicts the simulation results, where the final and initial decisions are labeled by the asterisks and the small circles,
respectively. Moreover, in Fig. 3, the solid lines and the dot–
dash lines are the evolutions of decisions and the boundaries
of local constraints, respectively. As shown in Fig. 3, the decisions of all players converge to the variational GNE (x∗1 =
col(1.23, 0.95), x∗2 = col(0.16, −0.27), x∗3 = col(−0.51, 1.51),
x∗4 = col(3.02, −0.97)), even if some initial decisions are out
of constraints. These simulation results verify the algorithm (5)
again.
7

Z. Deng, Y. Liu and T. Chen

Automatica 141 (2022) 110317
Gadjov, D., & Pavel, L. (2019). A passivity-based approach to Nash equilibrium seeking over networks. IEEE Transactions on Automatic Control, 64(3),
1077–1092.
Ghaderi, J., & Srikant, R. (2014). Opinion dynamics in social networks with
stubborn agents: Equilibrium and convergence rate. Automatica, 50(12),
3209–3215.
Godsil, C. D., & Royle, G. (2001). Algebraic graph theory. New York, NY, USA:
Springer.
Guo, Y., Hill, D. J., & Wang, Y. (2000). Nonlinear decentralized control of
large-scale power systems. Automatica, 36(9), 1275–1289.
Hobbs, B. F., & Pang, J. S. (2007). Nash-cournot equilibria in electric power
markets with piecewise linear demand functions and joint constraints.
Operations Research, 55(1), 113–127.
Hong, Y., Hu, J., & Gao, L. (2006). Tracking control for multi-agent consensus
with an active leader and variable topology. Automatica, 42(7), 1177–1182.
Huang, B., Zou, Y., & Meng, Z. (2021). Distributed-observer-based Nash equilibrium seeking algorithm for quadratic games with nonlinear dynamics. IEEE
Transactions on Systems, Man, and Cybernetics: Systems, 51(11), 7260–7268.
Kim, K. D., & Kumar, P. R. (2012). Cyber-physical systems: A perspective at the
centennial. Proceedings of the IEEE, 100, 1287–1308.
Li, Z., Li, Z., & Ding, Z. (2020). Distributed generalized Nash equilibrium seeking
and its application to femtocell networks. IEEE Transactions on Cybernetics,
1–13, in press.
Liu, S.-J., & Krstić, M. (2011). Stochastic Nash equilibrium seeking for games
with general nonlinear payoffs. SIAM Journal on Control and Optimization,
49(4), 1659–1679.
Lou, Y., Hong, Y., Xie, L., Shi, G., & Johansson, K. H. (2016). Nash equilibrium computation in subnetwork zero-sum games with switching communications.
IEEE Transactions on Automatic Control, 61(10), 2920–2935.
Mudumbai, R., Dasgupta, S., & Cho, B. B. (2012). Distributed control for optimal
economic dispatch of a network of heterogeneous power generators. IEEE
Transactions on Power Systems, 27(4), 1750–1760.
Rockafellar, R. T., & Wets, R. J.-B. (2004). Variational analysis. Berlin, Germany:
Springer.
Romano, A. R., & Pavel, L. (2019). Dynamic NE seeking for multi-integrator
networked agents with disturbance rejection. IEEE Transactions on Control
of Network Systems, 7(1), 129–139.
Ruszczyński, A. P. (2006). Nonlinear optimization. Princeton, N.J.: Princeton
University Press.
Tan, L., Zhu, Z., Ge, F., & Xiong, N. (2015). Utility maximization resource allocation
in wireless networks: Methods and algorithms. IEEE Transactions on Systems,
Man, and Cybernetics: Systems, 45(7), 1018–1034.
Tatarenko, T., Shi, W., & Nedić, A. (2021). Geometric convergence of gradient
play algorithms for distributed Nash equilibrium seeking. IEEE Transactions
on Automatic Control, 66(11), 5342–5353.
Tran, N.-T., Wang, Y.-W., Liu, X.-K., Xiao, J.-W., & Lei, Y. (2019). Distributed
optimization problem for second-order multi-agent systems with eventtriggered and time-triggered communication. Journal of the Franklin Institute,
356(17), 10196–10215.
Wang, X.-F., Sun, X.-M., Teel, A. R., & Liu, K.-Z. (2020). Distributed robust Nash
equilibrium seeking for aggregative games under persistent attacks: A hybrid
systems approach. Automatica, 122, Article 109255.
Yang, S., Liu, Q., & Wang, J. (2017). A multi-agent system with a proportionalintegral protocol for distributed constrained optimization. IEEE Transactions
on Automatic Control, 62(7), 3461–3467.
Yang, S., Liu, Q., & Wang, J. (2018). A collaborative neurodynamic approach
to multiple-objective distributed optimization. IEEE Transactions on Neural
Networks and Learning Systems, 29(4), 981–992.
Yang, H., Zhang, Z., & Zhang, S. (2011). Consensus of second-order multi-agent
systems with exogenous disturbances. International Journal of Robust and
Nonlinear Control, 21(9), 945–956.
Ye, M., & Hu, G. (2017). Distributed Nash equilibrium seeking by a consensus
based approach. IEEE Transactions on Automatic Control, 62(9), 4811–4818.
Yi, X., Yao, L., Yang, T., George, J., & Johansson, K. H. (2018). Distributed
optimization for second-order multi-agent systems with dynamic eventtriggered communication. In Proceedings of the 57th IEEE conference on
decision and control. Miami Beach, USA (pp. 3397–3402).
Yin, H., Shanbhag, U. V., & Mehta, P. G. (2011). Nash equilibrium problems
with scaled congestion costs and shared constraints. IEEE Transactions on
Automatic Control, 56(7), 1702–1708.
Zeng, X., Chen, J., Liang, S., & Hong, Y. (2019). Generalized Nash equilibrium
seeking strategy for distributed nonsmooth multi-cluster game. Automatica,
103, 20–26.
Zhang, Y., Liang, S., Wang, X., & Ji, H. (2020). Distributed Nash equilibrium
seeking for aggregative games with nonlinear dynamics under external
disturbances. IEEE Transactions on Cybernetics, 50(12), 4876–4885.

Fig. 3. The evolutions of the decisions of four players.

5. Conclusions
The constrained noncooperative games of second-order multiagent systems have been investigated in this paper. In the problem, the decisions of players are constrained by not only local
inequality constraints but also coupling inequality constraints.
Based on state feedback and gradient descent, a distributed algorithm has been designed for these second-order players to seek
the variational GNE autonomously. In the algorithm, the decisions
of other players are learned by an embedded strategy, rather than
be accessed directly. With the help of Lyapunov stability theory,
the asymptotic convergence of the algorithm has been proved.
Finally, the algorithm has been illustrated by two examples.
References
De Persis, C., & Grammatico, S. (2019). Distributed averaging integral Nash
equilibrium seeking on networks. Automatica, 110, Article 108548.
Deng, Z. (2021a). Distributed algorithm design for resource allocation problems
of second-order multiagent systems over weight-balanced digraphs. IEEE
Transactions on Systems, Man, and Cybernetics: Systems, 51(6), 3512–3521.
Deng, Z. (2021b). Distributed generalized Nash equilibrium seeking algorithm
for nonsmooth aggregative games. Automatica, 132, Article 109794.
Deng, Z., & Liang, S. (2019). Distributed algorithms for aggregative games of
multiple heterogeneous Euler–Lagrange systems. Automatica, 99, 246–252.
Deng, Z., & Nian, X. (2018). Distributed generalized Nash equilibrium seeking
algorithm design for aggregative games over weight-balanced digraphs. IEEE
Transactions on Neural Networks and Learning Systems, 30(3), 695–706.
Deng, Z., Nian, X., & Hu, C. (2019). Distributed algorithm design for nonsmooth resource allocation problems. IEEE Transactions on Cybernetics, 50(7),
3208–3217.
Esmalifalak, M., Shi, G., Han, Z., & Song, L. (2013). Bad data injection attack and
defense in electricity market using game theory study. IEEE Transactions on
Smart Grid, 4(1), 160–169.
Facchinei, F., & Kanzow, C. (2010). Generalized Nash equilibrium problems.
Annals of Operations Research, 175(1), 177–211.
Facchinei, F., & Pang, J.-S. (2003). Finite-dimensional variational inequalities and
complementarity problems. New York: Springer.
Facchinei, F., & Pang, J.-S. (2009). Nash equilibria: the variational approach. In
Convex optimization in signal processing and communications (pp. 443–493).
Cambridge: Cambridge University Press.
Fang, X., Wen, G., Huang, T., Fu, Z., & Hu, L. (2020). Distributed Nash equilibrium seeking over Markovian switching communication networks. IEEE
Transactions on Cybernetics, 1–13, in press.
Frihauf, P., Krstić, M., & Başar, T. (2011). Nash equilibrium seeking in
noncooperative games. IEEE Transactions on Automatic Control, 57(5),
1192–1207.
8

Z. Deng, Y. Liu and T. Chen

Automatica 141 (2022) 110317

Zhang, X., Papachristodoulou, A., & Li, N. (2018). Distributed control for reaching
optimal steady state in network systems: An optimization approach. IEEE
Transactions on Automatic Control, 63(3), 864–871.
Zhu, Y., Yu, W., Wen, G., & Chen, G. (2021). Distributed Nash equilibrium seeking
in an aggregative game on a directed graph. IEEE Transactions on Automatic
Control, 66(6), 2746–2753.

Yangyang Liu received the B.S. degree in automation
from Central South University of Forestry and Technology, Changsha, China, in 2019. She is currently
working toward the M.S. degree in control engineering
at Central South University, Changsha, China.
Her current research interests include multiagent systems, multi-cluster games, and distributed
optimization.

Zhenhua Deng received the B.S. degree in automation from Dalian Maritime University, Dalian, China, in
2011, the M.S. degree in control science and engineering from Central South University, Changsha, China,
in 2014, and the Ph.D. degree in operational research
and cybernetics from University of Chinese Academy of
Sciences, Beijing, China, in 2017.
He is currently an associate professor with Central South University. His current research interests
include multiagent systems, distributed optimization,
game theory, smart grids, and UAVs.

Tao Chen received the B.S. degree in automation from
Xiangtan University, Xiangtan, China, in 2020. He is
currently working toward the M.S. degree in electronic
information at Central South University, Changsha,
China.
His current research interests include multicluster games, multi-agent systems, and distributed
optimization.

9

