1. Introduction

1.1 Cognitive Radio Technology

Studies [1] performed by the Federal Communication Commission (FCC) have shown that the conventional fixed spectrum1 allocation policy is becoming insufficient to address today’s rapidly developing wireless communications, and there is a call for open spectrum allocation. To be specific, if parts of the radio spectrum were randomly scanned in a particular geographical area at a specific time, we would discover that [2]-[5]: (i) certain frequency bands, i.e., bands assigned to the PUs, in a specific spectrum are mainly unutilised most of the time; (ii) other frequency bands are only moderately utilised; and (iii) the remaining frequency bands are used to a significant extent by the PUs.

One way of improving the ineffectiveness of the traditional fixed spectrum allocation policy is to use dynamic spectrum access and cognitive radio technology. With the growth of cognitive radio technologies, spectrum utilisation can be improved by allowing CRs (also known as secondary users or unlicensed users) to access a spectrum hole2 that is unoccupied by the PUs (also called a licensed user).

Definition 1 [6]: A cognitive radio is a software-defined radio that can also interact with its neighbouring environment and respond based on its findings by changing its transmission parameters (e.g., transmit power, modulation technique and transmission frequency) using an intelligent approach with two primary objectives [2]: (i) reliable communication with good QoS and (ii) resourceful utilisation of the available spectrum.

From this definition, we recognise that the communication technique used in CR technology differs from that of conventional wireless devices in two main respects [2], [7]-[10]. The first is cognitive capability, which refers to the ability of an unlicensed user to sense, discover and gather information from its neighbouring region. This information can be either local information related to the node itself (i.e., CR) or global information related to the nearby PUs. Second, there is cognitive re-configurability, which refers to the ability of CR to dynamically take action upon gathering its findings by changing its transmission parameters. Cognitive capability and re-configurability give CRs the opportunity to capture the best available spectrum hole in a given spectrum band, which is considered an essential objective of CR technology [7], [8].

The coexistence between the users of cognitive radio networks (CRNs) and primary user networks (PUNs) gives rise to several challenges. Therefore, researchers must undertake more efforts to develop new resource allocation approaches to solve the problems that arise in such complicated heterogeneous environments3 [7], [11]. To facilitate the operation of CRs over licensed bands, three main scenarios have been recognised in the literature: (i) the underlay scenario, in which both CRs and PUs operate at the same time and try to access the same band; (ii) overlay scenario, in which CR users communicate in the absence of the PUs; and the (iii) interweave scenario, in which CRs must perform spectrum sensing first to check the availability of spectrum holes and then CR nodes are restricted to communicate over the detected free bands [2], [12].

1.2 Cognitive Radio Networks (CRNs): Functions and Architecture

In this section, we summarise the functions and architecture of CRNs and discuss a number of issues related to each element.

1.2.1 CRN Functions

In a CRN environment, cognitive radio users coexist with the primary users in a heterogeneous fashion, using the same spectrum band in either overlay or underlay scenarios. Additionally, a real-time interaction is necessary for users to adapt to the dynamic nature of the radio environment [8]. The main tasks that facilitate the interaction are illustrated in Fig. 1, which is referred to as cognitive cycle [2], [13]. The basic cognitive cycle pictured in Fig. 1. focuses on three cognitive tasks [2], [8]:

Fig. 1.Cognitive cycle

Definition 2 [2], [14-16]: The interference temperature is defined as the power of the radio frequency (RF) monitored at a receiving antenna and is commonly used to quantify and manage the cause of interference on the one hand and provide a precise measurement of the tolerable level of RF interference in the desired frequency band on the other, as shown in Fig. 2.

Fig. 2.Interference temperature model [1]

As long as CRs do not exceed the interference temperature limit by their transmissions in the frequency band of interest, they can use this spectrum band [8].

1.2.2 CRN Architecture

Cognitive radio networks (CRNs) can be classified as follows [8,10]: (i) infrastructure-based CR networks. Such networks feature a cognitive radio base station (CRBS) (or secondary base station), which facilitates communications among CR users. In this case, the monitoring and analysis of spectrum band utilisation and the decision on how to avoid interference with the PUs are the responsibilities of the CRBS. (ii) Ad-hoc based network: In this scenario, there is no permanent infrastructure, and the CRs can communicate among themselves in ad-hoc fashion over licensed or unlicensed spectrum bands. In AHCRNs, each CR is required to determine its best actions while avoiding the generation of interference to PUs based on local monitoring [10]. Common AHCRNs that operate over a licensed spectrum include the following entities:

When both types of users (i.e., CRs and PUs) transmit simultaneously, mutual interference occurs, as shown in Fig. 3. In this scenario, CR nodes must strictly control their transmit power to avoid any harmful interference with the active PUs. The solid lines in Fig. 3 indicate intended communication links among CR nodes and between primary user based station (PUBS) and its PU receiver with channel gain denoted by hc and hp respectively. Whereas the dotted lines denote the interference links from CR-to-PU and PU-to-CR with channel gains denoted by respectively. The superscripts ( c ) and ( p ) refer to CR and PU correspondingly. Without loss of generality, we assume that there is no interference generated among primary users.

Fig. 3.An example of underlay AHCRNs with three CR users (pairs) and one PU

1.3 Spectrum Sharing and Recourse Management in AHCRNs

In this section, we summarise the features and concepts of spectrum sharing and spectrum access in AHCRNs.

1.3.1 Spectrum Sharing in AHCRNs

A key challenge in an ad-hoc CRN is determining how to allocate transmission resources (e.g., power and channel) efficiently among CRs over a wide range of available spectra by considering the activity of neighbouring PUs [8]. To be specific, spectrum sharing in AHCRNs provides the capability to maintain good QoS for CRs while minimising the generated interference to the PUs by allocating the communication resources for CRs adaptively. In this context, spectrum sharing addresses two main problems [10]: (i) the resource allocation problem, which includes channel selection and power/rate control for CRs, and (ii) the spectrum access problem, which includes coordinating and sharing the available spectrum band of CRs. Two main spectrum sharing techniques [17] are summarised in Table 1

Table 1.Taxonomy of spectrum sharing in CRNs

1.3.2 Spectrum Access Techniques in AHCRNs

In the case of spectrum access in AHCRNs, we limit our discussion to underlay and overlay spectrum access scenarios only, as shown in Fig. 4.

Fig. 4.Classification of spectrum access

First, there is the overlay spectrum sharing scenario, in which CRs can access the available spectrum opportunistically/rationally when it is not being occupied by the PUs. Thus, a spectrum sensing procedure is required for CR nodes before making any attempt to use the spectrum to avoid potential interference with the licensed users. This procedure can be executed through innovative signal processing techniques (e.g., matched filtering, energy detection and coherent detection) [8]. The challenging point behind overlay spectrum access is how to keep continuous transmission for CR nodes. However, the solution to continuity of CR transmission in heterogeneous environment is proposed in [18]. Second, there is the underlay resource sharing approach, in which both PUs and CRs coexist over the same geographical area and across the same spectrum band. In this case, the transmission power must be well controlled and must be kept below a specific threshold to run smooth communication services with the PUs. The drawback of this type of spectrum access is that the CR nodes may suffer from bad performance because of the strict power constraint related to the CR nodes and the interference comes from PU. However, the performance of CR nodes in an underlay scenario can be improved by means of cooperation among the CR nodes and via relaying technique with interference cancellation, as presented in [19], [20]. Table 2 summarises the main features of the two mentioned techniques and issues associated with their implementation.

Table 2.A comparison between underlay and overlay spectrum sharing4

The analysis of spectrum access and resource allocation in AHCRNs has introduced several challenges to researchers due to the coexistence of two systems, CRs and PUs, with conflicting interests. Thus, game theory has been adopted as an effective tool for analysing situations involving systems with conflicting interests and to allocate transmission resources efficiently in AHCRNs, as presented in [21-23].

Our main contributions in this paper are: (i) to introduce the concept of spectrum access techniques in CRNs; (ii) to provide several models and scenarios for the application of game theory to underlay AHCRNs; (iii) to illustrate the advantages of applying game theory to AHCRNs; (iv) to demonstrate the application of game theory to underlay AHCRNs in different layers (PHY, MAC, and Network) in the protocol stack; (v) to propose definitions and models for “utility function” in each layer; and (vi) to review the recent work in the literature on underlay CRNs based on game theory and summarize its main features. To the best of our knowledge, this paper considers the first work that offer concrete descriptions related to resource allocation problem in underlay AHCRNs in different layers based on game theory.

 

2. Game Theory: Fundamentals and Application to AHCRNs

2.1 Fundamentals of Game Theory

Game theory was first introduced by J.V. Neumann and O. Morgenstern in 1944 [24]. It is a collection of mathematical tools aimed at understanding, analysing and modelling the interaction between decision makers [25-27]. Game theory has been extensively used in microeconomics, and only in recent years it has been widely acknowledged as a powerful tool to model the problem of resource allocation in AHCRNs.

Before we begin our discussion and analysis of AHCRNs based on game theory, let us formally define what we mean by the terms “game” and “game theory”.

Definition 3, “Game” [28]: A game is a model of the interactive decision problems among decision makers.

Definition 4,” Game theory” [25-27]: Game theory is a branch of applied mathematics that provides a means of understanding and analysing scenarios that feature conflicts among interacting decision makers.

Game theory can generally be divided into two main branches: (i) non-cooperative game theory, termed NCGT, and (ii) cooperative game theory, termed CGT. Table 3 summarises the main features of CGT and NCGT.

Table 3.Summary of main branches of game theory

Scenario 1: Resource allocation in AHCRNs can be represented as a game, in which the players are CR nodes competing with each other and with the owner of the spectrum (i.e., PUs) to access limited spectrum resources. In this scenario, the action of any CR affects the decisions of other players (i.e., opponent) and ultimately affects the performance of the primary users. Game theory is the preferred approach to investigating this scenario because it is generally applied to analyse situations in which the users’/players’ objectives are in conflict. However, in this paper, we limit our discussion to non-cooperative game theory and its application to the problem of resource allocation in AHCRNs.

2.2 Non-Cooperative AHCRNs Game Theory: Fundamental and Basic Concept

In recent years, the non-cooperative game theoretic approach has become one of the most favourable for modelling many problems in wireless ad-hoc networks, such as power control, spectrum sharing, MAC protocol design and multi-hop routing. Players in the non-cooperative game theoretic approach can be viewed as being rational6, assigning their resources independently without any means of collaboration with other nodes in a network.

Definition 5 [25], [26]: Non-cooperative game theory is an approach that can be used to resolve conflicts when rational decision makers interact with each other in a specific environment.

The normal or strategic form of AHCRN games consists of three main components (players, actions and utility functions), which can be mathematically represented as G = [24-27], where

1) N = {1,2,…, N} : is a finite set of decision makers (players);

2) A = A1 × A2 ×…× AN : is the Cartesian product of the sets of actions available to each player, with Ai being the action set for player i ;

3) ui : A → R : is the utility/payoff functions of player i which is a function of the action chosen by player and the actions chosen by all of the players in the game except those of player

Scenario 2: Non-cooperative game theory has been used to investigate the resource allocation problem in AHCRNs, in which the players are CR nodes; the players’ strategy corresponds to spectrum selection and the choice of power level for each link and route selection, and their utility function (or payoff function) denotes the level of satisfaction that each player (or decision maker) receives as a result of accessing the available spectrum bands [31]. Table 4 presents the details of scenario 2.

Table 4.Components of a non-cooperative game in AHCRNs

2.3 Nash Equilibrium and Pricing

In distributed non-cooperative game networks, such as AHCRNs, the players are rational and they think only about their own profit by choosing strategy that maximizes their utility function in order to reach to a resourceful outcome. This outcome is known as Nash equilibrium termed as (NE) which is the most general solution concept in non-cooperative game theory. However, players in a distributed network (e.g., AHCRNs) may behave selfishly or cheating from other’ strategy to maximize their benefit which results in poverty in the NE. In order to avoid such troublesome, pricing mechanism are used to punish players that not following the network’ rules. Thus, selfish players can be guided carefully toward a more stable point and the outcome of the network can be improved accordingly [31]. Nash equilibrium and pricing mechanism are discussed in the following sections.

2.3.1 Nash Equilibrium

One of the objectives of game theory is to predict what will occur when a game is played [27]. A reasonable prediction of the outcome of a game is called Nash equilibrium.

Definition 7 [25,26], [30]: The Nash equilibrium (termed NE) of a normal game G = is an action profile action ∈ A such that no individual player can obtain a better payoff from unilateral deviation. Mathematically, Nash equilibrium can be expressed as follows.

Alternatively, NE can be defined as the point at which the best response function of a player is reached [26]. For player CRi, the best response is the action profile chosen by player i that can maximise his/her utility function for the given action profiles of all other players. Mathematically, for player i is the best response that can be stated according to [26]

In the following, we present an example to illustrate the concept of NE.

Example 1 [27]: Consider the following game defined by Table 5. This is a two-CR-player game in which CR 1 chooses the row and CR 2 chooses the column. This game features a parameter δ , which varies between 0 and 1. The values in each cell indicate the preferences of each CR user, where the first number listed is the utility function of CR 1 and the second is the utility function of CR 2.

Table 5.Example of NE [27]

where X and Y are the action profiles available to CR users and (uCR1 ,uCR2 ) are the utility functions of CR 1 and CR 2, respectively. The game presented in the above example yields two NEs, given by the action profiles (X, Y) and (Y, X). Consider the first action profile (X, Y), where CR 1 plays action profile Y and receives a utility function of (1-δ) , while CR 2 plays action profile Y and receives a utility function of 0. In this scenario, CR 1 has no incentive to deviate because changing his or her action profile to Y would reduce the player’s utility function from a positive (1-δ) to 0; player CR 2 also has no incentive to deviate because changing his or her action profile to X would reduce the player’s utility from 0 to (-δ) . On the other hand, action profile (X, X) is a non-NE action profile because CR 1 could add to his or her utility function by unilaterally changing his or her action profile to Y. Such an independent deviation would change his or her action profile to (X, Y), thereby increasing the utility experienced by CR 1 from (-δ) to (1-δ) .

2.3.1.1 Existence and Uniqueness of NE

Dealing with non-cooperative game theory give us the opportunity to explore two important properties related to NE. These properties are (i) existence of NE and (ii) uniqueness of NE. The following sections present to readers the general approach of investigating and proving existence and uniqueness of NE.

2.3.2.1.1 Existence of NE

In an AHCRN game, numerous CR users with conflicting interests compete with each other and ultimately reach a stable point (i.e., NE). The key point is that in some games, there is no guarantee that Nash equilibrium exists. However, the existence of NE in non-cooperative AHCRN games can be achieved based on specific mathematical properties related to certain utility functions; such games can be shown to have a pure strategy Nash equilibrium (e.g., submodularity, supermodular games) [34].

Theorem 1 [35]: Nash equilibrium exists in a strategic game G = if the conditions ∀i ∈ N are met:

1) The strategy profile of player i is a nonempty, convex and compact subset of some Euclidean space.

2) The utility function ui is a continuous and quasi-concave function over its strategy set.

One question that may naturally arise at this stage is, “Do NE approaches that provide an efficient outcome for given game exist?” In this respect, Pareto optimality is frequently used to measure the efficiency of a game’s outcome [26], [34].

Definition 8 [24], [31]: Pareto optimality is an action profile action = (aʹ 1, aʹ2,..., aʹN ) such that no other action profile exists bction = (bʹ 1, bʹ2,..., bʹN ) for any given player could be better off without making another player becoming worse off. Mathematically speaking, an action profile action is Pareto optimal if and only if there is no other action profile bction such that ui(bction)≥ (action) ∀i ∈ N.

2.3.2.1.2 Uniqueness of NE

The first step in solving non-cooperative AHCRN game problems is to determine the existence of NE. In addition to existence, the uniqueness of NE is another important property that must be considered in a non-cooperative AHCRN game. The uniqueness of a NE can be guaranteed if a given non-cooperative game can be modelled through a special game technique and thus shown to reach a unique NE. Table 6 summarises the concepts of two special methods.

Table 6.A brief summary of potential game and standard function

2.3.3 Pricing Game

Pricing theory is a theory that was conceived in the field of economics. In the area of ad-hoc networks, pricing techniques play important roles in the resource allocation problem because they can guide selfish players to a more efficient operating point and thus improve the overall efficiency of the NE [31].

Definition 9: The pricing function in AHCRNs can be defined as the cost of harm that CRs generate to PUs in terms of the performance degradation that may take place in the primary networks (PNs). Generally, pricing is motivated by the following objectives [31]: (i) generate revenue for the AHCRNs, (ii) encourage CRs to use the available spectrum holes more efficiently and (iii) minimise the generated interference to PUs. Thus, pricing techniques have been adopted in the resource allocation problem in AHCRNs.

2.3.4 Motivations of using Game Theory in AHCRNs

Ad hoc cognitive radio network (AHCRN) has taken a paramount place in the area of CR due to the challenges associated with this type of network. The most important challenges are: (i) autonomous and multi-hop network: there is no central authority (i.e., CR base station) to take care on CR and all nodes must take their actions and decisions such as transmit power, spectrum allocation and packet forwarding independently (i.e., autonomous agent); (ii): nodes in AHCRNs are strictly energy and power constrained. Thus, a special technique must be considered to analyse the performance of AHCRNs, on the other hand, every protocol and algorithm related to the configuration and operation of AHCRNs must be absolutely distributed.

It should be noted that some techniques (e.g., Instantaneous allocation and Ergodic resource allocation) have been adopted certain tools from optimization theory to analyse and to study the problem of resource allocation in wireless network (see for example [38]). However, complexity and the doubt about convergence of the algorithms to a steady state point is the main drawback behind adopting these techniques. Game theory, on the other hand, has been shown in the literature as a powerful tool that can be adopted to analyse wireless networks and AHCRNs with better network performance and less complexity compared to ordinary optimization theory [27]. In the following we draw attention to certain advantages that can be gained in adopting game theory in AHCRNs:

 

3. Game Theory in AHCRNs: A Layered Viewpoint

Non-cooperative game theory can be applied to model and analyse AHCRNs at different layers, such as the physical layer, MAC layer and all other upper layers listed in the Open System Interconnection (OSI) layers. Fig. 6 shows the link between OSI layers and the corresponding applications. In this paper, we limit our discussion to three layers: (i) physical (PHY) layer, (ii) MAC layer and (iii) network Layer.

Fig. 6.AHCRNs with different OSI layers

In the following sections we will offer a variety of research works, based non-cooperative game theory, which related to PHY, MAC, and network layers respectively. Hence, the readers can have a sense on how to adopt/apply non-cooperative game theory to different problems.

3.1 Physical Layer variety

Cognitive radio technology is a simple idea that is based on the intelligent detection of white spectrum (i.e., spectrum hole). The implementation of cognitive radio technology requires a new type of functionality at the PHY layer to conduct spectrum sensing and allocation in heterogeneous AHCRNs where both PUs and CRs communicate at the same time [39]. Game theory has been shown to be an efficient technique for investigating and resolving the problems of resource allocation in AHCRNs, such as distributed power control and rate and channel allocation. The most commonly used performance metrics in AHCRNs at the physical layer include (i) the function of the estimated signal-to-interference-plus-noise ratio (SINR) and (ii) the bit error rate (BER). However, to design reliable performance metrics for AHCRNs, two main issues must be considered: (i) avoiding interference with the users of the primary networks and (ii) providing good QoS to the users of CRNs.

In game theory, the strategies of players in a PHY layer game can be defined as follows.

Definition 10: A PHY layer game in CRNs can be mathematically modelled as GPHY =

3.1.1 Non-cooperative Power Control Games (NCPCG)

Distributed power control is used to provide efficient interference management in AHCRNs. In other words, transmit power should be allocated carefully to maintain the generated interference to the PUs within a given threshold level. Thus, distributed power control has become one of the most important aspects in AHCRNs due to the significant performance achieved over entire networks when CR nodes limit their power, resulting in resourceful operation for both PUs and CRs. Game theory has been used to study the problem of power control in CRNs (see for example [40]-[47]). In NCGT, each CR node aims to maximise its own utility function by adjusting its own strategy space (i.e., power). However, without a proper distributed power control algorithm, the CR nodes pay no attention to the harm that they may cause the PUs by the interference they generate. One technique that is typically employed to provide better control over the cumulative interference generated by CRs is pricing. Thus, some researchers have adopted the pricing technique to address power control in AHCRNs. Table.7 shows some useful formula related to selected works.

Table 7.Formula related to selected work on non-cooperative power control in CRNs

In [40], a non-cooperative game with non-linear pricing power control was proposed for an MCCDMA cognitive radio system to provide smooth communication service between CRNs and PUNs. The main motivation in this work is the CR network has been modelled by using supermodular game, which resulted in a pure strategy NE. Existence of the NE has been proved mathematically, however, the authors didn’t show the uniqueness of NE.

More complicated non-linear costing function has been proposed in [41]. Power control problem in an underlay spectrum sharing scenario has been considered, where CRs trying to access part of the spectrum in the presence of PUs. Thus, interference temperature constraint has been adopted to control the transmission of the CRs. The novelty of this work comes from adopting the interference temperature constraint and setting of the pricing function that composed of two parts (as shown in Table.7) to indicate: (i) interference generated to other CR in the network, and (ii) an exponential part that reflect the harmful effect of CR’ power to the PUs. Therefore, adequate protection of PUs and fair spectrum sharing has been achieved in this work. Moreover, both existence and uniqueness of the NE has been verified mathematically. The authors in [42] proposed a non-cooperative power control algorithm with dynamic pricing function for a CDMA cognitive radio system. An equivalent bandwidth criterion was used to evaluate the interference among CRs and then dynamically modify the price to be charged for each CR node. The authors claim that the existing NE is unique through the definition of the standard function only. However, there is no evidence to show that the proposed power algorithm satisfies the properties of the standard function. In [40-42], pricing function is adopted to improve the efficiency of the NE for the chosen utility function. However, no NE-convergence results shown in [40]-[42], which consider one of the most important features when dealing with NCGT.

Novel scenario has been proposed in [43], where the problem of power allocation in CRNs is formulated with Stackelberg game model. The key feature in this work is that the PU guarantees QoS by jointly determining required power and the price to be charged from harmful CR. Thus, PU considered as a leader of the game and has the ability to control the transmission of CR and also to estimate power transmission demand for active CR. Two scenarios have been investigated in this work as follows: (i) single-PU single-CR scenario and (ii) single-PU multiple-CR scenario. Equilibrium of Stackelberg game has been derived in both scenarios. However, more sophisticated procedure is required in the second scenario, where multiple CRs are involved since interference from PU-CR and from CR-CR must be considered. Unlike [40]-[42], the authors in [43] proposed a low complexity algorithm to reach the equilibrium point. Moreover, the algorithm has been tested and proved via simulation. However, selfishness and cheating strategy of CR has been ignored from this work. Hence, certain modification must be taken to make the proposed algorithm suitable for more practical AHCRNs scenario.

In [44], the authors proposed a new and efficient power algorithm in CRNs with a novel cost function that considers not only the SINR requirement, but also the effect of power thresholds in CR nodes as shown in Table.7. Existence and uniqueness of the NE are validated mathematically and via simulation as well. The main advantages of this work, among others, comes from its simplicity in introducing the concept of non-cooperative game theory starting from problem formulation, proving existence and uniqueness of NE and finally showing the convergence of NE. Moreover, the proposed algorithm is distributed-based algorithm with fast convergence. Hence, the proposed algorithm is more suitable for practical applications in CRNs. Similar to [43] the scenario of selfishness and cheating strategy has been ignored.

In [45], the authors proposed a novel disturbed power control algorithm by introducing time-varying price to minimize the total interference in underlay spectrum sharing network. Moreover, the proposed algorithm is an extended version of the traditional water filling algorithm. Two scenarios have been proposed in this work: (i) communications, among CR node, with the aid of access point, and (ii) ad-hoc scenario. The novelty of this work comes from the proposed pricing function that resulted in equilibrium in the interference termed as “Interference Equilibrium -IE-”. Unlike [40-44], the pricing function in this work is dynamic and charged only when necessary. That means only if the CR’ interference is above the given threshold then the pricing factor works. Otherwise, the pricing factor is setting to zero and no punishment is needed. Hence, equilibrium and fairness exist in updating the prices function, which is another contribution in this work. The problem of power allocation is formulated neatly to maximize the surplus function as shown in Table.7. In the ad-hoc access model, the authors consider the selfishness of CR nodes, which is another major contribution compared to those in [40-44]. However, the convergence of the proposed distributed algorithm to a stable outcome is slower than that in [43] and [44]. Novel resource allocation scenario in CRNs has been proposed in [46], where the CRs compete among each other and sharing the spectrum concurrently with the PUs over frequencyselective channels. Moreover, more complex theory of finite-dimensional variational inequalities (VI) has been used to analyse and derive the NE. The problem has been formulated so that each CR maximizes his own rate, subject to two main constraints: (i) local power: to control the transmission power of CR node, and (ii) interference constraint: to manage and control the aggregate interference at the receiver of PU. The unique feature of this work comes from introducing new technique to solve and prove uniqueness and existence of the NE in noncooperative game. The proposed distributed algorithm converges to unique NE properly. However, convergence of the proposed game is slower compare to that in [45].

Finally, cooperative game theory was introduced in [47] to design a distributed power control algorithm for CRNs. The problem is formulated using constrained optimization technique and the optimal iterative power algorithm that updates the power strategies is derived using the Lagrangian technique and fixed-point method respectively. Existence and uniqueness of Nash Bargaining Solution (NBS) has been verified mathematically and through simulation as well.

The problem of power control using non-cooperative game theory can be defined as follows: Definition 11: A non-cooperative power control game in AHCRNs, termed an NPC-G, can be modelled as

where denotes the power profile of user CRi.

where is the SINR for the i-th user. The existence and uniqueness of NE in definition 11 can be quickly verified using the following theorem.

Theorem 2: NPC-G has at least one NE.

Proof: NPC-G satisfies the conditions set by theorem 1:

Theorem 3: NPC-G has a unique NE.

Proof: Please refer to [25] and [31] for similar details about the uniqueness of NE.

3.1.2 Joint Strategy: Power-channel and Power-rate Allocation

Power control is an efficient technique used to improve spectrum utilisation by allowing CRs to communicate smoothly in the presence of PUs while sustaining a certain QoS requirement to achieve reliable communication among CR nodes. Solving the channel allocation problem in AHCRNs helps solve the problem of allocating the available spectrum hole among multiple CR nodes. The main objective in this scenario is to maximise the utility function (e.g., achievable rate for CR) while maintaining the required SINR for all primary receivers. Thus, channel allocation in AHCRNs is best combined with power control so that spectrum allocation is performed with no extreme interference caused to the users of the primary networks. Joint channel and power allocation have been investigated in many studies (see, for example, [48], [49]). However, few studies have applied game theory for joint channel and power allocation in the area of AHCRNs due to the challenges presented by mathematical complexity and spectrum fluctuations commonly observed in such distributed-based AHCRNs (see, for example, [50], [51]).

Conversely, ad-hoc CRs require the support of varied services with different transmission rates. Thus, the efficient use of a network’s resources can be achieved by performing power and rate control jointly. Table 8 summarises related work on the development of joint rate-power control strategies using game theory.

Table 8.Related work to joint strategy

The problem of joint strategy can be modelled similarly to the problem of power control (i.e., definition 11) with a different utility function that encompasses strategy space with joint spectrum-power and power-rate issues.

3.2 MAC Layer

The resourceful design of MAC protocols8 in AHCRNs presents new challenges that did not previously exist in classical ad-hoc networks [10]. The decision of CRs to access a spectrum and determining which spectrum hole they must tune for their transmission without affecting the PUs are the most important challenges facing AHCRNs [56]. Furthermore, keeping time synchronisation for sensing coordination purposes and gathering the required information from nearby nodes are other factors that should be considered in MAC protocol design [57]. Several decentralised opportunistic MAC protocols have been proposed by the research community (see, for example, [56-60]). Table 9 provides a brief summary of MAC protocol research that has been performed in the area of AHCRNs.

Game theory offers simple tools with which to study medium access control problems in ad-hoc wireless networks and has been applied to the CSMA/CA [61], [62] and Aloha protocols [63]. Non-cooperative game theory, on the other hand, has been used to study the design of MAC protocols in CRNs (see, for example, [64-66]).

Table 9.Summary of MAC protocol research in the field of AHCRNs

Dynamic spectrum allocation (DSA)-driven MAC protocols for CRNs were proposed in [64]. The DSA-driven MAC protocol consists of four integral components [64]: (i) a DSA algorithm for data coordination among nodes, (ii) a negotiation mechanism to guide players to the right game strategy, (iii) a clustering algorithm to simplify the negotiation operation and (iv) a collision avoidance mechanism to minimise collision among clusters. Combining game theory with the DSA-MAC protocol results in a good improvement in spectrum utilisation because players can be guided to the right strategy and avoids any interference with PUs. Furthermore, the DSA-MAC protocol provides fair QoS support among CRs by adopting a pricing mechanism in the utility function. However, the nodes take time to respond to their partners because of the signalling among clusters. Thus, PUs may appear and claim their bands again, which results in a broken environment among CRs, especially in the areas where the spectrum is heavily used. Another MAC game was proposed in [65]. The interactions among CRs in an overlay spectrum sharing were modelled as a non-cooperative game. The authors proposed a more practical and novel scenario in which CR nodes have different utility functions for their selected strategies. This scenario makes CR nodes more aware about the band occupied by PUs and provides a better solution to the hidden node problem. However, there is no clear idea regarding how to manage the activities of PUs in such a heterogeneous environment. Moreover, the difficulty of analysing the complete design with different utility functions is another drawback of this protocol. An efficient MAC protocol using non-cooperative game theory in CRs was proposed in [66]. The novelty of this scheme lies in the ability of CR nodes to transmit simultaneously along different channels because CR nodes are equipped with multiple radios and compete with each other via CSMA/CA. Here, we will briefly study the model proposed in [66] as a non-cooperative game. The description of the MAC-Game in [66] is summarised in Fig. 7.

The MAC problem described in [66] is split into two sub-games: (i) a channel allocation game (CA), for the optimal allocation of the available spectrum, and (ii) a multiple accesses game (CSMA/CA), played among groups of CR players that share the same spectrum. Given that, and are the total time and the convergence time of the channel allocation and multiple access sub-games, respectively.

The CR players begin optimising the parameters of the CSMA/CA within only after the convergence of the CA sub-game take place as shown in Fig. 7.

Fig. 7.Description of MAC-Game proposed in [66]

In this model, selfish CRs first coordinate to obtain available spectra (i.e., F ). Then, fair spectrum allocation is achieved when CR players optimise the parameters of the CSMA/CA protocol on each channel. The objective of each selfish player9 (e.g., i) is to maximise his or her overall throughput ri in the CRNs. To provide a clear framework of this model, we propose the following definition.

Definition 12: The CSMA/CA game proposed in [66] can be mathematically modelled as GCSMA/CA = , where

where each player undertakes some effort to maximise his or her entire throughput ( ri ) through channel f.

The same approach can be applied to model the other MAC protocol as a non-cooperative game, with each game including different utility functions and different strategies for players.

One of the main advantages of this work is the improvement in spectrum utilisation, which is achieved because the CSMA/CA game allows for fair channel allocation for each radio. Another advantage is the splitting of the MAC game into two sub-games, which facilitates the analysis of NE and helps attain a unique NE that is Pareto-optimal. The approach also offers novel mechanisms that punish the selfish CR players and guide them to use the available spectrum efficiently. However, there is no practical scenario; thus, assumptions must be made about the activities of the PUs, the effect of the sudden appearance of the PUs on the performance of the CR nodes and, ultimately, the efficiency of the NE. All assumptions must be considered to make this model sufficient for practical CRNs.

3.3 Network Layer

One unique issue associated with AHCRNs is that CR nodes are considered temporary users of the available spectrum band. Thus, if the PUs appear suddenly and claim back the spectrum used by the CRs, the transmissions of the CR nodes must be moved to another spectrum hole. This problem in AHCRNs is called spectrum mobility, which gives rise to a new type of handoff in CRNs called spectrum handoff [8], [10]. In AHCRNs, spectrum handoff takes place in the following scenarios [10]: (i) the sudden appearance of Pus; (ii) a CR/relay node is unable to find a spectrum hole; or (iii) the spectrum hole in use by CRs cannot provide the minimum QoS for sufficient transmission. Thus, CR nodes need to communicate in a discontinuous fashion in their search for a new channel and route path to sustain their transmission. As a result, spectrum hole and routing path selection must be carried out jointly in AHCRNs.

3.3.1 Multi-hop Routing in AHCRNs

When CRNs operate in a multi-hop scenario, many challenges and issues arise at higher layers of the protocol stack. Clearly, routing is the most important issue to address to build CR multihop networks (CRMHNs). Multi-hop routing in CRNs presents the following new challenges compared relative to the traditional technology of multi-radio multichannel networks: (i) CR/relay nodes must carry out their transmissions over ON/OFF spectrum holes [67]10; (ii) cross layer optimisation over the PHY, MAC and network layers must be considered for spectrum sensing purposes and to address spectrum fluctuations; and (iii) relay nodes must perform spectrum sensing. To access available spectrum holes, a distributed algorithm, for relaying purposes, that considers the compensation for interference generated to PUs and the choice of the best path to the destination is required. Unfortunately, there is not much extensive research that has been performed in the area of multi-hop routing in AHCRNs because end-to-end routes can be broken by the sudden appearance of PUs.

Three possible routing approaches [68] are summarised in Table 10, which shows that a sufficient routing protocol can be developed depending on the performance of the PUs on their licensed bands compared to the holding time for CR transmission [69].

Table 10.Classification of routing approaches in AHCRNs

3.3.2 Joint Channel and Route Selection in AHCRNs

The problem of joint spectrum and route selection in multi-hop CRNs has been investigated in several research studies (see, for example, [69]-[71]). However, no optimal routing solution has been proposed in particular for the opportunistic routing environment. In [69], the authors proposed a comprehensive model for joint channel assignment and route selection in a semi-static multi-hop environment. In this work, the activities of PUs were assumed to be nearly stationary, and thus, the channel assignment and route selection algorithm could be modelled using a static MHCRNs scenario, as indicated in Table 8. Therefore, the iterative algorithm proposed in this work cannot be directly applied to dynamic, opportunistic and more practical scenarios in which the CRs share the same spectrum with PUs in a dynamic manner. Dynamic rate allocation, spectrum sharing and route selection for MHCRNs were investigated in [70]. The problem was formulated as a sequential decision process to address the average consumption of total power during each scheduling cycle. However, in this work, no clear method for handling the sudden appearance of ON/OFF spectrum holes was developed, which resulted in unstable route selection among CR nodes during the data session. An extended version of the traditional on-demand routing protocol was considered in [71] for MHCRNs. The authors proposed a spectrum-aware on-demand routing protocol for operating within a multi-flow scheduling environment. One of the most significant contributions of this work was the careful consideration of intersecting nodes using a novel scheduling scheme that provides these nodes the ability to evaluate the selected route and thus facilitate the flow of data. A routing metric that considers the delay of spectrum switching and backoff delay is another notable contribution. However, this work is not adequate for practical CRNs in which the opportunistic spectrum scenario occurs because the cumulative delay along the routing path will be uncontrollable.

A general multi hop ad-hoc CRN (MHAH-CRN) scenario is illustrated in Fig. 9. In this scenario, CRrn represent the relay nodes. Transmitters send data to the relay node to forward the received data to the receiver side. Without loss of generality, we assume that no interference is generated among PUs.

Fig. 9.Example of multi-hop routing in AHCRNs

To simplify the game theory analysis of the problem of ad-hoc MHCRs, we propose the following general definition.

Definition 12: A non-cooperative spectrum and routing game (NCSRG) can be mathematically modelled as NCRG = , where

where hi (Ri) is the number of routes available to player i, IE(ℓ, fi) is the interference level of link ℓ on channel fi that may disturb the transmission of PU and Ri is the set of paths from the source to sink.

To the best of our knowledge, not much extensive research has been conducted to analyse the problem of joint spectrum allocation and route selection in multi-hop CRNs using game theory. However, one of the pioneering studies on MHCRN resource allocation is [73]. In this study, end-to-end resource allocation in MHCRNs was performed, where both power and channels were allocated under the interference model to maximise the number of routes that could serve in the network. Three different games were implemented to this end: (i) a local flow games, (ii) a potential flow game and (iii) a cooperative link game, where the games varied in the selection of the utility function. However, no clear method for maintaining an active route among CR nodes throughout the entire data session was developed.

 

4. Challenges and Outlook into Upcoming Research

This survey discusses several non-cooperative game theoretic approaches to evaluating different choices in the existing problems associated with AHCRNs at different OSI layers, such as power control, channel/rate allocation in the PHY layer, the CSMA/CA game in the MAC layer and the channel allocation and routing problem in the network layer. The main game theoretic concepts invoked were those of non-cooperative game theory, which provide a strong basis for analysing Nash equilibrium. The following are unresolved issues related to the concepts presented in this survey.

4.2 Cooperative game theory: This overview focuses on non-cooperative game theory, in which each player is concerned about improving his/her utility function. Cooperative game theory, however, plays an important role in investigating the problem of resource allocation in AHCRNs, where players can communicate with one another to make a decision on how to play the game and how to allocate the resources efficiently. Cooperative game theory involves two main topics: bargaining game and coalitional games [74]. Efficient resource allocation in AHCRNs can be achieved by considering coalitional game theory, which represents one direction for future work.

4.3 Pricing mechanism: In this paper, the pricing mechanism was applied to the PHY layer only to improve the convergence of NE. The pricing mechanism can be applied to MHCRNs to address the problem of joint channel and route allocation to improve the performance of CRs by providing smooth communication service with the owner of the spectrum. In this scenario, CRs need to pay one price to PUs for spectrum sharing and another price to relay nodes in between for relaying purposes [75]11. Resource allocation (including channel, rate and route) in MHCRNs based on a pricing mechanism is another direction of research in the area of AHCRNs.

4.4 Security12 guarantee in AHCRNs: Guaranteeing security in AHCRNs is a challenging matter and also considered one of the greatest aspects in this field. In point of fact, the main challenges of securing AHCRNs come from the following reasons: (i) the distributed and self-organising nature of AHCRNs on the one hand and (ii) the highly dynamic spectrum available on the other. These unique features of AHCRNs require a new security technique because the traditional one can only be used for static spectrum allocation [35]. Several problems need to consider when dealing with security issues in AHCRs, for example, securing routing protocol, selfish misbehaviour and cheating strategy. However, the performance of security features in AHCRNs can be improved by means of collaboration among distributed nodes. Thus, in order to enhance the security aspect in AHCRNs it is important to figure out the trustworthiness (e.g., trust management) of contributing nodes since trust is the main key behind collaboration. Game theory, including cooperative and non-cooperative, can be applied to design and implement crucial security mechanisms to ensure the robustness of both network and nodes against attacks. However, in case of non-cooperative game, it is advisable to not directly model the problem as a NCGT. Instead, the players willingly collaborate with each other by exchanging security information. Furthermore, fictitious game can be adopted in non-cooperative scenario to facilitate the existence and uniqueness of NE. application of game theory to enhance security aspect in AHCRNs represents another direction for future work.

4.5 Spectrum handoff: Game theory can be applied to study and analyse the problem of spectrum handoff in AHCRNs where the available spectrum hole is fluctuating due to the frequent appearance of the PUs and changes in the available spectrum holes. The application of game theory to the problem of spectrum handoff must be considered by the research community.

 

5. Concluding Remarks

In this survey, we demonstrated how the fundamental concept of non-cooperative game theory can be applied to ad-hoc cognitive radio networks (AHCRNs). The studies presented in this review demonstrate that game theory can be used to better understand the complex interactions between CR nodes in this highly interfered and distributed environment. The main challenge facing AHCRNs is determining how to integrate spectrum decisions, spectrum sharing and spectrum mobility in the layers of the protocol stack using game theory so that each CR node can communicate efficiently in a distributed manner. In this context, pricing theory can be used to improve the efficiency of NE in AHCRNs by making selfish nodes aware of the inefficient NE.

Regarding the existing AHCRN Game-MAC solutions, various issues have not been studied. For example, the complete design of a non-cooperative game MAC protocol that includes (i) spectrum sensing, (ii) a decision-making techniques and (iii) spectrum mobility remains an open issue. Regarding multi-hop routing games in AHCRNs, we believe that there is more research that must be performed to design a game model that maintains the stability of both channels and routes while minimising the generated interference to PUs. The selection of utility functions in the MHCRNs joint spectrum and routing game is not an easy factor to consider because it depends on the availability of the spectrum and the sudden appearances of the PUs. Meanwhile, the utility function in multi-hop games follows the routing approaches described in Table 8. We hope that this survey offers a glance of the application of non-cooperative game theory to AHCRNs in different OSI layer stacks and offers help to interested researchers in their specific areas of study.