EO-1 Hyperion:

Hyperion (Pearlmann, 2003) is a hyperspectral sensor on board the NASA Earth

Observer 1 (EO-1) which functioned from November, 2001 to April, 2016. With a

242 channels ranging from the VNIR to SWIR, 400 to 2500 nm, Hyperion images at

a spatial resolution of 30m and a radiometric resolution of 12 bits. Hyperion

data is available as level 1, radiometrically calibrated product (Level 1R) as

well as radiometrically, geometrically corrected and georeferenced data (Level

1GST and 1T). Each of these datasets need to be further corrected before they

can be used to develop products of our interest. Of the 242 bands, some are

highly noisy and a redundancy exists due to the overlap of detecting regions by

the VNIR and SWIR detector. Hence we ignore these bands and the total usable

bands comes down to 196 (1-7, 58-78 and 225-242 are removed). This is followed

by correction due to effects of the atmosphere. The corrections often reduce

the SNR. To correct for this, attempts to separate the noise from the data are

performed using the MNF denoising technique.

RapidEye:

The RapidEye constellation is a set of 5 satellites designed and developed by

Surrey Satellite Technology Ltd in 2008 and later acquired by Planet in 2015.

It provides imagery of 5m/pixel spatial resolution in 5 bands (red, green,

blue, red-edge and near infrared). With an elevation of 630km and a 77km swath,

RapidEye has a revisit time of 5.5 days at nadir. All five satellite sensors

are calibrated similarly leaving no distinction between satellite imagery from

any sensor. The a mosaic of 5 level 3 ortho-corrected imagery was used for this

study. Atmospheric correction was performed using the Quick Atmospheric

Correction (QuAC) technique.

2.2

Study Area

The

study area is the Dhemaji and Lakhimpur region of Assam which is abundant in

rice crop cultivation. Hence, the red edge region is specifically used for

further discrimination within the vegetation class. Dhemaji district of Assam

is bound by Arunachal Pradesh in the north and the river Brahmaputra in the

south. It is a plain area with an average elevation of 104 m above sea level.

Numerous drainage systems originating from the hills of Arunachal Pradesh flow

through Dhemaji to drain into the Brahmaputra. Physiographically, Dhemaji is in

the form of three main sub districts: the piedmont zone where Dhemaji borders

the Arunachal Himalayas, the active

flood plains near the river Brahmaputra and its tributaries, and the low lying

alluvial belt. With a total geographic area of 323,700 Ha, Dhemaji covers a

variety of socio-geographic features including build up (208 Ha), horticultural

lands (2534 Ha), forest cover (53,225 Ha) and grasslands (97,167 Ha), making it

a perfect study area for the classification techniques aimed towards vegetation.

Five regions were selected across Dhemaji and Lakhimpur and data of Hyperion

and RapidEye taken on October, 2012 were analyzed and regions of interest were

selected based on the LULC map generated at the North Eastern Space

Applications Centre for 2011-2012. Each region and the subsequent classes we

have classified into are summarized below. The regions used for the study are

depicted in Figure 1. Details of training and test samples are available in

Table 2-4.Region 1: This

region depicts mainly the following features in the LULC map. A perennial and

non-perennial drainage system, agricultural lands, fallow lands and dense

plantations surrounding the rural build up near the water bodies. As the rural

build up was evident only in the form of a few pixels in the RapidEye image, we

have selected four classes for this region: (1) water body, (2) vegetation type

1 for agricultural lands, (3) vegetation type 2 for dense plantations

intermingled with rural build up and (4) fallow lands.

Region 2: This

region is geographically similar to region one. There exists a perennial water

body as well as waterlogged regions, agricultural lands, fallow lands and dense

plantations around rural build up. Five classes were selected: (1) water body,

(2) vegetation type 1 for agricultural lands, (3) vegetation type 2 for dense

plantations, (4) fallow lands and (5) water logged wastelands. The similarity

with region 1 was maintained so that accuracies could be checked in varying the

number of classes.

Region 3: Five

classes were selected for this region. However, a larger variety of vegetation

types were selected to check if the red edge bands could accurately assess the

class type. The classes selected are as follows: (1) water body, (2) vegetation

type 1 for agricultural lands, (3) vegetation type 2 for agricultural lands,

(4) dense plantations and (5) barren/fallow lands.

3 METHODOLOGY

A

Hyperion strip over the Dhemaji and Lakhimpur districts of Assam taken on

October, 2012 was used. RapidEye images over the same region for October, 2012

at similar local times were mosaicked and subsetted to the same areal extent as

the Hyperion data. This was followed by georeferencing the Hyperion data set to

the RapidEye data set using control points. Once both datasets were prepared,

the processing in the form of bad band removal was performed for Hyperion and

atmospheric corrections were performed on both Hyperion and RapidEye images. The

corrected and georeferenced images were fused using the Gram-Schmidt

hyperspectral sharpening method. See the original Rapid Eye image, Hyperion

image and fused image in Fig 2 a ,b and c respectively. The spectral

characteristic of the region is as observed in Fig 3 a ,b and c.Hyperspectral

datasets require significant denoising for enhanced spectral understanding.

Hence, the Minimum Noise Fraction Transformation as implemented by Green et al,

1998 was used. This is implemented using the concept of principle components.

The Eigen values and Eigen vectors of the image are obtained and ordered (See

Fig 2). The image is then projected onto the Eigen space to decorrelate the

bands and noise whitening is performed. The noise related Eigen value are

discarded and the signal rich Eigen-images are employed for the inverse that

gives us the denoised image that is used for classification.

A

spectral subset of the fused image and the Hyperion image in the red edge

region was also considered to understand if a red edge based classification

depicts superior results in the case of vegetation classification.Multiple

previous studies have been performed using fusion of hyperspectral or

multispectral images and panchromatic high resolution images (e.g. Pohl, 2013,

Akhtar et al, 2014, Yokoya et al, 2017). Yakoya et al, 2017 has further

performed a comparison of results for the different fusion methods for a

variety of datasets including AVIRIS, HyDICE etc. Many fusion algorithms exist

which vary in the accuracy either resulting in spatial or spectral distortions

(Zhang et al, 2007; Yakoya et al, 2012; Qian and Chen, 2012). Recent years have

seen more sophisticated attempts at the hyperspectral fusion challenge. Chen et

al., 2014 performed fusion by dividing the multispectral image into individual

bands and hyperspectral image into segments of bands centred around a

corresponding multispectral band and performed pan sharpening on each segment.

In this study, we use the Gram Schmidt spectral sharpening method (summarized

in Laben and Brower, 2000 and Maurer, 2013). This involves the hyperspectral

image combined via a linear combination by using weights to represent it as a

panchromatic image.

…

(1)

This

is followed by the Gram Schmidt orthonormalization of the vectors in the N

dimensional space where each band represents an N dimensional vector where N is

the number of pixels. This procedure decorrelates the bands. The

orthonormalization procedure involves the pan band taken as the first vector (v1).

It follows the general formula as follows where ui is the

orthonormal vector and vi is the original vector:

u1=v1

…

(2)

… (3)

This

is followed by replacing the averaged panchromatic image with the high spatial

resolution panchromatic image and performing an inverse Gram Schmidt transform

(similar to the forward transform). This gives us the fused product.

In

order to check for the validity of the fused product, point to point spectral

correlation was investigated between the fused data and the Hyperion data and

the correlation coefficient and regression coefficient were calculated.

3.2

Classification

Five

regions within the Dhemaji-Lakhimpur region was considered. Classification was

performed using ROIs derived from the LULC map as archived by the North Eastern

Space Applications Centre and visual examination of the images. Hyperspectral

classification techniques are adversely affected by the Houghes effect due to

which the required number of training samples for larger number of bands

increases to maintain the accuracy. Breunig et al, 2011 suggest that the SVM,

SID and SAM classifiers do not demonstrate a reduction in accuracy. Hence, two

classifiers were used for the purpose: Support Vector Machines (SVM) and

Spectral Angle Mapper (SAM). Classification was performed on the three segments

using RapidEye, Hyperion and the fused result of RapidEye and Hyperion and the

results were assessed by comparing the kappa coefficient and accuracy

assessment.

3.2.1 Spectral Angle Mapper (SAM) Classifier:

SAM is a classifier that compares the similarity between the test and training

samples by considering the spectrum to

be a D dimensional vector where D is given by the number of bands. The

training samples are either the laboratory spectra in the form of spectral

libraries that have been resampled to the dimensionality of the test samples.

Alternately, they are obtained from known regions within the satellite imagery

that is being classified. This study uses ROIs as obtained from a ground survey

performed in the Dhemaji and Lakhimpur district in 2012. Spectral similarity is

estimated by calculating the angle between the test and training spectrum

vectors.

…

(4)

Larger

angles suggest dissimilarity. An advantage SAM has over other traditional

classifiers is the independence from intensity values permitting regions of

shadow to also be classified accurately.

3.2.2 Support Vector machine (SVM)

Classifier: Support Vector Machine based

classification is a well recognized classification technique where an N-1

dimensional hyperplane is used to separate the data by maximizing the margin

between them. The hyperplane is called the optimal hyperplane and the data

points closes to the hyperplane are the constraining factors and are called the

support vectors. This suggests that SVM is a linear classifier. However, to

account for non linear classification, SVM can be used alongside kernels such

as polynomial, radial basis function, sigmoid etc. This study utilizes two

commonly used kernels which have demonstrated significant success in the past (Gordon,

2004). The mathematical form of the polynomial kernel is given as below:

… (5)

Here,

x is the input, xi is the support vector and d is the degree of the polynomial

to be used. See below for the radial basis function kernel expression:

… (6)

There

are two parameters of concern that can be tweaked: the gamma parameter and the penalty

parameter. The gamma parameter depicts the influence of the training sample

with smaller values causing far reaching influence. The penalty parameter or

the misclassification trade off parameter affects the smoothness of the

decision boundary. Larger values cause over fitting.

3.2.3 Maximum Likelihood Classifier

(MLC): MLC is based on the

Bayes law following posterior=prior*likelihood/evidence given by:

… (7)

… (8)

Generally, the prior probability of the for a class

? is assumed to be a constant or expected to be equal to each other and the

evidence, P(x), is usually common to all classes, therefore Lx is

dependent on P(x/?). Classification is performed such that the likelihood of x

belonging to a class ? is maximized. Sampling

should be such that the estimation of the mean and covariance is reflective of

that of the population. The maximum likelihood method is not useful when the distribution of the

population does not follow the normal distribution.

4. RESULTS AND DISCUSSION:

4.1

Spectral Characteristics of the Fused Product:

From Figure 3, we observe that the Gram Schmidt

spectral sharpening leads to an offset in the intensity values of the fused

product. In order to study if any distortion occurs or if the difference lies

solely in the offset, we have performed a correlation test on the data set. Ten

random regions were selected from both the Hyperion image and the fused image.

The average spectra of each region was computed and a scatter plot was used to

check for the correlation. Very high correlation was observed with an average

Pearson coefficient of correlation as 0.98 and a regression coefficient of

0.96.Multiple

previous studies have been performed using fusion of hyperspectral or

multispectral images and panchromatic high resolution images (e.g. Pohl, 2013,

Akhtar et al, 2014, Yokoya et al, 2017). Yakoya et al, 2017 has further

performed a comparison of results for the different fusion methods for a

variety of datasets including AVIRIS, HyDICE etc. Many fusion algorithms exist

which vary in the accuracy either resulting in spatial or spectral distortions

(Zhang et al, 2007; Yakoya et al, 2012; Qian and Chen, 2012). Recent years have

seen more sophisticated attempts at the hyperspectral fusion challenge. Chen et

al., 2014 performed fusion by dividing the multispectral image into individual

bands and hyperspectral image into segments of bands centred around a

corresponding multispectral band and performed pan sharpening on each segment.

In this study, we use the Gram Schmidt spectral sharpening method (summarized

in Laben and Brower, 2000 and Maurer, 2013). This involves the hyperspectral

image combined via a linear combination by using weights to represent it as a

panchromatic image.

…

(1)

This

is followed by the Gram Schmidt orthonormalization of the vectors in the N

dimensional space where each band represents an N dimensional vector where N is

the number of pixels. This procedure decorrelates the bands. The

orthonormalization procedure involves the pan band taken as the first vector (v1).

It follows the general formula as follows where ui is the

orthonormal vector and vi is the original vector:

u1=v1

…

(2)

… (3)

This

is followed by replacing the averaged panchromatic image with the high spatial

resolution panchromatic image and performing an inverse Gram Schmidt transform

(similar to the forward transform). This gives us the fused product.

In

order to check for the validity of the fused product, point to point spectral

correlation was investigated between the fused data and the Hyperion data and

the correlation coefficient and regression coefficient were calculated.

3.2

Classification

Five

regions within the Dhemaji-Lakhimpur region was considered. Classification was

performed using ROIs derived from the LULC map as archived by the North Eastern

Space Applications Centre and visual examination of the images. Hyperspectral

classification techniques are adversely affected by the Houghes effect due to

which the required number of training samples for larger number of bands

increases to maintain the accuracy. Breunig et al, 2011 suggest that the SVM,

SID and SAM classifiers do not demonstrate a reduction in accuracy. Hence, two

classifiers were used for the purpose: Support Vector Machines (SVM) and

Spectral Angle Mapper (SAM). Classification was performed on the three segments

using RapidEye, Hyperion and the fused result of RapidEye and Hyperion and the

results were assessed by comparing the kappa coefficient and accuracy

assessment.

3.2.1 Spectral Angle Mapper (SAM) Classifier:

SAM is a classifier that compares the similarity between the test and training

samples by considering the spectrum to

be a D dimensional vector where D is given by the number of bands. The

training samples are either the laboratory spectra in the form of spectral

libraries that have been resampled to the dimensionality of the test samples.

Alternately, they are obtained from known regions within the satellite imagery

that is being classified. This study uses ROIs as obtained from a ground survey

performed in the Dhemaji and Lakhimpur district in 2012. Spectral similarity is

estimated by calculating the angle between the test and training spectrum

vectors.

…

(4)

Larger

angles suggest dissimilarity. An advantage SAM has over other traditional

classifiers is the independence from intensity values permitting regions of

shadow to also be classified accurately.

3.2.2 Support Vector machine (SVM)

Classifier: Support Vector Machine based

classification is a well recognized classification technique where an N-1

dimensional hyperplane is used to separate the data by maximizing the margin

between them. The hyperplane is called the optimal hyperplane and the data

points closes to the hyperplane are the constraining factors and are called the

support vectors. This suggests that SVM is a linear classifier. However, to

account for non linear classification, SVM can be used alongside kernels such

as polynomial, radial basis function, sigmoid etc. This study utilizes two

commonly used kernels which have demonstrated significant success in the past (Gordon,

2004). The mathematical form of the polynomial kernel is given as below:

… (5)

Here,

x is the input, xi is the support vector and d is the degree of the polynomial

to be used. See below for the radial basis function kernel expression:

… (6)

There

are two parameters of concern that can be tweaked: the gamma parameter and the penalty

parameter. The gamma parameter depicts the influence of the training sample

with smaller values causing far reaching influence. The penalty parameter or

the misclassification trade off parameter affects the smoothness of the

decision boundary. Larger values cause over fitting.

3.2.3 Maximum Likelihood Classifier

(MLC): MLC is based on the

Bayes law following posterior=prior*likelihood/evidence given by:

… (7)

… (8)

Generally, the prior probability of the for a class

? is assumed to be a constant or expected to be equal to each other and the

evidence, P(x), is usually common to all classes, therefore Lx is

dependent on P(x/?). Classification is performed such that the likelihood of x

belonging to a class ? is maximized. Sampling

should be such that the estimation of the mean and covariance is reflective of

that of the population. The maximum likelihood method is not useful when the distribution of the

population does not follow the normal distribution.

4. RESULTS AND DISCUSSION:

4.1

Spectral Characteristics of the Fused Product:

From Figure 3, we observe that the Gram Schmidt

spectral sharpening leads to an offset in the intensity values of the fused

product. In order to study if any distortion occurs or if the difference lies

solely in the offset, we have performed a correlation test on the data set. Ten

random regions were selected from both the Hyperion image and the fused image.

The average spectra of each region was computed and a scatter plot was used to

check for the correlation. Very high correlation was observed with an average

Pearson coefficient of correlation as 0.98 and a regression coefficient of

0.96.