LOSSY COMPRESSION USING STATIONARY

0 downloads 0 Views 3MB Size Report
begin by thanking the people without whom it would not have been possible for me to submit this ... that the proposed approach resize the image in accordance with the measured rate of different ...... stationary wavelet transform are performed using MATLAB programming language. The ...... X = imresize(X, [SX SY]);. [ORX ...
Lossy Compression Using Stationary Wavelet Transform and Vector Quantization Thesis Submitted to Department of Information Technology Institute of Graduate Studies and Research Alexandria University

In Partial Fulfillment of the Requirements For the Degree Of

Master In

Information Technology By Omar Ghazi Abbood Khukre B.Sc. of Computer Science – 2011 Al-Turath University College, IRAQ-Baghdad

2016

Lossy Compression Using Stationary Wavelet Transform and Vector Quantization A Thesis Presented by Omar Ghazi Abbood Khukre For the Degree of Master In Information Technology Examiners’ Committee:

Approved

Prof. Dr. Mahmoud Mohamed Hassan Gabr

…………………….

Prof. of Mathematics, Faculty of science, Alexandria University

Prof. Dr. Abd El Baith Mohamed Mohamed Prof. of Computer Engineering Arab Academy for Science and Technology And Maritime Transport Department of computer engineering

…………………….

Prof. Dr. Shawkat K. Guirguis

…………………….

Prof. of Computer Science & Informatics Department of Information Technology Institute of Graduate Studies & Research Alexandria University

Date:

/

/

Advisor’s Committee:

Approved

Prof. Dr. Shawkat K. Guirguis Professor of Computer Science & Informatics and Vice Dean for Graduate Studies and Research Institute of Graduate Studies & Research Alexandria University

……………………….

Supervisor

Prof. Dr. Shawkat K. Guirguis Professor in Computer Science & Informatics Department of Information Technology Institute of Graduate Studies & Research Alexandria University

DECLARATION I declare that no part of the work referred to in this thesis has been submitted in support of an application for another degree or qualification of this or any other university or other institution of learning.

Name: Omar Ghazi Abbood Khukre Signature:

Acknowledgment To Allah, first and foremost, I bow, for he granted me the ability to complete this thesis, and his continuous help during all the steps of my work and my life. I would like to begin by thanking the people without whom it would not have been possible for me to submit this thesis. First, I would like to thank my principal supervisor, Prof. Dr. Shawkat K. Guirguis, professor of computer science, department of information technology, institute of graduate studies & research, Alexandria University, for his invaluable guidance, encouragement and great suggestions from the very early stages of this research. I am very grateful for his effort and his highly useful advice throughout the research study. I have benefited greatly from his experience and direction. I would like to record my gratitude and my special thanks to Dr. Hend Ali Elsayed Elsayed Mohammed, lecturer in communication and computer engineering department, faculty of engineering, delta university for science and technology, for her advice, guidance, invaluable comments, helpful discussions and her priceless suggestions that made this work interesting and possible. My deep pleasure goes to my older brother Dr. Mahmood A. Moniem, lecturer in Institute of Statistical Studies and Research, Cairo University. On encouragement and great suggestions in the stages of this research. I am very grateful for his effort and his highly useful advice throughout the research study. I have benefited greatly from his experience. Finally, I would like to thank my family, my father, and my mother whom I beseech Allah to protect, without whom I could not have made it here and achieved my dream, and all my best wishes to my brothers, sisters who gave me tips. I would like to express my thankfulness and gratitude to my friends who extend a helping hand to me and advice with continued support.

i

ABSTRACT Compression is the art of representing the information in a compact form rather than in its original or uncompressed form. In other words, using the data compression, the size of a particular file can be reduced. This is very useful when processing, storing or transferring a huge file, which needs lots of resources. If the algorithms used to encrypt work properly, there should be a significant difference between the original file and the compressed file. When the data compression is used in a data transmission application, speed is the primary goal. The speed of the transmission depends on the number of bits sent, the time required for the encoder to generate the coded message, and the time required for the decoder to recover the original ensemble. In a data storage application, the degree of compression is the primary concern. Compression can be classified as either lossy or lossless.

Image compression is a key technology in the transmission and storage of digital images because of vast data associated with them. This research suggests an effective approach for image compression using Stationary Wavelet Transform (SWT) and Vector Quantization which is a Linde Buzo Gray (LBG) vector quantization in order to compressed input images in four phases; namely preprocessing, image transformation, zigzag scan, and lossy/lossless compression. Preprocessing phase takes images as input, so that the proposed approach resize the image in accordance with the measured rate of different sizes to (8 × 8) And then converted from (RGB) to (gray scale). Image transformation phase received the resizable gray scale images and produced transformed images using SWT. Zigzag scan phase takes as an input the transformed images in 2D matrix and produced images in 1D matrix. Finally, in lossy/lossless compression phase takes 1D matrix and apply LBG vector quantization as lossy compression techniques and other lossless compression techniques such as Huffman coding and arithmetic coding. The result of our approach gives the highest possible compression ratio and less time possible than other compression approaches. Our approach is useful in the internet image compression.

ii

TABLE OF CONTENTS Acknowledgement ..................................................................................................................i Abstract ................................................................................................................................ ii Table of Contents ................................................................................................................. iii List of Figures .......................................................................................................................vi List of Tables...................................................................................................................... viii List of Symbols and Abbreviations ......................................................................................ix CHAPTER 1: INTRODUCTION ....................................................................................... 1 1.1 Lossy Compression.................................................................................................... 3 1.1.1 Transform Coding ............................................................................................. 4 1.1.2 Vector Quantization ........................................................................................... 5 1.2 Wavelet transforms .................................................................................................... 5 1.2.1 Discrete Wavelet Transform .............................................................................. 6 1.2.2 Lifting Wavelet Transform ................................................................................ 7 1.2.3 Stationary Wavelet Transform........................................................................... 8 1.3 Problem Statement ..................................................................................................... 9 1.4 Research Objective .................................................................................................. 10 1.5 Contribution of the thesis ........................................................................................ 10 1.6 Thesis Organization ................................................................................................. 10 CHAPTER 2: BACKGROUND AND LITERATURE REVIEW ................................. 11 2.1 Background .............................................................................................................. 11 2.1.1 Compression Techniques ................................................................................. 11 2.1.2 Lossy Compression using Vector Quantization............................................... 12 2.1.2.1 Linde-Buzo-Gray Algorithm ................................................................... 15 2.1.2.2 Equitz Nearest Neighbor Algorithm ....................................................... 16 2.1.2.3 Back Propagation Neural Network Algorithm ........................................ 18 2.1.2.4 Fast Back Propagation Algorithm ........................................................... 20 2.1.2.5 Joint Photopphic Experts Group ............................................................. 22 2.1.2.6 JPEG2000 ................................................................................................ 23

iii

2.1.3 Lossless Compression Techniques .................................................................. 24 2.1.3.1 Models and Code..................................................................................... 24 2.1.3.1.1 Huffman Coding ............................................................................... 24 2.1.3.1.2 Arithmetic Coding ............................................................................ 27 2.1.3.2 Dictionary Model .................................................................................... 31 2.1.3.2.1 Lempel Ziv Welch ........................................................................... 31 2.1.3.2.2 Run Length Encoding ....................................................................... 32 2.1.3.2.3 Fractal Encoding............................................................................... 33 2.1.4 Wavelet Transform .......................................................................................... 37 2.2

Literature Review for Various Techniques of Data Compression ....................... 39

2.2.1 Related Work ................................................................................................... 39 2.2.2 Previous Work ................................................................................................. 43 2.3 Summary ................................................................................................................. 48 CHAPTER 3: LOSSY COMPRESSION USING STATIONARY WAVELET TRANSFORMS AND VECTOR QUANTIZATION. .................................................... 49 3.1 Introduction 3.2 System Architecture ................................................................................................ 49 3.3 Preprocessing ........................................................................................................... 51 3.4 Image Transformation ............................................................................................. 52 3.4.1 Discrete Wavelet Transform… ........................................................................ 52 3.4.2 Lifting wavelet transform ................................................................................ 53 3.4.3 Stationary wavelet transform ........................................................................... 54 3.5 Zigzag Scan ............................................................................................................. 56 3.6 Lossy Compression, Vector quantization by Linde-Buzo-Gray ............................ 56 3.7 Lossless Compression.............................................................................................. 58 3.7.1 Arithmetic Coding ........................................................................................... 59 3.7.2 Huffman Coding .............................................................................................. 60 3.8 Compression Ratio .................................................................................................. 60 3.9 Summary.................................................................................................................. 60

iv

CHAPTER 4: EXPERIMENTS & RESULTS ANALYSIS .......................................... 61 4.1 Data Set and Its Characteristics ............................................................................... 61 4.2 Image formats used .................................................................................................. 61 4.3 PC Machine ............................................................................................................. 62 4.4 Experiments ............................................................................................................. 63 4.4.1 Experiment (1) ................................................................................................. 63 4.4.2 Experiment (2) ................................................................................................. 65 4.4.3 Experiment (3) ................................................................................................. 67 4.4.4 Average Compression Ratio ............................................................................ 69 4.5 Results Analysis ...................................................................................................... 71 CHAPTER 5: CONCLUSION AND FUTURE WORK ................................................ 72 5.1 Conclusion ............................................................................................................... 72 5.2 Future Work ............................................................................................................. 73 REFERENCE .................................................................................................................... 74 APPENDICES Appendix I: Implementation of lossy compression using stationary wavelet transform and vector quantization Appendix II: GUI of Implementation ARABIC SUMMARY

v

LIST OF FIGURES Page

Figure Figure 1.1:

Vector quantization encoder and decoder. ............................................. 3

Figure 1.2:

Lossy compression framework .............................................................. 4

Figure 1.3:

2D - Discrete wavelet transform ............................................................ 6

Figure 1.4:

Wire diagram of the forward transformation with the lifting scheme. .................................................................................................. 7

Figure 1.5:

Stationary wavelet decomposition of a two-dimensional image ........... 8

Figure 2.1:

Code words in 1-dimensional space ..................................................... 12

Figure 2.2:

Code words in 2-dimensional space ..................................................... 13

Figure 2.3:

The encoder and decoder in a vector quantizer .................................... 14

Figure 2.4:

Flowchart of Linde-Buzo-Gray algorithm ............................................ 16

Figure 2.5:

Back propagation neural network image compression system ............. 18

Figure2.6:

First level wavelet decomposition ........................................................ 37

Figure 2.7:

Conceptual diagram of the difference map generated by the vector quantization compressed ....................................................................... 40

Figure 2.8:

Block diagram of the proposed method (compression phase) .............. 42

Figure 2.9:

Block diagram of the proposed system ................................................. 42

Figure 2.10:

The structure of the wavelet transforms based compression ................. 43

Figure 2.11:

Extended hybrid system of discrete wavelet transform - vector quantization for image compression ..................................................... 44

Figure 2.12:

Block diagram of the proposed super resolution algorithm. ................. 45

Figure 2.13:

Flowchart of data folding. ..................................................................... 46

Figure 2.14:

Block diagram for wavelet–CPN based image compression ................ 47

Figure 3.1:

Architecture of the propose algorithm .................................................. 50

Figure 3.2:

Diagram conversion and downsizing .................................................... 52

Figure 3.3:

2D - Discrete wavelet transform ........................................................... 53

Figure 3.4:

Diagram lifting wavelet scheme transform ........................................... 54

vi

Figure 3.5:

3 level Stationary wavelet transform filter bank ................................... 54

Figure 3.6:

Stationary wavelet transform filters ...................................................... 54

Figure 3.7:

Zigzag scan ........................................................................................... 56

Figure 3.8:

Block diagram for lossy compression ................................................... 56

Figure 3.9:

Flowchart of Linde Buzo Gray algorithm............................................. 58

Figure 3.10:

Block diagram for lossless compression ............................................... 59

Figure 4.1:

Chart shows the result average compression ratio in level – 1 ............. 69

Figure 4.2:

Chart shows the result average compression ratio in level – 2 ............. 70

Figure 4.3:

Chart shows the result average compression ratio in level – 3 ............. 70

Figure 4.4:

Best path for lossy image compression................................................. 71

vii

LIST OF TABLES Table

Page

Table 2.1:

Comparison between lossy and lossless compression techniques ........... 12

Table 2.2:

Comparison of various algorithms of vector quantization....................... 21

Table 2.3:

Huffman coding ....................................................................................... 26

Table 2.4:

Huffman coding vs. Arithmetic coding ................................................... 30

Table 2.5:

Summarizing the advantages and disadvantages of various lossless compression algorithms ........................................................................... 36

Table 2.6:

Advantages and disadvantages of wavelet transform .............................. 39

Table 4.1:

Discrete wavelet transforms, vector quantization (Linde Buzo Gray), arithmetic and Huffman coding .................................................... 64

Table 4.2:

Lifting wavelet transforms, vector quantization (Linde Buzo Gray), arithmetic and Huffman coding ............................................................... 66

Table 4.3:

Stationary wavelet transforms, vector quantization (Linde Buzo Gray), arithmetic and Huffman coding .................................................... 68

viii

LIST OF SYMBOLS AND ABBREVIATIONS 2D

Two-dimensional space

AC

Arithmetic Coding

BPNN

Back Propagation Neural Network

BPP

Bits Per Pixel

CCITT

Comite Consultative International Telephonique of Telegraphique Graphique

CR

Compression Ratio

DCT

Discrete Cosine Transform

DWT

Discrete Wavelet Transform

ENN

Equitz Nearest Neighbor

FBP

Fast Back Propagation

GIF

Graphics Interchange Format

GLA

Generalized Lloyd Algorithm

HF

High Frequency

HH

High-High

HL

High-Low

IFS

Iterated Function System

IMWT

Integer Multi Wavelet Transform

JBIG

Joint Bi-level Image expert Group

JPEG

Joint Photographic Experts Group

JPEG2000

Joint Photographic Experts Group2000

LBG

Linde Buzo Gray

LF

Low Frequency

LH

Low-High

LL

Low-Low

LS

Lifting Scheme

LWT

Lifting Wavelet Transform

LZ

Lempel-Ziv

LZW

Lempel Ziv Welch

MFOCPN

Modified Forward-Only Counter Propagation Neural Network

MPEG

Motion Pictures Expert Group

ix

PNG

Portable Network Graphics

PSNR

Peak Signal-to-Noise Ratio

RAC

Randomized Arithmetic Coding

RLE

Run Length Encoding

SEC

Second

SNR

Signal-to-Noise Ratio

SPIHT

Set Partitioning In Hierarchical Trees

SWT

Stationary Wavelet Transform

TIE

Triangular Inequality Elimination

VQ

Vector Quantization

WPT

Wavelet Packet Transform

WT

Wavelet Transforms

x

Chapter 1

Introduction

INTRODUCTION Compression is the art of representing the information in a compact form rather than in its original or uncompressed form. In other words, using the data compression, the size of a particular file can be reduced. This is very useful when processing, storing or transferring a huge file, which needs lots of resources. If the algorithms used to encrypt works properly, there should be a significant difference between the original file and the compressed file. When the data compression is used in a data transmission application, speed is the primary goal. The speed of the transmission depends on the number of bits sent, the time required for the encoder to generate the coded message, and the time required for the decoder to recover the original ensemble. In a data storage application, the degree of compression is the primary concern. Compression can be classified as either lossy or lossless. Lossy compression is one in which compressing data and then decompressing it retrieves data that will be different from the original, but it is enough to be useful in some way. Lossy data compression is used frequently on the Internet and mostly in streaming media and telephony applications. In lossy data repeated compressing and decompressing, a file will cause it to lose quality. Lossless when compared with lossy data compression will retain the original quality, an efficient and minimum hardware implementation for the data compression and decompression needs to be used even though there are so many compression techniques which are faster, memory efficient which suits the requirements of the user [1]. In the decompression phase of lossy image compression, the output images are almost the same as the input images. In addition, this method is useful where a little information from each pixel is important. Lossless compression is to reconstruct the original data from the compressed file without any loss of data. Thus, the information does not change during the compression and decompression processes. These kinds of compression algorithms are called reversible compressions since the original message is reconstructed by the decompression process. Lossless compression techniques are used to compress medical images, text, and images preserved for legal reasons, computer executable file and so on [2].

1

Chapter 1

Introduction

The examples of the lossless compression techniques are run length encoding, Huffman encoding, LZW coding, area coding, and Arithmetic coding [3]. In the lossless compression scheme, after compression is numerically identical to the original image. It is used in many applications such as ZIP file format & in UNIX tool zip. It is important when the original & the decompressed data be identical. Some image file formats like PNG or GIF use only lossless compression. Most lossless compression programs do two things in sequence: the first step generates a statistical model for the input data, and the second step uses this model to map input data to bit sequences in such a way that "probable" (e.g. frequently encountered) data will produce shorter output than "improbable" data [4]. Discrete wavelet transform (DWT) is one of the wavelet transforms used in image processing. DWT decomposes an image into different sub band images, namely low-low (LL), low-high (LH), high-low (HL), and high-high (HH). A recent wavelet transform which has been used in a several image processing application that is the stationary wavelet transform (SWT).In short, SWT is similar to DWT but it does not use down-sampling, hence the sub bands will have the same size as the input image [6]. The stationary wavelet transform among the different tools of multiscale signal processing, the wavelet is a time-frequency analysis that has been widely used in the field of image processing such as DE noising, compression, and segmentation. Wavelet-based DE noising provides multi-scale treatment of noise, down-sampling of subband images during decomposition, and the threes holding of wavelet coefficients may cause edge distortion and artifacts in the reconstructed images [5]. Vector Quantization (VQ) is a block-coding technique that quantizes blocks of data instead of single samples. VQ exploits the correlation between neighboring signal samples by quantizing them together. VQ Compression contains two components: VQ encoder and decoder as shown in Figure 1.1. At the encoder, the input image is portioned into a set of non-overlapping image blocks. The closest code word in the code book is then found for each image block. Here, the closest code word for a given block is the one in the code book that has the minimum squared Euclidean distance from the input block. Next, the corresponding index for each searched closest code word is transmitted to the decoder. Compression is achieved because the indices of the closest code words in the code book sent to the decoder instead of the image blocks themselves [7].

2

Chapter 1

Introduction

Vector quantization (VQ) is a powerful method for image compression due to its excellent rate-distortion performance and its simple structure. Some efficient clustering algorithms are developed based on the VQ-like approach. However, the VQ algorithm still employs a full search method to ensure the best-matched code word and consequently results in the computational requirement is large. Therefore, many research efforts were paid on simplifying the search complexity for the encoding process. These approaches are further classified into two types in terms of simplified technique. One is the tree-structured VQ (TSVQ) techniques, and the other is the triangular inequality elimination (TIE) based approaches [8].

Figure 1.1: Vector quantization encoder and decoder This thesis focuses on lossy compression because it is the most popular category in real applications.

1.1 Lossy Compression Lossy compression works very differently. These programs simply eliminate "unnecessary" bits of information, tailoring the file so that it is smaller. This type of compression is used a lot for reducing the file size of bitmap pictures, which tend to be fairly bulky [9]. This may examine the color data for a range of pixels, and identifies subtle variations in pixel color values that are so minute that the human eye/brain is unable to

3

Chapter 1

Introduction

distinguish the difference between them. The algorithm may choose a smaller range of pixels whose color value differences fall within the boundaries of our perception, and substitute those for the others. The Lossy compression framework is shown in Figure (1.2).

Original Image Data

Compressed Image

Prediction/Transformat ion/ Decomposition

Quantization

Modeling and Encoding

Figure 1.2: Lossy compression framework To achieve this goal one of the following operations is performed. 1. Predicted image is formed by predicting pixels based on the values of neighboring pixels of the original image. Then the residual image is formed which is the difference between the predicted image and the original image. 2. Transformation a reversible process that reduces redundancy and/or provides an image representation that is more amenable to the efficient extraction and coding of relevant information. 3. Quantization process compresses a range of values to a single quantum value. When the number of discrete symbols in a given stream is reduced, the stream becomes more compressible. Entropy coding is then applied to achieve further compression. Major performance considerations of a lossy compression scheme are: a) the compression ratio (CR), the signal-to noise ratio (PSNR) of the reconstructed image with respect to the original, and c) the speed of encoding and decoding [9]. We will use the following techniques in the Lossy compression process:

1.1.1 Transform Coding Transform coding algorithm usually start by partitioning the original image into sub images (blocks) of small size (usually 8 x 8). For each block the transform coefficients are calculated, effectively converting the original 8 x 8 array of pixel values into an array of coefficients closer to the top-left corner usually contains most of the information needed to quantize and encode the image with little perceptual distortion. The resulting coefficients

4

Chapter 1

Introduction

are then quantized and the output of the quantized issued by a symbol encoding technique to produce the output bit stream representing the encoded image [9].

1.1.2 Vector Quantization The vector quantization is a classical quantization technique for signal processing and image compression, which allows the modelling of probability density functions by the distribution of prototype vectors. The main use of vector quantization (VQ) is for data compression [10] and [11]. It works by dividing a large set of values (vectors) into groups having approximately the same number of points closest to them. Each group is represented by its centroid value, as in LBG algorithm and some other algorithms [12]. The density matching property for vector quantization is powerful, especially in the case for identifying the density of large and high dimensioned data. Since data points are represented by their index to the closest centroid, commonly occurring data have less error and rare data have higher error. Hence VQ is suitable for lossy data compression. It can also be used for lossy data correction and density estimation. The methodology of vector quantization is based on the competitive learning paradigm, hence it is closely related to the self-organizing map model. Vector quantization (VQ) is used for lossy data compression, lossy data correction and density estimation [12]. Our approach is considered a lossy compression technique that enhances lossy compression technique by using stationary wavelet transform and vector quantization to solve the major problems of lossy compression techniques.

1.2 Wavelet Transforms (WT) Wavelets are signals which are local in time and scale and generally have an irregular shape. A wavelet is a waveform of effect limited duration that has an average value of zero. The term „wavelet‟ comes from the fact that they integrate to zero; they wave up and down across the axis. Many wavelets also display a property ideal for compact signal representation: orthogonally. This property ensures that data is not over represented. A signal can be decomposed into many shifted and scaled representations of the original mother wavelet. A wavelet transform can be used to decompose a signal into component wavelets. Once this is done the coefficients of the wavelets can be decimated to

5

Chapter 1

Introduction

remove some of the details. Wavelets have the great advantage of being able to separate the fine details in a signal. Very small wavelets can be used to isolate very fine details in a signal, while very large wavelets can identify coarse details. In addition, there are many different wavelets to choose from. Various types of wavelets are: Morlet, Daubechies, etc. One particular wavelet may generate a more sparse representation of a signal than another, so different kinds of wavelets must be examined to see which is most suited to image compression [13].

1.2.1 Discrete Wavelet Transform (DWT) The Discrete Wavelet Transform (DWT) of image signals produces a no redundant image representation, which provides better spatial and spectral localization of image formation, compared with other multi scale representations such as Gaussian and Laplacian pyramid. Recently, Discrete Wavelet Transform has attracted more and more interest in image fusion. An image can be decomposed into a sequence of different spatial resolution images using DWT. In case of a 2D image, an N level decomposition can be performed, resulting in 3N+1 different frequency bands and it is shown in Figure 1.3 Optimal decomposition level of the discrete, stationary, and dual tree complex [14].

Figure 1.3: 2D-Discrete wavelet transforms

6

Chapter 1

Introduction

1.2.2 Lifting Wavelet Transform (LWT) The lifting scheme (LS) has been introduced for the efficient computation of DWT .For image compression, it is very necessary that the selection of transform should reduce the size of the resultant data as compared to the original data set .So a new lossless image compression method is proposed. Wavelet using the lifting scheme significantly reduces the computation time, speed up the computation process. The lifting transforms even at its highest level is very simple. The lifting transform can be performed via two operations: Split, Predict and Update [15]. Suppose we have the one dimensional signal a0. The Lifting is done by performing the following sequence of operations: 1.Split 2. 3.

a0 into Even-1 and Odd-1

d-1 = Odd-1 – Predict (Even-1) a-1 = Even-1 + Update( d-1 )

These steps are repeated to construct multiple scales of the transform. The wiring diagram in Figure 1.4 shows the forward transform visually. The coefficients “a” are representing the averages in the signal that is Approximation coefficient, while the coefficients in “d” represent the differences in the signal that is Detailed Coefficient. Thus, these two sets also correspond to the low- pass and high- pass frequencies present in the signal [16].

Figure 1.4: Wire diagram of forward transformation with the lifting scheme

7

Chapter 1

Introduction

1.2.3 Stationary Wavelet Transform (SWT) Among the different tools of multi-scale signal processing, wavelet is a time-frequency analysis that has been widely used in the field of image processing such as denoising, compression, and segmentation. Wavelet-based denoising provides multi-scale treatment of noise, down-sampling of sub-band images during decomposition and the thresholding of wavelet coefficients may cause edge distortion and artifacts in the reconstructed images. To improve the limitation of the traditional wavelet transform, a multi-layer stationary wavelet transform (SWT) was adopted in this study, as illustrated in Figure 1.5. In Figure 1.5, Hj and Lj represent high-pass and low-pass filters at scale j, resulting from the interleaved zero padding of filters Hj-1 and Lj-1 (j>1). LL0 is the original image and the output of scale j, LLj, would be the input of scale j+1. LLj+1 denotes the lowfrequency (LF) estimation after the stationary wavelet decomposition, while LHj+1, HLj+1 and HHj+1 denote the high frequency (HF) detailed information along the horizontal, vertical and diagonal directions, respectively [5].

Figure 1.5: Stationary wavelet decomposition of a two-dimensional image

8

Chapter 1

Introduction

These sub-band images would have the same size as that of the original image because no down-sampling is performed during the wavelet transform. In this study, the Haar wavelet was applied to perform multi-layer stationary wavelet transform on a 2D image. Mathematically, the wavelet decomposition is defined as: LLj+1(χ, γ) = L[n] L [m] LLj (2j+1 m- χ, 2j+1 n- γ) LHj+1(χ, γ) = L[n] H [m] LLj (2j+1 m- χ, 2j+1 n- γ) (1.1) HLj+1(χ, γ) = H[n] L [m] LLj (2j+1 m- χ, 2j+1 n- γ) HHj+1(χ, γ) = H[n] H [m] LLj (2j+1 m- χ, 2j+1 n- γ) Where L[·] and H[·] represent the low pass and high pass filters respectively, and LL0(X,Y)=F(X,Y) Compare with the traditional wavelet transform, the SWT has several advantages. First, each sub-band has the same size, so it is easier to get the relationship between the sub-bands. Second, the resolution can be retained since the original data is not decimated. Also at the same time the wavelet coefficients contain much redundant information which helps to distinguish the noise from the feature. In this study, the image processing and stationary wavelet transform are performed using MATLAB programming language. The proposed method is tested using Standard images as well as image sets selected from Heath et al.‟s library. For the sake of thoroughness, the developed method is compared with the standard Sobel, Prewitt, Laplacian, and Canny edge detectors [5].

1.3

Problem Statement The large increase in the data lead to delays in access to the information required and

this leads to a delay in the time. Large data lead to data units and storage is full this leads to the need to buy a bigger space for storage and losing money. Large data lead to give inaccurate results for the similarity of data and this leads to getting inaccurate information. Also to show the difference between the types of transforms Stationary Wavelet Transforms and Discrete Wavelet Transform and Lifting Wavelet Transform because they are very similar at one level so we used three levels.

9

Chapter 1

1.4

Introduction

Research Objective In lossy compression, the compression ratio is unaccepted. The proposed system

suggests an image compression method of lossy image compression through the three types of transformations such as stationary wavelet transform, discrete wavelet transforms, and lifting wavelet transform and the comparison between the three types and the use of vector quantization (VQ) to improve the image compression process.

1.5

Contribution of the thesis Our thesis has two contributions. The first contribution is the lossy compressed

approach using stationary wavelet transforms and vector quantization has less compressed data than other wavelet transformation such as discrete wavelet transform and lifting wavelet transform. The second conclusion, when apply lossless compressors of the type of the arithmetic coding and Huffman encoding, the size of compressed data by arithmetic coding is better than Huffman coding. Our approach built to compress the data by using stationary wavelet transform (SWT) and vector quantization (VQ) and arithmetic coding.

1.6

Thesis Organization Chapter two introduces the details chapter illustrating previous studies on image

compression and the techniques used. Chapter three describes in detail the proposed system and how it improves the image compression with the lossy image compression technologies and lossless image compression techniques. Chapter four introduces the empirical results that applied on the proposed system, its effectiveness, and an analysis of results. It also compares between the results of the Used for a number of techniques as used in the image compression. Chapter five gives a general summary of the thesis, the research conclusions, and the top recommendations the researcher believes will be necessary for future research.

10

Chapter 2

Background and Literature Review

BACKGROUND AND LITERATURE REVIEW This chapter offers some important background related to the proposed system that including wavelet transform and vector quantization. It also introduces a taxonomy of image compression techniques, and covers a literature review on image compression algorithms.

2.1 Background 2.1.1 Compression Techniques Compression techniques come in two forms: lossy and lossless. Generally a lossy technique means that data are saved approximately rather than exactly. In contrast lossless techniques save data exactly. They look for sequences that are identical and code these. This type of compression has a lower compression rate than a lossy technique, but when the file is recovered it is identical to the original. Generally speaking, Lossless data compression is used as a component within lossy data compression technologies. Lossless compression is used in cases where it is important that the original and the decompressed data be identical, or where deviations from the original data could be deleterious. Typical examples are executable programs, text documents, and source code. Lossless compression methods may be categorized according to the type of data they are designed to compress. While, in principle, any general-purpose lossless compression algorithm can be used on any type of data, many are unable to achieve significant compression on data that are not of the form for which they were designed to compress [18]. In lossless compression schemes, the reconstructed image, after compression, is numerically identical to the original image. However, lossless compression can only achieve a modest amount of compression. An image reconstructed following lossy compression contains degradation relative to the original. Often this is because the compression scheme completely discards redundant information. However, lossy schemes are capable of achieving much higher compression. Under normal viewing conditions, no visible loss is perceived (visually lossless). Table 1 describes the comparison between loosy and lossless compression in some items [17].

11

Chapter 2

Background and Literature Review

Table 2.1: Comparison between lossy and lossless compression techniques Item

Lossy Compression

Lossless Compression

Reconstructed image

Contains degradation relative to the original image.

Numerically identical to the original image.

Compression rate

High compression (visually lossless).

2:1 (at most 3:1).

Application

Music, photos, video, medical images, scanned documents, fax machines.

Databases, emails, spreadsheets, office documents, source code.

2.1.2 Lossy Compression Vector Quantization

Figure 2.1: Code words in 1-dimensional space Vector quantization (VQ) is a lossy data compression method based on the principle of block coding. It is a fixed-to-fixed length algorithm. A VQ is nothing more than an approximate. The idea is similar to that of “rounding-off” (say to the nearest integer) [21] and [22]. The following example shown in Figure 2.1 represents every number less than -2 is approximated by -3, all numbers between -2 and 0 are approximated by -1, every number between 0 and 2 are approximated by +1, and every number greater than 2 are approximated by +3. The approximate values are uniquely represented by 2 bits. This is a 1-dimensional, called 2-bit VQ. It has a rate of 2 bits/dimension. In the above example, the stars are called code vectors [21]. A vector quantizer map k-dimensional vectors in the vector space vectors Y = {y: i = 1, 2, ... , N}. Each vector

a finite set of

is called a code vector or a code word, and

the set of all the codewords is called a codebook. Associated with each codeword

is a

nearest neighbor region called encoding region or Voronoi region [21] and [23] and it is defined by: |

||

||

} ................................... (1)

12

Chapter 2

Background and Literature Review

The set of encoding region's partition, the entire space ⋂



such that:

.......................... (2)

Thus the set of all encoding regions is called the partition of the space. In the following example, we take vectors in the two dimensional case without loss of generality in Figure 2.1 In the figure, Input vectors are marked with an x, code words are marked with solid circles, and the Voronoi regions are separated with boundary lines. The figure shows some vectors in space. Associated with each cluster of vectors is a representative code word. Each code word resides in its own Voronoi region. These regions are separated by imaginary boundary lines in Figure 2.2 given an input vector; the code word that is chosen to represent it is the one in the same Voronoi region. The representative code word is determined to be the closest in Euclidean distance from the input vector [21]. The Euclidean distance is defined by: √∑ Where code word

is the

....................................... (3)

component of the input vector, and

is the

component of the

. In Figure 2.2 there are 13 regions and 13 solid circles, each of which can be

uniquely represented by 4 bits. Thus, this is a 2-dimensional, 4-bit VQ. Its rate is also 2 bits/dimension [21].

Figure 2.2: Code words in 2-dimensional space

13

Chapter 2

Background and Literature Review

A vector quantizer is composed of two operations. The first is the encoder, and the second is the decoder [24]. The encoder takes an input vector and outputs the index of the code word that offers the lowest distortion. In this case the lowest distortion is found by evaluating the Euclidean distance between the input vector and each code word in the codebook. Once the closest code word is found, the index of that code word is sent through a channel (the channel could be computer storage, communications channel, and so on). When the decoder receives the index of the code word, it replaces the index with the associated code word. Figure 2.3 shows a block diagram of the operation of the encoder and the decoder [21].

Figure 2.3: The Encoder and decoder in a vector quantizer In Figure 2.3, an input vector is given, the closest code word is found and the index of the code word is sent through the channel. The decoder receives the index of the code word, and outputs the code word [21]. The drawback of vector quantization, this technique generates code book in very slow speed than bpp [25].

14

Chapter 2

Background and Literature Review

2.1.2.1 Linde-Buzo-Gray Algorithm: Generalized Lloyd Algorithm (GLA), which is also called, Linde-Buzo-Gray (LBG) Algorithm They used a mapping function to partition training vectors in N clusters. The mapping function is defined as in [10]. → CB Let X = (x1, x2,…,xk) be a training vector and d(X, Y) be the Euclidean Distance between any two vectors. The iteration of GLA for a codebook generation is given as follows: 1. LBG algorithm Step 1:

Randomly generate an initial codebook CB0.

Step 2:

i = 0.

Step 3:

Perform the following process for each training vector. Compute the Euclidean distances between the training vector and the code words in

. The Euclidean distance is defined as ∑

................................... (4)

Search the nearest code word among

.

Step 4:

Partition the codebook into N cells.

Step 5:

Compute the centroid of each cell to obtain the new codebook CBi+1.

Step 6:

Compute the average distortion for CBi+1. If it is changed by a small enough amount since the last iteration, the codebook may converge and the procedure stops. Otherwise, i = i + 1 and go to Step 3 [10].

LBG algorithm has the local optimization problem and the utility of each codeword in the codebook is low. The local optimization problem means that the codebook guarantees local minimum distortion, but not global minimum distortion [29].

15

Chapter 2

Background and Literature Review

Figure 2.4: Flowchart of Linde-Buzo-Gray Algorithm 2.1.2.2 Equitz Nearest Neighbor The selection of initial codebook by the LBG algorithm is poor, which results in an undesirable final codebook. Another algorithm Equitz Nearest neighbor (ENN) is used in which no need for selection of initial codebook. As the beginning of ENN algorithm, all training vectors are viewed as initial clusters (code vectors). Then, the two nearest vectors are found and merged by taking their average. A new vector is formed which replaces and reduce the number of clusters by one. The Process is going on until desired number of clusters is not obtained [30]. The steps for the implementation of the ENN algorithm are as follows: 2. ENN Algorithm 1. Initially, all the image vectors taken as the initial codewords. 2. Find each of two nearest codewords by the equation:

16

Chapter 2

Background and Literature Review

Od (X, Yi) =k-1Σj=0|Xj–Yi, j|............................... (2.2) Where “X” represents an input vector from the original image and “Y” represents a codeword, and merge them by taking their average where “k” represents the codeword length. 3. The new codeword replaces the two codeword and reduce the number of codewords by one. 4. Repeat step 2 and step 3 until desired number of codewords is reached. The ENN requires a long time and large number of iterations to design the codebook. Therefore, to decrease the number of iterations and time required to generate the codebook, an image block distortion threshold value (dth) is calculated [10]. The ENN algorithm is modified as: 1. Determine the desired number of codeword and the maximum number of gray levels in the image (max–gray) 2. Distortion threshold (dth) is calculated as: dth=k× (max-gray/64) ..................................... (5) Where „k‟ is codeword length 3. Calculate the distortion error between a taken codeword and a next codeword. If the distortion error is less than or equal to dth, then merge these two codeword and reduce the number of codeword by one. Otherwise, consider the next codeword. 4. Repeat the step 3 until we obtain the number of codewords equal to desired number of codewords. 5. Even, after all the codewords are compared and merged, the resultant number of codewords greater than desired number of codewords, then change the dth value as follows Dth=dth+k×(max-gray/256) .................................. (6) And then go to step 3.

17

Chapter 2

Background and Literature Review

2.1.2.3 Back Propagation Neural Network Algorithm BPNN algorithm helps to increase the performance of the system and to decrease the convergence time for the training of the neural network [31]. BPNN architecture is used for both image compression and also for improving VQ of images. A BPNN consists of three layers: input layer, hidden layer and output layer. The number of neurons in the input layer is equal to the number of neurons in the output layer. The number of neurons in the hidden layer should be less than that of the number of neurons in the input layer. Input layer neurons represent the original image block pixels and output layer neuron represents the pixels of the reconstructed image block. The assumption in hidden layer neurons is that the arrangement is in one-dimensional array of neurons, which represents the element of codeword. This process produces an optimal VQ codebook. The source image is divided into non-overlapping blocks of pixels such that block size equals the number of input layer neurons and the number of hidden layer neurons equals the codeword length. In the BP algorithm, to design the codebook, the codebook is divided into rows and columns in which rows represent the number of patterns of all images and columns represents the number of hidden layer units [10].

Figure 2.5: Back propagation neural network image compression system The implementation of BPNN VQ encoding can be summarized as follows:

18

Chapter 2

Background and Literature Review

2. BPNN algorithm 1. Divide the source image into non-overlapping blocks with predefined block dimensions (P), where (P×P) equals the number of neurons in the input layer. 2. Take one block from the image, normalize it, convert the image into pixels (rasterizing), and apply it to the input layer neurons of BPNN. 3. Execute one iteration BP in the forward direction to calculate the output of hidden layer neurons. 4. From the codebook file, find the codeword that best matches the outputs of hidden layer neurons. 5. Store index i.e. position of this codeword in codebook in the compressed version of source image file. 6. For all the blocks of source image, repeat the steps from step2 to step5 [10]. Number of bits required for indexing each block equals to log2M, where M is codebook length. The implementation of BPNN VQ decoding process can be described as follows: 1. Open compressed VQ file 2. Take one index from this file. 3. This index is then replaced by its corresponding codeword which is obtained from the codebook and this codeword is assumed to be the output of hidden layer neurons. 4. Execute one iteration BP in the forward direction to calculate the output of the output layer neurons, then de-rasterizing it, de-normalize it and store this output vector in a decompressed image file. 5. Repeat steps from step2 to step4 until the end of the compressed file. The BP algorithm is used to train the BPNN network to obtain the codebook with smaller size with improved performance of the system. The BPNN image compression system has the ability to decrease the errors that occur during transmission of compressed images through analog or digital channel. Practically, we can note that BPNN has the ability to enhance any noisy compressed image that has been corrupted during compressed image transmission through a noisy digital or analog channel. BPNN has the capacity to

19

Chapter 2

Background and Literature Review

compress untrained images, but not in the same performance of trained images. This can be done, especially when using a small number of image block dimension [33]. 2.1.2.4 Fast Back Propagation Algorithm The FBP algorithm is used for training the designed BPNN to reduce the convergence time of BPNN as possible as. The fast back propagation (FBP) algorithm is based on the minimization of an objective function after initial adaption cycles. This minimization can be obtained by reducing lambda (λ) from unity to zero during network training. The FBP algorithm differs from standard BP algorithm in the development of alternative training criterion. This criterion indicates that (λ) must change from 1 to 0 during training process i.e. λ approaches to zero as total error decreases. In each adaption cycle, λ should be calculated from the total error at that point, according to the equation: λ=λ (E), where E is error of network, indicates that λ≈1 when E˃˃1.When E˃˃1 for any positive integer n, 1/En approaches zero, therefore exp(-1/En) ≈1. When E˂˂1, 1/En is very large, therefore exp(-1/En)≈0. As a result, for the reduction of λ from 1 to 0, a suitable rule is as follows [32]: λ=λ(E)= exp (-μ/En).....................................(7) Where μ is a positive real number and n is a positive integer. When n is small, reduction of λ is faster, when E˃˃1. It has been experimentally verified that if λ is much smaller than unity during initial adaption cycles, algorithm may be trapped in local minimum. So, n should be greater than 1. Thus, λ is calculated during any network training according to equ. 6[32]: λ = λ(E)=exp(-μ/E2).......................................(8) In the FBP algorithm, all the hidden layer neurons and output layer neurons use hyperbolic tangent function instead of sigmoid functions in the BPNN architecture. So, the equation is modified for hyperbolic tangent function as follows [32]: F (NET j)= (

J-

J) / (

J-

J)............... (9)

And derivative of this function is as follows: F‫(׳‬NET j)=(1-(F(NETj)2).........................................(10) So that F (NET j) lies between -1 and 1.

20

Chapter 2

Background and Literature Review

Table 2.2 compares the previous algorithm for some parameters such as code book size, code word size, storage capacity, code generation time, complexity time, and performance. Table 2.2: Comparison of various algorithms of vector quantization Parameters

LBG

Very large Codebook size super codebook The size of each codeword in the codebook Codeword size is P×P, where P is the dimension of image block. It requires more storage Storage space space for the codebook.

ENN

BPNN FBP Codebook with Small codebook as Codebook size is same smaller size as compared to LBG as that of BPNN compared to algorithm. algorithm. ENN algorithm The size of each The size of codeword in the codeword in the codebook is The size of each codebook is P×P, equal to the codeword is same as where P is the number of BPNN algorithm dimension of image hidden layer block neurons. It requires less storage space for the codebook

It takes long It takes less time Codebook time for the for the generation generation time generation of of the codebook. the codebook. A complete design requires a Complexity time large number of computations

This algorithm reduces the computations dramatically

It requires less storage space as compared to ENN algorithm. It takes less time for the generation of the codebook as compare to ENN Computational load is less for encoding and decoding process as compared to ENN algorithm

Convergence time

Convergence Convergence Convergence time time is less than time is very is less than the that of the ENN large LBG algorithm algorithm.

Performance

The performance of this algorithm is not so good.

It requires same storage space as that of BPNN algorithm It takes less time for the generation of codebook than BPNN algorithm

Computational load is less for encoding and decoding process as compared to ENN algorithm FBP trains the BPNN image compression system to speed up the learning process and reduce the convergence time

Performance is Performance is better than LBG far much better Performance is better algorithm as LBG than ENN than BPNN algorithm selects the initial algorithm codebook randomly

21

Chapter 2

Background and Literature Review

2.1.2.5 Joint Photopphic Experts Group JPEG stands for Joint Photopphic Experts Group, it is indeed one of the most used standards in the field of the compression of photographic images and it was created at the beginning of the 90s. It tums out moreover very competitive when it is used in a weak or an average compression ratios. But the mediocre quality of the obtained images in a higher compression ratio as well as its lack of flexibility and features gave a clear evidence of its incapability to satisfy all the application requirements in the field of the digital image processing. Based on those facts, members of the PEG group recovered to develop a new standard for image coding offering more flexibility and functionalities: JPEG2000 [69]. 2. Joint Photopphic Experts Group (JPEG) Algorithm The algorithm behind JPEG is relatively straightforward and can be explained through the following steps [70]: 1. Take an image and divide it up into 8-pixel by 8-pixel blocks. If the image cannot be divided into 8-by-8 blocks, then you can add in empty pixels around the edges, essentially zero-padding the image. 2. For each 8-by-8 block, get image data such that you have values to represent the color at each pixel. 3. Take the Discrete Cosine Transform (DCT) of each 8-by-8 block. 4. After taking the DCT of a block, matrix multiply the block by a mask that will zero out certain values from the DCT matrix. 5. Finally, to get the data for the compressed image, take the inverse DCT of each block. All these blocks are combined back into an image of the same size as the original. As it may be unclear why these steps result in a compressed image, I'll now explain the mathematics and the logic behind the algorithm [70].

22

Chapter 2

Background and Literature Review

2.1.2.6 JPEG2000 A. Historic : With the continual expansion of multimedia and Internet applications, the needs and requirements of the technologies used, grew and evolved In March 1997 a new call for contributions were launched for the development of a new standard for the compression of still images, the JPEG2000 [69]. This project, JTC2 1.29.14 (15444), was intended to create a new image coding system for different types of still images (bi-level, gray-level, color, multicomponent). The standardization process, which is coordinated by the JTCI/SC29/WGI of ISO/lEC3, has produced the Final Draft International Standard (FDIS') in August 2000. The International Standard (Is) was ready by December 2000. Only editorial changes are expected at this stage and therefore, there will be no more technical or functional changes in Part 1 of the Standard B. Characteristics and features: The purpose of having a new standard was twofold. First, it would address a number of weaknesses in the existing standard second, it would provide a number of new features not available in the JPEG standard The preceding points led to several key objectives for the new standard, namely that it should enclose [69]: 1) Superior low bit-rate performance, 2) Lossless and lossy compression in a single code-stream, 3) Continuous-tone and bi-level compression, 4) Progressive transmission by pixel accuracy and resolution, 5) Fixed-rate, fixed-size, 6) Robustness to bit errors, 7) Open architecture, 8) Sequential build-up capability, 9) Interface with MPEG-4, 10) Protective image security, 11) Region of interest

23

Chapter 2

Background and Literature Review

2.1.3 Lossless Compression Techniques The extremely fast growth of data that needs to be stored and transferred has given rise to the demands of better transmission and storage techniques. Lossless data compressions categorized into two types are: models & code and dictionary models. Various lossless data compression algorithms have been proposed and used. Huffman Coding, Arithmetic Coding, Shannon Fano Algorithm, Run Length Encoding Algorithm are some of the techniques in use [34]. 2.1.3.1 Models and Code Model code divided to Huffman coding and Arithmetic coding 2.1.3.1.1 Huffman Coding A first Huffman coding algorithm was developed by David Huffman in 1951. Huffman coding is an entropy encoding algorithm used for lossless data compression. In this algorithm fixed length codes are replaced by variable length codes. When using variable-length code words, it is desirable to create a prefix code, avoiding the need for a separator to determine codeword boundaries. Huffman Coding uses, such prefix code [34]. Huffman procedure works as follow: 1. Symbols with a higher frequency are expressed using shorter encodings than symbols which occur less frequently. 2. The two symbols that occur least frequently will have the same length. The Huffman algorithm uses the greedy approach i.e. at each step the algorithm chooses the best available option. A binary tree is built up from the bottom up. To see how Huffman Coding works, let‟s take an example. Assume that the characters in a file to be compressed have the following frequencies: A: 25 B: 10 C: 99 D: 87 E: 9 F: 66 The processing of building this tree is: Create a list of leaf nodes for each symbol and arrange the nodes in the order from highest to lowest. C: 99 D:87 F:66 A:25 B:10 E:9

24

Chapter 2

Background and Literature Review

Select two leaf nodes with the lowest frequency. Create a parent node with these two nodes and assign the frequency equal to the sum of the frequencies of two child nodes.

Now add the parent node in the list and remove the two child nodes from the list. And repeat this step until you have only one node left.

25

Chapter 2

Background and Literature Review

Now label each edge. The left child of each parent is labeled with the digit 0 and right child with 1. The code word for each source letter is the sequence of labels along the path from root to the leaf node representing the letter. Huffman Codes are shown below in the table [34]. Table 2.3: Huffman Coding C

00

D

01

F

10

A

110

B

1110

E

1111

2. Huffman Encoding Algorithm Huffman Encoding Algorithm [52]. Huffman (W, n) //Here, W means weight and n is the no. of inputs Input: A list W of n (Positive) Weights. Output: An Extended Binary Tree T with Weights Taken from W that gives the minimum weighted path length. Procedure: Create list F from singleton trees formed from elements of W. While (F has more than 1 element) do Find T1, T2 in F that have minimum values associated with their roots // T1 and T2 are sub tree . Construct new tree T by creating a new node and setting T1 and T2 as its children Let, the sum of the values associated with the roots of T1 and T2 be associated with the root of T Add T to F Do Huffman-Tree stored in the F

26

Chapter 2

Background and Literature Review

2.1.3.1.2 Arithmetic Coding Arithmetic coding (AC) is a statistically lossless encoding algorithm with very high compression efficiency and is especially useful when dealing with the source with an alphabet of small size. Nowadays, AC is widely adopted in the image and video coding standards, such as JPEG2000 [22] and [39]. Recent researches on secure arithmetic coding primary focus on the two approaches: the interval splitting AC (ISAC) and the randomized AC (RAC). In [40] and [41], the strategy of key-based interval splitting had been successfully incorporated with arithmetic coding to construct a novel coder with the capabilities of compression and encryption. This approach has been studied deeply just for the floating-point arithmetic coding over the past few years. On compression efficiency, the authors illustrated that even the interval is just split (the total length kept unchanged) in an arithmetic coder, the code length will raise a little relative to the floating-point arithmetic coding [42] analyzed that the key-based interval splitting AC is vulnerable to known-plaintext attacks [43]. Further used message in distinguishability to prove that ISAC is still insecure under [36]. Cipher text-only attacks even under the circumstance that different keys are used to encrypt different messages. In order to enhance the security, [44] provided an extended version of ISAC, called Secure Arithmetic Coding (SAC), which applies two permutations to the input symbol sequence and the output codeword. However,[45] and [46] independently proved that it is still not secure under the chosen-plaintext attacks and known-plaintext attacks due to the regularities of permutations steps [47] presented a randomized arithmetic coding (RAC) algorithm, which achieves the capability of encryption by randomly swapping two symbol intervals during the process of binary AC. Although RAC does not suffer any loss of compression efficiency, its security problem does exist [48] proved that it is vulnerable to cipher-only attacks. Recently [49] presented a secure integer AC scheme (here called MIAC) that performs the compression and the encryption simultaneously. In this scheme, the size ratio D𝛼1,+1/λn of interval allocated to the symbol 𝛼1 will be far approximated to the probability P(𝛼1) and the size ratios D𝛼1,n+1/λn of interval allocated to the symbol 𝛼i will be far approximated to the probability P(𝛼i). In this paper, we further try to propose another secure arithmetic coding scheme with good compression efficiency and highest secrecy.

27

Chapter 2

Background and Literature Review

Illustrated Example of Arithmetic Encoding Arithmetic Coding works, let‟s take an example, we have a string: BE_A_BEE And we now compress it using arithmetic coding. Step 1: in the first step we do is look at the frequency count for the different letters: E

B

_

A

3

2

2

1

Step 2: In the second step we encode the string by dividing up the interval [0, 1] and allocate each letter an interval whose size depends on how often it count in the string. Our string start with a‟B‟, so we take the „B‟ interval and divide it up again in the same way:

The boundary between „BE‟ and „BB‟ is 3/8 of the way along the interval, which is itself 2/3 long and starts at 3/8. So boundary is 3/8 + (2/8) * (3/8) = 30/64. Similarly the boundary between „BB‟ and „B_‟ is 3/8+ (2/8) * (5/8) = 34/64, and so on. [51]. Step 3: In the third step we see next letter is now „E‟, so now we subdivide the”E‟ interval in the same way. We carry on through the message….And, continuing in this way, we eventually obtain:

28

Chapter 2

Background and Literature Review

And continuing in this way, we obtain:

So we represent the message as any number in the interval [7653888/16777216, 7654320/16777216] However, we cannot send numbers like 7654320/16777216 easily using a computer. In decimal notation, the rightmost digit to the left of the decimal point indicates the number of units; the one to its left gives the number of tens: the next one along gives the number of hundred, and so on. 7653888 = (7*106) + (6*105) + (5*104) + (3*103) + (8*102) + (8*10) + 8 Binary numbers are almost exactly the same, we only deal with powers of 2 instead of power of 10. The rightmost digit of a binary number is unitary (as before) the one to its left gives the number of 2s, the next one the number of 4s, and so on. 110100111 = (1*28) + (1*27) + (0*26) + (1*25) + (0*24) + (0*23) + (1*22) + (1*21) + 1 = 256 + 128 + 32 + 4 + 2 + 1 = 423 in denary (i.e. base 10) [51].

29

Chapter 2

Background and Literature Review

2. Arithmetic Encoding Algorithm BEGIN low = 0.0; high = 1.0; range = 1.0; While (symbol i= terminator) { get (symbol) ; Low = low + range * Range_low (symbol); Low = low + range * Range_high (symbol); range = high – low; } Output a code so that low