Image Algebra: An Overview - Science Direct

14 downloads 0 Views 2MB Size Report
ALGEBRA. 299 and nonlinear systems under the theory of mathematical morphology [33]. However, ..... associative and commutative binary operation on the value set F, then the global ...... multivalued or multidata image. We distinguish ...
COMPUTER

VISION,

GRAPHICS,

AND IMAGE

PROCESSING

49, 297-331 (1990)

Image Algebra: An Overview G. X. RITTER, J. N. WILSON, AND J. L. DAVIDSON Center

for Computer

Vision Research, Department of Computer University of Florida, Gainesville, Florida

and Injormation 3261 I

Sciences,

Received July 29,1988; revised April 17,1989 This paper is the first in a sequence of papers describing an algebraic structure for image processing that has become known as the AFATL Standard Image Algebra. This algebra provides a common mathematical environment for image processing algorithm development and methodologies for algorithm optimization, comparison, and performance evaluation. In addition, the image algebra provides a powerful algebraic language for image processing which, if properly embedded into a high level programming language, will greatly increase a programmer’s productivity as programming tasks are greatly simplified due to replacement of large blocks of code by short algebraic statements. The purpose of this paper is to familiarize the reader with the basic concepts of the algebra and to provide a general overview of its ’ 1990 Academrc Press, Inc. methodology. I. INTRODUCTION

Current practice in the development and implementation of image processing algorithms makes little use of any standardized, mathematically rigorous, efficient algebraic structure that is specifically designed for image manipulation. Yet vast increases in image processing activities in the military, industrial, and academic communities have resulted in a deluge of different image processing techniques, notation, and operations that all too often perform similar or identical tasks. Documentation of algorithms for image processing and rationale underlying algorithm design is often accomplished via word description or analogies that are extremely cumbersome and often ambiguous. The result of these ad hoc approaches has been a proliferation of non-standard notation and increased research and development cost. In response to this chaotic situation, the Air Force Armament Laboratory (AFATL) of the Air Force Systems Command in conjunction with DARPA called for the development of a highly structured mathematical foundation for image processing and image analysis with the intent that the fully developed structure would subsequently form the basis of a common image processing language. Several previous attempts to develop a unified algebraic approach to image processing met only partial success in expressing all transformations of gray level images. Thus, the goal of the Air Force Armament Laboratory was the development of a complete, unified algebraic structure that provides a common mathematical environment for image processing algorithm development, optimization, comparison, coding, and performance evaluation. The development of this structure proved highly successful, capable of fulfilling the tasks set forth by the Government and it is now officially referred to as the “AFATL Image Algebra.” The main purpose of this paper is to familiarize the reader with the basic concepts of the image algebra. Subsequent papers will discuss the mathematical properties of the algebra, its various substructures, applications, and software implementation issues. 297 0734-189X/90

$3.00

Copyright 0 1990 by Academic Press, Inc. All rights of reproduction in any form reserved.

298

RI’ITER, WILSON,

AND DAVIDSON

Several papers concerned with the image [l-8]. However, these papers were written algebra’s development. Since then, definitions degrees of refinements and adjustments. The reflects these changes and represents the Air

algebra have already been published during the formative stages of the and notation have undergone various image algebra presented in this paper Force’s accepted standard.

A. Related Work The idea of establishing a unifying theory for concepts and operations encountered in image and signal processing is not new. Thirty years ago, Unger proposed that many algorithms for image processing and analysis could be implemented in parallel using a “cellular array” machine [9]. These cellular array machines where inspired by the work of von Neumann in the 1950s [lo]. Realization of von Neumann’s cellular array machines was made possible with the advent of VLSI technology. NASA’s massively parallel processor or MPP [ll], and the CLIP series of computers developed by M. J. B. Duff and his colleagues [12, 131. represent the classic embodiment of von Neumann’s original automaton. A more general class of cellular array computers are pyramids [14] and Thinking Machines Corporation’s Connection Machine [15]. In an abstract sense, the Connection Machine is a universal cellular automaton with an additional mechanism added for nonlocal communication. Many operations performed by these cellular array machines can be expressed in terms of simple elementary operations. These elementary operations create a mathematical basis for the theoretical formalism capable of expressing a large number of algorithms for image processing and analysis. In fact, a common thread among designers of parallel image processing architectures is the belief that large classes of image transformations can be described by a small set of standard rules that induce these architectures. This belief led to the creation of mathematical formalisms which were then used to aid the design of special purpose parallel architectures. Matheron and Serra’s Texture Analyzer [16], ERIM’s (Environmental Research Institute of Michigan) Cytocomputer [17,18], and Martin Marietta’s GAPP [19] are examples of this approach. The formalism associated with these cellular architectures is that of pixel neighborhood arithmetic and mathematical morphology. Mathematical morphology is that part of image processing which is concerned with image filtering and analysis by structuring elements. It grew out of the early work of H. Minkowski and H. Hadwiger on geometric measure theory and integral geometry [20-221, and entered the modem era through the work of G. Matheron and J. Serra of the Ecole des Mines in Fountainbleu, France 123, 241. Matheron and Serra not only formulated the modem concepts of morphological image transformations but also designed and built the Texture Analyzer System. Since those early days, morphological operations and techniques have been applied from low level to intermediate to high level vision problems. Among some recent research papers on morphological image processing are Crimmins and Brown [25], Haralick, Lee, and Shapiro [26], Haralick, Stemberg, and Zhuang [27], and Maragos and Schafer [28-301. Serra and Stemberg were the first to unify morphological concepts and methods into a coherent algebraic theory specifically designed for image processing and image analysis. Stemberg was also the first to use the term “image algebra” [31, 32j. More recently, P. Maragos introduced a new theory unifying a large class of linear

IMAGE

ALGEBRA

299

and nonlinear systems under the theory of mathematical morphology [33]. However, despite these profound accomplishments, morphological methods have some wellknown limitations. For example, such fairly common image processing techniques as feature extraction based on convolution, Fourier-like transformations, chain-coding, histogram equalization transforms, image rotation, and image registration and rectification are-with the exception of a few simple cases-either extremely difficult or impossible to express in terms of morphological operations. The failure of a morphologically based image algebra to express a fairly straightforward U.S. Government furnished FLIR (forward looking infra red) algorithm was demonstrated by P. Miller at Perkin-Elmer [34]. The failure of morphological image algebra to provide a universal image processing algebra is due to its set-theoretic formulation which is based on the Minkowski addition and subtraction of sets [22]. These operations ignore the linear domain, transformations between different domains (spaces of different dimensionalities), and transformations between different value sets, e.g., sets consisting of real, complex, or vector valued numbers. On the other hand, the algebra presented in this paper includes these concepts and also incorporates and extends the morphological operations. Another algebraic structure with identical goals as the AFATL image algebra is the algebra formulated by Giardina and Dougherty [35]. It also defines a set of basic or primitive image operations. However, this algebra provides too low a level of specification, requiring complicated expressions to describe even simple image processing operations. In addition, due to lack of formal proofs, it is not clear if this algebra is capable of expressing all image to image transformations. As there is no basic exponentiation operator nor a provision for transforming the dimensionality of images, it is unclear how Fourier-like transforms or multi-to-single value image transforms can be expressed within the framework of this algebra. .It was for these reasons, among others, that this algebra was not accepted by AFATL as the standard image processing algebra. II.

IMAGE

ALGEBRA

A. Value Sets and Coordinate Sets The image algebra defined in this paper is capable of multivalue image manipulation and constitutes a heterogeneous algebra in the sense of Birkhoff [36]. This general form of the image algebra is discussed in Section III. In this section we restrict ourselves to single-valued image manipulation. The image algebra deals with six basic types of operands, namely, value sets, coordinate sets, the elements of each of these sets, images, and templates. The only value sets considered in this section are the sets of integers, real numbers, complex numbers, binary numbers of fixed length k, and extended real numbers (which include one of the symbols + 00 or - 00). We denote these sets by Z, R, C, Zp, and R =RU{+co}andR-, = R U { - oc}, respectively. These correspond to the vait& commonly encountered in most image processing routines and in the modeling of such routines. However, in its most general form, the image algebra allows for any semi-group F to be a value set. An unspecified value set will henceforth be denoted by F. The operations on and between elements of a given value set F E {Z,R, C, Zp, R,,, R-,} are the usual

300

RITTER,

WILSON,

AND

DAVIDSON

elementary operations associated with F. Thus, if F = R, then the operations are the usual arithmetic and logic operations of addition, multiplication, and maximum, and the complementary operations of subtraction, division, and minimum. In addition to these elementary operations on elements of value sets, the image algebra also includes the operations of union, intersection, set subtraction, choice function, and cardinality function on subsets of F. The choice function applied to a set returns (chooses) an arbitrary element of the set, while the cardinality function yields the number of elements in the set. Union, intersection, set subtraction, choice function, and cardinality function will be denoted by U, n , \ , choice, and card. respectively. Coordinate sets are subsets of n-dimensional Euclidean space R”. We reserve the letters X, Y, and W to denote coordinate sets. Elements of coordinate sets will be denoted by bold lower case letters. In particular, if x E X and X c R”, then x is of form x = (xi, x2,. . . ,x,,), where each coordinate xi (i = 1,2,. . . , n) is a real number. It follows from the definition that coordinate sets can be rectangular, hexagonal, toroidal discrete arrays as well as infinite subsets of R”. Providing coordinate sets with such wide varieties of shapes, sizes, and dimensions allows for a coherent mathematical approach to the modeling and manipulation of continuous as well as discrete images on any desired type of coordinate set. Image algebra operations acting on coordinate sets are operations on subsets of coordinate sets as well as operations between coordinate points. In particular, operations on subsets of coordinate sets are U , n , \ , choice function, and cardinality function. Image algebra operations on or between elements of coordinate sets are the usual operations between coordinate points, i.e., vector addition, scalar and vector multiplication, dot product, etc. B. Images Thus far we have defined two types of objects, value sets and coordinate sets. These sets and their elements constitute some important operands of the image algebra. However, the most fundamental of the algebra’s operands are images. The most general, yet useful, mathematical definition of an image involves the previously defined concepts of value sets and coordinate sets. Given coordinate and value sets X and F, respectively, then an F valued image a on X is the graph of a function a: X -+ F. Thus, an F valued image a on X is of the form a = {(x,a(x)):

x E X},

where a(x) E F. The set X is called the set of image coordinates of a, and the range of the function a (which is a subset of F) is the set of image values of a. An element (x, a(x)) of the image a is called a picutre element or pixel, where x is called the pixel location, and a(x) the pixel value at location x. The set of all F valued images on X is denoted by FX. Here we follow the usual mathematical convention of denoting the set of all functions from a set A to a set B by BA.

IMAGE

301

ALGEBRA

If the value set F = Z or F = R, then we are dealing with integer or real valued images, respectively. Similarly, replacing F by C, or Z2k, provides for complex or finite digital images, respectively. In Section III we extend the notion of value sets in order to include such objects as vector valued images. This fact and the above examples should make it clear that the various choices for F and X allow for a far greater variety of image operands than are currently used by the image processing community. C. Binary and Unaty Operations on Images

Operations on and between F valued images are the natural induced operations of the algebraic system F. For example, for real valued images (i.e., elements of Rx), the operations are the elementary operations induced by the vector lattice (a vector space which is also a lattice) R. Thus, the basic real valued image operations reflect the arithmetic and logic operations on R. In particular, the binary operations of addition, multiplication, and maximum on Rx are defined as follows: Let a, b E RX. Then a + b = { (x,c(x)):

c(x) = a(x) + b(x), x E X}

0)

a*b = {(x,c(x)):

c(x) = a(x) . b(x), x E X}

(2)

c(x) = a(x)

(3)

a

V

b = {(x,c(x)):

V

b(x), x E X}.

These are the basic binary operations for real valued images. As complex numbers are not endowed with a natural lattice structure, only operations 1 and 2 are basic operations between complex valued images. Analogous to the development of the algebra of real numbers, other binary and unary operations on real valued imaged can now be derived from the basic operations either directly or in terms of series expansion. However, instead of reinventing the wheel, we assume familiarity with the algebra of real numbers and let the remaining operations on Rx again be induced by the corresponding operations on R. Two of these operations-commonly used in image processing-are exponentiation and the computation of logarithms. In particular, if a and b are real valued images on X, then ab = {(x, c(x)) : c(x) = a(x)“)

if’ a(x) # 0, otherwise c(x) = 0, x E X).

(4)

As we are dealing with real valued images we follow the rules of real arithmetic and restrict this binary operation to those pairs of images a, b for which a(x)“@) E R whenever a(x) # 0. This prevents the creation of complex pixel values such as (-l)‘/*. The inverse of exponentiation is defined in the usual way by taking the logarithm, namely, log, b = {(x, c(x)): c(x) = log,,, b(x), x E X}. As for real numbers, log, b is defined only for those images a and b for which a(x) > 0 and b(x) > 0 for all x E X. The next basic binary operation, called the dot product, distinguishes itself from the above five in that its output is not an image but

302

RITTER, WILSON, AND DAVIDSON

a real number. Let X be finite, then the dot product is defined as a b = 1 a(x)b(x). l

(6j

XEX

An image a is called a constant image if all its gray values are the same; i.e., if a(x) = k for some real number k and for all x E X. Two constant images are of utmost importance in the algebra; these are the zero image, defined by 0 = {(x, 0): x E X}, and the unit image, defined by 1 = {(x, 1): x E X}. Suppose k E R and a is a constant image with a(x) = k. Then we define:

lok=b” kb = a*b log, b = log, b.

and and

kb=ab k+bra+b

In the definition of log we assume, of course, that k > 0 and b(x) > 0 for all x. We also note that exponentiation is defined even when a(x) = 0. Subtraction, division, and minimum are defined in terms of the basic operations and inverses. Specifically: a - b 5 a -t (-b)

and

a/b = a*b-‘,

where -b = {(x, - b(x)): a A b = -(-a

V

(x, b(x)) E b}

-b).

The images 0 and 1 have the obvious property a + 0 = a and a* 1 = a. On the other hand, b* b-’ does not necessarily equal 1. However, b* b-’ *b = b. For this reason b-’ is called the pseudo inverse of b. Inequalities between images are defined in terms of maximum and minimum. Thus, for example, a 2 b if and only if a v b = b. These observations show that the ring (Rx, +, *) and the lattice (Rx, v ) behave very much like the ring and lattice of real numbers. In view of the fact that the operations between real valued images are induced by the operations between real numbers, this should come as no great surprise. Similar observations hold for extended real valued images. Here the notion of an additive inverse or a multiplicative pseudo inverse is extended to that of an additive or multiplicative dual, respectively. Specifically, if a E R?, (or a E RX,,), then the additive dual of a is denoted a* and defined by a*(x) = -a(x), where we define -( - co) = + co and - ( + co) = - cc. Thus, (a*)* = a and if a E R? m, then a* is a member of the dual space R”, m. For a E (R:O,)X, where Rko = {r E R: r 2 0} and R$L = R” U { + m), the multiplicative dual of a is denoted by a and defined by if a(x) # 0 or a(x) # + a0 if a(x) = 0 ifa = $-cc). Obviously, if a(x) is finite for every x E X, then a* = - a and ii = a ~‘.

IMAGE

ALGEBRA

303

The use of the extended real number system in image processing may appear somewhat artificial. However, these value sets turn out to be of great theoretical as well as practical importance. They have proven useful in the description and optimization of a large class of non-linear image transforms as well in as providing a solid mathematical foundation for generalized morphological operations [37-391. There are various useful unary operations definable in terms of the basic binary operations. For example, we have already provided the definitions of kb and log, b, the exponential of an image and the logarithm of an image b to the base k, respectively. In particular, the exponential of an image b, exp(b) = eb = ab, and the natural logarithm of b is defined as In b = log, b, respectively, where a is the constant image defined by a(x) = e for all x E X. Similarly, the absolute value of an image a can be defined by ]a] = a V (-a). In view of these examples it is obvious that there are various operations on and between images of different degrees of complexity that can be derived from the operations defined thus far. Phrasing some of these in terms of the basic operations would result in complicated algebraic expressions or infinite series representation. However, this would defeat the goal of providing a simple language for image processing tasks. Instead, we follow our initial philosophy and define the operations on FX to be the operations induced by the (usually well-known) operations of the algebraic system F. In particular, the common unary operations on Rx are functions available in most high level programming languages. More generally, any function f: R + R induces a function Rx --) Rx, again denoted by f, and defined by

f(a) = {(x34~)) : 44 = f(a(x)) 1. For example, sin(a) = {(x, sin(a(x))): x E X}. Similarly, if xs denotes the characteristic function with respect to some set S c R, then x&4

= {(x7 xMx>)): x E X} = {(x, c(x)): c(x) = 1 if a(x) E S, otherwise c(x) = O}.

Excepting the formalism, many of the operations of the algebraic system Rx described thus far are not new and are well known to the image processing community. What is new is the concept of raising an image to the power of another image or taking the logarithm of an image to the base of another image. This generalizes the common pixel level operations of raising an image to a constant such as taking the square root of an image at each pixel or taking the natural logarithm of an image at each pixel. The idea of having more versatile pixel level operations has led to the following generalization of the characteristic function: Given a E FX and S E (2F)X, where 2F denotes the power set of F so that S(x) c F for each x E X, then xs(a)

= {(x,c(x)):

c(x) = 1 if a(x) E S(x), otherwisec(x)

= O}.

(7)

Pixel level image comparison provides a simple application example of the generalized characterized function. Given the image b E Rx, we detine Ssb E (2R)X by S,,(x) = {Y E R: r I b(x)}. The functions SKb, S,,, Ski,, and S, b are defined analogously. Thus, for example, S, b(~) = {r E R: r > b(x)} and S,,(x) = {r E R:

304

RITTER, WILSON,

r = b(x)}.

Substituting

AND DAVIDSON

these set functions for S in Eq. (7) yields

xs,,(a) = { (x,c(x)): C(X) = 1 if a(x) > b(x), else c(x) = 0} + xs,,(a) = { (x,c(x)): c(x) = 1 if a(x) s b(x), else c(x) = 0)) etc. In order to reduce and simplify notation,

we define x , b = xs, b, x ~ ,, = xs -rb. As alluded to earlier, these generalized Xib characteristic functions could have been defined in terms of the more elementary image operations. Specifically, we could have defined =

XSXb’

X2b

=

XS,~?

and

Xb

E Xs-,-

xzb(a)=[(a-b)vO]-‘*[(a-b)vO]. Obviously, if a = b, then (a - b) v 0 = 0 and x, b(a) = 0-l *0 = 0, since by definition of exponentiation, O- ’ = 0 . The function x b(a>l’?

X2bta)

=

[X

RITTER, WILSON, AND DAVIDSON

308

If t E (FT)x, then the generalized forward template operation or generalized left product of a with t is defined as t@a

= ((y&)):

b(y) = x~xa(xb~,(y)~

y E Y).

Note that for either the left or right template product, the input image a is an F, valued image on the coordinate set X while the output image b is an F valued image on the coordinate set Y. Thus template operations are capable of transforming images with certain range values defined over a given coordinate set into images with entirely different values defined over coordinate sets of different dimensionality. It is clear that substitution of different specific binary operations of y and o in the above definition of the generalized right and left template product will result in a wide variety of different distinct image transformations. The study of most of these transformations remains uncharted research territory. By examining over two hundred different image transformations from diverse areas of image processing and computer vision applications, we discovered only three basic image/template operations that governed these transforms. This does not mean that these are the only worthwhile operations between images and templates. Future research will undoubtedly uncover other useful combinations. The three basic operations between images and templates are denoted by @ . a. and 0, and called generalized convolution, additive maximum, and multiplicative maximum, respectively. The operation @ is defined for real and complex valued images as well as real or complex vector valued images. The operations q and @ are applied to extended real or extended real vector valued images. However, in this section we shall consider only single valued images and templates. Let X c R” be finite and Y c R”. Suppose a E FX and t E (F’)‘, where F E {C, R}. Then the generalized backward convolution is defined as a @ t = { (YNY)): If t E (F’)‘,

b(y) = xFxab)

. $(x>, y E Y)-

then the generalized forward convolution

is given by

t @a = ((y.b(y)):b(y)= xgxab). Lh), Y E

Y).

If a E RF m and t E (R?! -)‘, then the backward additive maximum is defined as

a ki t = ((y,b(y)): b(y) = xyxa(x) + t,(x), y E Y), where V xExa(x) + t,(x) = max{a(x) + t,(x): x E X}. For t E (RY_m)x we define the forward additive maximum transform by

t q a G (b,b(y)): b(y)= xyxab) + t,(y), y E Y).

IMAGE

ALGEBRA

309

In order to define a(x) + t,(x) when one or both of a(x) or t,(x) equals - cc, we employ the extended arithmetic convention r + - cc = - co + r = - co Vr E R-,. The operation of multiplicative maximum is somewhat analogous to that of the additive maximum. In particular, for a E (R:k)x and t E ((R:O,)x)y, we define the backward transform

The forward multiplicative

max transform is given by

t@a={(y,b(y)): b(y)=x’xa(x) -~,(Y),Y EY). where t E ((R:O,)‘)‘. Multiplication on R$L is extended to the symbol + cc by definingr.(+co)=(+cc).r= +ccVr>OandO.(+cc)=(+cc).O=O. The operations q and @ can be used to define the dual operations of additive minimum and multiplicative minimum a q t = (t* •I a*)*

and

A detailed study of the behavior of the operations q , q , 0, and 0, and of the mathematical structures induced by these operations can be found in [39]. Several important comments are now in order. First note that since t,,(x) = 0 whenever x 4 Y(t,), we have that XXExa(x) . t,(x) = C, EYc!,,a(x) * t,(x) and a(x) . t,(x). Here we use the convention C, E Yct,,a(x) . V xEx4x) . t,(x) = VxwE.SP(fyj t,(x) = 0 and Ly4”(tyj a(x) . t,,(x) = 0 whenever P(t,) = 0. Thus, the new pixel value b(y) in the forward convolution and multiplicative maximum transforms depends only on the values of a(x) and t,(x) for which x E Y(t,). Similarly, for t E (Rx,)’ we have that t,(x) = -cc whenever x CZY-,(t,). Thus, V xEx4x) + t,(x) = Lymm_,(tyj a(x) + t,(x). Here we define VxsyL,ct,ja(x) + t,(x) = - cc whenever 9’-,(t,) = 0. Hence, the action for computing the new pixel value b(y) using the operation CB, 0, or q takes place over the set Y(t Y) or K,(t,), respectively. Similar observations can be made for the forward as well as the dual transforms. For example, C, E xa(x) - t,(y) = C, E ,ct5ja(x) - t,(y). These observations are not only of theoretical significance but are also important in the implementation of image algebra operations. It may be apparent by now that the sets Y(t,) and L?&) generalize the notion of a convolution window or mask and a morphological structuring element, respectively. The application examples provided in this paper should clarify the analogy.

310

RI’I-TER,

WILSON,

AND

DAVIDSON

There is an easy generalization of the above defined operations between images and templates that proves useful when expressing algorithms in image algebra code. Suppose X c R” with X not necessarily finite, W c R”, Y c R”, a E Rx, and t E (Rw)’ with Y(t,) finite for each y E Y. Then we define a @t =

(y, b(y)): b(y)

=

1

C X‘EXW(f,)

a(x)

1 t,(x),

y E Y).

where b(y) = 0 whenever X n Y(t,) = 0. The operations q and @ are defined in a similar fashion. For example, for a we define the new pixel value b(y) = V, E xn 9mSP_,ctyJa(~)+ t,(x). In the above definitions we assume that Y(t,) and 9-,(t,) are finite for each y E Y. However, the definitions extend to continuous functions a(x) and t, on compact sets Y(ty) and Y-m(ts,), with the exception that in the formulation of a @ t the sum is replaced by an mtegral. That is, c(y)

=

1 W,)

a(x)

- t,(x)

dx.

Thus, image algebra operations can be used for expressing both continuous and discrete image transformations. As a final comment we remind the reader that while a is an image on X, a @ t, t @ a, a q t, a@ t, etc., are all images on the domain Y. Thus, since X and Y can be of different dimensions and shape, template operations may be used for changing the dimensionality or size and shape of images. In particular, in addition to the usual local or global convolutions-as occur in edge enhancement, local smoothing, morphological operations, and Fourier-like transformationstemplate operations also provide a tool for image rotation, zooming, image reduction, masked extraction, and matrix multiplication. At first glance, these operations may appear somewhat mysterious. However, the examples provided below and those given in [6, 2, 3, 41 illustrate the inherent simplicity and power of these operations. EXAMPLE 1. Local averaging. Let a be an image on a rectangular array X Let Y = 2’ and t E (R’)’ be the 3 X 3 neighborhood template defined as

c

2’.

lkl4 1 1 1 1 1 1

t=

1 1 1

Then $a @ t represents the image obtained from a by local averaging since the new pixel value is given by c(y) = $, E Y(tYj n xa(x). As an important remark, we note that the image a ~3 t is an image on all of Z2 with zero values outside of the array X c Z2. Obviously, computers are not capable of storing images defined on intinite arrays. Furthermore, in practice one is only interested in the image $(a 0 t) restricted to the array X, that is, $(a Q t)l,, where Ix denotes the restriction to X.

IMAGE

ALGEBRA

311

This problem could be solved as follows: Let s E (Rx)’ be defined by sr = (t,) Ix for each y E X, where t is the template defined in Example 1. Then $a @ s provides the desired finite image, since (a $ t)l, = a @ s. Thus, the question arises: Why not simply define t as a template from X to X instead from Z2 to Z2? The rationale for defining the template as we did is that this template can be used for smoothing any 2dimensional image independent of its array size X. The reason for this is that when defining an image b in a program, one is usually forced to declare its dimensions, i.e., the size of its underlying array X. In particular, an image algebra program statement of form b := a GBt means to replace b pointwise by a $ t so that the value of b at location y is the value of a @ t at location y. That is, the array on the left side of the program statement induces a restriction on the right side image array. In short, we make the convention that the image algebra equation b = (a @ t)lx, where X is the domain of b, corresponds to the image algebra program statement b := a $ t. Thus, a programmer is not faced with the task of redefining t for a different sized image, as would be the case if he had defined t E (Rx)’ for a given X. In fact, this is the way we have embedded image algebra into a variety of languages including FORTRAN and Common Lisp [41, 421. As an illustration we provide an image algebra FORTRAN (IAF) program for computing the local average. The main program contains three executable statements: one to read an image, a second to find the average of the image, and a third to write the result of the image. The program uses several different constructs of the IAF language. An array named img, to hold a 64 X 64 image, is declared on the fourth line. A translation invariant template named t is declared on the fifth line of the program. The statement img = (img !gcon t)/9.0 assigns the result of the image algebra expression :(img @ t) to array img. IAF uses mnemonics such as !gcon for generalized convolution, !amax for additive maximum, and so forth. program average parameter (M = 64, N = 64) integer M, N integer img(O : M - 1,O : N - 1) integer invariant t call rdimg( img) img = (img !gcon t) / 9.0 call wrimg( img) end integer template t( ) integer i, j do 1Oi = -1,l do2Oj = -1,l t(y1 + i,y2 + j) = 1 20 continue 10 continue end

Declarations

Image algebra expression assignment

Template support definition

IAF templates are defined using a program unit similar to a subprogram or function. The template program unit, given a target coordinate set element y =

312

RITTER,

WILSON,

AND

DAVIDSON

(yl,y2) will specify the support locations x = (xl,@ assigned by the template to (yl,y2) together with their values. At run time, the target location y = (yl,y2) of a template is available in the pseudo-variables yl and y2. To specify that t(,,, yz,(xl, xZ) = U, the template subprogram must execute an assignment statement t(x1,x2)

= v.

The template t of program average has its support for target pixel (yl,y2) the locations x shown in the figure of Example 1. Of course, the program statement b := $(a @ t) will produce a boundary effect. In particular, if a and b are m x n images with underlying coordinate set X = {(i, j j: l