On the Application of Kolmogorov Complexity to

0 downloads 0 Views 76KB Size Report
The existence of an universal element in the set of speci- fication methods ..... upcoming of new multimedia formats such as Macromedia's. Flash has allowed ...
On the Application of Kolmogorov Complexity to the Characterization and Evaluation of Computational Models and Complex Systems Carlos A. P. Campani and Paulo Blauth Menezes

Abstract— This paper presents a proposal for the application of Kolmogorov complexity to the characterization of systems and processes, and the evaluation of computational models. The methodology developed represents a theoretical tool to solve problems from systems science. Two applications of the methodology are presented in order to illustrate the proposal, both of which were developed by the authors. The first one is related to the software development process, the second to computer animation models. In the end a third application of the method is briefly introduced, with the intention of characterizing dynamic systems of chaotic behavior, which clearly demonstrates the potentials of the methodology. Keywords: Kolmogorov complexity, systems science, computational models, animation, image, evaluation.

approach the software development process is, in a formal sense, unpredictable and incomplete; The application of the method, based on machine simulation, in order to compare computer animation models (and in a sense evaluate them) with respect to data compression; A formal definition of lossy data compression; A proposal for the characterization of dynamic systems of chaotic behavior based on incompressibility. The methodology developed in this work represents a theoretical tool to solve systems science problems. This paper is an extended review from [1], [2], [4], introducing new results not covered elsewhere. II. KOLMOGOROV C OMPLEXITY

I. I NTRODUCTION Many problems in systems science are difficult to handle because their characterization is too complex. We frequently face the need to compare systems’ models or evaluate the practical possibility of achieving a specific result. In this case we meet the difficulties related to our own limited capacity of expressing such problems through some mathematical formalism that is expressive enough to allow some kind of inference over our problem. This article presents a proposal for the characterization of systems and processes and for model evaluation based on Kolmogorov complexity. The article is enriched by two applications, both developed by the authors (part of the approach was published in [1], [2]): the first one on software development, the second on computer animation. It is also presented a third and more comprehensive application proposal (section V-D), which is an additional result that demonstrates the potentials of the methodology. The proposed methodology makes comparisons and evaluations with reference to data compression [3]. The data compression ratio is an important parameter in our proposal, as it characterizes the information content of an object. The method is based on the computability theory and, in this context, one of the most important concepts is that of machine simulation. The main results shown are: The definition of the software development process as data compression. According to our This work was partially supported by CNPq (Projects HoVerCAM, GRAPHIT, E-Automaton), FINEP/CNPq (Project Hyper-Seed), and CAPES/UFPel in Brazil. Carlos A. P. Campani is at Universidade Federal de Pelotas and Universidade Federal do Rio Grande do Sul. Paulo Blauth Menezes is at Universidade Federal do Rio Grande do Sul.

Kolmogorov complexity is an algorithmic randomness theory that deals with the amount of information of individual objects, as measured by the size of its smallest algorithmic description [5], [6], [7]. The objects over which Kolmogorov complexity is defined are binary strings (or natural numbers, since the set of binary strings is enumerable). Thus, Kolmogorov complexity associates to each binary string (or natural number) a value that represents its “complexity”. This section of the article presents the definition for Kolmogorov complexity and some well known results in this area. The Kolmogorov complexity of a binary string (or natural number) is the size of its smallest effective (computable) description relative to a specification method previously chosen (reference Turing machine [8]). Objects and descriptions are represented indistinguishably as binary strings or natural numbers. We define |.| as the size of a binary string (number of binary digits). In Definition 1, f : {0, 1}∗ → {0, 1}∗ is any given partial function. {0, 1}∗ denotes the set of all binary strings of size greater than or equal to zero (including the empty string). Definition 1: The complexity of object x ∈ {0, 1}∗ with respect to a specification method f is defined as Cf (x) = minf (p)=x |p|. If there is no description p it is said, by definition, that Cf (x) = ∞. Definition 2: We say that method f minorizes (or additively minorizes) a method g if there exists a constant c > 0 such that, for all x, Cf (x) ≤ Cg (x) + c. Definition 3: (Universal or Asymptotically Optimal Method) Let F be a subclass of the partial functions over the set of the natural numbers. A function f is universal (or asymptotically optimal) for F if it belongs to F and for every function g ∈ F , there exists a constant cf,g > 0

that depends solely on f and g, such that for every x, Cf (x) ≤ Cg (x) + cf,g . The existence of an universal element in the set of specification methods allows its usage as a reference method for the definition of complexity, thus rendering the complexity measure independent of the specification method chosen, if only by a constant. In this way it allows the measure of complexity to be an attribute inherent to the object, which makes it an objective concept [1]. Nevertheless, the class of partial functions does not contain an universal element. But, we can prove that there is an universal element in the class of the recursive partial functions. This function is called the universal recursive partial function [7]. Hence, Kolmogorov complexity is defined using Turing’s machine and restricting the specification methods to recursive partial functions. Theorem 1: (Invariance Theorem) There exists a recursive partial function u, called universal, such that for any recursive partial function f and binary string x, and for some c > 0, Cu (x) ≤ Cf (x) + c and c depends only on f . In Theorem 1’s proof, the concept of machine simulation and the existence of a universal Turing machine (that simulates all other machines) play an important role. By fixing an universal recursive partial function u, called universal of reference, we can define Cu (x) = C(x). To sum it up, we can say that the Kolmogorov complexity of an object represents the information that must be supplied so that the object can be computationally (algorithmically) constructed (or computed). Thus, Kolmogorov complexity represents quantitatively the information immanent in this object. In the same way, we can define conditional complexity of x given y, denoted by C(x|y), as the length of the smallest program that computes x from y. We observe that some strings may be algorithmic compressed, but others do not. This is the concept of incompressibility. For a constant c > 0, we say x is c-incompressible if C(x) ≥ |x| − c. It means that x can not be algorithmically compressed more than its literal size, but only by some constant c bits. If one string x is algorithmically incompressible means that we can not find regularities in the string in an effective way. It is interesting to observe that the majority of the information in our world is highly compressible (fact that permits to compress data using programs like gzip, winzip, compress, etc.). It means that we live in a world imersed into regularity, which we can interpret as general laws governing the phenomena around us. There is a more robust version of Kolmogorov complexity, called prefix complexity [7], based on auto-delimited descriptions. An auto-delimited description has the remarkable property that string’s code contains its own length. We denote the prefix complexity as K(.). III. S YSTEMS S CIENCE Systems science (or systems theory, or cybernetics) is a fuzzy academic domain and difficult to define (see [9]). The research area is characterized by an wide range of applications.

One possible definition is: “Systems science is the study and application of general problem solution methods and general principles of control of systems of a large range of types (mechanical, chemical, biological, electronical, economical, computational, etc.)”. The research area received an important contribution from Claude Shannon’s information theory [10]. Systems science does not try to study separately the components that compose complex systems, as traditionally the reductionism try to do. Otherwise, it try an holistic approach to the problem. It constitutes a transdisciplinary study of the system’s abstract properties, investigating general principles shared by almost all complex systems. Systems science argues that we can find in the reality we try out with our physical sensations, seemed by us as diverse and complex, types of organization (or regularity). Hence, founded these general laws governing these systems, we can apply the laws to new domains. A system may be viewed as an entity executing a process that results in an observable behaviour. We can apply this definition to almost all systems, from an unicellular organism (like a bacteria), to the economy of a modern industrialized country. We can think systems science research area as strongly connected to the modern complexity theory. Complexity theory is a powerfull tool to develop models and find solutions for problems related to complex systems. It is exactly an application of computational complexity theory in the context of systems science the focus of our interest in this paper.

IV. S YSTEM AND P ROCESS C HARACTERIZATION AND M ODEL E VALUATION The main contribution of this article is a proposal for the application of Kolmogorov complexity as a general tool for the characterization and evaluation of computational models and complex systems. Kolmogorov complexity is adequate to this task, since one of the most important characteristic of models and systems is their information content. The general idea is that the amount of information characterizes a process or a system. Eventually, we may stipulate maximum levels of complexity which are “acceptable”. If a process is not “acceptable” it is not feasible, because there are not enough resources allocated for its realization. Beyond characterizing such processes, we can compare computational models and systems by the criteria of being “more complex” or “simpler”. In this context, we identify as a central characteristic of the objects under investigation the quantity of information inherent in those objects, suppressing all other aspects and details of them. Thus we consider, in stead of dynamic aspects, only the static aspects related to the construction of the object (or its computation), despising other necessary resources such as execution time, except for the information supplied for its computation. The following concepts are fundamental in our approach: incompressibility; minorization and machine simulation; and non-computability of complexity.

A. Incompressibility The original purpose of Kolmogorov complexity was to formally define random sequences, thus creating the basis for a mathematical theory of probabilities [7]. A binary string is said to be random if its complexity is approximately equal to the size of the string. In this way, we define simple strings as being the ones that are regular or compressible, and random strings or complex ones as the ones that contain irregularity, or that are incompressible. It is interesting to observe that the problem of determining if a string is incompressible is an undecidable problem. Suppose that the set of incompressible strings is enumerable. Let f be a recursive partial function that enumerates the incompressible strings. Thus f (n) denotes the first string x0 with complexity greater than n. Then, C(x0 ) > n and C(x0 ) = C(f (n)) ≤ C(n) + c = log n + c. So C(x0 ) > n and C(x0 ) ≤ log n + c, for any arbitrary n, which is a contradiction. Observe that log denotes the base two logarithm. It means that we cannot formally determine how much information is necessary to compute a string, this being a limitation of the formal systems. B. Minorization and Machine Simulation As we saw earlier, minorization (see Definition 2) indicates that a specification method is “more efficient” in data compression than another one. Let sf be a string described by method f , and sg the same string described by a method g. Then, if for all s, |sf | ≤ |sg |+c (“f minorizes g”), it means that asymptotically method f is more efficient at data compression than method g. We can associate these methods with recursive partial functions (Turing machines). In this way, if a method f simulates method g, then f minorizes g. Machine simulation is a central and well known concept in computability theory. Hence, the proposed method inherits many consolidated and well known results. We shall remember that machine simulation induces an hierarchy of computational power and, consequently, an hierarchy of languages (the best known one being Chomsky’s hierarchy). C. Non-Computability of Complexity Thinking about C(.) as a function over binary strings, remains one question: Is it possible to compute the function C? As a matter of fact, the answer is no [11], [7]. Theorem 2: Function C is not computable. To determine C(x) we must execute every program and collect all that stop and compute x, choosing the one that has the smallest size. Trivially, from the halting problem, we cannot decide if a program stops or not [8]. Definition 4: A function f is semi-computable if there exists a recursive (total) function φ such that φ(x, t) is monotonic in t and limt→∞ φ(x, t) = f (x). Theorem 3: Function C is semi-computable. This means that function C, although it cannot be exactly computed, can be approximated. This has certain interesting effects as we shall see below.

V. A PPLICATIONS OF THE P ROPOSED M ETHODOLOGY A. Characterization of Software Development One of the main goals of software development methodologies is to obtain optimal code and correct programs with the least possible cost. The metrics and methods from software engineering are important in order to predict efforts and costs of the development process. We defend that the size of the program is an important parameter for software development, as it has influence over the management of the software project. The use of Kolmogorov complexity shows that the size of the program is related to the measure of complexity of the problem that will be solved by the program being developed, independently of other factors such as the programming language chosen or the ability of the developer (by the Invariance Theorem the complexity is invariant with respect to the specification method adopted). The choice of the size of the smallest program, among other measurements, defines an adequate measure of complexity, an universal complexity based on program size [1]. Our approach is related to a vision of the software development process as data compression. According to our approach, we can argue that formal methods have the property of “incompleteness”, in a very similar mathematical sense to the proof of arithmetics’ incompleteness done by G¨odel in 1931. This conclusion is based on Chaitin’s work about the incompleteness of the formal systems [12] (see the argument in Section IV-A). Another important aspect of software development is predictability. The predictability of software development is the ability to foresee development time, efforts and the development process costs. A fundamental parameter to anticipate costs and predict efforts is the size of the source code [1]. The average time it takes developers to write programs can be estimated by the average developer productivity. We can estimate the total number of lines of code and use this information to assess the necessary resources and deadlines for the software delivery. We are proposing the use of Kolmogorov complexity as a software development metric. If we accept that this metric is adequate, as it does not depend on the programming language used nor on developer ability, we will observe a problem related to the non-computability of complexity. We suggest that the software development process is, in this sense, formally unpredictable [1]. By the semi-computability of complexity we can affirm that the best solution, with regards to code size, can only be approximated. According to the authors [1], these results (incompleteness of the formal methods and the unpredictability of the software development process) suggest that software development is, in spite of the efforts towards its formalization, more subjective and empirical than objective and formal. The limits presented characterize the software development process as both experimental and based on heuristics such as, for example, the scientific development in physics and chemistry. Those conclusions, according to the authors, suggest that software engineering is a scientific area not fully characterized by typical engineering work, but also by experimental sciences

methodology [1]. B. Comparison and Evaluation of Animation Models An interesting application of Kolmogorov complexity recently developed and which illustrates our methodology is its usage as a measurement for comparing and, in a sense, evaluating computer animation models [13] (see a close approach/application in [14]). This comparison/evaluation is based on the data compression ratio obtained in an animation model. The development of a set of concepts and methods for evaluations of such type is very useful for Web applications development, where there is a great need for sophisticated multimedia resources and, consequently, an optimized and economical use of the communication channel between server and client. In a general sense, multimedia formats have data compression algorithms that enhance the application’s performance when exchanging image, sound, animation and video files. Such applications offer resources for manipulating multimedia objects such as rotation, translation, synchronization between events, and so on. The most commonly used animation format on Web pages are GIF (Graphics Interchange Format) animations. A GIF animation file is composed by a stream of compressed images which are presented in sequence, thus forming an animation which is a very useful resource for expressing ideas. The recent upcoming of new multimedia formats such as Macromedia’s Flash has allowed even greater flexibility in Web page construction. An animation is formed by a sequence of frames shown on screen at time intervals. We define “animation” as a pair (s, t) where s is a sequence s = s1 s2 s3 · · · sn , each si being a frame, and t is a sequence of time intervals t = t1 t2 t3 · · · tn , each ti ∈ N. In this way, we have defined an animation as a binary string “computed” in some way. This represents its “dynamic behavior”, i.e., the result of the animation’s execution, but says nothing about how to store and produce the animation. Hence, we must define a mechanism that “computes” the animation. Definition 5: An animation descriptor is a pair φ = (α, ∆), where α is an indexed set of images (or actors) and ∆ is a function ∆ : α → a, i.e., ∆ maps α in the target animation a. For example, in the GIF format the ∆ function is very simple. Every frame is stored literally inside the GIF file and the animation runs sequentially, from the first frame to the last one. Definition 6: Let s, s0 ∈ {0, 1}∗ be binary strings. We call δ a data decompression algorithm if, for all s0 there exists some s such that s0 = δ(s), where s is the code for s0 , with |s| ≤ |s0 | + c. We call γ a data compression algorithm if, for all s, s = δ(γ(s)), and γ must be an injective function. We call the pair Γ = (γ, δ) a compression scheme. It is important to notice that Definition 6 deals with lossless data compression, not including the case of lossy data compression algorithms. We can now formally define the concept of “better compression”. Definition 7: Let Γ1 = (γ1 , δ1 ) and Γ2 = (γ2 , δ2 ) be two data compression schemes. We say that Γ1 is better than Γ2 if, for all s and a certain c > 0, |γ1 (s)| ≤ |γ2 (s)| + c.

In this last Definition, we can find the link between our problem and Kolmogorov complexity. |γ(s)| is the code size for s in a compression scheme (γ, δ). A compression scheme Γ1 is better than Γ2 if and only if Γ1 minorizes Γ2 . Definition 8: An animation scheme is a pair G = (Γ, φ), where Γ is a compression scheme and φ is an animation descriptor. Observe that algorithm γ is applied over α, the set of images (or actors) from Definition 5. Let G be an animation scheme and a be a target animation. We define G(a) as being animation a as executed by scheme G, and define |G(a)| as the number of bits needed to store animation a using scheme G. It can be shown that, for some c > 0, |G(a)| = |γ(α)| + |∆| + c ,

(1)

where |γ(α)| is the number of bits needed to store all images (or actors) in α using compression γ, e |∆| is the size of function ∆. The meaning of |∆| is that we must describe (or specify) function ∆ in some effective way. In that way, we count the number of bits of that description. Definition 9: Let G1 and G2 be two animation schemes. G1 is better than G2 if, for every target animation a and a certain c > 0, |G1 (a)| ≤ |G2 (a)| + c and ¬∃c0 ∀a|G2 (a)| ≤ |G1 (a)| + c0 . The first means that G1 is not worse than G2 . The second, fixing c0 , is equivalent to ∃a|G2 (a)| > |G1 (a)| + c0 , that is, it is enough to show a0 that holds |G2 (a0 )| > |G1 (a0 )| + c0 . That is, G1 is better than G2 if G1 minorizes G2 in the Kolmogorov complexity sense of minorization. We call G1 and G2 respectively G1 -machine and G2 -machine in order to emphasize this relationship between the concepts. Hence, G1 machine is better than G2 -machine if G1 -machine simulates G2 -machine. As we have formalized the concept of “better animation” as machine simulation, induced by Kolmogorov complexity, we must formally define animations as machines. We observe that the compression ratio obtained is caused by two different aspects: The compression algorithm used to compress the images used in the animation; and the way in which the animation is computed from this set of images. These two different parts are represented in Equation (1) as |γ(α)| and |∆|, respectively. The concept of machine simulation (minorization) is central in our approach, allowing the comparison of the models with regards to the data compression obtained. The method inherits well known results from computability theory’s machine simulation (since the method works by the construction of proofs of machine simulation). In the same way machine simulation induces an hierarchy of machines (and of languages), in our method it induces an hierarchy of animation methods. C. Formalizing Lossy Data Compression In the previous Section we formalized lossless data compression (as we can see by the formula s = δ(γ(s))). But, many compression codecs (specially those designed for audio and video compression) compress not preserving all data (some bits may “change”), with the purpose of better compress ratio.

An example of this kind of compression is MPEG compression. MPEG is the designation for a group of audio and video standards. It is a generic coding of moving pictures and associated audio. MPEG use discrete cosine transform and code data with Huffman coding (an asymptotically optimal coding). We are not interested in how a specific standard for lossy data compression choose what will be discarded when data is compressed. We focus on a generic mathematical definition to describe and evaluate lossy data compression [4]. The main idea come from [15]. The authors suggest that partial information storage may be defined as a probability distribution. Choosing a sample from this distribution and applying a function over the sample, we compute a string similar to the original uncompressed string, if and only if the probability distribution is representative of the string we are compressing. But, these method is not practical and it does not define the quality of the compression. In order to get the definition, we need to use information distance [16] to measure difference between information before and after compression. Distance may be defined as d(x, y) = max(K(x|y),K(y|x)) max(K(x),K(y)) . It is important to note that d(., .) has the properties we expect: d(x, y) ≥ 0 and d(x, y) = 0 if and only if x = y; d(x, y) = d(y, x) (symmetry); d(x, y) ≤ d(x, z) + d(z, y) (triangle inequality). By the symmetry of algorithmic information [7], we know that K(x|y) ≈ K(xy) − K(y), allowing to measure information distance in a practical way (using some computable approximation of K(.), for example Lempel-Ziv coding, we can do empirical tests and evaluations on real MPEG streams). Suppose s0 = δ(γ(s)) and s 6= s0 (lossy data compression). We say that s0 is a ²-approximation of s if d(s, s0 ) ≤ ². The lossy data compression, r-compressed with ²approximation, applyed to a string s, may be defined as a program p that computes s0 , d(s, s0 ) ≤ ² and r = |s|/|p|, in such a way that, for some c, |p| ≤ |s| + c and K(s0 ) ≤ K(s) + O(1). Last Equation follows from a theorem stating that K(f (x)) ≤ K(x) + O(1), where f is any computable function over binary strings [7]. In our case, f = δ◦γ. It means that the compressed version of the string has less information than the original one. D. Characterization of Chaotic Systems One of the classic postulates of modern science is that every phenomenon can be predicted, as long as one knows precisely its initial conditions and its transformation rule (general law that rules the phenomenon’s behavior). Nevertheless, many phenomena are difficult to predict as they are exceedingly sensible to these initial conditions and to rounding errors in intermediary real values in mathematical calculations. Thus, any imprecision in determining these conditions and any rounding in calculations causes an inordinately big discordance between the prediction and the actual phenomenon (this effect is called the butterfly effect). Such behavior is called chaotic behavior [17]. The study of chaos theory can be defined as the qualitative study of dynamic, non-linear systems of aperiodic and unstable behavior. The typical characteristics

of chaotic systems are: non-linearity; determinism; sensitivity to initial conditions; irregularity and behavioral aperiodicity; impossibility of long term prediction caused by rounding in mathematical calculations with real numbers. A dynamic system can be defined in terms of a phase space, whose points represent possible system states, and an evolution law (or mapping) that describes the subsequent states of the system [18]. The system’s orbit is the trajectory, or set of points from the phase space, described by the system. The concept of chaos suggests the absence of organization, a disorder in which uncertainty and unpredictability predominate. Nonetheless, chaos theory studies that which can be called “the order of disorder”. The concept of chaos is related to that of complexity. Complex systems can exhibit a chaotic behavior, which is of difficult expression in simple terms. The patterns in a chaotic system are present, but are not regular or easy to predict. This does not mean that, in some important cases, a reasonable short term prediction cannot be done. The main practical applications of prediction and control (a subject not discussed in this article) of chaotic systems are, among others: more efficient wings for airplanes; better turbines; chemical reactions in chemical industries; implantable defibrillators; economical planning and financial market prediction; and computer networks [17]. We shall represent a system’s behavior as a binary string. One example of a chaotic system is the doubling map [19]. Consider the binary expansion of a real number, with regards only to its fractional part. The phase space of this chaotic system is composed of the real numbers in the interval [0; 1). The mapping of this system is defined by the formula xn+1 = 2xn mod 1. The observable states are the intervals [0; 21 ) and [ 21 ; 1). We assume that the initial state is obtained by choosing randomly a real number in the interval [0; 1), according to the uniform distribution. Then, the observable behavior cannot be better predicted than a series of flips of a fair coin [19]. Observe that the orbit described by the system consists of the bit stream of the binary expansion of the real number chosen initially. Although some may define a random sequence as a “chaotic sequence” (see [20]), we are proposing a significantly different chaos characterization. We must consider chaotic behavior as being in between the extremes of the line connecting Kolmogorov’s low complexity zone and Kolmogorov’s high complexity zone. We postulate that the proximity to one or the other extreme is a good measure that characterizes how chaotic a given system is. According to our approach, the proximity to the low Kolmogorov complexity zone represents a smaller irregularity and aperiodicity, defining linear dynamic systems or non-linear dynamic systems of a quasi-chaotic behavior. The proximity to the extreme of high Kolmogorov complexity represents dynamic systems of a very chaotic behavior or random (non-chaotic) behavior. Thus, we must consider, in our characterization of the chaotic systems, the concept of irregularity (aperiodicity), which is related to the concept of incompressibility, as seen above. The latter is related to the concept of security. A sequence is secure if it is impossible to discern it from a random sequence generated by a fair coin (i.e. it is unpredictable).

Consider a simple chaotic system (pseudo-random number generator) [17]. It is based in a generalization of the doubling map, xn+1 = cxn mod m, where c, m ∈ N. The constants c and m must be chosen with a certain criteria so that the generated sequence may be of good quality. Usually the initial value of the sequence (random seed) is taken from a relatively unpredictable value (for example, from the computer’s clock or from a previously stored random value). We know that the sequence generated by this algorithm presents a small Kolmogorov complexity, since it’s algorithmic description is short. Nevertheless it is empirically accepted that such sequences are difficult to predict (a good part of the commercial transactions on the Internet are based on cryptographic keys generated in a very similar way). Our approach proposes to characterize such sequences by how secure (or unpredictable) they are. According to our approach, the difficulty in predicting the behavior of a chaotic system is due to the non-computability of Kolmogorov complexity (difficulty in finding the minimal description). Such characterization presents the advantage of bringing into the field of chaotic systems a whole series of results from the Kolmogorov complexity area, which are mathematically wellfounded and expressive. VI. C ONCLUSION We presented a proposal for the application of Kolmogorov complexity as a tool to solve problems from systems science. Our approach seeks to characterize processes and systems, and to evaluate computational models based on the idea of data compression (understood as a measurement of the amount of information). The methodology was applied to the characterization of the software development process and to the comparison of computer animation models. It was also presented a proposal for the characterization of dynamic systems of chaotic behavior. The method is theoretical and more qualitative than empirical and quantitative. Future works shall cover ways to apply the methodology in more comprehensive contexts. Finally, we shall in the near future compare the proposed methodology to other methods from systems science. R EFERENCES [1] C. A. P. Campani and P. B. Menezes, “Characterizing the software development process: A new approach based on Kolmogorov complexity,” in Computer Aided Systems Theory - EUROCAST’2001, 8th International Workshop on Computer Aided Systems Theory, ser. Lecture Notes in Computer Science, Moreno-D´ıaz, Buchberger, and Freire, Eds., vol. 2178. Springer, 2001, pp. 242–256. [2] ——, “Aplicac¸a˜ o da complexidade de Kolmogorov na caracterizac¸a˜ o e avaliac¸a˜ o de modelos computacionais e sistemas complexos,” in 5th Workshop on Formal Methods, A. Martini and D. D´eharbe, Eds. Brazil: Sociedade Brasileira de Computao (SBC), Instituto de Inform´atica/UFRGS, 2002, pp. 100–112. [3] D. Lelewer and D. Hirschberg, “Data compression,” ACM Computing Surveys, vol. 19, no. 3, pp. 261–296, Sep 1987. [4] C. A. P. Campani and P. B. Menezes, “Evaluating computer animation models with lossy data compression using Kolmogorov complexity,” in Proceedings of the 2003 International Conference Imaging Science, Systems, and Technology (CISST’03), H. R. Arabnia and Y. Mun, Eds. CSREA Press, 2003, pp. 721–725.

[5] A. N. Kolmogorov, “Three approaches to the quantitative definition of information,” Problems of Information Transmission, vol. 1, pp. 4–7, 1965. [6] ——, “Logical basis for information theory and probability theory,” IEEE Transactions on Information Theory, vol. 14, pp. 662–664, 1968. [7] M. Li and P. Vit´anyi, An Introduction to Kolmogorov Complexity and its Applications. New York: Springer, 1997. [8] S. Kleene, Mathematical Logic. New York: Wiley, 1967. [9] R. Ashby, An Introduction to Cybernetics. London: Chapman and Hall, 1957. [10] C. Shannon, “A mathematical theory of communication,” Bell Sys. Tech. Journal, vol. 27, pp. 379–423, 623–656, 1948. [11] P. G´acs, “Lecture notes on descriptional complexity and randomness,” Boston University, Computer Science Dept., Boston, Tech. Rep., Dec 1993. [Online]. Available: http://www.cs.bu.edu/faculty/gacs/papers/aitnotes.ps.Z [12] G. J. Chaitin, “Godel’s theorem and information,” International Journal of Theoretical Physics, vol. 22, pp. 941–954, 1982. [13] R. Parent, Computer Animation: Algorithms and Techniques. Morgan Kaufmann Publishers, ISBN: 1558605797. [14] S. Subbaramu, A. Gates, and V. Kreinovich, “Application of Kolmogorov complexity to image compression: It is possible to have a better compression, but it is not possible to have the best one,” Bulletin of the European Association for Theoretical Computer Science, vol. 69, pp. 150–154, Oct 1998. [15] V. Kreinovich and L. Longpre, “Human visual perception and Kolmogorov complexity: Revisited,” University of Texas at El Paso, Computer Science Department, Tech. Rep., 1998. [16] C. H. Bennett, P. G´acs, M. Li, P. Vit´anyi, and W. H. Zurek, “Information distance,” in 25th ACM Symp. Theory of Comput., 1993, pp. 21–30. [17] W. Ditto and T. Munakata, “Principles and applications of chaotic systems,” Communications of the ACM, pp. 96–102, Nov 1995. [18] R. Devaney, An Introduction to Chaotic Dynamical Systems. Reading: Addison-Wesley, 1989, 336 p. [19] P. Vit´anyi, “Randomness,” to be published. [Online]. Available: http://www.cwi.nl/ paulv/ency.ps [20] V. Uspensky, A. Semenov, and A. Shen, “Can an individual sequence of zeros and ones be random?” Russian Math. Surveys, vol. 45, no. 1, pp. 121–189, 1990.