A FOUR COMPONENT CRYPTOGRAPHIC HASH FRAMEWORK

by Jeremy A. Hansen A Dissertation Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Engineering at The University of Wisconsin-Milwaukee May, 2009

A FOUR COMPONENT CRYPTOGRAPHIC HASH FRAMEWORK

by Jeremy A. Hansen A Dissertation Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Engineering at The University of Wisconsin-Milwaukee May, 2009

Major Professor

Date

Graduate School Approval

Date

ii

ABSTRACT A FOUR COMPONENT CRYPTOGRAPHIC HASH FRAMEWORK by Jeremy A. Hansen The University of Wisconsin-Milwaukee, 2009 Under the supervision of George I. Davida

The traditional method for describing cryptographic hash algorithms is to offer two components: the compression function and an iteration scheme. This paper shows that describing cryptographic hash algorithms as being comprised of four components more easily and more accurately illustrates the operation of the algorithm and simplifies analysis. In the four-component framework, cryptographic hash algorithms are divided into preprocessing, iterative structure, combining function and finalization components. As part of the iterative structure, a global state is initialized, updated according to the preprocessed input, and finalized into the output of the algorithm: the hash value. Each component lends itself to providing or preserving security properties in the overall algorithm. The four components are individually examined for the properties they provide and attacks they may prevent. Existing techniques for each component (that may not have been initially framed as such) are presented and analyzed and

iii

a few novel techniques are introduced. An extensive list of algorithms, including all published candidates for SHA-3, the forthcoming United States Advanced Hash Standard, are analyzed for their choices of each of their four components.

Major Professor

Date

iv

c Copyright by Jeremy A. Hansen, 2009

All Rights Reserved

v

Table of Contents

List of Figures . . . . . . . . . List of Tables . . . . . . . . . . List of Symbols/Nomenclature Acknowledgements . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

ix x xi xii

1 Cryptographic Hash Algorithms 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . 1.2 Properties . . . . . . . . . . . . . . . . . . . . . . . 1.2.1 Collision Resistance . . . . . . . . . . . . . 1.2.2 f -Resistance . . . . . . . . . . . . . . . . . 1.2.3 Preimage and Second Preimage Resistance 1.3 Applications . . . . . . . . . . . . . . . . . . . . . . 1.3.1 Password Obfuscation . . . . . . . . . . . . 1.3.2 Data Integrity . . . . . . . . . . . . . . . . 1.3.3 Challenge-Response Protocols . . . . . . . . 1.3.4 Digital Signatures . . . . . . . . . . . . . . 1.3.5 Bit Commitment Schemes . . . . . . . . . . 1.4 Attacks . . . . . . . . . . . . . . . . . . . . . . . . 1.4.1 Length Extension . . . . . . . . . . . . . . . 1.4.2 Fixed Points . . . . . . . . . . . . . . . . . 1.4.3 Second Preimages in Long Messages . . . . 1.5 Contributions and Thesis . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

1 1 2 2 4 6 7 7 8 9 10 10 11 11 12 12 13

2 The 2.1 2.2 2.3 2.4

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

16 16 17 19 19 21 21 22 23 24

3 Preprocessing 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Locality . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Content . . . . . . . . . . . . . . . . . . . . . . . . 3.2.3 Preprocessing as a Domain Transformation . . . . 3.3 Properties and Observations . . . . . . . . . . . . . . . . . 3.3.1 Preprocessors and Property Preservation . . . . . 3.3.2 Preprocessing as Encoding and Differential Attacks

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

25 25 26 26 28 30 31 31 32

2.5

Framework Overview . . . . . . . . . . . . . Previous Frameworks . . . . . . . The Four Component Framework Components of the Framework . 2.4.1 Preprocessor . . . . . . . 2.4.2 Iterative structure . . . . 2.4.3 Combining Functions . . . 2.4.4 Finalization . . . . . . . . Application . . . . . . . . . . . .

. . . .

. . . . . . . . .

. . . . . . . . .

vi

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

3.4

3.5

3.3.3 Preprocessor versus Iterative Structure Schemes . . . . . . . . . . . . . . . . . . . . . 3.4.1 Padding . . . . . . . . . . . . . . . . . 3.4.2 Message Length/Final Block Length . 3.4.3 Salt . . . . . . . . . . . . . . . . . . . 3.4.4 Bits So Far/Blocks So Far . . . . . . . 3.4.5 Whitening . . . . . . . . . . . . . . . . 3.4.6 Message Self-Interleaving . . . . . . . 3.4.7 Message Duplication . . . . . . . . . . 3.4.8 Block Position . . . . . . . . . . . . . 3.4.9 Block Permutation . . . . . . . . . . . 3.4.10 Hash Length and other Parameters . . 3.4.11 Tagging . . . . . . . . . . . . . . . . . 3.4.12 Checksums and Error-Detecting Codes 3.4.13 Lossless Compression . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . .

block size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4 Iterative Structure and Combining Functions 4.1 Introduction . . . . . . . . . . . . . . . . . . . . 4.2 Properties and Observations . . . . . . . . . . . 4.2.1 Combining Functions . . . . . . . . . . . 4.2.2 Iterative Structure . . . . . . . . . . . . 4.3 Iterative Structure Schemes . . . . . . . . . . . 4.3.1 Linear Iterative Schemes . . . . . . . . . 4.3.2 Tree Iterative Schemes . . . . . . . . . . 4.3.3 Modifying Features . . . . . . . . . . . . 4.3.4 Proposed Structures . . . . . . . . . . . 4.4 Combining Function Schemes . . . . . . . . . . 4.4.1 Execution Strategy . . . . . . . . . . . . 4.4.2 Subcomponents . . . . . . . . . . . . . . 4.5 Conclusion . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

33 33 34 35 35 36 37 38 38 38 39 39 39 40 40 41

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

42 42 44 44 49 50 50 53 54 55 58 58 60 62

5 Finalization 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . 5.2 Properties and Observations . . . . . . . . . . . . 5.3 Finalization Schemes . . . . . . . . . . . . . . . . 5.3.1 Subset-of-State . . . . . . . . . . . . . . . 5.3.2 Final Permutation and Final Compression 5.3.3 No-Input Iterations . . . . . . . . . . . . . 5.3.4 Pumping Rounds . . . . . . . . . . . . . . 5.3.5 Pseudorandom Number Generators . . . . 5.4 Conclusion . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

65 65 65 66 66 67 67 68 68 68

. . . .

70 70 70 72 75

6 Analysis and Conclusions 6.1 Negative Features . . . . . 6.2 Algorithm Analysis . . . . 6.2.1 Preprocessing . . . 6.2.2 Iterative Structure

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

vii

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

6.3

6.4 6.5

6.2.3 Combining Function . . . . 6.2.4 Finalization . . . . . . . . . Algorithm Analysis: SANDstorm . 6.3.1 Preprocessing . . . . . . . . 6.3.2 Iterative Structure . . . . . 6.3.3 Combining Function . . . . 6.3.4 Finalization . . . . . . . . . Recommendations . . . . . . . . . 6.4.1 Recommended Construction Conclusion . . . . . . . . . . . . . 6.5.1 Future Directions . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

78 81 84 84 84 84 85 85 85 86 87

References

88

Appendix: Application Programming Interface

99

Curriculum Vitae

102

viii

List of Figures

2.1 2.2 2.3 2.4

The Merkle-Damg˚ ard Iterative Structure Sponge-style “Absorb” Phase . . . . . . Sponge-style “Squeeze” Phase . . . . . . Four Component Framework . . . . . .

. . . .

17 18 19 20

3.1

Preprocessors as Domain Transformations . . . . . . . . . . . . . . . . . . .

30

4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 4.10 4.11

A “supercombiner” . . . . . . . . . . . . . . . . . . . . . The squeezing phase of a sponge function as finalization A Narrow-pipe Linear Iterative Structure . . . . . . . . An `-pipe Linear Iterative Structure for ` = 2 . . . . . . Tandem Davies-Meyer . . . . . . . . . . . . . . . . . . . A Merkle Hash Tree . . . . . . . . . . . . . . . . . . . . A Modified XOR Hash Tree . . . . . . . . . . . . . . . . The Enveloped Merkle-Damg˚ ard Iterative Structure . . Shoup’s Transform . . . . . . . . . . . . . . . . . . . . . The 3C Hash’s Iterative Structure . . . . . . . . . . . . A Balanced Feistel Network . . . . . . . . . . . . . . . .

45 47 51 52 53 54 55 56 57 58 60

ix

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . . . . . . . . .

. . . .

. . . . . . . . . . .

. . . .

. . . . . . . . . . .

. . . .

. . . . . . . . . . .

. . . .

. . . . . . . . . . .

. . . .

. . . . . . . . . . .

. . . .

. . . . . . . . . . .

. . . .

. . . . . . . . . . .

. . . .

. . . . . . . . . . .

. . . .

. . . . . . . . . . .

. . . . . . . . . . .

List of Tables

3.1 3.2

Examples of preprocessor content types in both localities . . . . . . . . . . Breakdown of various preprocessing schemes . . . . . . . . . . . . . . . . . .

26 34

6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9 6.10 6.11 6.12 6.13 6.14 6.15 6.16 6.17 6.18 6.19 6.20

Requirements for Property Preservation . . . . . . . . . . . . Legend for Preprocessor Abbreviations . . . . . . . . . . . . . Preprocessing Techniques in Common Algorithms . . . . . . . Preprocessing Techniques in Other Hash Algorithms . . . . . Preprocessing Techniques in SHA-3 Candidates, Part 1 . . . . Preprocessing Techniques in SHA-3 Candidates, Part 2 . . . . Iterative Structure Techniques in Common Algorithms . . . . Iterative Structure Techniques in Other Algorithms . . . . . . Iterative Structure Techniques in SHA-3 Candidates, Part 1 . Iterative Structure Techniques in SHA-3 Candidates, Part 2 . Legend for Combining Function Abbreviations . . . . . . . . Combining Function Techniques in Common Algorithms . . . Combining Function Techniques in Other Algorithms . . . . . Combining Function Techniques in SHA-3 Candidates, Part 1 Combining Function Techniques in SHA-3 Candidates, Part 2 Legend for Finalization Function Abbreviations . . . . . . . . Finalization Techniques in Common Algorithms . . . . . . . . Finalization Techniques in Other Algorithms . . . . . . . . . Finalization Techniques in SHA-3 Candidates, Part 1 . . . . . Finalization Techniques in SHA-3 Candidates, Part 2 . . . . .

71 72 73 73 73 74 76 76 76 77 78 79 79 79 80 81 82 82 82 83

x

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

List of Symbols/Nomenclature

|| ⊕ ∨ ∧ h n H M mi S w L π b

Concatenation of bit-strings The exclusive-OR (XOR) operation The bitwise-OR operation The bitwise-AND operation The hash value returned by the algorithm The length, in bits, of the hash value h The entire hash algorithm The entire input message The ith block of M The internal state of the algorithm The length, in bits, of the state S The length, in bits, of the message M The preprocessing function component The size of a message block operated upon by a local preprocessor I∆ The iterated structure component which uses combining functions in ∆ δi The ith combining function in ∆ dδi The length of a message block operated upon by combining function δi ∆ A collection of combining functions Φ The finalization function component O(x) “Big O” notation: the number of operations is asymptotically at most x Ω(x) “Big Omega” notation: the number of operations is asymptotically at least x HD(x, y) The Hamming Distance between x and y - the number of positions where the bits of x and y differ HW(x) The Hamming Weight of x - the number of bits set to 1 in x

xi

Acknowledgements

I would like to dedicate this dissertation foremost to Cara and JJ, whose patience and supportiveness are undoubtedly the reason I was able to finish this work. For all of the hours that we have not spent together due to my spending time working on this, I love you and thank you both. I would also like to thank my parents and grandparents whose work over the years enabled me to accomplish my goals. Thanks to George Davida who, throughout my many strange and tangential ideas, managed to help me stay focused and graduate in a reasonable amount of time. Thanks to Bruce Barr, my first academic mentor, who managed to get the gears in my head turning in the right direction all those years ago. I have not forgotten you. Additional thanks to my employers over the course of my graduate studies who have remained tolerant and supportive of my activities outside of work, and all of my friends of whom I now expect to see more.

xii

1

Chapter 1 Cryptographic Hash Algorithms

1.1

Introduction

The design and analysis of cryptographic hash algorithms is made easier by defining them as the composition of four parts. Cryptographic hash algorithms are critical pieces of modern cryptographic systems that provide a transformation unlike those provided by other cryptographic primitives like block ciphers and stream ciphers. They are typically used to guarantee integrity in applications such as bit-commitment protocols, challenge-response protocols, user password protection, and digital signature schemes. This essential part of a cryptographer’s toolkit is new relative to block and stream ciphers and is not as well understood. Our work seeks to partially remedy that by introducing a novel framework which allows cryptographers to more easily demonstrate that a particular cryptographic hash algorithm possesses necessary security properties and consequently can be used in all appropriate applications. Unlike stream or block ciphers whose outputs are the same length as their inputs, hash algorithms produce as output a fixed-length bit string (commonly referred to as a hash value, hash, or message digest) from an arbitrarily-long bit string. Determining the hash value of an input is often referred to as “hashing” the input. The entire hash algorithm can be represented as the function H operating on its arbitrary-length preimage M to produce its image h: H(M ) → h The security of the algorithm begins with the property that, if the message changes, the hash value h will almost certainly change. Because of this, the hash value is often

2 used as a shorter, surrogate representation of the preimage, but this “compression” should not be confused with reversible compression since the preimage should not be retrievable from the hash value. Cryptographic hash algorithms are expected to be one-way—that is, it should be difficult to invert the algorithm given an image to produce a preimage. Note, though, that there must be other “colliding” preimages which hash to the same message digest whenever the preimage bit-string is longer than the image. Resistance to both attacks which generate colliding preimages and attacks that produce preimages given only an image is the most common measurement of the security of cryptographic hash algorithms. Both of these properties are described in more detail in the next section.

1.2

Properties

To be useful in its role, a cryptographic hash algorithm is expected to possess certain properties. We first assume according to basic cryptographic principles that the entire algorithm specification is published and otherwise publicly available. We will also assume that the algorithm and all of its inputs are deterministic. There are algorithms which offer a powerful tool in randomized hashing [65], but we opt to work only with algorithms whose inputs are non-random. Three properties are typically cited as essential: collision resistance, preimage resistance, and second preimage resistance. We also discuss some lesser-used security properties which are related to the “traditional” properties. The following definitions of security properties are consistent with those of Menezes et al. [97].

1.2.1 Collision Resistance The first essential property of cryptographic hash algorithms, collision resistance, guarantees that it is difficult for any party to produce two inputs (called collisions or a colliding pair ) which hash to the same value. Hash algorithms which possess this property are often considered to be members of a distinct category: collision

3 resistant hash functions. We avoid using the terms “collision freedom” or “collision free” as found elsewhere in the cryptologic literature since the word freedom implies that some algorithm may in fact be free from collisions. As we deal here only with concrete instantiations of hash algorithms, collisions must exist due to the pigeonhole principle, as described by the “foundations of hashing dilemma” [115,17]. Since collisions are unavoidable, security properties like collision resistance are usually argued by asserting that there exist no attacks against a particular algorithm that are more efficient than brute force. In the case of collision resistance, every hash algorithm which produces an n-bit hash is vulnerable to a brute force attack which produces a n

colliding pair of inputs in approximately 2 2 operations due to the birthday paradox [97]. It is usually said that a hash algorithm H is collision resistant if determining inputs x and x0 such that H(x) = H(x0 ) is computationally infeasible. In the concrete case of a collision resistant algorithm, finding x and x0 would require no fewer than n

2 2 operations, the same number expected by the algorithm-independent birthday n

attack. Attacks which generate collisions in fewer than 2 2 operations are usually considered to be breaks of the algorithm, even if the attacks may not be applicable in a reasonable amount of time. Despite their infeasibility, collision attacks that are faster than the general birthday attack may be indicative of larger undiscovered flaws in the algorithm which may in turn need lead to even more efficient attacks.

1.2.1.1

Multicollisions

Finding additional collisions in the typical Merkle-Damg˚ ard iterative structure turns out to not be much more difficult than finding the first. As proven in [90], in an iterated cryptographic hash algorithm with an ideal compression function (as described in more detail in Chapter 4) and an internal state that is w bits long, an

4 attacker requires w

Ω(min{2 2 , 2n(K−1)/K }) operations to find K colliding messages. This type of attack illustrates the difference between collisions on the internal state (“state collisions”) and collisions on the final w

hash value (“output collisions”). The first quantity above, 2 2 , indicates the difficulty of finding internal collisions, while 2n(K−1)/K represents the difficulty of finding K output collisions in an ideal hash function producing an n-bit hash. It follows then, that when w ≥ 2n, multicollisions are of a similar difficulty as the ideal hash.

1.2.2 f -Resistance Collision resistance alone may not be sufficient for certain protocols where other specific security requirements apply. For example, the algorithm VSH [38], while provably collision resistant, is vulnerable to attacks against its algebraic structure [119] which may make it unsuitable for certain typical applications of cryptographic hash algorithms such as those that apply algebraic operations similar to those used internally in the algorithm. In [3], Anderson describes several properties of cryptographic hash algorithms which involve arithmetic operations, such as addition freedom, which is defined as a property of a hash function H where it is infeasible to find x, x0 , and y, such that H(x) + H(x0 ) = H(y). We recast this property as “addition resistance” by defining it as the property such that an attack against an addition-resistant algorithm requires no fewer operations than an algorithm-independent brute force attack to find x, x0 , and y, such that H(x) + H(x0 ) = H(y). A cryptographic hash algorithm is f -resistant if, for any fixed function f of v variables, an attacker requires more operations than an algorithm-independent brute force attack to determine all xi for all values of i ∈ {1, ..., v} and y such that f (H(x0 ), ..., H(xv )) = H(y) An f -attack against H is an attack which produces a set of values for all xi and

5 y that violates H’s f -resistance. Using the previous notation, it is straightforward to describe other security properties. The function for multiplication resistance, for example, can be written as f (H(x0 ), H(x1 )) = H(x0 ) · H(x1 ) = H(y) The f for addition resistance can be written similarly as f (H(x0 ), H(x1 )) = H(x0 ) + H(x1 ) = H(y) and for collision resistance written as f (H(x0 )) = H(x0 ) = H(y) With this generalized resistance property notation, cryptographers can enumerate the functions to which specific hash algorithms are resistant (or weak). This is useful since certain applications may require complex f -resistances. An example of a more complex resistance requirement is in the Yen-Laih signature scheme [3]: given A, B, and q, it should be difficult to determine x and y such that f (H(x)) =

A H(x) (mod q) = H(y) B n

Lemma 1.2.1 The algorithm-independent brute force f -attack requires O(2 2 ) operations. Proof Fix all x0 , ..., xv−1 to arbitrary values, then launch a birthday attack to find n

xv and y to find f (H(x0 ), ..., H(xv )) = H(y), which requires O(2 2 ) operations. This brute force attack establishes an upper bound for all choices of f , but does not preclude an algorithm-independent attack against a particular f which requires n

substantially fewer operations than 2 2 .

6

1.2.3 Preimage and Second Preimage Resistance The other two essential properties, preimage resistance and second preimage resistance, are usually more important to applications than collision resistance, and are fortunately more resistant to brute force attack. Whereas the previous resistance properties allow an attacker flexibility with the inputs and outputs, these two properties require that certain values remain fixed. Preimage resistance in a cryptographic hash algorithm guarantees that, given only a hash value, it is difficult to determine a preimage which hashes to that value. More formally, we say that a cryptographic hash algorithm H is preimage resistant if, given y, an attacker requires no fewer operations than an algorithm-independent brute force attack to determine x such that H(x) = y. The brute force attack on a hash function with an output that is n bits long simply selects inputs until a preimage that hashes to y is found. Since the probability of finding a preimage this way is 2−n , an attacker can expect to select O(2n ) inputs before he finds one which hashes to y. Second preimage resistance is similar, except that an attacker also has a preimage x to work from in addition to the hash value. A cryptographic hash algorithm H is second preimage resistant if, given x and y, an attacker requires no fewer operations than an algorithm-independent brute force attack to determine x0 such that H(x) = H(x0 ) = y. While the brute force second preimage attack requires the same number of operations as preimage resistance (that is, O(2n )), second-preimage attacks are usually considered to be easier than preimage attacks since the additional information provided by x may allow for differential attacks to succeed. In typical iterated hash algorithms, longer messages provide more opportunities for second-preimage attacks, such that the difficulty of finding a second preimage is reduced significantly [80]. Refer to section 1.4.3 for more details.

7

1.2.3.1

Multipreimages

Like attacks which can quickly produce additional collisions after the first, it may be possible to generate K preimages from a given hash in less than O(K2n ) time. As proven in [90], an attacker requires w

Ω(min{2 2 , K2n }) operations to find K-way preimages or K-way second preimages in an ideal iterated hash algorithm with a w-bit internal state and n-bit hash value.

1.3

Applications

Cryptographic hash algorithms are useful for many applications such as password obfuscation, data integrity, challenge-response protocols, digital signatures, and bit commitment schemes. The majority of these applications require hash algorithms to be (second) preimage resistant, though attacks on some applications can exploit functions which are not fully collision resistant [71]. Each of these applications are described below.

1.3.1 Password Obfuscation To avoid storing plaintext passwords, hashes of passwords are often stored instead. This prevents a malicious party who gains access to the password file from being able to use the passwords directly. When a legitimate user logs in, the user provides their password p, the system hashes the user’s input (H(p)), and compares it to the previously stored hash h. If they are equal, the user is successfully authenticated. An attacker without access to this hash has only one hope of compromising this system: correctly guess the user’s password while interacting directly with the target system. With the hash, however, an attacker can devote resources outside of the authenticating system to determining the password. This is called an offline attack, whereas the previous guessing-based attack is an online attack. In a na¨ıve offline

8 attack, an attacker produces many possible values for p, hashes each, then compares each hash to the value extracted from the authorizing host. When he finds a match, the attacker knows that he has a value for p that can be successfully used to log on and impersonate the user. To improve the speed of an offline attack, an attacker may be able to precompute lookup tables of the hashes of common passwords. Another method to speed the password search is to use rainbow tables [106], which allow for the construction of efficient lookup tables which offer a trade off between the time spent searching and the size of the table. If the hash algorithm that underlies this application is not preimage resistant, the attacker can work backwards directly from h to determine p. If the difficulty of doing so is less than guessing all possible passwords, this preimage attack results in an attack that undermines the security of the algorithm.

1.3.2 Data Integrity Hash functions are often used to guarantee the integrity of data in transit and in storage. When used to guarantee the integrity of data in transit, cryptographic hashes behave much like error-detecting codes but have the additional benefit of being more resistant to malicious attack than typical error-detecting codes. Applications using hashes for integrity verification operate in two distinct steps: evaluation and verification. The evaluation step uses a hash algorithm H to produce a hash value h from an input string x such that h = H(x). This value h is stored for later retrieval or sent along with the data string. The verification step then re-evaluates the data string, which we refer to as x0 , to determine if any changes have taken place. Assuming that h itself was unchanged in transit or during storage, we can conclude with high probability that, if h = H(x0 ), then x = x0 . Note that the problem remains of how to keep h itself from being modified between the evaluation and verification

9 steps. Tripwire [82] is an implementation of this scheme which verifies the integrity of important files. It stores hashes of important system files which are re-verified periodically. Execution-time whitelisting systems such as the system described in [66] maintain a list of hashes of all executable files on a system and only allows those applications with hashes in the list to execute, regardless of the user’s permissions. In this application, second preimage resistance is the most important property. Without it, an attacker can replace a legitimate message or file with one he generates as part of a second preimage attack that will pass unnoticed through the verification step.

1.3.3 Challenge-Response Protocols Authentication protocols like CHAP [127] which are based on a challenge-response exchange can be implemented using cryptographic hash algorithms. The challenge portion begins with the user requesting access to a resource. The authenticator responds by sending the challenge C, a random string of data, to the user. The user concatenates this random string with their password pu and hashes the result to generate the response Ru = H(C||pu ) The user sends Ru to the authenticator, which, with its copy of the expected password pe , can calculate the expected response to the challenge as Re = H(C||pe ). If Re = Ru , the authenticator can conclude with a high degree of certainty that the user knows his or her password. The primary benefit of this scheme is that the password is never transmitted over an insecure channel, and malicious modification of the exchanged messages will almost always result in the authenticator rejecting the user’s request. Hash algorithms subject to preimage attacks may be exploited by an attacker to discover pu given C and Ru in a similar manner to attacks against password obfuscation applications. Non-collision-resistant algorithms may also be subject to attack, such as those described in [121] and [136].

10

1.3.4 Digital Signatures Digital signatures are used on digital documents for similar reasons as physical signatures on physical documents, the most straightforward of which is to demonstrate the authenticity of the item that is being signed. Digital signatures are also more powerful than physical signatures in that modification of the signed message becomes evident to any party verifying the signature. For this reason, digital signatures are also effective at guaranteeing non-repudiation, so a party who has produced a digital signature can not return at a later time and claim that their signature is invalid. Hash algorithms are used in digital signatures both to improve their security and increase their speed. Originally, RSA was used to sign (represented by function S) an entire message M with a private key k to generate a signature s = S(k, M ). In [46], Davida shows a weakness in this signature scheme that effectively discloses the private key if an attacker is allowed to choose the message to be signed. By first hashing M so that the operation becomes s = S(k, H(M )), Davida’s attack is foiled, provided that the cryptographic hash used is not subject to the same homomorphisms as exploited by Davida’s attack1 . Since the RSA signing operation requires a fair number of exponentiation operations on extremely large numbers, reducing the size of the input to a short hash value dramatically improves the performance of the operation. Weaknesses in the hash function, particularly those producing collisions, can lead to attacks which allow two different messages to bear the same valid digital signature.

1.3.5 Bit Commitment Schemes Bit commitment allows a party to commit to a particular value (typically 0 or 1) with an untrusted second party so that neither party can cheat in an exchange. That is, if the first party “Alice”, chooses the value 1 and committed this to the second party, “Bob”, Alice cannot convince Bob that she committed to the value 0 and Bob 1

That is, H(M1 ∗ M2 ) 6= H(M1 ) ∗ H(M2 )

11 cannot renege on an agreement that depended upon Alice choosing 1. There are two phases of a commitment scheme: the commit phase and the reveal phase. The commit phase involves Alice choosing the value 0 or 1, generating a random string r, and publishing her commitment c. Supposing that Alice chooses 1, she calculates c = H(1||r), and provides c to Bob. In the reveal phase, Bob publishes the true result of the exchange, 0 or 1. Alice reveals her choice by publishing 1 and r, at which point, Bob can easily verify that the value of c that Alice provided earlier corresponds to H(1||r). Alice can cheat in this scheme if she can produce collisions in the underlying hash algorithm that differ in their first bits: c∗ = H(1||r0 ) = H(0||r1 ) By committing to c∗ , she can select r0 or r1 and submit it as r, depending on which value she chooses after the commit phase. Bob can cheat if he can create preimages that hash to Alice’s commitment c, since he can claim that the preimage he created is in fact Alice’s choice with a different value of r.

1.4

Attacks

There are many techniques for attacking cryptographic hash algorithms, most of which are algorithm-specific, such as differential cryptanalysis, described in Chapter 4. There are other, more generic attacks, though, that can be applied to algorithms in the general case, or to at least a large subset of algorithms, say, those that use the Merkle-Damg˚ ard iterative structure. We discuss a few of these here.

1.4.1 Length Extension In a linearly-iterated hash algorithm, it is possible for an attacker to generate a new hash value with no knowledge of the original message M aside from its length L and its hash h = H(M ). Since most hash algorithms append some padding and the

12 length L to the message before hashing, we can represent h as h = H(h0 , M ||pad||L) An attacker can append an arbitrary second message M 0 to this by first calculating its length: L0 = length(M ||pad||L) Then the attacker can construct h0 = H(h, M 0 ||pad0 ||L0 ) without direct knowledge of M .

1.4.2 Fixed Points A fixed point is a message block x that results in the following condition with a compression function C and chaining value hi : hi+1 = C(hi , x) = hi Since x does not update the internal state, it may be repeated infinitely many times to generate colliding messages: H(M ) = H(x||M ) = H(x||x||M ) Fortunately, this attack can be foiled by appending the message’s length L, since H(M ||L) 6= H(x||M ||length(x||M ))

1.4.3 Second Preimages in Long Messages In iterated hash algorithms that produce an n-bit hash, long messages are susceptible to a second preimage attack with difficulty less than 2n . Since the message is usually broken into blocks to be processed by compression functions that have a

13 fixed-size inputs, we say that the message M is made up of t blocks. While computing H(M ), each of the t intermediate values is stored and indexed for later use. At this point, the attacker simply hashes random values until an output matches one of the stored intermediate values. The probability that a random value hashes to one of the stored values is t 2n Thus, the expected number of trials this requires is 2n O t

Once a colliding block is found, this block can be used to replace the entire message up to the colliding intermediate value. Supposing that the collision is found after the k th block of the message (mk−1 ) is processed and the intermediate value at that point is hk . With this attack, the colliding block µ can replace the string {m0 ||...||mk−1 } so that the new message is M 0 = µ||mk ||...||mlast As in the fixed-point attack, appending the message length foils this attack, since H(M ||L) 6= H(µ||mk ||...||mlast ||length(µ||mk ||...||mlast )) even if they collide without the length.

1.5

Contributions and Thesis

The Hash Futures Panel at the Second NIST Hash Workshop in 2006 came to the consensus that cryptographers do not really know which features are desirable in a hash algorithm [72]. The framework described in this dissertation and in its predecessors [47,48] serves to resolve some of this uncertainty. We follow a practical approach to analyzing and designing cryptographic hash algorithms and break down these algorithms into preprocessing, iterative structure, combining functions, and

14 finalization components. A structured approach like that present in this model allows weaknesses in existing algorithms to be mitigated by adding or replacing components and gives new algorithms the opportunity to select strong individual components at the time of their design. We attempt to decompose the complexities of hash algorithms into smaller pieces which are individually easier to analyze. In the next chapter, we continue our discussion by formally defining the framework and its components. We limit our discussion to real-world algorithms (the standard model), but do discuss hash properties in the ideal model when interesting results apply to actual instantiations of cryptographic hash algorithms2 . Each component is described individually with lists of existing techniques, the attacks that they can foil, and known weaknesses. We then consider the security of a wide selection of modern cryptographic hash algorithms (including all the first-round candidates in the SHA-3 competition) by analyzing the properties of their individual components. In addition to analyzing the SHA-3 candidates, we also provide a description of how algorithms might be generated a la carte and on the fly based on the security properties that are required. We also discuss in the Appendix how the generic Application Programming Interface (API) [105] provided by NIST for the SHA-3 competition is well-suited to this framework. The four component model supports the typical cryptanalytic methodology of analyzing hash algorithms by inspecting and breaking weakened versions. Rather than approaching each hash algorithm as a monolithic structure, though, cryptanalysts can devote their energies to attacking one component at a time. If a piece of an algorithm is analyzed and found to be weak, one may be able to exploit the entire algorithm. On the other hand, attacks which affect a single component may be demonstrated to not be applicable by virtue of constraints imposed by other components. These 2

The ideal model offers one such construct called a random oracle (discussed more in Chapter 4), which represents a perfect cryptographic hash algorithm and can be used to describe upper bounds on the security of a particular construction.

15 other components have an opportunity, then, to shield the overall algorithm from some of the the weaknesses of the compromised component. This provides designers the ability to provide defense in depth within the algorithm itself. To borrow a term from [90], combining components that provide some redundancy in their security assurances leads to better, failure tolerant algorithms.

16

Chapter 2 The Framework

2.1

Overview

This framework divides cryptographic hash algorithms into four distinct components, which, while different than those typically selected in the construction of hash algorithms, provide a general, concrete method encompassing all cryptographic hash algorithms including (but not limited to) those following the Merkle-Damg˚ ard structure [102,45], the (extended) sponge framework [62], streaming hashes, and tree-based hashes. We begin with a short discussion of each of the individual components, then explore the application of the framework to the analysis, design, and classification of cryptographic hashes. Previous analytical frameworks have focused on specific components, like the compression function [43] or the iterative structure [111], while others have relied on certain assumptions, some concrete [102,45], some not [95,21]. This framework, on the other hand, can be used to represent any cryptographic hash algorithm. That is not to say, though, that these more specific frameworks are without a use, or that they were attempting to accommodate all algorithms in this way. Each will be described in the corresponding subsection of the four component framework as tools to analyze individual components. In the simplest terms, a hash algorithm H produces a fixed-length bit string (a hash value) of length n from an arbitrarily-long input bit string. If we have an input M , a collection of parameters p0 , ..., pk ∈ P , and the final hash value h, we describe a hash algorithm at a high level as H(M, P ) → h

17

m0

m1

mk-1 Final hash value

C

h0

C

h1

h2

hk-1

C

hk

Figure 2.1 The Merkle-Damg˚ ard Iterative Structure

To avoid repetition of the symbol P in every function, we assume hereafter that the collection of parameters is passed to every component or is otherwise available globally.

2.2

Previous Frameworks

The original framework for constructing a hash algorithm is the Merkle-Damg˚ ard (MD) iterated scheme, which can be described as the linear application of a compression function to a message block and the output of the previous compression and is illustrated in Figure 2.1. More concisely, the ith message block mi and the ith chaining value hi are passed as input to the compression function C to return the next chaining value: C(mi ||hi ) → hi+1 In this scheme, the only state per se is the chaining value passed to the next iteration. Once the final message block is processed in this way, the final chaining value is returned as the hash value for the entire algorithm. Sponge functions depart from the MD methodology, though they remain linear constructions that process the input message sequentially in blocks. Sponge functions begin with a large state into which message blocks are incorporated, then the state undergoes a permutation after which the next message block is added and the state is permuted again. Supposing that the state after the ith message block is called Si+1 , the function which combines the message block with the state is C, and the function

18

mi

Si

mi+1

C

P

Si+1

C

P

Si+2

Algorithm state

Figure 2.2 Sponge-style “Absorb” Phase

which permutes the state is P , this “absorbing” phase of a sponge function can be seen in Figure 2.2 and described as P (C(Si , mi )) → Si+1 This continues until the last message block mk−1 is added and the state undergoes its final permutation. At this point, the state (Sk ) undergoes the “squeezing” phase which involves calling the permutation function a number of times without message input and is illustrated in Figure 2.3. Every time the state is permuted, a part of the state is extracted via the function X and concatenated to the final hash value. (For the sake of illustration, X returns the least significant bits of S.) When the hash value reaches the desired size, the hash is returned. h = X(P (Sk ))||X(P 2 (Sk ))||X(P 3 (Sk ))|| . . . The Extended Sponge Framework (ESF) generalizes the notion of a sponge function by expanding the ways that a message can be introduced into the state as well as how the state produces the final hash value. Where the original sponge functions only combine message blocks with the exclusive OR (XOR) operation, the ESF offers the option of using either XOR or simply overwriting some part of the state. For generating output, sponge functions only use squeezing rounds, while the ESF offers squeezing rounds or truncation.

19

P

Sk

Sk+1

P

lsbk

Sk+2

Sk+b

lsbk+1

lsbk+b

Final hash value

h

Figure 2.3 Sponge-style “Squeeze” Phase

2.3

The Four Component Framework

Where the previous frameworks apply specific constraints on the operation, sequence, or internal state of cryptographic hash algorithms, this four component framework attempts to accommodate any structures that algorithm designers might introduce. This framework describes a hash algorithm as having four distinct components: a preprocessor, an iterative structure, a set of one or more combining functions, and a finalization. In a very similar fashion as shown in Figure 9.2b in [97], the function H is defined as the composition of several other functions, including a preprocessor π, an iteration scheme I∆ which uses combining functions δ0 , ..., δq ∈ ∆, and a finalization function Φ: H(M ) = Φ(I∆ (π(M )))

2.4

Components of the Framework

We now define each component in a way which clearly delimits “responsibility” for transforming the input into the output. While the combining function has long been considered to be the most important part of a hash algorithm and while its importance should not be understated, the other parts of calculating a cryptographic hash also prove to be very important.

20

M

Preprocessor π

M

S

Combining functions Δ

Iterative Structure IΔ

S Finalization Φ Final hash value h

Figure 2.4 Four Component Framework

21

2.4.1 Preprocessor The preprocessor is the first of four components to execute. In cryptographic hash algorithms like MD5 and SHA-1, the preprocessing is limited to a technique called MD-Strengthening, which pads the message and then appends the length of the message so that the processed message is a multiple of the block size used by the combining function, and prevents attacks such as fixed-point attacks which introduce additional data into the message. In the context of the overall framework, the preprocessing step operates only upon the message, returning a preprocessed message, but does not modify the internal state of the algorithm. We define the preprocessing step (π) as any portion of the algorithm that modifies the input message M without updating the internal state. ˆ π(M ) → M Like the security provided by the hashing step in an RSA digital signature, preprocessing can serve to disrupt homomorphisms or other exploitable algebraic structures in the algorithm’s combining function. Similarly, the preprocessor can “screen out” high-probability characteristics (see Chapter 4 for details) by inserting fixed or duplicated data. We discuss specific preprocessors in Chapter 3 and explain the security properties each provides.

2.4.2 Iterative structure The iterative structure along with its combining functions makes up the core of the cryptographic hash algorithm. We define the iterative structure (I∆ ) as a function which applies any number of combining functions from ∆ to a state S of length w and an input bit string M of a priori unknown length. In practice, most hash algorithms use only a single combining function and not a collection of functions for speed, memory, or complexity reasons. It stands to reason, though, that an

22 algorithm designer might choose to use multiple combining functions, as wee see later that several of the SHA-3 candidate designers did. In the event that the combining functions have a fixed-size domain, it is the responsibility of the iterative structure to act as a domain extension for these functions so that the algorithm may process an input of arbitrary length. When all message data is processed into the state, the “semi-final” state Sˆ is then returned for finalization. ˆ ) → Sˆ I∆ (M The Merkle-Damg˚ ard iterative structure is the best-known scheme, and it represents a linear iterative structure, since it applies its combining functions to sequential blocks of the message input. Tree-based iterative structures are also possible, where message blocks are placed at the leaves of the tree, and evaluated until the root of the tree is reached. These various iterative structures, along with their strengths and weaknesses, are described in detail in Chapter 4.

2.4.3 Combining Functions The combining functions in ∆ operate upon a subset of the message and a subset of the state to modify the state. We define the combining function as a function which takes a fixed-length message block and the current hash state as arguments and returns a modified hash state. (Note that unless we explicitly note otherwise, we assume that all inputs to a particular function are concatenated.) If some combining function δi operates on a message block of length d and the entire state (of length w), we can describe it as: δi : {0, 1}d × {0, 1}w → {0, 1}w In the MD iterative structure, the iterated function is called a compression function, a name which makes sense given that the normal input to this function is a reasonably small chaining value concatenated with a reasonably small message block,

23 and these functions simply compress the input to the length of the chaining value. We find this terminology limiting, since other schemes like the sponge framework [21] and RadioGat´ un [20] do not use their internal iterated functions the same way. Since both compression functions and a sponge’s iterated absorbing functions are responsible for combining the input with the chaining value or state, we decided that the term combining function was more apt. An MD-style compression function can be translated into a combining function by defining the subset of the message as the block mi and the subset of the state as the entire n bits of the state. (We use n bits here instead of w to illustrate that the typical “narrow-pipe” MD iterative structure has a state the same size as the final hash value.) Most cryptographers would conclude that the combining function is the basis of security in a cryptographic hash algorithm, and in many ways this is true, as it is usually the case that an attack on the combining function results in an attack on the entire algorithm. Combining functions are discussed alongside iterative structures in Chapter 4.

2.4.4 Finalization The finalization is responsible for transforming the algorithm state returned by the iterative structure into a final hash value of the desired length. When the state is the same length as the desired hash value (that is, w = n), as in MD5 [114], no finalization is necessary, but when the state is larger than the desired hash value, the state must be converted into a bit string of the appropriate size, typically by way of truncation. We define the finalization (Φ) as any operation which modifies the semi-final hash state Sˆ after the entire input stream has been processed by the iterative structure and produces a hash value of the appropriate size from any portion of the hash state

24 and input. ˆ →h Φ(S) The finalization function’s two chief responsibilities are to process Sˆ to create h in a similar way that a cryptographic hash algorithm processes M to create h, and second to produce a hash value of the desired length.

2.5

Application

We expand upon the assertion of [16] that one designs a hash algorithm in two steps: designing a compression function and specifying a domain extension transformation. We instead claim that one should design a cryptographic hash algorithm by choosing the four components that most closely fit the security and usability requirements of the application(s). This notion is explained in much more detail in Chapter 6 where many real hash algorithms are expressed using this model.

25

Chapter 3 Preprocessing

3.1

Introduction

Some recent attacks against common cryptographic hash algorithms SHA-1, MD5, and HAVAL (by Wang et al. in particular) have shown that it is possible to construct colliding messages for each of them in less time than the na¨ıve brute force method of simply hashing random blocks until two colliding inputs are found [137,138,129]. Several of these attacks dramatically reduce the search space for collisions by selecting messages with specific bitwise differences. For example, Suzuki and Kurosawa’s twoblock collision attack against 3-round HAVAL [129] can find a single collision in roughly 35 computations. The attack first requires that a “near collision” (a collision in some, but not all of the output bits) be found, using a particular input difference in the message block pairs mi ||mi+1 and m0i ||m0i+1 which can then be used to find fully colliding block pairs. This chapter focuses on preprocessing, the transformations of the input which operate strictly before the message is processed by the iterated combining functions. Preprocessing is a powerful way to prevent several known attacks against cryptographic hash algorithms and offers a method to “patch” vulnerabilities in hash algorithms after they are discovered, as in the case of the recent attacks. We show that preprocessing can not only reinforce resistance properties, but also reinstate some of these properties in broken algorithms. We can prevent Suzuki and Kurosawa’s attack by preprocessing the message in such a way that no message block can be constructed that produces the near collision required. In this example, mi ||mi+1 must differ from m0i ||m0i+1 in exactly one place

26 Content Positional Message Characteristic Fixed Parametric

Global example Padding Message Length Padding Prepend hash length n

Local example Block Position Bits So Far Whitening Per-block salt

Table 3.1 Examples of preprocessor content types in both localities

per message block: the most significant bit of the fifth word. To do this, we use “targeted whitening” as described in Section 3.4.5 and inject a word containing all zeros after the fourth word of every block, in a similar way that Szydlo and Yin suggested strengthening SHA-1 against a related attack [130]. This way, the near collision requirement would never be satisfied, regardless of the original input. The only drawback to this technique is that the speed of HAVAL decreases by about 3 percent.

3.2

Definitions

Within the four component framework, the preprocessor simply modifies the ˆ directly to the iterative structure. stream of input bits and passes its output M We define a preprocessor as the function π which takes a message M as an argument: ˆ π(M ) → M All the preprocessors described in this chapter can be categorized based on two qualities: locality and content. While there do not seem to be overarching security properties associated with these qualities themselves, the designations help in comparing similar preprocessors, particularly when the security of one is known better than the other.

3.2.1 Locality We define the locality of a preprocessor according to the size of its domain. Two distinct types were introduced in [130]: local preprocessors and global preprocessors.

27 Local preprocessors operate upon fixed-size blocks and can return their output without access to the entire message. Global preprocessors on the other hand operate on the entire message, and may require the entire message to produce any output whatsoever. An example of local preprocessing is message self-interleaving (see Section 3.4.6), which outputs two identical copies of each input byte, while MerkleDamg˚ ard Strengthening (see Section 3.4.2) is a global preprocessor since the entire message must be processed before its length is known. Note that the length of the blocks processed by a local preprocessor (b, and its output, b+) may not be the same as d, the length of the block processed by the compression function, as is described below in Section 3.2.3. Definition In a cryptographic hash algorithm, a local preprocessor is a function π λ : {0, 1}b → {0, 1}b+ which takes as an input a single b-bit block mi and returns a processed (b + )-bit block m ˆ i . may be any positive or negative integer or zero, depending on whether the preprocessor expands, contracts or permutes the message. This definition differs from that found in [130] in that we do not restrict the construction to only preprocessors which expand the input blocks. The local preprocessor may act as an expansion function, as a compression function itself and shorten the block, or return an output of the same length. The processed blocks are then passed to the iterated compression function in the place of the original message. Though by definition they are separate in the framework, local preprocessors producing blocks of length d (so b + = d) can be seen as extensions of the combining functions if such a distinction is necessary or desired. Conversely, portions of the combining function that operate on the message without updating the state, such as SHA-1-style message expansion, can be defined as local preprocessing.

28 Definition In a cryptographic hash algorithm, a global preprocessor is a function π γ : {0, 1}∗ → {0, 1}∗+ which takes as an input message M of indeterminate length L and returns a processed ˆ of length L ˆ = L + . may be any positive or negative integer or zero, message M depending on whether the preprocessor expands, contracts or permutes the message. The primary difference between local and global preprocessors is that a global preprocessor operates upon the entire message before it can return its processed result, while its local counterpart requires only a single input block. Applications of hash algorithms which have inputs of indeterminate length at the time of evaluation are said to have a streaming requirement. While the streaming requirement typically means that only local preprocessors are appropriate, it is possible that a global preprocessor only reads a block at a time until it produces its result. In particular, a global preprocessor of this sort that maintains counters or state variables of its own can be used with applications having the streaming requirement.

3.2.2 Content The content quality describes how the preprocessor produces its output. Since we expect that most preprocessors return parts of the message unmodified, we ignore those portions and instead focus on the added or modified parts. We divide content into four types: positional, message characteristic, fixed, and parametric. Positional preprocessors operate in tandem with the iterative structure and modify input blocks based on their position in the overall structure. Thus, a preprocessor which adds a field to each block that equals “1” when it is the last block and “0” otherwise would be positional. Positional preprocessors can prevent attacks such as length extension which disrupt the position of message blocks. Since the positional style can be combined with any of the other content styles, some preprocessors (like

29 the previous example, which is combined with the fixed quality) can be said to have multiple qualities. Preprocessors that use message characteristics are those that use some property of the input message or the message itself. An example of this is Merkle-Damg˚ ard Strengthening, which appends the length of the message to the message itself. Message characteristic preprocessors introduce redundancy and dependencies among the input bits. In the case of Merkle-Damg˚ ard Strengthening, it makes any attack that changes the length of the message affect the length field in the last block, which it turns out is sufficient to defeat several generic attacks. We mention here for clarification that we would not consider a preprocessor of this type which returns an ˆ ) as a message characteristic preprocessor. unmodified message (that is, M = M Fixed preprocessors inject data that is constant, or at least unrelated to the inputs or parameters of the algorithm. Whitening, for example, simply inserts a fixed data block at intervals into the message. This behavior is unchanged if the message itself or algorithm parameters change. Fixed preprocessors produce inputs where an attacker has no control over some subset of the message bits, which can often disrupt differential attacks and other attacks that require carte blanche access to the input bits. Parametric preprocessors modify the data in a way that is either specified by algorithm parameters or contains the parameters themselves. A straightforward parametric preprocessor is one that prepends the hash length n to the message. The serves to make invocations of the algorithm with different values of n produce different hash values, and may defend against attacks in the first block in a similar way as a fixed preprocessor. Like positional preprocessors, those using parameters can be combined with other qualities. For example, a parametric/message characteristic preprocessor might be a input byte repetition scheme (“message self-interleaving” - see Section 3.4.6) where the parameter defines the length of the byte blocks to be repeated.

30

M

M

Permutation

M

M Encoding

(with 2 impossible values)

M

M

Compression (with 2 colliding values)

Figure 3.1 Preprocessors as Domain Transformations

3.2.3 Preprocessing as a Domain Transformation Modifying a string by preprocessing can have three distinct effects on its length: increase it ( > 0), decrease it ( < 0), or leave it the same ( = 0). When ≥ 0, the preprocessor should be an injective function. That is, there should be no inputs which produce the same output. If there are, a many-to-one relationship like this can result in trivial collisions in the overall algorithm and should be avoided. When < 0, the preprocessor must be collision resistant itself and should have b ≥ n to avoid introducing collisions into the overall algorithm. Since finding a collision in a b

function with an output of b bits requires O(2 2 ) operations, algorithm designers using preprocessors with < 0 must be sure that the difficulty of finding a collision in the preprocessor is at least as difficult as finding them in the overall algorithm. One might argue that preprocessing the input block just shifts the attacks to other input choices. This is true for preprocessors that simply permute the input block to other values and can be inverted, but those that inject redundant data or compress the block invariably transform the domain of the compression function in ways that may preclude particular attacks.

31

3.3

Properties and Observations

3.3.1 Preprocessors and Property Preservation When describing the security of a cryptographic hash algorithm, the underlying components are often assumed to exhibit some ideal behavior so that the other parts of the algorithm can be shown to preserve the desirable properties of the ideal component. In most cases, the component assumed to be ideal is the combining function and the component to be analyzed is the iterative structure. A particular iterative structure is said to be X-preserving (or just X-Pr) if, when the combining functions possess the property X, an algorithm using this iterative structure also possesses the property X. In Chapter 4, this notion of “property preservation” is applied to the iterative structure as just mentioned, but we can also use it to describe the benefits of a particular preprocessor. The preservation of security properties like collision resistance is strongly dependent on the preprocessor function. It is usually claimed that the Merkle-Damg˚ ard structure is collision resistancepreserving (CR-Pr), though that claim is somewhat misleading. The “basic” MD structure simply iterates the compression function until the last message block, and without some preprocessing, it is simple to produce collisions in this basic structure. For example, if we assume that the final compression function operates on an incomplete block and fills in any incomplete parts with “0” bits, a message M and the message M 0 = M ||0 collide when hashed. If padding is added to the end of the ˆ that is unambiguous (usually padding with a “1” message M that produces an M bit, followed by as many “0” bytes as are necessary to fill the block), this is no longer a problem. We now have ˆ = M ||1||000...000 6= M ˆ 0 = M ||0||1||00...000 M and can expect that H(M ) 6= H(M 0 ). An even better preprocessor would include an encoding of the length of the message in the padding to prevent slightly more sophis-

32 ticated attacks. These brief counterexamples demonstrate that while the iterative structure is necessary to preserve the collision resistance of the underlying combining function, the preprocessing step is also necessary. In fact, many schemes offered as iterative structures or domain extensions for combining functions are little more than the “basic” MD structure with additional preprocessing. HAIFA [25], for example, supplements the MD structure with salt and “bits so far” fields in every block, which improves the algorithm’s resistance to second preimage attacks [28]. Preprocessors can also contribute to the preservation of other properties, as described in [51].

3.3.2 Preprocessing as Encoding and Differential Attacks Preprocessors play an important role in the resistance of cryptographic hash algorithms to differential attacks [130]. Existing differential attacks require the attacker to determine candidate preimages that satisfy certain conditions based on how the combining function mixes the input bits. (See Chapter 4 for more details on differential attacks.) Attackers typically seek inputs where changes in specific bits are canceled out later in the combining function by changes in other input bits. By introducing additional conditions through preprocessing, the difficulty of attacks can be increased proportional to the number of bits added to the input by the preprocessing. Control over the precise formats of inputs can be made more difficult by introducing constants and dependencies between input bits. A simple preprocessor is often sufficient to defeat a known attack and ensure f -resistance for a particular f . Theorem 3.3.1 An f -attack against a combining function δ which requires some fixed, finite set of conditions in any subset of bits in an input block can be prevented by a local preprocessor. Proof Suppose we have an f -attack called A on a combining function δ. Assuming that, in order to find a set of inputs which violate δ’s f -freedom, A requires that some subset (of e bits) of the d input bits be constant or a function of the remaining input bits d − e. To defeat A, we choose values for each of the e specific bits that will

33 explicitly violate A’s criteria. For example, if A specifies that the ith and j th bits of the input had to always be equal, a simple preprocessor (covered in more detail in Section 3.4.5) could inject constant bits into the input (and shift the remaining input bits) such that the ith bit is always zero and the j th is always one. Similar constructions can be formulated for any set of attack criteria.

3.3.3 Preprocessor versus Iterative Structure block size When b + 6= n, attackers seeking differential attacks may be able to find blocks where the bytes affected by the preprocessor are shifted into a more favorable position. For example, if the preprocessor adds a zero byte after every eighth byte, we have b = 64 and = 8. If d = 80, the fixed byte will be in the ninth position in the first block, in the eighth in the second, the seventh position in the third block, and so on. A worse situation occurs when b + > n, since there will be blocks which will have no injected bytes and therefore lose the benefit of the preprocessor against differential attack. This “staggering” of the preprocessed block boundaries with the combining function’s block boundaries may be beneficial in some cases, but in attacks where the attacker requires control over the structure of the blocks, this staggering may allow attackers to find appropriate block replacement points.

3.4

Schemes

In this section we describe some straightforward local and global preprocessing techniques. Note that most of these techniques insert redundant data into the message which can make differential cryptanalytic attacks more difficult. Due to the way the four component framework is laid out, preprocessing steps which set initial states or other algorithm constants by way of parameters (and not message bits) are not included. That type of operation is the responsibility of the component using the constants in question.

34 Scheme Padding Message Length Salt Bits So Far Whitening Message Self-Interleaving Message Duplication Block Position Block Permutation Parameters Local Tagging Global Tagging Checksums/EDC/ECC Compression

Locality Global Global Global/Local Local Local Local Global Local Local Global/Local Local Global Global/Local Global/Local

Content Fixed/Message/Positional Message Parametric Message Fixed Message Message Positional Message Parametric Message Message Message Message

Table 3.2 Breakdown of various preprocessing schemes

3.4.1 Padding Since the length of most messages are not conveniently a multiple of the length of a message block in an iterated hash scheme, messages must be padded so that the iterated scheme can be applied to complete message blocks. This padding method must be unambiguous - that is, it should never be the case that for two messages M and M 0 and padding function π where M 6= M 0 , π(M ) = π(M 0 ), or collisions can be found trivially, regardless of the strength of the iterated combining functions. One straightforward, unambiguous padding scheme is appending a “1” bit, then as many “0” bits as necessary to fill a block. A second unambiguous (though slower, and less frequently used) scheme is to pad the message with z zeroes to fill a block, leaving room for an r-length binary representation of z. Padding schemes need not be fixed preprocessors, though. For example, the padding used by the Blender algorithm [29] repeats input bits from the message as padding.

35

3.4.2 Message Length/Final Block Length Appending the message length to the message itself (after a suitable amount of padding as described above) is a well-established technique used by most modern cryptographic hash algorithms, and is often referred to as Merkle-Damg˚ ard Strengthening or MD-Strengthening [98,45]. MD-Strengthening defends against second preimage, trivial free-start collision (where the attacker can choose the previous state), and fixed-point second preimage and collision attacks. It also prevents collision attacks which find colliding messages of different lengths. Some might argue that, since this is a part of the MD5 and SHA-1 standards, it is not appropriate to refer to it as a preprocessor, but in algorithms without such a feature, it can be bolted on in a similar way as any other preprocessor. An alternate method, which requires very short messages or non-streaming access to the message is to prepend the message’s length to the message. This produces a prefix-free encoding of the message, compared to MD-Strengthening which produces a suffix-free encoding. Another alternate method is to append only the final block’s length to the end of the message, the benefit of which is that the algorithm needs not track the overall length of the message. To achieve the same level of security as MD-Strengthening, though, using the final block length in this way should only be used in conjunction with techniques that protect against block insertion and deletion like a “Blocks So Far” field as described below (which, of course, requires a counter to track the length of the message).

3.4.3 Salt Salts are random, short strings of bits that were originally used to increase the difficulty of dictionary attacks against stored passwords whose hashes were publicly accessible. Each bit of the salt increases the difficulty of an offline dictionary attack by a factor of two. Salts also defend against multicollisions [25] and herding attacks

36 [79] requiring precomputation of intermediate state values. Several algorithms in the SHA-3 competition include the salt as a parameter so that the application is not required to explicitly include the salt as part of the message. Depending on the implementation, the salt may also be used as a parameter, perhaps passed to the combining functions by the iterative structure to simulate a “family” of hash algorithms as required for certain security proofs.

3.4.4 Bits So Far/Blocks So Far Introduced as part of the HAIFA Framework [25], “Bits So Far” (bSF) involves inserting a fixed-size bit string of length containing the number of input bits that the algorithm has processed to that point into each block. “Blocks So Far” (BSF) is a variant which instead counts the number of message blocks, and can be viewed as a block index or counter, though its effect on the security of the algorithm is nearly the same as a bitwise counter. Length extension attacks such as those described in [49] and its generalization in [80] can be prevented by this sort of message preprocessing. It has also been shown in [28] that the bSF field improves the second preimage resistance of the Merkle-Damg˚ ard construction from 2n−τ to 2n , where τ is the number of message blocks (and intermediate state values) in the message. Assuming that differential attacks can be found in the combiner function, the bSF preprocessor can introduce additional delays to those attacks. The colliding blocks must first be identical in the bytes allocated to the counter, since replacing a single block does not modify the block or bit count. Therefore, single-block differential attacks may be possible if the message is long enough. As the number of possible block values increases, so does the number of the attacker’s possible insertion points. Bits so far remedies this to a certain extent since some bit count values may be impossible to produce without being at the last block. A field counting bits is usually more dynamic between blocks than one counting blocks since half the time, the block count

37 will change in only one bit. To illustrate this, if we assume that one of the 2 possible values in the BSF field are chosen uniformly at random when an attacker produces a candidate colliding input block, this block will only be useful in the message block mi where i is equal to the random value in the BSF field. If it is instead a bSF field, this block can only be used as the final block unless the value of the bSF field is divisible by d − . Thus, an attacker with a sufficiently long message M can much more easily find a collision while using a BSF field than a bSF field, which will tend to allow collisions only in the final block. It may even be a better idea to have the bits or blocks count be the seed for a pseudo-random number generator, since attackers may not know where their candidate colliding blocks would actually fit in the message, unless they are able to easily determine the seed from the generator.

3.4.5 Whitening Whitening is the process of adding blocks of fixed bytes at certain intervals or at certain positions within a message block and was introduced in [130]. This is a straightforward method to introduce roadblocks to attackers seeking differential attacks, but this defense is at the expense of the algorithm’s speed. For example, introducing one fixed byte after every message byte would effectively double the size of the message and require roughly twice as much time to evaluate. The argument can be made that whitening improves diffusion by spreading out the message data across more combining function iterations. Two basic types of whitening exist: targeted and periodic. Periodic whitening inserts the fixed block of data every so many bytes, until the end of the message is reached. Targeted whitening focuses on particular portions of the input block and inserts the fixed block at specific positions, usually with the intent of breaking up known differential attacks.

38

3.4.6 Message Self-Interleaving Also introduced in [130], message self-interleaving resembles whitening except that blocks of the message are duplicated and inserted (or self-interleaved) instead of fixedvalue blocks. The redundancy introduced by repeating blocks of the message serves to foil differential attacks against the combining function which rely on particular characteristics or differential trails. As with whitening, the drawback to this method is that the rate of the algorithm decreases as the number of added bits increases. Pair block chaining is a special case of message self-interleaving where each message block is concatenated with the following message block, as described in [14] and depicted below: πPBC (m0 ||m1 ||...||mk ) → m0 ||m1 ||m1 ||m2 ||...||mk The message blocks are paired up so that each message block will be operated upon by two combining function iterations. This scheme is also used for “incremental hashing” where it is necessary to have the ability to update the hash value when the message changes without requiring a complete re-hashing of the message.

3.4.7 Message Duplication This could be described as a special case of message self-interleaving, where the entire message is appended to itself (before or after padding) and the entirety is hashed. This preprocessor ensures that every message bit affects at least two different combining function invocations, which may defeat typical collision and preimage attacks.

3.4.8 Block Position This is an addition to a message block that indicates its position in an iterative or tree structure. In algorithms which use linear iterative structures like DCH [141] and Sg`ail [96], Block Position differs from Blocks So Far in that the positional indicator

39 bytes are only added to the final block or other blocks in key positions, which can produce a prefix-free encoding [39,40]. This technique defends against block insertions which shift the position of the marked block and as a prefix-free encoding, it can assist in the preservation of properties in the combining function and iterative structure.

3.4.9 Block Permutation Selective block permutation based on the block’s position can be considered to be a part of the preprocessing step when it operates independently from the compression function such as in Spectral Hash [120], which permutes only the first block. If the permutation is difficult to invert, attacks which rely on backtracking to a previous state may be more difficult. Aside from the operations required to perform the actual permutation, this preprocessor does not affect the number of combining function invocations.

3.4.10 Hash Length and other Parameters As previously mentioned, parameters are often used to determine IVs and other constants in the algorithm. According to our framework, however, that would put this use of parameters into the realm of the iterative structure. Injecting the block size, number of rounds, height of a hash tree, or the like into the message serves not only to defend against attacks requiring known states, but also differential attacks targeting the block containing the parameters.

3.4.11 Tagging Tagging, introduced in [78], inserts redundancy into the input stream in a slightly different way than the previous techniques and offers the option for both global and local preprocessing. The premise behind local tagging is that each block (or every xth block) will be passed through a (possibly different) hash function or MAC, the output of which is both prepended and appended to the message block. Global tagging is

40 similar in that it hashes the entire message first, then prepends and appends the resulting hash value to the original message at which point the processed message is then hashed again to produce the final hash. As a global preprocessor, tagging, like message duplication above, is only feasible if the algorithm has non-streaming access to the message, can read the message twice in its entirety, or has enough memory to cache the entire message.

3.4.12 Checksums and Error-Detecting Codes Checksums, error-detecting codes, and error-correcting codes are the integrityguaranteeing ancestors of modern cryptographic hash algorithms. Encoding schemes can be seen as a generalization of preprocessors that expand the message block. For example, message self-interleaving can be described as a repetition code, and Blender [29] appends checksums based on sums of all message words. The redundancy added by encoding a message can improve the resilience of the hash algorithm against differential attacks as they introduce dependencies between input bits and consequently increase the dependence of output bits on the input bits. We can apply one of many error-correcting codes into each message block to add a well-understood form of redundancy to the message. The benefit of this method over other techniques like message self-interleaving is that an error-correcting code requires many fewer bits. The drawback, of course, is that the code might not change if the number of bits a differential attack changes exceeds the number of bits the code was designed to correct or detect. This drawback can be mitigated by choosing codes with a high minimum distance that are efficient to calculate, as discussed in [76].

3.4.13 Lossless Compression While this scheme does not make sense as a local preprocessor, an algorithm designer might opt to run a compressor like bzip2 [124] over the entire input before passing it to the iterative structure. Provided that the compression algorithm can

41 process streaming data, this global preprocessor can be used on input which are only available a portion at a time. Besides potentially reducing the length of the message, compression algorithms generate unambiguous outputs. Additionally, compression may result in a prefix- or suffix-free encoding of the message.

3.5

Conclusion

We look to preprocessors to achieve certain goals: 1. They should not increase the probability of a collision in the algorithm 2. They should defeat any specific known or trivial attacks 3. In conjunction with the iterative structure, they should preserve desired security properties of the combining function 4. They should reduce the probability of differential attacks Combining function designers should work to find probable characteristics in their functions, if not to redesign and fix the flaws, then to explicitly preprocess away the vulnerabilities. The “patch” on the base algorithm that preprocessing offers may come at the cost of a reduction in the algorithm’s speed, since the preprocessor may inflate the input or have a nontrivial execution time. One unanswered question remains about how to maintain f -resistance with the least amount of preprocessing overhead. We posit without proof that a careful choice of preprocessors and a reasonably secure combining function can guarantee any fixed finite set of arithmetic freedom properties. An interesting research direction that would answer this would be to prove which choice of π can prevent specific f -attacks. Similarly, we might also conclude that if the combining function does not provide the desired security properties, preprocessing steps can be introduced that explicitly destroy any structure in the overall hash algorithm which might violate the security properties.

42

Chapter 4 Iterative Structure and Combining Functions

4.1

Introduction

Previous works discussing the practical construction of cryptographic hash algorithms have described the process as occurring in two steps: choosing a compression function, then selecting a mode of operation which allows the algorithm to accept inputs of arbitrary lengths and preserve the desirable properties of the compressor. These two components have been largely considered to be the whole of the algorithm, a perspective that this work seeks to shift by removing the preprocessing and finalization steps from the iterative structure (with which they are usually bundled). We instead define the iterative structure and its combining functions as the core of every cryptographic hash function, but not the entirety. Definition A combining function is a function δ which takes as inputs a dδ -bit message block and an wδ -bit state and outputs an updated wδ -bit state, or more formally: δ : {0, 1}dδ × {0, 1}wδ → {0, 1}wδ Where it is not explicitly noted, we maintain the order of the combining function arguments with the message input first, the state second. Though combining functions may be defined as operating on all w bits of the state (where wδ = w), it is often the case that the iterative structure only presents a subset of the state bits (so wδ < w) to the combining function for either performance or strictly architectural reasons. Note also that different combining functions may operate on different length message blocks, so we might have dδi 6= dδj with two combining functions δi 6≡ δj .

43 Since the combining functions are tasked with the majority of the “heavy lifting” in the algorithm, it stands to reason that the combining functions must possess the security properties requisite in a cryptographic hash algorithm. Each of these functions are therefore expected to possess at a minimum the security properties discussed in Chapter 1: collision resistance, preimage resistance, and second preimage resistance. Given that a combining function has no persistent state of its own and fixed-length inputs and outputs, these security properties need to be redefined to be appropriate to these functions. This is typically done by defining the combining function-specific properties pseudo-collision resistance and pseudo-preimage resistance which give attackers control not only over the message input, but also the state input. A pseudo-collision attack on δ is one that produces a wδ -bit state S and two dδ -bit message blocks mi and mj such that δ(mi , S) = δ(mj , S) in fewer than 2wδ /2 operations [97]. Similarly, a pseudo-preimage attack, given a state S, produces a state S 0 and a message block mi such that δ(mi , S 0 ) = S in fewer than 2wδ operations [97]. Note that in these examples, we assume that wδ ≤ dδ , though if this is not the case, the attack complexity is bounded instead by the message block size dδ . While these attacks do not always result in a direct attack against the overall algorithm, it has been proven that a pseudo-preimage attack on the combining function (that uses the entire w bits of the state) requiring fewer than 2n−2 operations can be translated into an effective attack on algorithms using the MD iterative structure [97]. Definition An iterative structure is a function which takes an arbitrary-length input and updates an w-bit internal state by calling one or more combining functions from the set δ0 , ..., δq ∈ ∆ as black boxes. I∆ : {0, 1}∗ → {0, 1}w In addition to calling combining functions to process the input, the iterative structure is also responsible for performing any necessary initialization and then maintain-

44 ing the internal state between combining function invocations. The purpose of the iterative structure is to use the combining functions in the most secure manner to update the state while maintaining performance goals. This component creates a variable input length (VIL) transformation from one or more fixed input length (FIL) transformations. Iterative structures can be differentiated primarily by the way they call the combining functions and secondarily in the size of their states and the way they choose or compute their constants. The size of the internal state is extremely important, since it provides an upper bound on the resistance of the algorithm against attack as discussed in Chapter 1. Also important is the number of state bits passed to each iteration of the combiner functions, because it may be possible to produce pseudo-collisions via brute force attack if the combiner’s inputs and outputs are too small. The algorithm’s constants are also an important factor in its security, as it was shown in [36] that ignoring the constants in an AES-based cryptographic hash algorithm resulted in collision attacks faster than the birthday attack.

4.2

Properties and Observations

4.2.1 Combining Functions The design of compression functions has been extensively studied [110,35], possibly more than any other aspect of cryptographic hash algorithms, largely because of their similarity to block ciphers, which are far better understood. It is not our intention to break new ground here in the design and analysis of combining functions. Instead, we discuss in a general sense the properties which make a combining function suitable for use in a secure cryptographic hash algorithm. For most of the rest of this paper, though, the combining functions are treated as black boxes in our four component framework. Clearly, a combining function plays a large role in establishing the resistances of a hash algorithm, as a weak combiner may expose the algorithm to single-block attacks for which the other components might not be able to compensate without undue

45

mi

d bits

Si-1

wδ bits

δ

δ

δ

δ

δ

Si

δ* Figure 4.1 A “supercombiner” transforming five iterations of δ into a single iteration of δ ∗

performance degradation. Most algorithms’ security is therefore defined in terms of the security of their combining functions and how the other components support this security, as discussed further in Section 4.5. When evaluating combining functions, it is helpful to expand their boundaries beyond the structures described in algorithm specifications. We investigated this briefly in Chapter 3 when we encountered error-correcting codes as preprocessors and their near-equivalence with combining functions’ message expansion steps. Transformations within combining functions are often executed iteratively a certain number of times, with each iteration called a round. We might reuse this notion of internal iteration by joining multiple combining functions into what we call a “supercombiner” as depicted in Figure 4.1. Each combiner within the supercombiner has the same access to the message input, but each in turn passes its modified internal state to the next. This provides a convenient structure to evaluate the supercombiner as a whole object rather than a few separate objects which individually might not have the desired properties. We can use this idea to show how structures like sponge-based hash algorithms can be massaged into a more familiar construction. Theorem 4.2.1 A sponge-style hash algorithm (as defined previously) can be instantiated as a wide-pipe Merkle-Damg˚ ard iterated algorithm with a finalizaton function. Proof In the sponge construction, each message block is combined with the state in

46 two distinct steps. The first step, which we will call “injection” and denote by δINJ : {0, 1}d × {0, 1}d → {0, 1}d where the first argument is the d message bits, and the second is the d-bit portion of the state with which the message bits will be combined. As defined in [21], this combining function applies the XOR operator to the two inputs to return the updated portion of the state. The second step is denoted by δPERM : {0, 1}w → {0, 1}w which applies a permutation to the state that has had the message block injected. Note that this combining function requires no input message bits. We then define the supercombiner δSPONGE to be the sequential application of first δINJ to a d-bit portion of the state, and then the application of δPERM to the entire state. If we then define I as the Merkle-Damg˚ ard construction, and ∆ = {δSPONGE }, I∆ → Sˆ is a representation of the iterative structure of a sponge function. Since sponge functions have a much more elaborate output function than MD (which in fact has no output processing in narrow-pipe constructions), we define a separate finalization function ˆ →h ΦSQUEEZE (S) This function wraps the entire “squeezing” phase of the sponge construction to abstract away the details of how the state is transformed into the output hash value.

4.2.1.1

Basic Properties

Combining functions have inherited several important basic requirements from block ciphers, including completeness, the Strict Avalanche Criterion (SAC), and balance, all of which are discussed thoroughly in Chapter 8 of [110]. A function is complete when every output bit depends on every input bit [77]. The SAC takes this a

47

S

δ

PERM

S

δ

PERM

lsb

S

S

lsb

lsb

h

h

ΦSQUEEZE

Figure 4.2 The squeezing phase of a sponge function as a single finalization function ΦSQUEEZE

step further and requires that, when a single input bit is complemented, every output bit should change with a probability that approaches one-half [140]. A function’s balance is measured by evaluating the Hamming weight of all possible outputs. A balanced function is one whose outputs are made up of equal numbers of ones and zeroes. In a balanced function f : {0, 1}n → {0, 1}n , we expect that HW(f ) = 2n−1 . Combining functions that do not produce a high level of diffusion (dependency of the output bits on the input bits) and therefore do not possess these three properties will often fall victim to differential attacks, as described in the next section.

4.2.1.2

Differential Properties

Differential attacks are attempts to determine collisions in specific combining functions based upon how the combiner produces output bits from its input bits. We first define a delta (not to be confused with δ as used above to describe a combining function) as the bitwise difference between two binary strings, usually by applying the exclusive-OR (XOR) operator. Differential attacks seek predictable correlations between deltas in input bits and deltas in output bits. These correlations are referred to as characteristics [23] and are expressed in terms of the probability that a desired output delta will be effected by an input delta. Combining functions which have characteristics of high probability are at risk of attack by differential attacks.

48 Since most combining functions operate as a few fast operations repeated for a certain number of rounds, differential attacks tend to be focused upon the changes that occur in the intermediate outputs between rounds. Inputs which tend to produce the same intermediate values after a certain number of rounds may be used as a starting point for finding inputs which result in collisions in the final hash value. In combining functions based on block ciphers or having the same sort of internal iterations, increasing the number of rounds often increases the difficulty of differential attacks, as more and more dependencies are introduced between the input and output bits. The task of finding optimal characteristics has been shown to be equivalent to finding minimal weight codewords in a linear code [94], which is known to be an NPComplete problem, so cryptanalysts often search for shortcuts like so-called neutral bits which do not affect the differences in the intermediate states for a certain number of initial rounds. By removing these bits from consideration in the characteristic search, the probability of finding a good characteristic is increased. Using this neutral bit technique, Biham and Chen produced a near-collision attack on SHA-0 in 240 operations [24]. De Canniere et al. [32] improved upon this and offered a technique for the automated search for differential paths.

4.2.1.3

Algebraic Structures and Homomorphisms

Homomorphic cryptographic hash algorithms are those algorithms which satisfy the following for some function f : H(f (M0 , M1 , ..., Mq )) = f (H(M0 ), H(M1 ), ..., H(Mq )) As previously mentioned, the provably collision resistant hash function VSH1 is susceptible to attack due to its homomorphic structure [119]. Suppose M0 , M1 , and 1 VSH is collision resistant if one assumes that finding nontrivial modular square roots of a very smooth number is a difficult problem

49 M2 are input messages of equal length, M0 ∧ M1 = M2 , and M2 is a string of all zeroes, then: H(M2 )H(M0 ∨ M1 ) ≡ H(M0 )H(M1 ) (mod n) Functions like this are clearly not f -resistant for a reasonably simple f . By mixing operations using different algebraic structures, such as modular arithmetic, exclusiveOR operations, and nonlinear AND and OR operations, the combining function may be shielded against all but the most complex attacks which exploit homomorphisms.

4.2.2 Iterative Structure The iterative structure defines the algorithm’s state and how the set of functions represented by ∆ operates upon processed message bits to modify the state. The long message attack (as outlined in Chapter 1) that produces second preimages is a good example of an attack that can be foiled by the proper choice of iterative structure. Since the attack relies on finding state collisions between combining function iterations, the iterative structure must contain a state that is sufficiently large so that brute force attacks against these intermediate states are of higher complexity than 2n so they cannot be effectively used to attack the algorithm as a whole. The ordering of the message block bits and the state bits is also part of the operation of the iterative structure. We might, for example, effectively create a new combining function from an existing one by rearranging the input bits or might provide domain separation by using either a different set of constants or a separate state in a combining function iteration. This is precisely the technique that the “Enveloped Merkle-Damg˚ ard” and “Enveloped Shoup Transform” use to create a different final combining function from the primary combiner. On the other hand, we might use the previous supercombiner construction to simplify the iterative structure’s internal function calls.

50

4.3

Iterative Structure Schemes

Many proposed iterative schemes require a certain preprocessor or finalization function, but when these schemes are stripped of these other components, the number of unique schemes is reduced dramatically. Proposed schemes can be divided roughly into linear and tree-based schemes and then further distinguished by the presence of other modifying features.

4.3.1 Linear Iterative Schemes Linear iterative schemes are the original and most common iterative structures, and involve applying combining functions to sequential blocks of the message (usually all of the same size). The typical linear scheme, the MD Construction, is well-known to preserve the collision resistance of its combining function, though it does not guarantee the preservation of preimage resistance or second preimage resistance [7]. Linear constructions which do preserve these properties are briefly discussed below and described in more detail in [16,7,6].

4.3.1.1

Narrow-pipe Schemes

We differentiate the basic linear schemes by the size of their states. Narrow-pipe constructions (like MD, depicted in Figure 4.3) have a state that is roughly equal to longest possible hash length. Schemes that maintain a large state where w ≥ 2n are usually called wide-pipe [90]. Designers of cryptographic hash algorithms have traditionally assumed that the length of the chaining value (what we describe here as the state) was the same length as the final hash value. This may have been due to the fact that block ciphers were used as compression functions and it was easier to prove properties on structures where w = n. These narrow-pipe hashes have some significant drawbacks because of the small size of their internal states. The length of the state, w, sets an upper bound on the security of the algorithm against attacks

51

m0

m1

d bits

S0

n bits

δ

mk-1

δ

S1

S2

Sk-1

δ

S*

Figure 4.3 A Narrow-pipe Linear Iterative Structure

such as multicollisions and second-preimages in long messages which require about w

w

min{2 2 , 2n(K−1)/K } and min{2 2 , K2n } operations, respectively.

4.3.1.2

Wide-pipe Schemes

To maintain the same basic Merkle-Damg˚ ard iterative structure while increasing the size of the state and subsequently the resistance against these attacks, wide-pipe and `-pipe constructions were developed. Wide-pipe constructions utilize a combining function (or supercombiner made of up of several combining functions) which operate upon the entire w-bit state, where w > n, and a final compression reduces the final state/chaining value to the requisite n bits. (Finalization functions such as this final compression step are discussed in detail in Chapter 5.) `-pipe constructions are a special case of the wide-pipe design which uses the original n-bit combining functions, but arrange them in ` parallel levels to accommodate a state where w = `n, as depicted in Figure 4.4. In the double-pipe (` = 2) example given in [90], the final message block and the two parallel chaining values are passed to a single combining function, and the final n-bit hash is returned. If the iterative structure organizes its state and combining functions as two parallel but separate “tracks” (unlike the `-pipe construction), the final “cascaded” hash value becomes vulnerable to multicollision attacks on the output as if there were two distinct hashes [75]. This illustrates the importance of not keeping these sub-states separate and instead ensuring that there

52

m0

m1

mk-1

d bits

S0

n bits

δ

S1

δ

δ

S' 1

δ

n bits

Sk-1 S' k-1

δ

S*

d bits n bits

S' 0

n bits

Figure 4.4 An `-pipe Linear Iterative Structure for ` = 2

is a fair amount of data crossing over between the different portions of the state. In iterative structures that have w = 2n, and are part of “double block length” algorithms [69], the double block combining function may be made up of a pair of block cipher-based combining functions each defined as the mapping δBC : {0, 1}d × {0, 1}n → {0, 1}n (where d = 2n) Several constructions fit this double block length model, including Tandem-DaviesMeyer, Abreast-Davies-Meyer [88], and MDC-2 [1]. It has been recently proven that the Tandem-Davies-Meyer construction (using AES as the underlying block cipher) provides a lower bound on the complexity of collision attacks at 2120.4 operations, which is reasonably close to the upper (birthday) bound of 2128 [56]. The approach in [56] (based upon [128]) to evaluating lower bounds on attacks against this sort of construction is an endorsement for reserving the cryptographic hash algorithm’s complexity in the combining function, rather than in the iterative structure, and also provides a strong endorsement for constructions of this type.

53

Si-1

δ

+

Si

+

S' i

Mi

δ

S' i-1

Figure 4.5 Tandem Davies-Meyer

4.3.1.3

Sponge Construction

Structures based on the sponge construction are also linear but use injection followed by permutation combiners with a large internal state instead of compression functions (see Theorem 4.2.1). The sponge construction is usually described as producing its output by “squeezing” as described in Chapter 2, but that falls into the discussion on finalization functions, where it is covered in more detail. These constructions can be very efficient given their large states, though they may be vulnerable to “slide attacks” [62] without other countermeasures in place.

4.3.2 Tree Iterative Schemes A simple “hash tree” was originally described by Merkle in [100] to capitalize on its parallelizable structure for speed while maintaining collision resistance preservation. The Merkle hash tree arranges message blocks at the leaves of a binary tree where they are hashed. The results are then concatenated and are passed as input to the parent nodes, as shown in Figure 4.6. Andreeva et al. [7] improved upon this and

54

m0

d bits

δ δ

m1

d bits

δ δ

m2

d bits

S*

δ δ

m3

d bits

δ Figure 4.6 A Merkle Hash Tree

offer a different (keyed) construction, the “modified XOR tree”, which preserves both collision resistance and (second) preimage resistance. Though these proposals use binary trees exclusively, there is no reason to restrict the discussion to only binary tree structures. Indeed, SHA-3 proposals MD6 [112] and SANDstorm [131] introduce 4-ary and 10-ary trees, respectively, which result in shallower and wider trees, and improve their parallelizability.

4.3.3 Modifying Features 4.3.3.1

Multiple Combiners

Most algorithms use just a single combining function, but some use more. In the Enveloped MD structure, for example, a different combining function (actually, the same combiner is used, but the inputs are shifted) is used for the final block to produce iterative structures which are pseudorandom oracle-preserving [16]. Other iterative structures might choose to use sequences of different and unique combining functions so that the two functions disrupt one another’s algebraic structures.

55

m0 +

δ

L

m1 +

key0

δ

+

S*

m2 +

δ

key1

key2

m3

Figure 4.7 A Modified XOR Hash Tree

4.3.3.2

Periodic Permutations

State permutations may improve mixing in algorithms that use combiners on fixed subsets of state bits. Sponge functions, for example, can be described as applying a combining function, then a periodic permutation between the combining function invocations. Note that the difference between this permutation and the block permutation preprocessor in Section 3.4.9 is that here the entire state is subject to permutation rather than just a portion of the message in the preprocessor.

4.3.4 Proposed Structures Given the above schemes, we now discuss some of the iterative structures discussed in [17] and [39] to see how they break down on this framework. (We do not analyze the actual algorithms until Chapter 6)

4.3.4.1

Strengthened Merkle-Damg˚ ard

This structure is simply a narrow-pipe linear construction with a Merkle-Damg˚ ardStrengthening preprocessor applied to the input.

56

m0

m1

mk-1

d-n bits

d bits

IV1

n bits

δ

S1

δ

Sk-1 S2 IV2

n bits

δ

S*

Figure 4.8 The Enveloped Merkle-Damg˚ ard Iterative Structure

4.3.4.2

Prefix-free Merkle-Damg˚ ard

This scheme uses the Merkle-Damg˚ ard iterative structure and applies a prefix-free encoding preprocessor to the message.

4.3.4.3

Enveloped Merkle-Damg˚ ard

As depicted in Figure 4.8, this is otherwise identical to Strengthened MD, but the final combining function is different from those iterated through all the previous message blocks [16].

4.3.4.4

Shoup’s Transform

This is also a linear construction, but a fixed number of keys (or a expanded and scheduled single key) are XORed with the state before every compression function iteration. (See Figure 4.9) If we cast this XOR operation as an easily-invertible permutation (given the keys), this scheme can be described as an otherwise typical Merkle-Damg˚ ard-style iterative struction with a parameter-driven periodic permutation before every combining function iteration. A variation is the “Enveloped Shoup Transform” which takes the same basic structure and adds a different final combiner, as described in Enveloped MD above. Andreeva et al. [5] produced a generic second preimage attack on Shoup-style constructions and other similar “dithered” hash function structures. A later proposal

57

m0

m1

δ

S0

+

κ0

+

mk-1

δ

+

κ1

δ

S*

κk-1

Figure 4.9 Shoup’s Transform

from Andreeva et al. [8], relying more heavily on keying and using message bits from multiple blocks in single combining function invocations, is immune to such attacks.

4.3.4.5

NMAC and HMAC

The NMAC construction, as described in [39], is simply the MD construction with a finalization function. The HMAC construction, discussed in the same work, adds a full block worth of zeroes to the message during preprocessing, then applies the MD iterative structure, then applies a finalization using the same combining function as in the iterative structure.

4.3.4.6

3C

In the 3C structure, for example, one half of its 2n state is equal to k M

δ(mi , Si )

i=0

as shown in Figure 4.10. Once the final message block is processed, this “checksum block” is processed as an additional message block. It has been shown, however, that adding XOR-based (or other linear) checksums in this way adds virtually no additional security [58]. Attacks similar to finding multicollisions can be launched on these constructions to determine colliding values for the entire algorithm in approximately

58

m0

m1

mk-1 Finalization

d bits

S0 S' 0

δ

n bits

n bits

δ

S1

⊕

S' 1

S2

⊕

δ

Sk-1

⊕

δ

h

S' k-1

Figure 4.10 The 3C Hash’s Iterative Structure

as many operations as a collision attack against the state without the additional checksum bits. We note that the GOST hash algorithm uses a similar structure as 3C, though since it replaces the intermediate chaining values with its message blocks, its checksum can be categorized as preprocessing, as described in Section 3.4.12:

4.4

Combining Function Schemes

These are by no means exhaustive lists. Of all of the four components, combining functions are the most difficult to define and enumerate exhaustively, since the majority of the algorithm’s complexity resides here. We expect combining functions to behave as FIL versions of the entire algorithm. With that in mind, combining functions tend to have their own set of parameters, structure, and styles. Combining functions might vary in the number of rounds they execute, the size of their input block (d), the strategy they employ to produce their output, or the specific internal components they use.

4.4.1 Execution Strategy Traditional cryptographic hash algorithms have reused block ciphers as compression functions, but that is just one of several execution strategies that algorithm designers have proposed. Here are several.

59

4.4.1.1

Block Cipher-Based Combiners

Block cipher-based combiners use their underlying cipher to encrypt the algorithm’s state with each message block. Preneel notes in [110] that combining functions based upon block ciphers may be vulnerabile to attacks afflicting the block cipher. Since block ciphers are bijections (with a fixed key) and any party with access to the message blocks (like an attacker seeking second preimages) can apply the decryption operation with the original key, it is fairly easy for an attacker to “backtrack” through the message from the hash value unless other countermeasures are in place. These vulnerabilities may be exploited to produce collisions, preimages, or f -attacks in less time than brute force. A detailed analysis of the most secure block cipher-based constructions to avoid attacks such as this can be found in [111].

4.4.1.2

Inject-Permute Combiners

Where block ciphers combine the message block and state in a single step, combiners in sponge functions operate in two distinct steps: injection, where the message is typically exclusively-ORed with some equal-sized portion of the state followed by permutation, where the newly-updated state is permuted to diffuse the inserted message data.

4.4.1.3

Concatenate-Permute-Truncate Combiners

This style of combiner found in both the Snefru [101] and Grindahl [84] algorithms closely resembles the sponge-style combiners. They differe in that they operate with three phases: concatenation, permutation, and truncation. The message block is first concatenated to the internal state, the result is permuted, and a fixed-size portion of the result of the permutation is returned as the next state.

60

Li

Ri

f

k i (m)

+ Li+1

Ri+1

Figure 4.11 A Balanced Feistel Network

4.4.1.4

Belt-And-Mill Combiners

These combiners separate the state into two substates - the belt and the mill [20]. After the message bits are added, the belt portion of the state undergoes a linear transformation, while the mill undergoes a nonlinear transformation. During each iteration of these transformations, bits from each of these substates are exchanged so that they are tightly coupled and not subject to the same sort of multicollision-style attacks previously mentioned.

4.4.2 Subcomponents Within each combiner, designers tend to use a handful of tried-and-true subcomponents, most of which were enumerated in Fleischmann et al’s evaluation of the SHA-3 candidates [55]. We briefly discuss each of them below.

4.4.2.1

Balanced and Unbalanced Feistel Network

Commonly used in block ciphers like DES, balanced Feistel networks [123] divide the state into two halves, each of which is successively transformed via a keyed non-

61 linear function f , then the two are combined and the halves are swapped, as depicted in Figure 4.11. A typical round might be described as Si → Li ||Ri (fki (m) (Ri ) ⊕ Li )||Li → Si+1 This continues for a certain number of successive rounds until the state is sufficiently combined with the message, which acts as the key when Feistel networks are used in hash algorithms. Unbalanced Feistel networks differ in that the input is divided into two parts of unequal length.

4.4.2.2

Message Expansion

Message expansion is a technique used in the compression functions of algorithms like SHA-1 and MD5 to improve the diffusion of the message bits across the internal rounds. It is desirable for this message expansion to be an encoding with a high minimum distance (as briefly mentioned in Chapter 3). Though finding the minimum distance of a linear code is NP-hard [133], codes suitable for use in message expansion with good performance and high minimum distances have been described in [76].

4.4.2.3

Maximum Distance Separable Matrix

Maximum Distance Separable (MDS) matrices [117] provide a linear substitution and are used to produce a high level of diffusion in the output bits. This technique was popularized in the Rijndael algorithm [44] which was chosen as the Advanced Encryption Standard, and applied to cryptographic hash algorithms in the Whirlpool algorithm [13].

4.4.2.4

S-boxes

The design and analysis of the Data Encryption Standard block cipher spurred many to evaluate DES’ method for introducing nonlinearity: substitution boxes. For performance considerations, they are usually implemented as lookup tables, where

62 the number of input bits exceeds the number of output bits. In this way, they can be seen as lightweight non-collision-resistant compression functions.

4.4.2.5

Feedback Shift Registers

The output of each iteration of a feedback shift register (FSR) is determined by a transformation of the input and the previous state. In an nonlinear FSR, this transform function is nonlinear, and in a linear FSR, the transform is linear [122,86].

4.4.2.6

Transforms With Provable Properties

By leveraging known-hard problems, several proposals for combining functions have included proofs of security (often for collision resistance) based on the assumption that the problems are in fact computationally infeasible to solve. Specific problems like the knapsack problem, discrete logarithms over elliptic curves, finding short vectors in lattices, finding isogenies between elliptic curves, and decoding random linear codes have been propsed in [99], [31], [9], [37], and [11], respectively.

4.5

Conclusion

In order to describe the security of a cryptographic hash algorithm’s iterative structure and combining functions, we must analyze the security of how the components interoperate. A common method for evaluating the overall security of a domain extension of combining functions is to describe its capability for property preservation [39,16,7,17]. Proofs of property preservation usually begin with the assumption that the underlying combining function possesses some ideal security property, say collision resistance, then show that the overall algorithm possesses the same security property when a particular iterative scheme is used. Let δ be an X-resistant combining function. If I is an X-resistance preserving iterative structure/domain extension that uses δ, then Iδ must be collision resistant. It follows then, that to design a cryptographic hash algorithm with a set of prop-

63 erties X0 , ..., Xq , we should first strive for a combining function that provides these properties (or nearly provides these properties, if it is possible to correct the behavior with preprocessing or finalization). Once we have this combining function possessing X0 , ..., Xq , we should then build an iterative structure, preprocessor, and finalization function that provably preserves (or reinstates) each of these properties. We have described previously how the choice of preprocessor can reinforce the security of the “core” of the iterative structure and combining functions. The first and arguably most important of the properties to preserve is collision resistance, though that property seems to be preserved only by choosing an appropriate preprocessor. While the MD structure with MD-Strengthening preserves the collision resistance of a compression function, it does not necessarily maintain its preimage resistance or balance [15]—structures such as the aforementioned `-pipe hash, on the other hand, do help to enforce the algorithm’s preimage resistance. The iterative structure itself has the capability to preserve at least one specific property of its combining functions: indifferentiability from a random oracle [95]. Random oracles are often used as idealized (and not possible to instantiate in reality) hash algorithms when described as variable-input-length random oracles (VILROs) and as idealized combining functions when described as a fixed-input-length random oracles (FIL-ROs). A random oracle is a theoretical construction which, used as a public function and given an input, produces an infinitely-long output of random bits that is truncated as necessary. Should the random oracle be provided with the same input as one it has previously operated upon, it will produce the same infinitely-long string of bits since its state is maintained across all queries. Though random oracles cannot be instantiated in the real world, we hope to be able to create combining functions which simulate all of their important behaviors. We say that such a function is indifferentiable from a random oracle, and call such functions “pseudorandom” oracles. We say that an iterative structure that preserves this prop-

64 erty is pseudorandom oracle-preserving (PRO-Pr). While finalization functions like truncation (see Section 5.3.1) can produce a PRO-Pr algorithm, the iterative structure can also be PRO-Pr by using a different final combiner, as seen in the EMD and Enveloped Shoup constructions. An important distinction to make is that though a combining function that is a pseudorandom oracle is also collision resistant, an iterative structure that is PRO-Pr is not necessarily collision resistance preserving [16].

65

Chapter 5 Finalization

5.1

Introduction

Finalization is the process of converting the state produced by the iterative structure into the final hash value h. It is the simplest component, as its security requirements are less stringent and it has known fixed-length inputs and outputs. In fact, most algorithms in use today (MD5 and SHA-1, for example) do not even include a finalization component. The length of the final hash value, n, is an extremely important parameter of cryptographic hash algorithms, and in a similar way as the size of the state, the upper bound on the security of any hash algorithm is based upon n. When n = w, finalization is not necessary, but if n < w, one or more of the techniques described in Section 5.3 are required. A finalization function operates only upon the semi-final state Sˆ and any parameters and produces the final hash value h: ˆ →h Φ(S) Any finalization function that requires part of the message is not by our definition a finalization and belongs in the iterative structure.

5.2

Properties and Observations

Intuitively, it seems that it would be beneficial for the final state to be as securely generated as the output of any other FIL compression function. The reality, though, is that the simplest construction, truncation, can produce an indifferentiable algorithm with a compression function simulating a random oracle and the typical MerkleDamg˚ ard iterative structure [51]. In the case where the underlying functions are not pseudorandom oracles, however, other more complicated finalization functions which

66 have better security properties (and which may be pseudorandom oracles themselves) may supplement the properties of the other three components. As described in [30], the goals of finalization functions can be split into introducing “discontinuity” into Sˆ and producing the output in the form of a hash value of the appropriate size. The advantages provided by having a state larger than the hash value, namely improved resistance to generic multicollision and preimage attacks, necessitate finalization to reduce the large state to the desired n-bit hash value h. Finalization does not positively affect the collision resistance of the algorithm if colliding inputs producing the same Sˆ can be found, since the finalization will produce the same h. Finalization can, however, mitigate the risk of near-collisions in the state. Even if it is possible to produce a collision in some of the state bits, applying a transformation resembling a compression function dramatically reduces the probability that the near-collision in Sˆ will result in a near (or full) collision in h. On the other hand, the security of some iterative structure/combining function combinations can be compromised by finalization such as truncation. [83] shows that, in certain double block length constructions, truncation can reduce the difficulty of collision and preimage attacks to 2m/3 and 2m/2 , respectively, where m represents the number of bits that ˆ are retained from S.

5.3

Finalization Schemes

5.3.1 Subset-of-State In those algorithms that produce a state which is larger than the desired hash value, a simple way of producing the hash is by returning some subset of the state bits unmodified. This finalization’s most straightforward variant is truncation (sometimes called the “chop construction”) which returns the most or least significant n bits of Sˆ as h. Other algorithms might select a more complicated subset of state bits such as choosing every xth byte of the state. In [40], the authors explain that finalization by truncation can defeat attacks on

67 MACs which are based on length extension by hiding a portion of the final hash. Returning only a portion of the state bits in this way illustrates the importance of having combining functions that also produce outputs which are collision resistant in the specific bits that are retained after truncation.

5.3.2 Final Permutation and Final Compression Applying a permutation to the state introduces discontinuity, which may be useful in disrupting algebraic attacks as described in Section 1.2.2. Since permutations of the state do not actually reduce its size, other finalization functions that produce the proper length output are required. Applying a final compression is one way to do that, and in addition to reusing combining functions from the iterative structure, other simpler options like XORing portions of Sˆ with itself may also be fast and effective. Final compressions, like truncation, can also produce property-preserving behavior. For example, the HMAC construction applies the internal combiner as a final compression, with the state as the message and the initial state S0 as the state to produce a pseudorandom oracle-preserving algorithm. Another benefit of a final compression is that it makes an attacker’s job of working backwards from h to determine a candidate Sˆ as difficult as inverting the compression function, which is a property useful in cryptographic hash algorithms being used as MACs, as mentioned in the previous section.

5.3.3 No-Input Iterations No-input iterations are a special case of a final permutation that applies combining functions without message input a certain number of times. Since it is a permutation, its sole purpose is to separate the hash value as much as possible from the semi-final state. Variants of these no-input iterations include “blank rounds”, which substitute all-zero blocks for the message inputs, “constant rounds” which, like blank rounds, use a fixed value other than zero, and “incremented rounds” which use a counter value

68 that increments after every block. The benefit of using iterations of this sort as a permutation is that a dedicated permutation function does not need to be developed and existing combining functions can be reused.

5.3.4 Pumping Rounds Pumping rounds (also known as “squeezing rounds” in sponge functions, as depicted in Figure 2.3) are otherwise identical to the above no-input iterations except some subset of the intermediate state is output as part of the hash value after each iteration. The algorithm typically calls only enough pumping rounds to produce a hash value of the desired length.

5.3.5 Pseudorandom Number Generators Pseudorandom number generators (PRNGs) may not possess security properties that would allow them to be used as hash functions themselves, but they may be effective in translating Sˆ into h. A PRNG uses Sˆ as a seed, then produces as many output bits as are desired, in a manner resembling pumping rounds.

5.4

Conclusion

Finalization can be seen as a variation of combining functions, since they operate on inputs and outputs of a fixed size. They do not, however, require all of the same properties as a combining function. Truncation is known to be sufficient for indifferentiability, and a final compression (provided that it is a pseudorandom oracle itself) is also sufficient for PRO-Pr. Looking at the ratio of bits in Sˆ compared to the number of bits in h leads to an interesting question. How much should the semi-final state be reduced to produce the hash value? It seems fairly clear that having w ≥ 2n is a good starting point, since generic multicollisions can be of ideal difficulty, and the chop construction’s requirements for having a “super-logarithmic” [51] number of the state bits is also satisfied. Whether that number is too high (or too low) remains

69 an open question. We might ask whether it is better to calculate a large internal state and truncate it or generate as much output as needed using pumping rounds. These produce outputs with the same maximum security, as a semi-final state of a particular size is still required for a secure h. We note that finalization techniques like pumping rounds can ˆ but any additional bits beyond the first w do produce an h which is larger than S, not factor into the security of the output. Put another way, the algorithm’s security is bound above by the smaller of w and n.

70

Chapter 6 Analysis and Conclusions

We have detailed some specific techniques that each component can use to contribute to the overall security of the algorithm. Table 6.1 shows some of the requirements to preserve certain properties in the combining functions. This approach to decomposing cryptographic hash algorithms into their component parts allows us to present high-level comparisons of very different algorithms and can allow us to quickly pick out those that do not have certain desirable features.

6.1

Negative Features

It is worthwhile to look at the other side of the coin and also examine in which circumstances a component could reduce the security of an algorithm to such a degree that the other components may not be able to compensate. A preprocessor can damage an algorithm by removing any bit from the input in the production of its output. The iterative structure can damage an algorithm by making any single bit of the input not affect the state or by insecurely calling combining functions. Combining functions can affect security negatively by not possessing the required security properties, or by making any output bit independent from any input bit. The finalization function can harm the algorithm by producing a hash value with bits that are not ˆ dependent on S.

6.2

Algorithm Analysis

We show now that all SHA-3 candidates and a handful of other well-known algorithms can be expressed in this generalized framework. Note the following about the tables:

71 Property to be Preserved Requirements Collision Resistance Preprocessor: MD-Strengthening or Bits So Far Preimage Resistance Finalization: Truncation Second Preimage Resistance Iterative Structure: w ≥ 2n Indifferentiability (PRO-Pr) Preprocessor: Prefix-free encoding or Iterative Structure: Different Final Combiner or Finalization: Truncation or Final Compression Table 6.1 Requirements for Property Preservation in the Four Component Framework

Constants and their derivations are not mentioned, even when parameters are used. We consider constants to be part of their respective components, but otherwise outside of the scope of these tables. An initial state S0 , for example, is derived as part of the iterative structure. The size of the states and specific output lengths are not specified, except in general terms. Only adjectives like “narrow pipe” and “wide pipe” are used to indicate the size of the state compared to the length of the final hash value. The hash value’s length is not indicated at all, except when explicitly part of the algorithm’s name (e.g. RIPEMD-160, which produces a 160-bit output). Note, though, that all SHA-3 candidates support output lengths of 224, 256, 384, and 512 bits. The exact divisions that are made here may not exactly match those specified by the algorithm designers. For example, functions described as “finalization” in several candidates (as a different final combiner) are included in the iterative structure component by virtue of their use of message bits and updating of the state. A true ˆ finalization similar to this would be “final compression”, which is only applied to S. Another potentially confusing distinction arises in SHABAL’s three final permutations that occur after the message is fully processed [30]. In this four component model, the permutations are categorized as part of the preprocessor since part of the input is used, then later passed to a typical iteration of the combining function. If

72 Abbreviation Pad MDS S BSF Wh MSI BPos Pe Pa ECC

Preprocessing Technique Padding Merkle-Damg˚ ard Strengthening (or equivalent) Salt Bits/Blocks So Far Whitening Message Self-Interleaving/Repeated Final Block Block Position Periodic Permutation Hash Length/Parameters Checksum/Error Correcting and Detecting Codes

Table 6.2 Legend for Preprocessor Abbreviations

the inputs to these functions were instead fixed or not based on the message, the finalizations would be named appropriately.

6.2.1 Preprocessing We begin with evaluating the preprocessing techniques in use by over 60 different cryptographic hash algorithms. Traditional algorithms like MD5 use only padding and MD-Strengthening, but many algorithms go beyond these two techniques. Table 6.2 provides an explanation to the abbreviations used in all of the following tables describing the algorithms’ preprocessors.

73 Algorithm MD5 [114] SHA-1 [103] SHA-224 [125] SHA-256 [125] SHA-384 [125] SHA-512 [125]

Pad X X X X X X

MDS X X X X X X

S

BSF

Wh

MSI

BPos

Pe

Pa

ECC

Pa

ECC

Table 6.3 Preprocessing Techniques in Common Algorithms Algorithm RIPEMD-128 [50] RIPEMD-160 [50] RIPEMD-256 [50] RIPEMD-320 [50] Whirlpool [13] Grindahl [84] MD4 [113] HAVAL [145] PANAMA [42] RadioGat´ un [20] Tiger [4]

Pad X X X X X X X X X X X

MDS X X X X X X X X

S

BSF

Wh

MSI

BPos

Pe

X

X

Table 6.4 Preprocessing Techniques in Other Hash Algorithms Algorithm Abacus [126] ARIRANG [34] AURORA [74] BLAKE [12] Blender [29] Blue Midnight Wish [60] Boole [116] Cheetah [81] CHI [68] CRUNCH [63] CubeHash [19] DCH [141] Dynamic SHA [143] Dynamic SHA2 [144] ECHO [18] ECOH [31] Edon-R [61] EnRUPT [107] ESSENCE [92] FSB [11] Fugue [64] Grøstl [59] Hamsi [108] JH [142] Keccak [22] Khichidi-1 [135] LANE [73] Lesamnta [70] Luffa [33] LUX [104] MCSSHA-3 [93]

Pad X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

MDS X X X X X X X X X X

S X

BSF

Wh

MSI

BPos

Pe

Pa X

X X

X X X X

X

X X X X

X

X

X

X X

X X X X X X X X X X X

ECC

X X

X

X X

X

X X

Table 6.5 Preprocessing Techniques in SHA-3 Candidates, Part 1

74

Algorithm MD6 [112] MeshHash [52] NaSHA [91] SANDstorm [131] Sarmal [134] Sg` ail [96] Shabal [30] SHAMATA [10] SHAvite-3 [26] SIMD [89] Skein [53] Spectral Hash [120] StreamHash [132] SWIFFTX [9] Tangle [2] TIB3 [109] Twister [54] Vortex [85] WaMM [139] Waterfall [67]

Pad X X X X X X X X X X X X X X X X X X X

MDS X X X X

X X X

S

BSF

X

X

X X X X X X X

X X X

MSI

BPos X

Pe

X

X X X X

Wh

X X X X

X X X

X X

X

X

X

X X

Pa X X

X

Table 6.6 Preprocessing Techniques in SHA-3 Candidates, Part 2

ECC

75

6.2.2 Iterative Structure The typical iterative structure in use is Merkle-Damg˚ ard, a linear, narrow-pipe construction. Exactly half of the proposed SHA-3 algorithms that are linear are narrow-pipe, with the other half wide-pipe. Only a small handful use the tree construction, multiple combiners, or periodic permutation. Moreso than any other component, the SHA-3 candidates are fairly homogeneous in their iterative structures.

76 Algorithm MD5[114] SHA-1[103] SHA-224[125] SHA-256[125] SHA-384[125] SHA-512[125]

Lin. X X X X X X

Tree

N. Pipe X X X X X X

W. Pipe

> 1 Comb.

Permutations

Table 6.7 Iterative Structure Techniques in Common Algorithms Algorithm RIPEMD-128[50] RIPEMD-160[50] RIPEMD-256[50] RIPEMD-320[50] Whirlpool[13] Grindahl[84] MD4[113] HAVAL[145] PANAMA[42] RadioGat´ un[20] Tiger[4]

Lin. X X X X X X X X X X X

Tree

N. Pipe X X X X X

W. Pipe

> 1 Comb.

Permutations

X X X X X X

Table 6.8 Iterative Structure Techniques in Other Algorithms Algorithm Abacus[126] ARIRANG[34] AURORA[74] BLAKE[12] Blender[29] Blue Midnight Wish[60] Boole[116] Cheetah[81] CHI[68] CRUNCH[63] CubeHash[19] DCH[141] Dynamic SHA[143] Dynamic SHA2[144] ECHO[18] ECOH[31] Edon-R[61] EnRUPT[107] ESSENCE[92] FSB[11] Fugue[64] Grøstl[59] Hamsi[108] JH[142] Keccak[22] Khichidi-1[135] LANE[73] Lesamnta[70] Luffa[33] LUX[104] MCSSHA-3[93]

Lin. X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Tree

N. Pipe

W. Pipe X

> 1 Comb.

Permutations

X X

X X

X X X X X

X

X X X X X X X X X X

X X X X X X X X X X

X X X

X

Table 6.9 Iterative Structure Techniques in SHA-3 Candidates, Part 1

77

Algorithm MD6[112] MeshHash[52] NaSHA[91] SANDstorm[131] Sarmal[134] Sg` ail[96] Shabal[30] SHAMATA[10] SHAvite-3[26] SIMD[89] Skein[53] Spectral Hash[120] StreamHash[132] SWIFFTX[9] Tangle[2] TIB3[109] Twister[54] Vortex[85] WaMM[139] Waterfall[67]

Lin.

Tree X

X X

N. Pipe

> 1 Comb.

Permutations

X X

X X X X X X X X X X X X X X X X

W. Pipe X X X

X X X X X X X X X X X X

X X

X X

X X

Table 6.10 Iterative Structure Techniques in SHA-3 Candidates, Part 2

78 Abbreviation BC IP CPT BFN UFN ME MDS SB FSR Pr

Combining Function Technique Block cipher-based Inject-Permute (sponge style) Concatenate-Permute-Truncate Balanced Feistel network Unbalanced Feistel network Message expansion Maximum distance separable matrix Substitution box (Non)Linear feedback shift register Combining function with provable properties

Table 6.11 Legend for Combining Function Abbreviations

6.2.3 Combining Function Table 6.11 provides an explanation to the abbreviations used in all of the following tables describing the algorithms’ combining functions. The majority of the content in these tables (in fact, columns 4 through 9) were originally found in [55]. The first three “execution style” columns and the last “provable” column are novel. The vast majority of proposed algorithms are block-cipher-based, and most also use s-boxes and message expansion. MDS matrices are also popular, with 17 algorithms using structures resembling (or directly using) AES’ matrices.

79 Algorithm MD5[114] SHA-1[103] SHA-224[125] SHA-256[125] SHA-384[125] SHA-512[125]

BC X X X X X X

IP

CPT

BFN

UFN X X X X X X

ME X X X X X X

MDS

SB

FSR

Pr

FSR

Pr

FSR

Pr

Table 6.12 Combining Function Techniques in Common Algorithms Algorithm RIPEMD-128[50] RIPEMD-160[50] RIPEMD-256[50] RIPEMD-320[50] Whirlpool[13] Grindahl[84] MD4[113] HAVAL[145] PANAMA[42] RadioGat´ un[20] Tiger[4]

BC X X X X X

IP

CPT

BFN

UFN X X X X

ME X X X X X

X X X

X X

X

X

X

MDS

SB

X X

X

X X X

X

Table 6.13 Combining Function Techniques in Other Algorithms Algorithm Abacus[126] ARIRANG[34] AURORA[74] BLAKE[12] Blender[29] Blue Midnight Wish[60] Boole[116] Cheetah[81] CHI[68] CRUNCH[63] CubeHash[19] DCH[141] Dynamic SHA[143] Dynamic SHA2[144] ECHO[18] ECOH[31] Edon-R[61] EnRUPT[107] ESSENCE[92] FSB[11] Fugue[64] Grøstl[59] Hamsi[108] JH[142] Keccak[22] Khichidi-1[135] LANE[73] Lesamnta[70] Luffa[33] LUX[104] MCSSHA-3[93]

BC

IP X

CPT

BFN

X X X X X

UFN X X X

ME X X X X

MDS X X

SB

X

X X

X X X X X X X X X X X X X

X X

X X X

X X

X X X

X

X X X X

X X X

X X

X

X X X

X X

X X

X X X

X

X X X

X

X

X X X

X X

X X

X

X X X X

X X X X X

Table 6.14 Combining Function Techniques in SHA-3 Candidates, Part 1

80

Algorithm MD6[112] MeshHash[52] NaSHA[91] SANDstorm[131] Sarmal[134] Sg` ail[96] Shabal[30] SHAMATA[10] SHAvite-3[26] SIMD[89] Skein[53] Spectral Hash[120] StreamHash[132] SWIFFTX[9] Tangle[2] TIB3[109] Twister[54] Vortex[85] WaMM[139] Waterfall[67]

BC X

IP

CPT

BFN

UFN

ME

MDS

SB

X X

X X X X X

X X X X X X X X X X X X X X X X X X

X X X

X X X

X

X X X X X X

Pr

X

X X X X

X X X X

X

FSR X

X X

X X X X X X X X X

Table 6.15 Combining Function Techniques in SHA-3 Candidates, Part 2

X

81 Abbreviation Trunc SoS FP FC NII PR

Finalization Function Technique Truncation Subset of state (not truncation) Final permutation Final compression No-input iteration Pumping rounds

Table 6.16 Legend for Finalization Function Abbreviations

6.2.4 Finalization Table 6.16 provides an explanation to the abbreviations used in all of the following tables describing the algorithms’ finalization functions. Finalization is not usually necessary for traditional cryptographic hash algorithms, since they all operate in the narrow-pipe model and their internal state was returned unmodified as the final hash value. The most common finalization of the SHA-3 candidates is truncation, with 38 algorithms using it in some capacity.

82 Algorithm MD5[114] SHA-1[103] SHA-224[125] SHA-256[125] SHA-384[125] SHA-512[125]

Trunc

SoS

FP

FC

NII

PR

X X

Table 6.17 Finalization Techniques in Common Algorithms Algorithm RIPEMD-128[50] RIPEMD-160[50] RIPEMD-256[50] RIPEMD-320[50] Whirlpool[13] Grindahl[84] MD4[113] HAVAL[145] PANAMA[42] RadioGat´ un[20] Tiger[4]

Trunc

SoS

X X

X

FP

FC

NII

PR

X X X X

X X

X

Table 6.18 Finalization Techniques in Other Algorithms Algorithm Abacus[126] ARIRANG[34] AURORA[74] BLAKE[12] Blender[29] Blue Midnight Wish[60] Boole[116] Cheetah[81] CHI[68] CRUNCH[63] CubeHash[19] DCH[141] Dynamic SHA[143] Dynamic SHA2[144] ECHO[18] ECOH[31] Edon-R[61] EnRUPT[107] ESSENCE[92] FSB[11] Fugue[64] Grøstl[59] Hamsi[108] JH[142] Keccak[22] Khichidi-1[135] LANE[73] Lesamnta[70] Luffa[33] LUX[104] MCSSHA-3[93]

Trunc X X X

SoS

FP

FC

NII X

PR X

X

X

X

X X X X X X X X X X X X X X X X X X

X

X X X X

X X X X

X X

X X

X

Table 6.19 Finalization Techniques in SHA-3 Candidates, Part 1

83

Algorithm MD6[112] MeshHash[52] NaSHA[91] SANDstorm[131] Sarmal[134] Sg` ail[96] Shabal[30] SHAMATA[10] SHAvite-3[26] SIMD[89] Skein[53] Spectral Hash[120] StreamHash[132] SWIFFTX[9] Tangle[2] TIB3[109] Twister[54] Vortex[85] WaMM[139] Waterfall[67]

Trunc X

SoS

FP

FC

NII

PR X

X X X X X X X X X X X X X X X

X

X X

X

Table 6.20 Finalization Techniques in SHA-3 Candidates, Part 2

84

6.3

Algorithm Analysis: SANDstorm

The SANDStorm algorithm [131] is a submission to the SHA-3 competition by Sandia National Labs, and serves here as a good example of a candidate which, while complex and very different from traditional algorithms, still fits in well within this four component framework. While we describe the components of SANDstorm in more detail than the other SHA-3 candidates, we do not discuss all of the specifics of the algorithm’s operation, but instead opt to explain its components generally.

6.3.1 Preprocessing SANDstorm uses the usual MD-Strengthening of padding followed by appending the message length. It also includes a counter on blocks that increments from 1 to 4 which, while it is described in the algorithm specification as being part of the chaining value, is classified as a very short “Blocks So Far” field in this framework.

6.3.2 Iterative Structure The structure is a 10-ary tree, whose depth is expanded to a maximum of 4, which only occurs when the message is longer than 1000 blocks (of 512 or 1024 bits, depending on the desired output length). This structure not only improves parallelizability, but also offers the ability to do fast incremental hashing [14] should the hashes of large inputs which differ in only a few blocks be required. The drawback to this structure, like other tree structures, is the additional storage overhead required to track the intermediate states in parent nodes.

6.3.3 Combining Function The combining function used in SANDstorm is a fairly straightforward block cipher-based function that uses message expansion and s-boxes.

85

6.3.4 Finalization At the root of the tree, the accumulated state bits have a final compression applied to them to generate a hash value of the desired size.

6.4

Recommendations

We envision two purposes for the classification: as a suggestion for an algorithm construction technique and to focus cryptanalyses on components rather than on monolithic algorithms, particularly when so many share structural components. As they seek the best candidate in the SHA-3 competition, our recommendation to NIST is that they choose the best combining function(s) from all of the existing candidates, then select the other three components a la carte. This will allow a candidate with a strong combining function but weak components to lend its strength to an overall stronger algorithm with an iteration style, preprocessor, or finalization quite different from its own proposal.

6.4.1 Recommended Construction We suggest the following very loose sequence for designing iterated cryptographic hash algorithms: 1. Determine the security and usability properties desired based on the applications’ requirements. 2. Determine the hash length(s) and length of the internal state based on the desired properties of the algorithm. 3. Choose or design an iterative structure which supports the chosen state size and preserves the relevant security properties 4. Choose or design combining functions which provide the relevant security properties and can operate on the portions of the state determined by the selection of the iterative structure.

86 5. Choose or design preprocessing steps (that interoperate with the iterative structure, if necessary) that preserve the relevant security properties. 6. Choose or design finalization steps which operate upon the semi-final output, provide the required hash length, and preserve the relevant security properties.

6.5

Conclusion

This paper describes a modular model for cryptographic hash algorithms which allows the algorithm to be evaluated for security properties by comparing their individual components: preprocessor, iterative structure, combining function, and finalization. The structure allows designers to design the algorithm with “defense in depth” in mind, so that individual components found to be weak do not necessarily compromise the entire algorithm. It also allows for drop-in replacement of any piece in a way that should not severely impact applications already using the algorithm. Cryptanalysis also benefits from the division into four components since reducedstrength constructions become much more straightforward for any algorithm. Theorem 6.5.1 According to the definitions in Chapter 1, the assertion of [97] that: ...a compression function secure against preimage, 2nd-preimage, and collision (fixed IV) attacks is necessary and sometimes, but not always sufficient for a secure iterated hash. is false. Proof We show that we can construct a secure iterated hash which uses a compression function that is insecure against collision attacks. Suppose a compression function that takes an input of d bits insecurely combines the first two bits of the input such that when the two bits are equal, they cancel each other out and do not affect the output. The function otherwise behaves as a pseudorandom oracle. Collisions become trivial, since any inputs 1 1 || {0, 1}d−2 and 0 0 || {0, 1}d−2 where the final d − 2 bits are the same produce collisions when the combining function is applied. By introducing a preprocessor which inserts bits such that the first bit of every

87 message block is a zero, the second is a one, and the remaining bits are unchanged, the overall algorithm can be collision resistant while the combining function is not.

6.5.1 Future Directions This framework does not account for specific interdependencies between components, and determining a generalized way for components to securely interconnect is an interesting future research topic. Simple interdependencies like the block length or input size are easier to decouple and fit within the framework, but the RadioGat´ un [20] hash algorithm, for example, tightly couples its iterative structure to its combining functions which can complicate analysis. The fact that the components are distinct and logically separate in this paper should not be understood to mean that we believe that all hash algorithm designs must follow this pattern. We do believe, though, that this divide-and-conquer approach will allow cryptographers to construct strong cryptographic hash algorithms from vetted and well-known building blocks. When a component is found to be weak or unsuitable in a particular application, it can be replaced by one which is known to be strong. That is also not to say that there are any guarantees that creating a new hash algorithm from known strong components will yield a strong overall algorithm, but best practices should be assembled and researched to determine the best methods for constructing algorithms piecewise. More detailed research into proofs for the specific security properties provided by each choice of component is another ongoing future research direction.

88

References [1] Accredited Standards Committee X9. Working Draft: American National Standard X9.31-1992: Public Key Cryptography Using Reversible Algorithms for the Financial Services Industry: Part 2: The MDC-2 Hash Algorithm, June 4, 1993. [2] Rafael Alvarez, Gary McGuire, and Antonio Zamora. The Tangle Hash Function. Submitted to NIST, 2008. [3] Ross Anderson. The classification of hash functions. In IMA Conference in Cryptography and Coding, 1993. [4] Ross J. Anderson and Eli Biham. TIGER: A fast new hash function. In Dieter Gollmann, editor, FSE, volume 1039 of Lecture Notes in Computer Science, pages 89–97. Springer, 1996. [5] Elena Andreeva, Charles Bouillaguet, Pierre-Alain Fouque, Jonathan J. Hoch, John Kelsey, Adi Shamir, and S´ebastien Zimmer. Second preimage attacks on dithered hash functions. In Nigel P. Smart, editor, EUROCRYPT, volume 4965 of Lecture Notes in Computer Science, pages 270–288. Springer, 2008. [6] Elena Andreeva, Gregory Neven, Bart Preneel, and Thomas Shrimpton. Sevenproperty-preserving iterated hashing: ROX. In ASIACRYPT, pages 130–146, 2007. [7] Elena Andreeva, Gregory Neven, Bart Preneel, and Thomas Shrimpton. Threeproperty-preserving iterations of keyless compression functions. In ECRYPT Workshop on Hash Functions, 2007. [8] Elena Andreeva and Bart Preneel. A new three-property-secure hash function. In Selected Areas in Cryptography, 2008. [9] Yuriy Arbitman, Gil Dogon, Vadim Lyubashevsky, Daniele Micciancio, Chris Peikert, and Alon Rosen. SWIFFTX: A Proposal for the SHA-3 Standard. Submitted to NIST, 2008. [10] Adem Atalay, Orhun Kara, Ferhat Karakoc, and Cevat Manap. SHAMATA HASH FUNCTION ALGORITHM SPECIFICATIONS. Submitted to NIST, 2008. [11] Daniel Augot, Matthieu Finiasz, Philippe Gaborit, St´ephane Manuel, and Nicolas Sendrier. SHA-3 proposal: FSB. Submitted to NIST, 2008. [12] Jean-Phillipe Aumasson, Luca Henzen, Willi Meier, and Raphael C.-W. Phan. SHA-3 proposal BLAKE. Submitted to NIST, 2008.

89 [13] Paulo S. L. M. Barreto and Vincent Rijmen. The WHIRLPOOL hash function. World-Wide Web document, 2001. [14] Mihir Bellare, Oded Goldreich, and Shafi Goldwasser. Incremental cryptography with application to virus protection. In Proc. 27th ACM Symp. on Theory of Computing, pages 45–56, Las Vegas, 1995. ACM. [15] Mihir Bellare and Tadayoshi Kohno. Hash function balance and its impact on birthday attacks. In Christian Cachin and Jan Camenisch, editors, EUROCRYPT, volume 3027 of Lecture Notes in Computer Science, pages 401–418. Springer, 2004. [16] Mihir Bellare and Thomas Ristenpart. Multi-property-preserving hash domain extension and the EMD transform. In Lai and Chen [87], pages 299–314. [17] Mihir Bellare and Thomas Ristenpart. Hash functions in the dedicated-key setting: Design choices and MPP transforms. Cryptology ePrint Archive, Report 2007/271, 2007. http://eprint.iacr.org/. [18] Ryad Benadjila, Olivier Billet, Henri Gilbert, Gilles Macario-Rat, Thomas Peyrin, Matt Robshaw, and Yannick Seurin. SHA-3 Proposal: ECHO. Submitted to NIST, 2008. [19] Dan Bernstein. CubeHash Specification (2.B.1). Submitted to NIST, 2008. [20] Guido Bertoni, Joan Daemen, Gilles Van Assche, and Micha¨el Peeters. Radiogat´ un, a belt-and-mill hash function. NIST - Second Cryptographic Hash Workshop, August 24-25, 2006. [21] Guido Bertoni, Joan Daemen, Micha¨el Peeters, and Gilles Van Assche. Sponge functions. In ECRYPT Workshop on Hash Functions, 2007. [22] Guido Bertoni, Joan Daemen, Micha¨el Peeters, and Gilles Van Assche. Keccak specifications. Submitted to NIST, 2008. [23] E. Biham and A. Shamir. Differential cryptanalysis of Snefru, Khafre, REDOCII, LOKI and Lucifer. In J. Feigenbaum, editor, CRYPTO ’91: Proceedings of the 11th Annual International Cryptology Conference on Advances in Cryptology, pages 156–171. Springer, 1992. Lecture Notes in Computer Science No. 576. [24] Eli Biham and Rafi Chen. Near-collisions of sha-0. In Franklin [57], pages 290–305. [25] Eli Biham and Orr Dunkelman. A framework for iterative hash functions: HAIFA. In NIST’s Second Cryptographic Hash Workshop, 2006. [26] Eli Biham and Orr Dunkelman. The SHAvite-3 Hash Function. Submitted to NIST, 2008.

90 [27] Dan Boneh and Xavier Boyen. On the impossibility of efficiently combining collision resistant hash functions. In CRYPTO, pages 570–583, 2006. [28] Charles Bouillaguet, Orr Dunkelman, Pierre-Alain Fouque, and Antoine Joux. On the security of iterated hashing based on forgery-resistant compression functions. Cryptology ePrint Archive, Report 2009/077, 2009. http://eprint. iacr.org/. [29] Colin Bradbury. BLENDER: A Proposed New Family of Cryptographic Hash Algorithms. Submitted to NIST, 2008. [30] Emmanuel Bresson, Anne Canteaut, Benoˆıt Chevallier-Mames, Christophe Clavier, Thomas Fuhr, Aline Gouget, Thomas Icart, Jean-Fran¸cois Misarsky, Mar`ıa Naya-Plasencia, Pascal Paillier, Thomas Pornin, Jean Ren´e Reinhard, C´eline Thuillet, and Marion Videau. Shabal, a Submission to NIST’s Cryptographic Hash Algorithm Competition. Submitted to NIST, 2008. [31] Daniel R. L. Brown, Adrian Antipa, Matt Campagna, and Rene Struik. ECOH: the Elliptic Curve Only Hash. Submitted to NIST, 2008. [32] Christophe De Canni`ere and Christian Rechberger. Finding sha-1 characteristics: General results and applications. In Lai and Chen [87], pages 1–20. [33] Christophe De Canniere, Hisayoshi Sato, and Dai Watanabe. Hash Function Luffa: Specification. Submitted to NIST, 2008. [34] Donghoon Chang, Seokhie Hong, Changheon Kang, Jinkeon Kang, Jongsung Kim, Changhoon Lee, Jesang Lee, Jongtae Lee, Sangjin Lee, Yuseop Lee, Jongin Lim, and Jaechul Sung. ARIRANG. Submitted to NIST, 2008. [35] Donghoon Chang, Mridul Nandi, Jesang Lee, Jaechul Sung, and Seokhie Hong. Compression function design principles supporting variable output lengths from a single small function. Cryptology ePrint Archive, Report 2007/416, 2007. http://eprint.iacr.org/. [36] Donghoon Chang and Moti Yung. Do we need to vary the constants? (methodological investigation of block-cipher based hash functions). Cryptology ePrint Archive, Report 2006/467, 2006. http://eprint.iacr.org/. [37] Denis Charles, Eyal Goren, and Kristen Lauter. Cryptographic hash functions from expander graphs. Cryptology ePrint Archive, Report 2006/021, 2006. http://eprint.iacr.org/. [38] Scott Contini, Arjen K. Lenstra, and Ron Steinfeld. VSH, an efficient and provable collision-resistant hash function. In Serge Vaudenay, editor, EUROCRYPT, volume 4004 of Lecture Notes in Computer Science, pages 165–182. Springer, 2006.

91 [39] Jean-S´ebastien Coron, Yevgeniy Dodis, C´ecile Malinaud, and Prashant Puniya. Merkle-Damg˚ ard revisited: How to construct a hash function. In Victor Shoup, editor, CRYPTO, volume 3621 of Lecture Notes in Computer Science, pages 430–448. Springer, 2005. [40] Jean-S´ebastien Coron, Yevgeniy Dodis, C´ecile Malinaud, and Prashant Puniya. A new design criteria for hash-functions, 2005. [41] Joan Daemen. Cipher and Hash Function Design, Strategies based on linear and differential cryptanalysis. PhD thesis, Katholieke Universiteit Leuven, 1995. [42] Joan Daemen. Fast hashing and stream encryption with PANAMA. In Fast Software Encryption, LNCS 1372, pages 60–74, 1998. [43] Joan Daemen and Vincent Rijmen. The wide trail design strategy. In Bahram Honary, editor, IMA Int. Conf., volume 2260 of Lecture Notes in Computer Science, pages 222–238. Springer, 2001. [44] Joan Daemen and Vincent Rijmen. The Design of Rijndael. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2002. [45] I. Damg˚ ard and Ivan Bjerre. A design principle for hash functions. In G. Brassard, editor, Advances in Cryptology–Crypto’89. LNCS, vol.435, pages 416–427. Springer, 1990. Lecture Notes in Computer Science No. 435. [46] George I. Davida. Chosen signature cryptanalysis of the RSA public key cryptosystem. Technical Report TR-CS-82-2, Dept of EECS, University of Wisconsin, Milwaukee, 1982. [47] George I. Davida and Jeremy A. Hansen. A four-component framework for designing and analyzing cryptographic hash algorithms. Cryptology ePrint Archive, Report 2007/240, 2007. http://eprint.iacr.org/. [48] George I. Davida and Jeremy A. Hansen. A structural comparison of SHA-3 candidates. In 12th Information Security Conference (ISC 2009), submitted, 2009. [49] Richard Drews Dean. Formal aspects of mobile code security. PhD thesis, Princeton University, 1999. Adviser-Andrew Appel. [50] Hans Dobbertin, Antoon Bosselaers, and Bart Preneel. Ripemd-160: A strengthened version of ripemd. In Dieter Gollmann, editor, FSE, volume 1039 of LNCS, pages 71–82. Springer, 1996. [51] Yevgeniy Dodis and Prashant Puniya. Getting the best out of existing hash functions; or what if we are stuck with SHA? In Steven M. Bellovin, Rosario Gennaro, Angelos D. Keromytis, and Moti Yung, editors, ACNS, volume 5037 of Lecture Notes in Computer Science, pages 156–173, 2008.

92 [52] Bj¨orn Fay. MeshHash. Submitted to NIST, 2008. [53] Niels Ferguson, Stefan Lucks, Bruce Schneier, Doug Whiting, Mihir Bellare, Tadayoshi Kohno, Jon Callas, and Jesse Walker. The Skein Hash Function Family. Submitted to NIST, 2008. [54] Ewan Fleischmann, Christian Forler, and Michael Gorski. The Twister Hash Family. Submitted to NIST, 2008. [55] Ewan Fleischmann, Christian Forler, and Michael Gorski. Classification of the SHA-3 Candidates, 2009. http://www.uni-weimar.de/cms/fileadmin/ medien/medsicherheit/Research/SHA3/Classification_of_the_SHA-3_ Candidates.pdf. [56] Ewan Fleischmann, Michael Gorski, and Stefan Lucks. On the security of Tandem-DM. Cryptology ePrint Archive, Report 2009/054, 2009. http: //eprint.iacr.org/. [57] Matthew K. Franklin, editor. Advances in Cryptology - CRYPTO 2004, 24th Annual International CryptologyConference, Santa Barbara, California, USA, August 15-19, 2004, Proceedings, volume 3152 of Lecture Notes in Computer Science. Springer, 2004. [58] Praveen Gauravaram and John Kelsey. Linear-xor and additive checksums don’t protect damg˚ ard-merkle hashes from generic attacks. In Tal Malkin, editor, CTRSA, volume 4964 of Lecture Notes in Computer Science, pages 36–51. Springer, 2008. [59] Praveen Gauravaram, Lars R. Knudsen, Krystian Matusiewicz, Florian Mendel, Christian Rechberger, Martin Schl¨affer, and Søren S. Thomsen. Grøstl – a SHA3 candidate. Submitted to NIST, 2008. [60] Danilo Gligoroski, Vlastimil Klima, Svein Johan Knapskog, Mohamed ElHadedy, Jørn Amundsen, and Stig Frode Mjølsnes. Cryptographic Hash Function BLUE MIDNIGHT WISH. Submitted to NIST, 2008. [61] Danilo Gligoroski, Rune Steinsmo Ødeg˚ ard, Marija Mihova, Svein Johan Knapskog, Ljupco Kocarev, and Ale˘s Dr´aal. Cryptographic Hash Function EDON-R. Submitted to NIST, 2008. [62] Michael Gorski, Stefan Lucks, and Thomas Peyrin. Slide attacks on a class of hash functions. In Josef Pieprzyk, editor, ASIACRYPT, volume 5350 of Lecture Notes in Computer Science, pages 143–160. Springer, 2008. [63] Louis Goubin, Mickael Ivascot, William Jalby, Olivier Ly, Valerie Nachef, Jacques Patarin, Joana Treger, and Emmanuel Volte. CRUNCH. Submitted to NIST, 2008.

93 [64] Shai Halevi, William E. Hall, and Charanjit S. Jutla. The Hash Function Fugue. Submitted to NIST, 2008. [65] Shai Halevi and Hugo Krawczyk. Strengthening digital signatures via randomized hashing. In Cynthia Dwork, editor, CRYPTO, volume 4117 of Lecture Notes in Computer Science, pages 41–59. Springer, 2006. [66] Jeremy A. Hansen. Cryptographic authentication of processes and files to protect a trusted base. Master’s thesis, University of Wisconsin - Milwaukee, May 2005. [67] Bob Hattersley. Waterfall Hash - Algorithm Specification and Analysis. Submitted to NIST, 2008. [68] Phil Hawkes and Cameron McDonald. Submission to the SHA-3 Competition: The CHI Family of Cryptographic Hash Algorithms. Submitted to NIST, 2008. [69] Shoichi Hirose. Provably secure double-block-length hash functions in a blackbox model. In Choonsik Park and Seongtaek Chee, editors, ICISC, volume 3506 of Lecture Notes in Computer Science, pages 330–342. Springer, 2004. [70] Shoichi Hirose, Hidenori Kuwakado, and Hirotaka Yoshida. SHA-3 Proposal: Lesamnta. Submitted to NIST, 2008. [71] P. Hoffman and B. Schneier. RFC 4270: Attacks on Cryptographic Hashes in Internet Protocols. Network Working Group, November 2005. [72] Paul Hoffman. Notes from the hash futures panel. In NIST’s Second Cryptographic Hash Workshop, 2006. [73] Sebastiaan Indesteege, Elena Andreeva, Christophe De Canni`ere, Orr Dunkelman, Emilia Ksper, Svetla Nikova, Bart Preneel, and Elmar Tischhauser. The LANE hash function. Submitted to NIST, 2008. [74] Tetsu Iwata, Kyoji Shibutani, Taizo Shirai, Shiho Moriai, and Toru Akishita. AURORA: A Cryptographic Hash Algorithm Family. Submitted to NIST, 2008. [75] Antoine Joux. Multicollisions in iterated hash functions. application to cascaded constructions. In Franklin [57], pages 306–316. [76] Charanjit S. Jutla and Anindya C. Patthak. Provably good codes for hash function design. In Eli Biham and Amr M. Youssef, editors, Selected Areas in Cryptography, volume 4356 of Lecture Notes in Computer Science, pages 376–393. Springer, 2006. [77] J.B. Kam and G.I. Davida. Structured design of substitution-permutation encryption networks. IEEE Transactions on Computers, 28(10):747–753, 1979. [78] Neil Kauer, Tony Suarez, and Yuliang Zheng. Enhancing the MD-strengthening and designing scalable families of one-way hash algorithms, 2005.

94 [79] John Kelsey and Tadayoshi Kohno. tradamus attack, 2005.

Herding hash functions and the nos-

[80] John Kelsey and Bruce Schneier. Second preimages on n-bit hash functions for much less than 2n work. In Ronald Cramer, editor, EUROCRYPT, volume 3494 of LNCS, pages 474–490. Springer, 2005. [81] Dmitry Khovratovich, Alex Biryukov, and Ivica Nicoli´c. The Hash Function Cheetah: Specification and Supporting Documentation. Submitted to NIST, 2008. [82] G. H. Kim and E. H. Spafford. The design and implementation of tripwire: A file system integrity checker. In 2ACMConfSec, 1994. [83] Lars R. Knudsen and Fr´ed´eric Muller. Some attacks against a double length hash proposal. In Roy [118], pages 462–473. [84] Lars R. Knudsen, Christian Rechberger, and Søren S. Thomsen. The grindahl hash functions. In Alex Biryukov, editor, FSE, volume 4593 of Lecture Notes in Computer Science, pages 39–57. Springer, 2007. [85] Michael Kounavis and Shay Gueron. Vortex: A New Family of One Way Functions based on Rijndael Rounds and Carry-less Multiplication. Submitted to NIST, 2008. [86] Hugo Krawczyk. LFSR-based hashing and authentication. In Yvo G. Desmedt, editor, CRYPTO ’94: Proceedings of the 14th Annual International Cryptology Conference on Advances in Cryptology, pages 129–139. Springer, 1994. Lecture Notes in Computer Science No. 839. [87] Xuejia Lai and Kefei Chen, editors. Advances in Cryptology - ASIACRYPT 2006, 12th International Conference on the Theory and Application of Cryptology and Information Security, Shanghai, China, December 3-7, 2006, Proceedings, volume 4284 of Lecture Notes in Computer Science. Springer, 2006. [88] Xuejia Lai and James L. Massey. Hash functions based on block ciphers. Lecture Notes in Computer Science, 658:55–70, 1993. [89] Ga¨etan Leurent, Charles Bauillaguet, and Pierre-Alain Fouque. SIMD Is a Message Digest. Submitted to NIST, 2008. [90] Stefan Lucks. A failure-friendly design principle for hash functions. In Roy [118], pages 474–494. [91] Smile Markovski and Aleksandra Mileva. (NaSHA). Submitted to NIST, 2008.

2.B.1 Algorithm Specification

[92] Jason Worth Martin. ESSENCE: A Candidate Hashing Algorithm for the NIST Competition. Submitted to NIST, 2008.

95 [93] Mikhail Maslennikov. SECURE HASH ALGORITHM MCSSHA-3. Submitted to NIST, 2008. [94] Krystian Matusiewicz and Josef Pieprzyk. Finding good differential patterns for attacks on sha-1. In Øyvind Ytrehus, editor, WCC, volume 3969 of Lecture Notes in Computer Science, pages 164–177. Springer, 2005. [95] Ueli Maurer, Renato Renner, and Clemens Holenstein. Indifferentiability, impossibility results on reductions, and applications to the random oracle methodology. In Theory of Cryptography - TCC 2004, Lecture Notes in Computer Science, pages 21–39. Springer-Verlag, 2004. [96] Peter Maxwell. The Sg`ail Cryptographic Hash Function. Submitted to NIST, 2008. [97] Alfred J. Menezes, Paul C. van Oorschot, and Scott A. Vanstone. Handbook of Applied Cryptography. CRC Press, 1997. http://www.cacr.math.uwaterloo. ca/hac/. [98] Ralph Merkle. One-way hash functions and DES. In Advances in Cryptology– Crypto’89. LNCS, vol.435, pages 428–446. Springer-Verlag, 1990. [99] Ralph Merkle and Martin Hellman. Hiding information and signatures in trapdoor knapsacks. In IEEE Trans. Inform. Theory, volume IT-24, pages 525–530, Sep 1978. [100] Ralph C. Merkle. Protocols for public key cryptosystems. In IEEE Symposium on Security and Privacy, pages 122–134, 1980. [101] Ralph C. Merkle. Fast software encryption functions. In CRYPTO ’90: Proceedings of the 10th Annual International Cryptology Conference on Advances in Cryptology, pages 476–501, London, UK, 1991. Springer-Verlag. [102] Ralph Charles Merkle. Secrecy, authentication, and public key systems. PhD thesis, Stanford University, Stanford, CA, USA, 1979. [103] National Institute of Standards and Technology (NIST). FIPS Publication 180: Secure Hash Standard (SHS), May 11, 1993. [104] Ivica Nikoli´c, Alex Biryukov, and Dmitry Khovratovich. Hash family LUX Algorithm Specifications and Supporting Documentation. Submitted to NIST, 2008. [105] NIST. ANSI C Cryptographic API Profile for SHA-3 Candidate Algorithm Submissions, 2008. [106] Philippe Oechslin. Making a faster cryptanalytic time-memory trade-off. In CRYPTO, pages 617–630. Springer, 2003.

96 [107] Sean O’Neil, Karsten Nohl, and Luca Henzen. EnRUPT Hash Function Specification. Submitted to NIST, 2008. ¨ ul K¨ [108] Ozg¨ uc¨ uk. The Hash Function Hamsi. Submitted to NIST, 2008. [109] Daniel Penazzi and Miguel Montes. The TIB3 Hash. Submitted to NIST, 2008. [110] Bart Preneel. Analysis and Design of Cryptographic Hash Functions. PhD thesis, Katholieke Universiteit Leuven, 1993. [111] Bart Preneel, Ren´e Govaerts, and Joos Vandewalle. Hash functions based on block ciphers: a synthetic approach. In Douglas R. Stinson, editor, CRYPTO ’93: Proceedings of the 13th Annual International Cryptology Conference on Advances in Cryptology, pages 368–378. Springer, 1994. Lecture Notes in Computer Science No. 773. [112] Ron Rivest, Benjamin Agre, Daniel V. Bailey, Christopher Crutchfield, Yevgeniy Dodis, Kermin Elliott Fleming, Asif Khan, Jayant Krishnamurthy, Yuncheng Lin, Leo Reyzin, Emily Shen, Jim Sukha, Drew Sutherland, Eran Tromer, and Yiqun Lisa Yin. The MD6 hash function – A proposal to NIST for SHA-3. Submitted to NIST, 2008. [113] Ronald L. Rivest. The MD4 message digest algorithm. Technical Report MIT/LCS/TM-434, MIT Laboratory for Computer Science, October 1990. [114] Ronald L. Rivest. The MD5 message-digest algorithm. Internet Request for Comments, April 1992. RFC 1321. [115] Phillip Rogaway. Formalizing human ignorance: Collision-resistant hashing without the keys. Cryptology ePrint Archive, Report 2006/281, 2006. http: //eprint.iacr.org/. [116] Greg G. Rose. Design and Primitive Specification for Boole. Submitted to NIST, 2008. [117] Joachim Rosenthal and Roxana Smarandache. Maximum distance separable convolutional codes. In Appl. Algebra Engrg. Comm. Comput, volume 10, pages 15–32, 1999. [118] Bimal K. Roy, editor. Advances in Cryptology - ASIACRYPT 2005, 11th International Conference on the Theory and Application of Cryptology and Information Security, Chennai, India, December 4-8, 2005, Proceedings, volume 3788 of Lecture Notes in Computer Science. Springer, 2005. [119] Markku-Juhani O. Saarinen. Security of VSH in the real world, 2006. [120] Gokay Saldamlı, Cevahir Demirkıran, Megan Maguire, Carl Minden, Jacob Topper, Alex Troesch, Cody Walker, and C ¸ etin Kaya Ko¸c. Spectral Hash. Submitted to NIST, 2008.

97 [121] Yu Sasaki, Go Yamamoto, and Kazumaro Aoki. Practical password recovery on an md5 challenge and response. Cryptology ePrint Archive, Report 2007/101, 2007. http://eprint.iacr.org/. [122] B. Schneier. Applied Cryptography: Protocols, Algorithms, and Source Code in C. John Wiley & Sons, New York, 1993. [123] Bruce Schneier and John Kelsey. Unbalanced feistel networks and block cipher design. In Proceedings of the Third International Workshop on Fast Software Encryption, pages 121–144, London, UK, 1996. Springer-Verlag. [124] Julian Seward. bzip2 and libbzip2, version 1.0.5: A program and library for data compression, 2009. [125] Secure Hash Standard. National Institute of Standards and Technology, Washington, 2002. Federal Information Processing Standard 180-2. [126] Neil Sholer. Abacus: A Candidate for SHA-3. Submitted to NIST, 2008. [127] W. Simpson. RFC 1996: PPP Challenge Handshake Authentication Protocol (CHAP). Network Working Group, August 1996. [128] John P. Steinberger. The collision intractability of mdc-2 in the ideal-cipher model. In Moni Naor, editor, EUROCRYPT, volume 4515 of Lecture Notes in Computer Science, pages 34–51. Springer, 2007. [129] Kazuhiro Suzuki and Kaoru Kurosawa. How to find many collisions of 3-pass HAVAL, 2007. [130] Michael Szydlo and Yiqun Lisa Yin. Collision-Resistant Usage of MD5 and SHA-1 Via Message Preprocessing. In David Pointcheval, editor, CT-RSA, volume 3860 of Lecture Notes in Computer Science, pages 99–114. Springer, 2006. [131] Mark Torgerson, Richard Schroeppel, Tim Draelos, Nathan Dautenhahn, Sean Malone, Andrea Walker, Michael Collins, and Hilarie Orman. The SANDstorm Hash. Submitted to NIST, 2008. [132] Michal Trojnara. StreamHash Algorithm Specifications and Supporting Documentation. Submitted to NIST, 2008. [133] Alexander Vardy. The intractability of computing the minimum distance of a code. IEEE Transactions on Information Theory, 43(6):1757–1766, 1997. ¨ [134] Kerem Varıcı, Onur Ozen, and C ¸ elebi Kocair. Sarmal: SHA-3 Proposal. Submitted to NIST, 2008. [135] Natarajan Vijayarangan. A NEW HASH ALGORITHM: Khichidi-1. Submitted to NIST, 2008.

98 [136] Lei Wang, Kazuo Ohta, and Noboru Kunihiro. Password recovery attack on authentication protocol md4(password——challenge). In ASIACCS ’08: Proceedings of the 2008 ACM symposium on Information, computer and communications security, pages 3–9, New York, NY, USA, 2008. ACM. [137] Xiaoyun Wang, Yiqun Lisa Yin, and Hongbo Yu. Finding collisions in the full sha-1. In CRYPTO, pages 17–36, 2005. [138] Xiaoyun Wang and Hongbo Yu. How to break md5 and other hash functions. In Eurocrypt 2005, volume 3494, pages 19–35. Lecture Notes in Computer Science, May 2005. [139] John Washburn. WAMM: A CANDIDATE ALGORITHM FOR THE SHA-3 COMPETITION. Submitted to NIST, 2008. [140] A. F. Webster and S. E. Tavares. On the design of S-boxes. In H. C. Williams, editor, Advances in Cryptology — Crypto ’85, pages 523–534, New York, 1986. Springer-Verlag. [141] David A. Wilson. The DCH Hash Function. Submitted to NIST, 2008. [142] Hongjun Wu. The Hash Function JH. Submitted to NIST, 2008. [143] Zijie Xu. Dynamic SHA. Submitted to NIST, 2008. [144] Zijie Xu. Dynamic SHA2. Submitted to NIST, 2008. [145] Y. Zheng, J. Pieprzyk, and J. Seberry. HAVAL - A one-way hashing algorithm with variable length of output. In J. Seberry and Y. Zheng, editors, Advances in Cryptology — Auscrypt ’92, pages 83–104, Berlin, 1993. Springer-Verlag.

99

Appendix: Application Programming Interface Introduction Aside from the demonstration of the applicability on individual algorithms found in Chapter 6, we describe here how this four-component framework easily fits NIST’s required Application Programming Interface (API) for all SHA-3 candidates. NIST broke down the operation of proposed algorithms into the following three basic functions: HashReturn Init(hashState *state, int hashbitlen); HashReturn Update(hashState *state, const BitSequence *data, DataLength databitlen); HashReturn Final(hashState *state, BitSequence *hashval);

Init() The purpose of this function is to initialize the internal state and any algorithm constants based only on algorithm parameters, with the length of the final hash value as the only required parameter. In our framework, this function will do exactly that, setting necessary initial values in each component as necessary. The state S and any required “global” variables like bits hashed so far are stored directly in the state structure.

Update() This function is called to introduce new message data (in the data variable) and update the state appropriately. Three of the four components are represented in this function. The preprocessor operates on the message data and any parameters, then the usual iterated structure updates S. We can represent this in C-style code as follows, with the functions IS and PP representing the iterative structure and preprocessor, respectively:

100 DataLength PP(hashState *state, const BitSequence *data, DataLength databitlen, BitSequence *ppData, bool final); This function returns the length of the preprocessed message that it allocates at ppData, and applies global preprocessors if final is set to true. HashReturn IS(hashState *state, const BitSequence *data, DataLength databitlen); The IS function invokes the combining functions “behind the scenes” as appropriate. Effectively a clone of the Init function, this is meant to be called with the output from the PP function as follows: HashReturn Update(hashState *state, const BitSequence *data, DataLength databitlen) { BitSequence *ppData; DataLength ppLen = PP(state, data, databitlen, ppData, false); HashReturn retVal = IS(state, ppData, ppLen); free(ppData); return retVal; }

Final() This function is responsible for any preprocessors operating upon the entire message, like padding or MD-strengthening, processing any remaining message bits, and applying the finalization function FF. The FF function applies its finalization transformations to the state and places the hash value into hashval. HashReturn FF(hashState *state, BitSequence *hashval); The Final function might look like the following: HashReturn Final(hashState *state, BitSequence *hashval) { BitSequence *ppData; DataLength ppLen = PP(state, NULL, 0, ppData, true); HashReturn retVal = IS(state, ppData, ppLen); free(ppData); if (retVal == SUCCESS)

101 return FF(state, hashval); else return retVal; }

102

CURRICULUM VITAE

Jeremy A. Hansen Place of Birth: Manitowoc, Wisconsin, United States Education M.S., University of Wisconsin - Milwaukee, May 2005 Major: Computer Science B.S., University of Wisconsin - Madison, 1998 Major: Computer Science Dissertation Title: A Four Component Cryptographic Hash Framework Conference Publications “A Four-Component Framework for Cryptographic Hash Algorithms,” (with George I. Davida) 16th USENIX Security Symposium (Poster), August 2007. “A Preliminary Exploration of Striped Hashing: A Probabilistic Scheme to Speed Up Existing Hash Algorithms,” (with George I. Davida) Second International Conference on e-Business and Telecommunication Networks (Poster), October 2005. “A Structural Comparison of SHA-3 Candidates,” (with George I. Davida) 12th Information Security Conference (ISC 2009) (Submitted), September 2009. Awards Chancellor’s Graduate Student Award (2006-2007) University Service UW-Milwaukee Graduate Faculty Committee (2007-2008) UW-Milwaukee Academic Policy Committee (2007-2008)

Major Professor

Date

by Jeremy A. Hansen A Dissertation Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Engineering at The University of Wisconsin-Milwaukee May, 2009

A FOUR COMPONENT CRYPTOGRAPHIC HASH FRAMEWORK

by Jeremy A. Hansen A Dissertation Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Engineering at The University of Wisconsin-Milwaukee May, 2009

Major Professor

Date

Graduate School Approval

Date

ii

ABSTRACT A FOUR COMPONENT CRYPTOGRAPHIC HASH FRAMEWORK by Jeremy A. Hansen The University of Wisconsin-Milwaukee, 2009 Under the supervision of George I. Davida

The traditional method for describing cryptographic hash algorithms is to offer two components: the compression function and an iteration scheme. This paper shows that describing cryptographic hash algorithms as being comprised of four components more easily and more accurately illustrates the operation of the algorithm and simplifies analysis. In the four-component framework, cryptographic hash algorithms are divided into preprocessing, iterative structure, combining function and finalization components. As part of the iterative structure, a global state is initialized, updated according to the preprocessed input, and finalized into the output of the algorithm: the hash value. Each component lends itself to providing or preserving security properties in the overall algorithm. The four components are individually examined for the properties they provide and attacks they may prevent. Existing techniques for each component (that may not have been initially framed as such) are presented and analyzed and

iii

a few novel techniques are introduced. An extensive list of algorithms, including all published candidates for SHA-3, the forthcoming United States Advanced Hash Standard, are analyzed for their choices of each of their four components.

Major Professor

Date

iv

c Copyright by Jeremy A. Hansen, 2009

All Rights Reserved

v

Table of Contents

List of Figures . . . . . . . . . List of Tables . . . . . . . . . . List of Symbols/Nomenclature Acknowledgements . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

ix x xi xii

1 Cryptographic Hash Algorithms 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . 1.2 Properties . . . . . . . . . . . . . . . . . . . . . . . 1.2.1 Collision Resistance . . . . . . . . . . . . . 1.2.2 f -Resistance . . . . . . . . . . . . . . . . . 1.2.3 Preimage and Second Preimage Resistance 1.3 Applications . . . . . . . . . . . . . . . . . . . . . . 1.3.1 Password Obfuscation . . . . . . . . . . . . 1.3.2 Data Integrity . . . . . . . . . . . . . . . . 1.3.3 Challenge-Response Protocols . . . . . . . . 1.3.4 Digital Signatures . . . . . . . . . . . . . . 1.3.5 Bit Commitment Schemes . . . . . . . . . . 1.4 Attacks . . . . . . . . . . . . . . . . . . . . . . . . 1.4.1 Length Extension . . . . . . . . . . . . . . . 1.4.2 Fixed Points . . . . . . . . . . . . . . . . . 1.4.3 Second Preimages in Long Messages . . . . 1.5 Contributions and Thesis . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

1 1 2 2 4 6 7 7 8 9 10 10 11 11 12 12 13

2 The 2.1 2.2 2.3 2.4

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

16 16 17 19 19 21 21 22 23 24

3 Preprocessing 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Locality . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Content . . . . . . . . . . . . . . . . . . . . . . . . 3.2.3 Preprocessing as a Domain Transformation . . . . 3.3 Properties and Observations . . . . . . . . . . . . . . . . . 3.3.1 Preprocessors and Property Preservation . . . . . 3.3.2 Preprocessing as Encoding and Differential Attacks

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

25 25 26 26 28 30 31 31 32

2.5

Framework Overview . . . . . . . . . . . . . Previous Frameworks . . . . . . . The Four Component Framework Components of the Framework . 2.4.1 Preprocessor . . . . . . . 2.4.2 Iterative structure . . . . 2.4.3 Combining Functions . . . 2.4.4 Finalization . . . . . . . . Application . . . . . . . . . . . .

. . . .

. . . . . . . . .

. . . . . . . . .

vi

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

3.4

3.5

3.3.3 Preprocessor versus Iterative Structure Schemes . . . . . . . . . . . . . . . . . . . . . 3.4.1 Padding . . . . . . . . . . . . . . . . . 3.4.2 Message Length/Final Block Length . 3.4.3 Salt . . . . . . . . . . . . . . . . . . . 3.4.4 Bits So Far/Blocks So Far . . . . . . . 3.4.5 Whitening . . . . . . . . . . . . . . . . 3.4.6 Message Self-Interleaving . . . . . . . 3.4.7 Message Duplication . . . . . . . . . . 3.4.8 Block Position . . . . . . . . . . . . . 3.4.9 Block Permutation . . . . . . . . . . . 3.4.10 Hash Length and other Parameters . . 3.4.11 Tagging . . . . . . . . . . . . . . . . . 3.4.12 Checksums and Error-Detecting Codes 3.4.13 Lossless Compression . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . .

block size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4 Iterative Structure and Combining Functions 4.1 Introduction . . . . . . . . . . . . . . . . . . . . 4.2 Properties and Observations . . . . . . . . . . . 4.2.1 Combining Functions . . . . . . . . . . . 4.2.2 Iterative Structure . . . . . . . . . . . . 4.3 Iterative Structure Schemes . . . . . . . . . . . 4.3.1 Linear Iterative Schemes . . . . . . . . . 4.3.2 Tree Iterative Schemes . . . . . . . . . . 4.3.3 Modifying Features . . . . . . . . . . . . 4.3.4 Proposed Structures . . . . . . . . . . . 4.4 Combining Function Schemes . . . . . . . . . . 4.4.1 Execution Strategy . . . . . . . . . . . . 4.4.2 Subcomponents . . . . . . . . . . . . . . 4.5 Conclusion . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

33 33 34 35 35 36 37 38 38 38 39 39 39 40 40 41

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

42 42 44 44 49 50 50 53 54 55 58 58 60 62

5 Finalization 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . 5.2 Properties and Observations . . . . . . . . . . . . 5.3 Finalization Schemes . . . . . . . . . . . . . . . . 5.3.1 Subset-of-State . . . . . . . . . . . . . . . 5.3.2 Final Permutation and Final Compression 5.3.3 No-Input Iterations . . . . . . . . . . . . . 5.3.4 Pumping Rounds . . . . . . . . . . . . . . 5.3.5 Pseudorandom Number Generators . . . . 5.4 Conclusion . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

65 65 65 66 66 67 67 68 68 68

. . . .

70 70 70 72 75

6 Analysis and Conclusions 6.1 Negative Features . . . . . 6.2 Algorithm Analysis . . . . 6.2.1 Preprocessing . . . 6.2.2 Iterative Structure

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

vii

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

6.3

6.4 6.5

6.2.3 Combining Function . . . . 6.2.4 Finalization . . . . . . . . . Algorithm Analysis: SANDstorm . 6.3.1 Preprocessing . . . . . . . . 6.3.2 Iterative Structure . . . . . 6.3.3 Combining Function . . . . 6.3.4 Finalization . . . . . . . . . Recommendations . . . . . . . . . 6.4.1 Recommended Construction Conclusion . . . . . . . . . . . . . 6.5.1 Future Directions . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

78 81 84 84 84 84 85 85 85 86 87

References

88

Appendix: Application Programming Interface

99

Curriculum Vitae

102

viii

List of Figures

2.1 2.2 2.3 2.4

The Merkle-Damg˚ ard Iterative Structure Sponge-style “Absorb” Phase . . . . . . Sponge-style “Squeeze” Phase . . . . . . Four Component Framework . . . . . .

. . . .

17 18 19 20

3.1

Preprocessors as Domain Transformations . . . . . . . . . . . . . . . . . . .

30

4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 4.10 4.11

A “supercombiner” . . . . . . . . . . . . . . . . . . . . . The squeezing phase of a sponge function as finalization A Narrow-pipe Linear Iterative Structure . . . . . . . . An `-pipe Linear Iterative Structure for ` = 2 . . . . . . Tandem Davies-Meyer . . . . . . . . . . . . . . . . . . . A Merkle Hash Tree . . . . . . . . . . . . . . . . . . . . A Modified XOR Hash Tree . . . . . . . . . . . . . . . . The Enveloped Merkle-Damg˚ ard Iterative Structure . . Shoup’s Transform . . . . . . . . . . . . . . . . . . . . . The 3C Hash’s Iterative Structure . . . . . . . . . . . . A Balanced Feistel Network . . . . . . . . . . . . . . . .

45 47 51 52 53 54 55 56 57 58 60

ix

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . . . . . . . . .

. . . .

. . . . . . . . . . .

. . . .

. . . . . . . . . . .

. . . .

. . . . . . . . . . .

. . . .

. . . . . . . . . . .

. . . .

. . . . . . . . . . .

. . . .

. . . . . . . . . . .

. . . .

. . . . . . . . . . .

. . . .

. . . . . . . . . . .

. . . .

. . . . . . . . . . .

. . . . . . . . . . .

List of Tables

3.1 3.2

Examples of preprocessor content types in both localities . . . . . . . . . . Breakdown of various preprocessing schemes . . . . . . . . . . . . . . . . . .

26 34

6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9 6.10 6.11 6.12 6.13 6.14 6.15 6.16 6.17 6.18 6.19 6.20

Requirements for Property Preservation . . . . . . . . . . . . Legend for Preprocessor Abbreviations . . . . . . . . . . . . . Preprocessing Techniques in Common Algorithms . . . . . . . Preprocessing Techniques in Other Hash Algorithms . . . . . Preprocessing Techniques in SHA-3 Candidates, Part 1 . . . . Preprocessing Techniques in SHA-3 Candidates, Part 2 . . . . Iterative Structure Techniques in Common Algorithms . . . . Iterative Structure Techniques in Other Algorithms . . . . . . Iterative Structure Techniques in SHA-3 Candidates, Part 1 . Iterative Structure Techniques in SHA-3 Candidates, Part 2 . Legend for Combining Function Abbreviations . . . . . . . . Combining Function Techniques in Common Algorithms . . . Combining Function Techniques in Other Algorithms . . . . . Combining Function Techniques in SHA-3 Candidates, Part 1 Combining Function Techniques in SHA-3 Candidates, Part 2 Legend for Finalization Function Abbreviations . . . . . . . . Finalization Techniques in Common Algorithms . . . . . . . . Finalization Techniques in Other Algorithms . . . . . . . . . Finalization Techniques in SHA-3 Candidates, Part 1 . . . . . Finalization Techniques in SHA-3 Candidates, Part 2 . . . . .

71 72 73 73 73 74 76 76 76 77 78 79 79 79 80 81 82 82 82 83

x

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

List of Symbols/Nomenclature

|| ⊕ ∨ ∧ h n H M mi S w L π b

Concatenation of bit-strings The exclusive-OR (XOR) operation The bitwise-OR operation The bitwise-AND operation The hash value returned by the algorithm The length, in bits, of the hash value h The entire hash algorithm The entire input message The ith block of M The internal state of the algorithm The length, in bits, of the state S The length, in bits, of the message M The preprocessing function component The size of a message block operated upon by a local preprocessor I∆ The iterated structure component which uses combining functions in ∆ δi The ith combining function in ∆ dδi The length of a message block operated upon by combining function δi ∆ A collection of combining functions Φ The finalization function component O(x) “Big O” notation: the number of operations is asymptotically at most x Ω(x) “Big Omega” notation: the number of operations is asymptotically at least x HD(x, y) The Hamming Distance between x and y - the number of positions where the bits of x and y differ HW(x) The Hamming Weight of x - the number of bits set to 1 in x

xi

Acknowledgements

I would like to dedicate this dissertation foremost to Cara and JJ, whose patience and supportiveness are undoubtedly the reason I was able to finish this work. For all of the hours that we have not spent together due to my spending time working on this, I love you and thank you both. I would also like to thank my parents and grandparents whose work over the years enabled me to accomplish my goals. Thanks to George Davida who, throughout my many strange and tangential ideas, managed to help me stay focused and graduate in a reasonable amount of time. Thanks to Bruce Barr, my first academic mentor, who managed to get the gears in my head turning in the right direction all those years ago. I have not forgotten you. Additional thanks to my employers over the course of my graduate studies who have remained tolerant and supportive of my activities outside of work, and all of my friends of whom I now expect to see more.

xii

1

Chapter 1 Cryptographic Hash Algorithms

1.1

Introduction

The design and analysis of cryptographic hash algorithms is made easier by defining them as the composition of four parts. Cryptographic hash algorithms are critical pieces of modern cryptographic systems that provide a transformation unlike those provided by other cryptographic primitives like block ciphers and stream ciphers. They are typically used to guarantee integrity in applications such as bit-commitment protocols, challenge-response protocols, user password protection, and digital signature schemes. This essential part of a cryptographer’s toolkit is new relative to block and stream ciphers and is not as well understood. Our work seeks to partially remedy that by introducing a novel framework which allows cryptographers to more easily demonstrate that a particular cryptographic hash algorithm possesses necessary security properties and consequently can be used in all appropriate applications. Unlike stream or block ciphers whose outputs are the same length as their inputs, hash algorithms produce as output a fixed-length bit string (commonly referred to as a hash value, hash, or message digest) from an arbitrarily-long bit string. Determining the hash value of an input is often referred to as “hashing” the input. The entire hash algorithm can be represented as the function H operating on its arbitrary-length preimage M to produce its image h: H(M ) → h The security of the algorithm begins with the property that, if the message changes, the hash value h will almost certainly change. Because of this, the hash value is often

2 used as a shorter, surrogate representation of the preimage, but this “compression” should not be confused with reversible compression since the preimage should not be retrievable from the hash value. Cryptographic hash algorithms are expected to be one-way—that is, it should be difficult to invert the algorithm given an image to produce a preimage. Note, though, that there must be other “colliding” preimages which hash to the same message digest whenever the preimage bit-string is longer than the image. Resistance to both attacks which generate colliding preimages and attacks that produce preimages given only an image is the most common measurement of the security of cryptographic hash algorithms. Both of these properties are described in more detail in the next section.

1.2

Properties

To be useful in its role, a cryptographic hash algorithm is expected to possess certain properties. We first assume according to basic cryptographic principles that the entire algorithm specification is published and otherwise publicly available. We will also assume that the algorithm and all of its inputs are deterministic. There are algorithms which offer a powerful tool in randomized hashing [65], but we opt to work only with algorithms whose inputs are non-random. Three properties are typically cited as essential: collision resistance, preimage resistance, and second preimage resistance. We also discuss some lesser-used security properties which are related to the “traditional” properties. The following definitions of security properties are consistent with those of Menezes et al. [97].

1.2.1 Collision Resistance The first essential property of cryptographic hash algorithms, collision resistance, guarantees that it is difficult for any party to produce two inputs (called collisions or a colliding pair ) which hash to the same value. Hash algorithms which possess this property are often considered to be members of a distinct category: collision

3 resistant hash functions. We avoid using the terms “collision freedom” or “collision free” as found elsewhere in the cryptologic literature since the word freedom implies that some algorithm may in fact be free from collisions. As we deal here only with concrete instantiations of hash algorithms, collisions must exist due to the pigeonhole principle, as described by the “foundations of hashing dilemma” [115,17]. Since collisions are unavoidable, security properties like collision resistance are usually argued by asserting that there exist no attacks against a particular algorithm that are more efficient than brute force. In the case of collision resistance, every hash algorithm which produces an n-bit hash is vulnerable to a brute force attack which produces a n

colliding pair of inputs in approximately 2 2 operations due to the birthday paradox [97]. It is usually said that a hash algorithm H is collision resistant if determining inputs x and x0 such that H(x) = H(x0 ) is computationally infeasible. In the concrete case of a collision resistant algorithm, finding x and x0 would require no fewer than n

2 2 operations, the same number expected by the algorithm-independent birthday n

attack. Attacks which generate collisions in fewer than 2 2 operations are usually considered to be breaks of the algorithm, even if the attacks may not be applicable in a reasonable amount of time. Despite their infeasibility, collision attacks that are faster than the general birthday attack may be indicative of larger undiscovered flaws in the algorithm which may in turn need lead to even more efficient attacks.

1.2.1.1

Multicollisions

Finding additional collisions in the typical Merkle-Damg˚ ard iterative structure turns out to not be much more difficult than finding the first. As proven in [90], in an iterated cryptographic hash algorithm with an ideal compression function (as described in more detail in Chapter 4) and an internal state that is w bits long, an

4 attacker requires w

Ω(min{2 2 , 2n(K−1)/K }) operations to find K colliding messages. This type of attack illustrates the difference between collisions on the internal state (“state collisions”) and collisions on the final w

hash value (“output collisions”). The first quantity above, 2 2 , indicates the difficulty of finding internal collisions, while 2n(K−1)/K represents the difficulty of finding K output collisions in an ideal hash function producing an n-bit hash. It follows then, that when w ≥ 2n, multicollisions are of a similar difficulty as the ideal hash.

1.2.2 f -Resistance Collision resistance alone may not be sufficient for certain protocols where other specific security requirements apply. For example, the algorithm VSH [38], while provably collision resistant, is vulnerable to attacks against its algebraic structure [119] which may make it unsuitable for certain typical applications of cryptographic hash algorithms such as those that apply algebraic operations similar to those used internally in the algorithm. In [3], Anderson describes several properties of cryptographic hash algorithms which involve arithmetic operations, such as addition freedom, which is defined as a property of a hash function H where it is infeasible to find x, x0 , and y, such that H(x) + H(x0 ) = H(y). We recast this property as “addition resistance” by defining it as the property such that an attack against an addition-resistant algorithm requires no fewer operations than an algorithm-independent brute force attack to find x, x0 , and y, such that H(x) + H(x0 ) = H(y). A cryptographic hash algorithm is f -resistant if, for any fixed function f of v variables, an attacker requires more operations than an algorithm-independent brute force attack to determine all xi for all values of i ∈ {1, ..., v} and y such that f (H(x0 ), ..., H(xv )) = H(y) An f -attack against H is an attack which produces a set of values for all xi and

5 y that violates H’s f -resistance. Using the previous notation, it is straightforward to describe other security properties. The function for multiplication resistance, for example, can be written as f (H(x0 ), H(x1 )) = H(x0 ) · H(x1 ) = H(y) The f for addition resistance can be written similarly as f (H(x0 ), H(x1 )) = H(x0 ) + H(x1 ) = H(y) and for collision resistance written as f (H(x0 )) = H(x0 ) = H(y) With this generalized resistance property notation, cryptographers can enumerate the functions to which specific hash algorithms are resistant (or weak). This is useful since certain applications may require complex f -resistances. An example of a more complex resistance requirement is in the Yen-Laih signature scheme [3]: given A, B, and q, it should be difficult to determine x and y such that f (H(x)) =

A H(x) (mod q) = H(y) B n

Lemma 1.2.1 The algorithm-independent brute force f -attack requires O(2 2 ) operations. Proof Fix all x0 , ..., xv−1 to arbitrary values, then launch a birthday attack to find n

xv and y to find f (H(x0 ), ..., H(xv )) = H(y), which requires O(2 2 ) operations. This brute force attack establishes an upper bound for all choices of f , but does not preclude an algorithm-independent attack against a particular f which requires n

substantially fewer operations than 2 2 .

6

1.2.3 Preimage and Second Preimage Resistance The other two essential properties, preimage resistance and second preimage resistance, are usually more important to applications than collision resistance, and are fortunately more resistant to brute force attack. Whereas the previous resistance properties allow an attacker flexibility with the inputs and outputs, these two properties require that certain values remain fixed. Preimage resistance in a cryptographic hash algorithm guarantees that, given only a hash value, it is difficult to determine a preimage which hashes to that value. More formally, we say that a cryptographic hash algorithm H is preimage resistant if, given y, an attacker requires no fewer operations than an algorithm-independent brute force attack to determine x such that H(x) = y. The brute force attack on a hash function with an output that is n bits long simply selects inputs until a preimage that hashes to y is found. Since the probability of finding a preimage this way is 2−n , an attacker can expect to select O(2n ) inputs before he finds one which hashes to y. Second preimage resistance is similar, except that an attacker also has a preimage x to work from in addition to the hash value. A cryptographic hash algorithm H is second preimage resistant if, given x and y, an attacker requires no fewer operations than an algorithm-independent brute force attack to determine x0 such that H(x) = H(x0 ) = y. While the brute force second preimage attack requires the same number of operations as preimage resistance (that is, O(2n )), second-preimage attacks are usually considered to be easier than preimage attacks since the additional information provided by x may allow for differential attacks to succeed. In typical iterated hash algorithms, longer messages provide more opportunities for second-preimage attacks, such that the difficulty of finding a second preimage is reduced significantly [80]. Refer to section 1.4.3 for more details.

7

1.2.3.1

Multipreimages

Like attacks which can quickly produce additional collisions after the first, it may be possible to generate K preimages from a given hash in less than O(K2n ) time. As proven in [90], an attacker requires w

Ω(min{2 2 , K2n }) operations to find K-way preimages or K-way second preimages in an ideal iterated hash algorithm with a w-bit internal state and n-bit hash value.

1.3

Applications

Cryptographic hash algorithms are useful for many applications such as password obfuscation, data integrity, challenge-response protocols, digital signatures, and bit commitment schemes. The majority of these applications require hash algorithms to be (second) preimage resistant, though attacks on some applications can exploit functions which are not fully collision resistant [71]. Each of these applications are described below.

1.3.1 Password Obfuscation To avoid storing plaintext passwords, hashes of passwords are often stored instead. This prevents a malicious party who gains access to the password file from being able to use the passwords directly. When a legitimate user logs in, the user provides their password p, the system hashes the user’s input (H(p)), and compares it to the previously stored hash h. If they are equal, the user is successfully authenticated. An attacker without access to this hash has only one hope of compromising this system: correctly guess the user’s password while interacting directly with the target system. With the hash, however, an attacker can devote resources outside of the authenticating system to determining the password. This is called an offline attack, whereas the previous guessing-based attack is an online attack. In a na¨ıve offline

8 attack, an attacker produces many possible values for p, hashes each, then compares each hash to the value extracted from the authorizing host. When he finds a match, the attacker knows that he has a value for p that can be successfully used to log on and impersonate the user. To improve the speed of an offline attack, an attacker may be able to precompute lookup tables of the hashes of common passwords. Another method to speed the password search is to use rainbow tables [106], which allow for the construction of efficient lookup tables which offer a trade off between the time spent searching and the size of the table. If the hash algorithm that underlies this application is not preimage resistant, the attacker can work backwards directly from h to determine p. If the difficulty of doing so is less than guessing all possible passwords, this preimage attack results in an attack that undermines the security of the algorithm.

1.3.2 Data Integrity Hash functions are often used to guarantee the integrity of data in transit and in storage. When used to guarantee the integrity of data in transit, cryptographic hashes behave much like error-detecting codes but have the additional benefit of being more resistant to malicious attack than typical error-detecting codes. Applications using hashes for integrity verification operate in two distinct steps: evaluation and verification. The evaluation step uses a hash algorithm H to produce a hash value h from an input string x such that h = H(x). This value h is stored for later retrieval or sent along with the data string. The verification step then re-evaluates the data string, which we refer to as x0 , to determine if any changes have taken place. Assuming that h itself was unchanged in transit or during storage, we can conclude with high probability that, if h = H(x0 ), then x = x0 . Note that the problem remains of how to keep h itself from being modified between the evaluation and verification

9 steps. Tripwire [82] is an implementation of this scheme which verifies the integrity of important files. It stores hashes of important system files which are re-verified periodically. Execution-time whitelisting systems such as the system described in [66] maintain a list of hashes of all executable files on a system and only allows those applications with hashes in the list to execute, regardless of the user’s permissions. In this application, second preimage resistance is the most important property. Without it, an attacker can replace a legitimate message or file with one he generates as part of a second preimage attack that will pass unnoticed through the verification step.

1.3.3 Challenge-Response Protocols Authentication protocols like CHAP [127] which are based on a challenge-response exchange can be implemented using cryptographic hash algorithms. The challenge portion begins with the user requesting access to a resource. The authenticator responds by sending the challenge C, a random string of data, to the user. The user concatenates this random string with their password pu and hashes the result to generate the response Ru = H(C||pu ) The user sends Ru to the authenticator, which, with its copy of the expected password pe , can calculate the expected response to the challenge as Re = H(C||pe ). If Re = Ru , the authenticator can conclude with a high degree of certainty that the user knows his or her password. The primary benefit of this scheme is that the password is never transmitted over an insecure channel, and malicious modification of the exchanged messages will almost always result in the authenticator rejecting the user’s request. Hash algorithms subject to preimage attacks may be exploited by an attacker to discover pu given C and Ru in a similar manner to attacks against password obfuscation applications. Non-collision-resistant algorithms may also be subject to attack, such as those described in [121] and [136].

10

1.3.4 Digital Signatures Digital signatures are used on digital documents for similar reasons as physical signatures on physical documents, the most straightforward of which is to demonstrate the authenticity of the item that is being signed. Digital signatures are also more powerful than physical signatures in that modification of the signed message becomes evident to any party verifying the signature. For this reason, digital signatures are also effective at guaranteeing non-repudiation, so a party who has produced a digital signature can not return at a later time and claim that their signature is invalid. Hash algorithms are used in digital signatures both to improve their security and increase their speed. Originally, RSA was used to sign (represented by function S) an entire message M with a private key k to generate a signature s = S(k, M ). In [46], Davida shows a weakness in this signature scheme that effectively discloses the private key if an attacker is allowed to choose the message to be signed. By first hashing M so that the operation becomes s = S(k, H(M )), Davida’s attack is foiled, provided that the cryptographic hash used is not subject to the same homomorphisms as exploited by Davida’s attack1 . Since the RSA signing operation requires a fair number of exponentiation operations on extremely large numbers, reducing the size of the input to a short hash value dramatically improves the performance of the operation. Weaknesses in the hash function, particularly those producing collisions, can lead to attacks which allow two different messages to bear the same valid digital signature.

1.3.5 Bit Commitment Schemes Bit commitment allows a party to commit to a particular value (typically 0 or 1) with an untrusted second party so that neither party can cheat in an exchange. That is, if the first party “Alice”, chooses the value 1 and committed this to the second party, “Bob”, Alice cannot convince Bob that she committed to the value 0 and Bob 1

That is, H(M1 ∗ M2 ) 6= H(M1 ) ∗ H(M2 )

11 cannot renege on an agreement that depended upon Alice choosing 1. There are two phases of a commitment scheme: the commit phase and the reveal phase. The commit phase involves Alice choosing the value 0 or 1, generating a random string r, and publishing her commitment c. Supposing that Alice chooses 1, she calculates c = H(1||r), and provides c to Bob. In the reveal phase, Bob publishes the true result of the exchange, 0 or 1. Alice reveals her choice by publishing 1 and r, at which point, Bob can easily verify that the value of c that Alice provided earlier corresponds to H(1||r). Alice can cheat in this scheme if she can produce collisions in the underlying hash algorithm that differ in their first bits: c∗ = H(1||r0 ) = H(0||r1 ) By committing to c∗ , she can select r0 or r1 and submit it as r, depending on which value she chooses after the commit phase. Bob can cheat if he can create preimages that hash to Alice’s commitment c, since he can claim that the preimage he created is in fact Alice’s choice with a different value of r.

1.4

Attacks

There are many techniques for attacking cryptographic hash algorithms, most of which are algorithm-specific, such as differential cryptanalysis, described in Chapter 4. There are other, more generic attacks, though, that can be applied to algorithms in the general case, or to at least a large subset of algorithms, say, those that use the Merkle-Damg˚ ard iterative structure. We discuss a few of these here.

1.4.1 Length Extension In a linearly-iterated hash algorithm, it is possible for an attacker to generate a new hash value with no knowledge of the original message M aside from its length L and its hash h = H(M ). Since most hash algorithms append some padding and the

12 length L to the message before hashing, we can represent h as h = H(h0 , M ||pad||L) An attacker can append an arbitrary second message M 0 to this by first calculating its length: L0 = length(M ||pad||L) Then the attacker can construct h0 = H(h, M 0 ||pad0 ||L0 ) without direct knowledge of M .

1.4.2 Fixed Points A fixed point is a message block x that results in the following condition with a compression function C and chaining value hi : hi+1 = C(hi , x) = hi Since x does not update the internal state, it may be repeated infinitely many times to generate colliding messages: H(M ) = H(x||M ) = H(x||x||M ) Fortunately, this attack can be foiled by appending the message’s length L, since H(M ||L) 6= H(x||M ||length(x||M ))

1.4.3 Second Preimages in Long Messages In iterated hash algorithms that produce an n-bit hash, long messages are susceptible to a second preimage attack with difficulty less than 2n . Since the message is usually broken into blocks to be processed by compression functions that have a

13 fixed-size inputs, we say that the message M is made up of t blocks. While computing H(M ), each of the t intermediate values is stored and indexed for later use. At this point, the attacker simply hashes random values until an output matches one of the stored intermediate values. The probability that a random value hashes to one of the stored values is t 2n Thus, the expected number of trials this requires is 2n O t

Once a colliding block is found, this block can be used to replace the entire message up to the colliding intermediate value. Supposing that the collision is found after the k th block of the message (mk−1 ) is processed and the intermediate value at that point is hk . With this attack, the colliding block µ can replace the string {m0 ||...||mk−1 } so that the new message is M 0 = µ||mk ||...||mlast As in the fixed-point attack, appending the message length foils this attack, since H(M ||L) 6= H(µ||mk ||...||mlast ||length(µ||mk ||...||mlast )) even if they collide without the length.

1.5

Contributions and Thesis

The Hash Futures Panel at the Second NIST Hash Workshop in 2006 came to the consensus that cryptographers do not really know which features are desirable in a hash algorithm [72]. The framework described in this dissertation and in its predecessors [47,48] serves to resolve some of this uncertainty. We follow a practical approach to analyzing and designing cryptographic hash algorithms and break down these algorithms into preprocessing, iterative structure, combining functions, and

14 finalization components. A structured approach like that present in this model allows weaknesses in existing algorithms to be mitigated by adding or replacing components and gives new algorithms the opportunity to select strong individual components at the time of their design. We attempt to decompose the complexities of hash algorithms into smaller pieces which are individually easier to analyze. In the next chapter, we continue our discussion by formally defining the framework and its components. We limit our discussion to real-world algorithms (the standard model), but do discuss hash properties in the ideal model when interesting results apply to actual instantiations of cryptographic hash algorithms2 . Each component is described individually with lists of existing techniques, the attacks that they can foil, and known weaknesses. We then consider the security of a wide selection of modern cryptographic hash algorithms (including all the first-round candidates in the SHA-3 competition) by analyzing the properties of their individual components. In addition to analyzing the SHA-3 candidates, we also provide a description of how algorithms might be generated a la carte and on the fly based on the security properties that are required. We also discuss in the Appendix how the generic Application Programming Interface (API) [105] provided by NIST for the SHA-3 competition is well-suited to this framework. The four component model supports the typical cryptanalytic methodology of analyzing hash algorithms by inspecting and breaking weakened versions. Rather than approaching each hash algorithm as a monolithic structure, though, cryptanalysts can devote their energies to attacking one component at a time. If a piece of an algorithm is analyzed and found to be weak, one may be able to exploit the entire algorithm. On the other hand, attacks which affect a single component may be demonstrated to not be applicable by virtue of constraints imposed by other components. These 2

The ideal model offers one such construct called a random oracle (discussed more in Chapter 4), which represents a perfect cryptographic hash algorithm and can be used to describe upper bounds on the security of a particular construction.

15 other components have an opportunity, then, to shield the overall algorithm from some of the the weaknesses of the compromised component. This provides designers the ability to provide defense in depth within the algorithm itself. To borrow a term from [90], combining components that provide some redundancy in their security assurances leads to better, failure tolerant algorithms.

16

Chapter 2 The Framework

2.1

Overview

This framework divides cryptographic hash algorithms into four distinct components, which, while different than those typically selected in the construction of hash algorithms, provide a general, concrete method encompassing all cryptographic hash algorithms including (but not limited to) those following the Merkle-Damg˚ ard structure [102,45], the (extended) sponge framework [62], streaming hashes, and tree-based hashes. We begin with a short discussion of each of the individual components, then explore the application of the framework to the analysis, design, and classification of cryptographic hashes. Previous analytical frameworks have focused on specific components, like the compression function [43] or the iterative structure [111], while others have relied on certain assumptions, some concrete [102,45], some not [95,21]. This framework, on the other hand, can be used to represent any cryptographic hash algorithm. That is not to say, though, that these more specific frameworks are without a use, or that they were attempting to accommodate all algorithms in this way. Each will be described in the corresponding subsection of the four component framework as tools to analyze individual components. In the simplest terms, a hash algorithm H produces a fixed-length bit string (a hash value) of length n from an arbitrarily-long input bit string. If we have an input M , a collection of parameters p0 , ..., pk ∈ P , and the final hash value h, we describe a hash algorithm at a high level as H(M, P ) → h

17

m0

m1

mk-1 Final hash value

C

h0

C

h1

h2

hk-1

C

hk

Figure 2.1 The Merkle-Damg˚ ard Iterative Structure

To avoid repetition of the symbol P in every function, we assume hereafter that the collection of parameters is passed to every component or is otherwise available globally.

2.2

Previous Frameworks

The original framework for constructing a hash algorithm is the Merkle-Damg˚ ard (MD) iterated scheme, which can be described as the linear application of a compression function to a message block and the output of the previous compression and is illustrated in Figure 2.1. More concisely, the ith message block mi and the ith chaining value hi are passed as input to the compression function C to return the next chaining value: C(mi ||hi ) → hi+1 In this scheme, the only state per se is the chaining value passed to the next iteration. Once the final message block is processed in this way, the final chaining value is returned as the hash value for the entire algorithm. Sponge functions depart from the MD methodology, though they remain linear constructions that process the input message sequentially in blocks. Sponge functions begin with a large state into which message blocks are incorporated, then the state undergoes a permutation after which the next message block is added and the state is permuted again. Supposing that the state after the ith message block is called Si+1 , the function which combines the message block with the state is C, and the function

18

mi

Si

mi+1

C

P

Si+1

C

P

Si+2

Algorithm state

Figure 2.2 Sponge-style “Absorb” Phase

which permutes the state is P , this “absorbing” phase of a sponge function can be seen in Figure 2.2 and described as P (C(Si , mi )) → Si+1 This continues until the last message block mk−1 is added and the state undergoes its final permutation. At this point, the state (Sk ) undergoes the “squeezing” phase which involves calling the permutation function a number of times without message input and is illustrated in Figure 2.3. Every time the state is permuted, a part of the state is extracted via the function X and concatenated to the final hash value. (For the sake of illustration, X returns the least significant bits of S.) When the hash value reaches the desired size, the hash is returned. h = X(P (Sk ))||X(P 2 (Sk ))||X(P 3 (Sk ))|| . . . The Extended Sponge Framework (ESF) generalizes the notion of a sponge function by expanding the ways that a message can be introduced into the state as well as how the state produces the final hash value. Where the original sponge functions only combine message blocks with the exclusive OR (XOR) operation, the ESF offers the option of using either XOR or simply overwriting some part of the state. For generating output, sponge functions only use squeezing rounds, while the ESF offers squeezing rounds or truncation.

19

P

Sk

Sk+1

P

lsbk

Sk+2

Sk+b

lsbk+1

lsbk+b

Final hash value

h

Figure 2.3 Sponge-style “Squeeze” Phase

2.3

The Four Component Framework

Where the previous frameworks apply specific constraints on the operation, sequence, or internal state of cryptographic hash algorithms, this four component framework attempts to accommodate any structures that algorithm designers might introduce. This framework describes a hash algorithm as having four distinct components: a preprocessor, an iterative structure, a set of one or more combining functions, and a finalization. In a very similar fashion as shown in Figure 9.2b in [97], the function H is defined as the composition of several other functions, including a preprocessor π, an iteration scheme I∆ which uses combining functions δ0 , ..., δq ∈ ∆, and a finalization function Φ: H(M ) = Φ(I∆ (π(M )))

2.4

Components of the Framework

We now define each component in a way which clearly delimits “responsibility” for transforming the input into the output. While the combining function has long been considered to be the most important part of a hash algorithm and while its importance should not be understated, the other parts of calculating a cryptographic hash also prove to be very important.

20

M

Preprocessor π

M

S

Combining functions Δ

Iterative Structure IΔ

S Finalization Φ Final hash value h

Figure 2.4 Four Component Framework

21

2.4.1 Preprocessor The preprocessor is the first of four components to execute. In cryptographic hash algorithms like MD5 and SHA-1, the preprocessing is limited to a technique called MD-Strengthening, which pads the message and then appends the length of the message so that the processed message is a multiple of the block size used by the combining function, and prevents attacks such as fixed-point attacks which introduce additional data into the message. In the context of the overall framework, the preprocessing step operates only upon the message, returning a preprocessed message, but does not modify the internal state of the algorithm. We define the preprocessing step (π) as any portion of the algorithm that modifies the input message M without updating the internal state. ˆ π(M ) → M Like the security provided by the hashing step in an RSA digital signature, preprocessing can serve to disrupt homomorphisms or other exploitable algebraic structures in the algorithm’s combining function. Similarly, the preprocessor can “screen out” high-probability characteristics (see Chapter 4 for details) by inserting fixed or duplicated data. We discuss specific preprocessors in Chapter 3 and explain the security properties each provides.

2.4.2 Iterative structure The iterative structure along with its combining functions makes up the core of the cryptographic hash algorithm. We define the iterative structure (I∆ ) as a function which applies any number of combining functions from ∆ to a state S of length w and an input bit string M of a priori unknown length. In practice, most hash algorithms use only a single combining function and not a collection of functions for speed, memory, or complexity reasons. It stands to reason, though, that an

22 algorithm designer might choose to use multiple combining functions, as wee see later that several of the SHA-3 candidate designers did. In the event that the combining functions have a fixed-size domain, it is the responsibility of the iterative structure to act as a domain extension for these functions so that the algorithm may process an input of arbitrary length. When all message data is processed into the state, the “semi-final” state Sˆ is then returned for finalization. ˆ ) → Sˆ I∆ (M The Merkle-Damg˚ ard iterative structure is the best-known scheme, and it represents a linear iterative structure, since it applies its combining functions to sequential blocks of the message input. Tree-based iterative structures are also possible, where message blocks are placed at the leaves of the tree, and evaluated until the root of the tree is reached. These various iterative structures, along with their strengths and weaknesses, are described in detail in Chapter 4.

2.4.3 Combining Functions The combining functions in ∆ operate upon a subset of the message and a subset of the state to modify the state. We define the combining function as a function which takes a fixed-length message block and the current hash state as arguments and returns a modified hash state. (Note that unless we explicitly note otherwise, we assume that all inputs to a particular function are concatenated.) If some combining function δi operates on a message block of length d and the entire state (of length w), we can describe it as: δi : {0, 1}d × {0, 1}w → {0, 1}w In the MD iterative structure, the iterated function is called a compression function, a name which makes sense given that the normal input to this function is a reasonably small chaining value concatenated with a reasonably small message block,

23 and these functions simply compress the input to the length of the chaining value. We find this terminology limiting, since other schemes like the sponge framework [21] and RadioGat´ un [20] do not use their internal iterated functions the same way. Since both compression functions and a sponge’s iterated absorbing functions are responsible for combining the input with the chaining value or state, we decided that the term combining function was more apt. An MD-style compression function can be translated into a combining function by defining the subset of the message as the block mi and the subset of the state as the entire n bits of the state. (We use n bits here instead of w to illustrate that the typical “narrow-pipe” MD iterative structure has a state the same size as the final hash value.) Most cryptographers would conclude that the combining function is the basis of security in a cryptographic hash algorithm, and in many ways this is true, as it is usually the case that an attack on the combining function results in an attack on the entire algorithm. Combining functions are discussed alongside iterative structures in Chapter 4.

2.4.4 Finalization The finalization is responsible for transforming the algorithm state returned by the iterative structure into a final hash value of the desired length. When the state is the same length as the desired hash value (that is, w = n), as in MD5 [114], no finalization is necessary, but when the state is larger than the desired hash value, the state must be converted into a bit string of the appropriate size, typically by way of truncation. We define the finalization (Φ) as any operation which modifies the semi-final hash state Sˆ after the entire input stream has been processed by the iterative structure and produces a hash value of the appropriate size from any portion of the hash state

24 and input. ˆ →h Φ(S) The finalization function’s two chief responsibilities are to process Sˆ to create h in a similar way that a cryptographic hash algorithm processes M to create h, and second to produce a hash value of the desired length.

2.5

Application

We expand upon the assertion of [16] that one designs a hash algorithm in two steps: designing a compression function and specifying a domain extension transformation. We instead claim that one should design a cryptographic hash algorithm by choosing the four components that most closely fit the security and usability requirements of the application(s). This notion is explained in much more detail in Chapter 6 where many real hash algorithms are expressed using this model.

25

Chapter 3 Preprocessing

3.1

Introduction

Some recent attacks against common cryptographic hash algorithms SHA-1, MD5, and HAVAL (by Wang et al. in particular) have shown that it is possible to construct colliding messages for each of them in less time than the na¨ıve brute force method of simply hashing random blocks until two colliding inputs are found [137,138,129]. Several of these attacks dramatically reduce the search space for collisions by selecting messages with specific bitwise differences. For example, Suzuki and Kurosawa’s twoblock collision attack against 3-round HAVAL [129] can find a single collision in roughly 35 computations. The attack first requires that a “near collision” (a collision in some, but not all of the output bits) be found, using a particular input difference in the message block pairs mi ||mi+1 and m0i ||m0i+1 which can then be used to find fully colliding block pairs. This chapter focuses on preprocessing, the transformations of the input which operate strictly before the message is processed by the iterated combining functions. Preprocessing is a powerful way to prevent several known attacks against cryptographic hash algorithms and offers a method to “patch” vulnerabilities in hash algorithms after they are discovered, as in the case of the recent attacks. We show that preprocessing can not only reinforce resistance properties, but also reinstate some of these properties in broken algorithms. We can prevent Suzuki and Kurosawa’s attack by preprocessing the message in such a way that no message block can be constructed that produces the near collision required. In this example, mi ||mi+1 must differ from m0i ||m0i+1 in exactly one place

26 Content Positional Message Characteristic Fixed Parametric

Global example Padding Message Length Padding Prepend hash length n

Local example Block Position Bits So Far Whitening Per-block salt

Table 3.1 Examples of preprocessor content types in both localities

per message block: the most significant bit of the fifth word. To do this, we use “targeted whitening” as described in Section 3.4.5 and inject a word containing all zeros after the fourth word of every block, in a similar way that Szydlo and Yin suggested strengthening SHA-1 against a related attack [130]. This way, the near collision requirement would never be satisfied, regardless of the original input. The only drawback to this technique is that the speed of HAVAL decreases by about 3 percent.

3.2

Definitions

Within the four component framework, the preprocessor simply modifies the ˆ directly to the iterative structure. stream of input bits and passes its output M We define a preprocessor as the function π which takes a message M as an argument: ˆ π(M ) → M All the preprocessors described in this chapter can be categorized based on two qualities: locality and content. While there do not seem to be overarching security properties associated with these qualities themselves, the designations help in comparing similar preprocessors, particularly when the security of one is known better than the other.

3.2.1 Locality We define the locality of a preprocessor according to the size of its domain. Two distinct types were introduced in [130]: local preprocessors and global preprocessors.

27 Local preprocessors operate upon fixed-size blocks and can return their output without access to the entire message. Global preprocessors on the other hand operate on the entire message, and may require the entire message to produce any output whatsoever. An example of local preprocessing is message self-interleaving (see Section 3.4.6), which outputs two identical copies of each input byte, while MerkleDamg˚ ard Strengthening (see Section 3.4.2) is a global preprocessor since the entire message must be processed before its length is known. Note that the length of the blocks processed by a local preprocessor (b, and its output, b+) may not be the same as d, the length of the block processed by the compression function, as is described below in Section 3.2.3. Definition In a cryptographic hash algorithm, a local preprocessor is a function π λ : {0, 1}b → {0, 1}b+ which takes as an input a single b-bit block mi and returns a processed (b + )-bit block m ˆ i . may be any positive or negative integer or zero, depending on whether the preprocessor expands, contracts or permutes the message. This definition differs from that found in [130] in that we do not restrict the construction to only preprocessors which expand the input blocks. The local preprocessor may act as an expansion function, as a compression function itself and shorten the block, or return an output of the same length. The processed blocks are then passed to the iterated compression function in the place of the original message. Though by definition they are separate in the framework, local preprocessors producing blocks of length d (so b + = d) can be seen as extensions of the combining functions if such a distinction is necessary or desired. Conversely, portions of the combining function that operate on the message without updating the state, such as SHA-1-style message expansion, can be defined as local preprocessing.

28 Definition In a cryptographic hash algorithm, a global preprocessor is a function π γ : {0, 1}∗ → {0, 1}∗+ which takes as an input message M of indeterminate length L and returns a processed ˆ of length L ˆ = L + . may be any positive or negative integer or zero, message M depending on whether the preprocessor expands, contracts or permutes the message. The primary difference between local and global preprocessors is that a global preprocessor operates upon the entire message before it can return its processed result, while its local counterpart requires only a single input block. Applications of hash algorithms which have inputs of indeterminate length at the time of evaluation are said to have a streaming requirement. While the streaming requirement typically means that only local preprocessors are appropriate, it is possible that a global preprocessor only reads a block at a time until it produces its result. In particular, a global preprocessor of this sort that maintains counters or state variables of its own can be used with applications having the streaming requirement.

3.2.2 Content The content quality describes how the preprocessor produces its output. Since we expect that most preprocessors return parts of the message unmodified, we ignore those portions and instead focus on the added or modified parts. We divide content into four types: positional, message characteristic, fixed, and parametric. Positional preprocessors operate in tandem with the iterative structure and modify input blocks based on their position in the overall structure. Thus, a preprocessor which adds a field to each block that equals “1” when it is the last block and “0” otherwise would be positional. Positional preprocessors can prevent attacks such as length extension which disrupt the position of message blocks. Since the positional style can be combined with any of the other content styles, some preprocessors (like

29 the previous example, which is combined with the fixed quality) can be said to have multiple qualities. Preprocessors that use message characteristics are those that use some property of the input message or the message itself. An example of this is Merkle-Damg˚ ard Strengthening, which appends the length of the message to the message itself. Message characteristic preprocessors introduce redundancy and dependencies among the input bits. In the case of Merkle-Damg˚ ard Strengthening, it makes any attack that changes the length of the message affect the length field in the last block, which it turns out is sufficient to defeat several generic attacks. We mention here for clarification that we would not consider a preprocessor of this type which returns an ˆ ) as a message characteristic preprocessor. unmodified message (that is, M = M Fixed preprocessors inject data that is constant, or at least unrelated to the inputs or parameters of the algorithm. Whitening, for example, simply inserts a fixed data block at intervals into the message. This behavior is unchanged if the message itself or algorithm parameters change. Fixed preprocessors produce inputs where an attacker has no control over some subset of the message bits, which can often disrupt differential attacks and other attacks that require carte blanche access to the input bits. Parametric preprocessors modify the data in a way that is either specified by algorithm parameters or contains the parameters themselves. A straightforward parametric preprocessor is one that prepends the hash length n to the message. The serves to make invocations of the algorithm with different values of n produce different hash values, and may defend against attacks in the first block in a similar way as a fixed preprocessor. Like positional preprocessors, those using parameters can be combined with other qualities. For example, a parametric/message characteristic preprocessor might be a input byte repetition scheme (“message self-interleaving” - see Section 3.4.6) where the parameter defines the length of the byte blocks to be repeated.

30

M

M

Permutation

M

M Encoding

(with 2 impossible values)

M

M

Compression (with 2 colliding values)

Figure 3.1 Preprocessors as Domain Transformations

3.2.3 Preprocessing as a Domain Transformation Modifying a string by preprocessing can have three distinct effects on its length: increase it ( > 0), decrease it ( < 0), or leave it the same ( = 0). When ≥ 0, the preprocessor should be an injective function. That is, there should be no inputs which produce the same output. If there are, a many-to-one relationship like this can result in trivial collisions in the overall algorithm and should be avoided. When < 0, the preprocessor must be collision resistant itself and should have b ≥ n to avoid introducing collisions into the overall algorithm. Since finding a collision in a b

function with an output of b bits requires O(2 2 ) operations, algorithm designers using preprocessors with < 0 must be sure that the difficulty of finding a collision in the preprocessor is at least as difficult as finding them in the overall algorithm. One might argue that preprocessing the input block just shifts the attacks to other input choices. This is true for preprocessors that simply permute the input block to other values and can be inverted, but those that inject redundant data or compress the block invariably transform the domain of the compression function in ways that may preclude particular attacks.

31

3.3

Properties and Observations

3.3.1 Preprocessors and Property Preservation When describing the security of a cryptographic hash algorithm, the underlying components are often assumed to exhibit some ideal behavior so that the other parts of the algorithm can be shown to preserve the desirable properties of the ideal component. In most cases, the component assumed to be ideal is the combining function and the component to be analyzed is the iterative structure. A particular iterative structure is said to be X-preserving (or just X-Pr) if, when the combining functions possess the property X, an algorithm using this iterative structure also possesses the property X. In Chapter 4, this notion of “property preservation” is applied to the iterative structure as just mentioned, but we can also use it to describe the benefits of a particular preprocessor. The preservation of security properties like collision resistance is strongly dependent on the preprocessor function. It is usually claimed that the Merkle-Damg˚ ard structure is collision resistancepreserving (CR-Pr), though that claim is somewhat misleading. The “basic” MD structure simply iterates the compression function until the last message block, and without some preprocessing, it is simple to produce collisions in this basic structure. For example, if we assume that the final compression function operates on an incomplete block and fills in any incomplete parts with “0” bits, a message M and the message M 0 = M ||0 collide when hashed. If padding is added to the end of the ˆ that is unambiguous (usually padding with a “1” message M that produces an M bit, followed by as many “0” bytes as are necessary to fill the block), this is no longer a problem. We now have ˆ = M ||1||000...000 6= M ˆ 0 = M ||0||1||00...000 M and can expect that H(M ) 6= H(M 0 ). An even better preprocessor would include an encoding of the length of the message in the padding to prevent slightly more sophis-

32 ticated attacks. These brief counterexamples demonstrate that while the iterative structure is necessary to preserve the collision resistance of the underlying combining function, the preprocessing step is also necessary. In fact, many schemes offered as iterative structures or domain extensions for combining functions are little more than the “basic” MD structure with additional preprocessing. HAIFA [25], for example, supplements the MD structure with salt and “bits so far” fields in every block, which improves the algorithm’s resistance to second preimage attacks [28]. Preprocessors can also contribute to the preservation of other properties, as described in [51].

3.3.2 Preprocessing as Encoding and Differential Attacks Preprocessors play an important role in the resistance of cryptographic hash algorithms to differential attacks [130]. Existing differential attacks require the attacker to determine candidate preimages that satisfy certain conditions based on how the combining function mixes the input bits. (See Chapter 4 for more details on differential attacks.) Attackers typically seek inputs where changes in specific bits are canceled out later in the combining function by changes in other input bits. By introducing additional conditions through preprocessing, the difficulty of attacks can be increased proportional to the number of bits added to the input by the preprocessing. Control over the precise formats of inputs can be made more difficult by introducing constants and dependencies between input bits. A simple preprocessor is often sufficient to defeat a known attack and ensure f -resistance for a particular f . Theorem 3.3.1 An f -attack against a combining function δ which requires some fixed, finite set of conditions in any subset of bits in an input block can be prevented by a local preprocessor. Proof Suppose we have an f -attack called A on a combining function δ. Assuming that, in order to find a set of inputs which violate δ’s f -freedom, A requires that some subset (of e bits) of the d input bits be constant or a function of the remaining input bits d − e. To defeat A, we choose values for each of the e specific bits that will

33 explicitly violate A’s criteria. For example, if A specifies that the ith and j th bits of the input had to always be equal, a simple preprocessor (covered in more detail in Section 3.4.5) could inject constant bits into the input (and shift the remaining input bits) such that the ith bit is always zero and the j th is always one. Similar constructions can be formulated for any set of attack criteria.

3.3.3 Preprocessor versus Iterative Structure block size When b + 6= n, attackers seeking differential attacks may be able to find blocks where the bytes affected by the preprocessor are shifted into a more favorable position. For example, if the preprocessor adds a zero byte after every eighth byte, we have b = 64 and = 8. If d = 80, the fixed byte will be in the ninth position in the first block, in the eighth in the second, the seventh position in the third block, and so on. A worse situation occurs when b + > n, since there will be blocks which will have no injected bytes and therefore lose the benefit of the preprocessor against differential attack. This “staggering” of the preprocessed block boundaries with the combining function’s block boundaries may be beneficial in some cases, but in attacks where the attacker requires control over the structure of the blocks, this staggering may allow attackers to find appropriate block replacement points.

3.4

Schemes

In this section we describe some straightforward local and global preprocessing techniques. Note that most of these techniques insert redundant data into the message which can make differential cryptanalytic attacks more difficult. Due to the way the four component framework is laid out, preprocessing steps which set initial states or other algorithm constants by way of parameters (and not message bits) are not included. That type of operation is the responsibility of the component using the constants in question.

34 Scheme Padding Message Length Salt Bits So Far Whitening Message Self-Interleaving Message Duplication Block Position Block Permutation Parameters Local Tagging Global Tagging Checksums/EDC/ECC Compression

Locality Global Global Global/Local Local Local Local Global Local Local Global/Local Local Global Global/Local Global/Local

Content Fixed/Message/Positional Message Parametric Message Fixed Message Message Positional Message Parametric Message Message Message Message

Table 3.2 Breakdown of various preprocessing schemes

3.4.1 Padding Since the length of most messages are not conveniently a multiple of the length of a message block in an iterated hash scheme, messages must be padded so that the iterated scheme can be applied to complete message blocks. This padding method must be unambiguous - that is, it should never be the case that for two messages M and M 0 and padding function π where M 6= M 0 , π(M ) = π(M 0 ), or collisions can be found trivially, regardless of the strength of the iterated combining functions. One straightforward, unambiguous padding scheme is appending a “1” bit, then as many “0” bits as necessary to fill a block. A second unambiguous (though slower, and less frequently used) scheme is to pad the message with z zeroes to fill a block, leaving room for an r-length binary representation of z. Padding schemes need not be fixed preprocessors, though. For example, the padding used by the Blender algorithm [29] repeats input bits from the message as padding.

35

3.4.2 Message Length/Final Block Length Appending the message length to the message itself (after a suitable amount of padding as described above) is a well-established technique used by most modern cryptographic hash algorithms, and is often referred to as Merkle-Damg˚ ard Strengthening or MD-Strengthening [98,45]. MD-Strengthening defends against second preimage, trivial free-start collision (where the attacker can choose the previous state), and fixed-point second preimage and collision attacks. It also prevents collision attacks which find colliding messages of different lengths. Some might argue that, since this is a part of the MD5 and SHA-1 standards, it is not appropriate to refer to it as a preprocessor, but in algorithms without such a feature, it can be bolted on in a similar way as any other preprocessor. An alternate method, which requires very short messages or non-streaming access to the message is to prepend the message’s length to the message. This produces a prefix-free encoding of the message, compared to MD-Strengthening which produces a suffix-free encoding. Another alternate method is to append only the final block’s length to the end of the message, the benefit of which is that the algorithm needs not track the overall length of the message. To achieve the same level of security as MD-Strengthening, though, using the final block length in this way should only be used in conjunction with techniques that protect against block insertion and deletion like a “Blocks So Far” field as described below (which, of course, requires a counter to track the length of the message).

3.4.3 Salt Salts are random, short strings of bits that were originally used to increase the difficulty of dictionary attacks against stored passwords whose hashes were publicly accessible. Each bit of the salt increases the difficulty of an offline dictionary attack by a factor of two. Salts also defend against multicollisions [25] and herding attacks

36 [79] requiring precomputation of intermediate state values. Several algorithms in the SHA-3 competition include the salt as a parameter so that the application is not required to explicitly include the salt as part of the message. Depending on the implementation, the salt may also be used as a parameter, perhaps passed to the combining functions by the iterative structure to simulate a “family” of hash algorithms as required for certain security proofs.

3.4.4 Bits So Far/Blocks So Far Introduced as part of the HAIFA Framework [25], “Bits So Far” (bSF) involves inserting a fixed-size bit string of length containing the number of input bits that the algorithm has processed to that point into each block. “Blocks So Far” (BSF) is a variant which instead counts the number of message blocks, and can be viewed as a block index or counter, though its effect on the security of the algorithm is nearly the same as a bitwise counter. Length extension attacks such as those described in [49] and its generalization in [80] can be prevented by this sort of message preprocessing. It has also been shown in [28] that the bSF field improves the second preimage resistance of the Merkle-Damg˚ ard construction from 2n−τ to 2n , where τ is the number of message blocks (and intermediate state values) in the message. Assuming that differential attacks can be found in the combiner function, the bSF preprocessor can introduce additional delays to those attacks. The colliding blocks must first be identical in the bytes allocated to the counter, since replacing a single block does not modify the block or bit count. Therefore, single-block differential attacks may be possible if the message is long enough. As the number of possible block values increases, so does the number of the attacker’s possible insertion points. Bits so far remedies this to a certain extent since some bit count values may be impossible to produce without being at the last block. A field counting bits is usually more dynamic between blocks than one counting blocks since half the time, the block count

37 will change in only one bit. To illustrate this, if we assume that one of the 2 possible values in the BSF field are chosen uniformly at random when an attacker produces a candidate colliding input block, this block will only be useful in the message block mi where i is equal to the random value in the BSF field. If it is instead a bSF field, this block can only be used as the final block unless the value of the bSF field is divisible by d − . Thus, an attacker with a sufficiently long message M can much more easily find a collision while using a BSF field than a bSF field, which will tend to allow collisions only in the final block. It may even be a better idea to have the bits or blocks count be the seed for a pseudo-random number generator, since attackers may not know where their candidate colliding blocks would actually fit in the message, unless they are able to easily determine the seed from the generator.

3.4.5 Whitening Whitening is the process of adding blocks of fixed bytes at certain intervals or at certain positions within a message block and was introduced in [130]. This is a straightforward method to introduce roadblocks to attackers seeking differential attacks, but this defense is at the expense of the algorithm’s speed. For example, introducing one fixed byte after every message byte would effectively double the size of the message and require roughly twice as much time to evaluate. The argument can be made that whitening improves diffusion by spreading out the message data across more combining function iterations. Two basic types of whitening exist: targeted and periodic. Periodic whitening inserts the fixed block of data every so many bytes, until the end of the message is reached. Targeted whitening focuses on particular portions of the input block and inserts the fixed block at specific positions, usually with the intent of breaking up known differential attacks.

38

3.4.6 Message Self-Interleaving Also introduced in [130], message self-interleaving resembles whitening except that blocks of the message are duplicated and inserted (or self-interleaved) instead of fixedvalue blocks. The redundancy introduced by repeating blocks of the message serves to foil differential attacks against the combining function which rely on particular characteristics or differential trails. As with whitening, the drawback to this method is that the rate of the algorithm decreases as the number of added bits increases. Pair block chaining is a special case of message self-interleaving where each message block is concatenated with the following message block, as described in [14] and depicted below: πPBC (m0 ||m1 ||...||mk ) → m0 ||m1 ||m1 ||m2 ||...||mk The message blocks are paired up so that each message block will be operated upon by two combining function iterations. This scheme is also used for “incremental hashing” where it is necessary to have the ability to update the hash value when the message changes without requiring a complete re-hashing of the message.

3.4.7 Message Duplication This could be described as a special case of message self-interleaving, where the entire message is appended to itself (before or after padding) and the entirety is hashed. This preprocessor ensures that every message bit affects at least two different combining function invocations, which may defeat typical collision and preimage attacks.

3.4.8 Block Position This is an addition to a message block that indicates its position in an iterative or tree structure. In algorithms which use linear iterative structures like DCH [141] and Sg`ail [96], Block Position differs from Blocks So Far in that the positional indicator

39 bytes are only added to the final block or other blocks in key positions, which can produce a prefix-free encoding [39,40]. This technique defends against block insertions which shift the position of the marked block and as a prefix-free encoding, it can assist in the preservation of properties in the combining function and iterative structure.

3.4.9 Block Permutation Selective block permutation based on the block’s position can be considered to be a part of the preprocessing step when it operates independently from the compression function such as in Spectral Hash [120], which permutes only the first block. If the permutation is difficult to invert, attacks which rely on backtracking to a previous state may be more difficult. Aside from the operations required to perform the actual permutation, this preprocessor does not affect the number of combining function invocations.

3.4.10 Hash Length and other Parameters As previously mentioned, parameters are often used to determine IVs and other constants in the algorithm. According to our framework, however, that would put this use of parameters into the realm of the iterative structure. Injecting the block size, number of rounds, height of a hash tree, or the like into the message serves not only to defend against attacks requiring known states, but also differential attacks targeting the block containing the parameters.

3.4.11 Tagging Tagging, introduced in [78], inserts redundancy into the input stream in a slightly different way than the previous techniques and offers the option for both global and local preprocessing. The premise behind local tagging is that each block (or every xth block) will be passed through a (possibly different) hash function or MAC, the output of which is both prepended and appended to the message block. Global tagging is

40 similar in that it hashes the entire message first, then prepends and appends the resulting hash value to the original message at which point the processed message is then hashed again to produce the final hash. As a global preprocessor, tagging, like message duplication above, is only feasible if the algorithm has non-streaming access to the message, can read the message twice in its entirety, or has enough memory to cache the entire message.

3.4.12 Checksums and Error-Detecting Codes Checksums, error-detecting codes, and error-correcting codes are the integrityguaranteeing ancestors of modern cryptographic hash algorithms. Encoding schemes can be seen as a generalization of preprocessors that expand the message block. For example, message self-interleaving can be described as a repetition code, and Blender [29] appends checksums based on sums of all message words. The redundancy added by encoding a message can improve the resilience of the hash algorithm against differential attacks as they introduce dependencies between input bits and consequently increase the dependence of output bits on the input bits. We can apply one of many error-correcting codes into each message block to add a well-understood form of redundancy to the message. The benefit of this method over other techniques like message self-interleaving is that an error-correcting code requires many fewer bits. The drawback, of course, is that the code might not change if the number of bits a differential attack changes exceeds the number of bits the code was designed to correct or detect. This drawback can be mitigated by choosing codes with a high minimum distance that are efficient to calculate, as discussed in [76].

3.4.13 Lossless Compression While this scheme does not make sense as a local preprocessor, an algorithm designer might opt to run a compressor like bzip2 [124] over the entire input before passing it to the iterative structure. Provided that the compression algorithm can

41 process streaming data, this global preprocessor can be used on input which are only available a portion at a time. Besides potentially reducing the length of the message, compression algorithms generate unambiguous outputs. Additionally, compression may result in a prefix- or suffix-free encoding of the message.

3.5

Conclusion

We look to preprocessors to achieve certain goals: 1. They should not increase the probability of a collision in the algorithm 2. They should defeat any specific known or trivial attacks 3. In conjunction with the iterative structure, they should preserve desired security properties of the combining function 4. They should reduce the probability of differential attacks Combining function designers should work to find probable characteristics in their functions, if not to redesign and fix the flaws, then to explicitly preprocess away the vulnerabilities. The “patch” on the base algorithm that preprocessing offers may come at the cost of a reduction in the algorithm’s speed, since the preprocessor may inflate the input or have a nontrivial execution time. One unanswered question remains about how to maintain f -resistance with the least amount of preprocessing overhead. We posit without proof that a careful choice of preprocessors and a reasonably secure combining function can guarantee any fixed finite set of arithmetic freedom properties. An interesting research direction that would answer this would be to prove which choice of π can prevent specific f -attacks. Similarly, we might also conclude that if the combining function does not provide the desired security properties, preprocessing steps can be introduced that explicitly destroy any structure in the overall hash algorithm which might violate the security properties.

42

Chapter 4 Iterative Structure and Combining Functions

4.1

Introduction

Previous works discussing the practical construction of cryptographic hash algorithms have described the process as occurring in two steps: choosing a compression function, then selecting a mode of operation which allows the algorithm to accept inputs of arbitrary lengths and preserve the desirable properties of the compressor. These two components have been largely considered to be the whole of the algorithm, a perspective that this work seeks to shift by removing the preprocessing and finalization steps from the iterative structure (with which they are usually bundled). We instead define the iterative structure and its combining functions as the core of every cryptographic hash function, but not the entirety. Definition A combining function is a function δ which takes as inputs a dδ -bit message block and an wδ -bit state and outputs an updated wδ -bit state, or more formally: δ : {0, 1}dδ × {0, 1}wδ → {0, 1}wδ Where it is not explicitly noted, we maintain the order of the combining function arguments with the message input first, the state second. Though combining functions may be defined as operating on all w bits of the state (where wδ = w), it is often the case that the iterative structure only presents a subset of the state bits (so wδ < w) to the combining function for either performance or strictly architectural reasons. Note also that different combining functions may operate on different length message blocks, so we might have dδi 6= dδj with two combining functions δi 6≡ δj .

43 Since the combining functions are tasked with the majority of the “heavy lifting” in the algorithm, it stands to reason that the combining functions must possess the security properties requisite in a cryptographic hash algorithm. Each of these functions are therefore expected to possess at a minimum the security properties discussed in Chapter 1: collision resistance, preimage resistance, and second preimage resistance. Given that a combining function has no persistent state of its own and fixed-length inputs and outputs, these security properties need to be redefined to be appropriate to these functions. This is typically done by defining the combining function-specific properties pseudo-collision resistance and pseudo-preimage resistance which give attackers control not only over the message input, but also the state input. A pseudo-collision attack on δ is one that produces a wδ -bit state S and two dδ -bit message blocks mi and mj such that δ(mi , S) = δ(mj , S) in fewer than 2wδ /2 operations [97]. Similarly, a pseudo-preimage attack, given a state S, produces a state S 0 and a message block mi such that δ(mi , S 0 ) = S in fewer than 2wδ operations [97]. Note that in these examples, we assume that wδ ≤ dδ , though if this is not the case, the attack complexity is bounded instead by the message block size dδ . While these attacks do not always result in a direct attack against the overall algorithm, it has been proven that a pseudo-preimage attack on the combining function (that uses the entire w bits of the state) requiring fewer than 2n−2 operations can be translated into an effective attack on algorithms using the MD iterative structure [97]. Definition An iterative structure is a function which takes an arbitrary-length input and updates an w-bit internal state by calling one or more combining functions from the set δ0 , ..., δq ∈ ∆ as black boxes. I∆ : {0, 1}∗ → {0, 1}w In addition to calling combining functions to process the input, the iterative structure is also responsible for performing any necessary initialization and then maintain-

44 ing the internal state between combining function invocations. The purpose of the iterative structure is to use the combining functions in the most secure manner to update the state while maintaining performance goals. This component creates a variable input length (VIL) transformation from one or more fixed input length (FIL) transformations. Iterative structures can be differentiated primarily by the way they call the combining functions and secondarily in the size of their states and the way they choose or compute their constants. The size of the internal state is extremely important, since it provides an upper bound on the resistance of the algorithm against attack as discussed in Chapter 1. Also important is the number of state bits passed to each iteration of the combiner functions, because it may be possible to produce pseudo-collisions via brute force attack if the combiner’s inputs and outputs are too small. The algorithm’s constants are also an important factor in its security, as it was shown in [36] that ignoring the constants in an AES-based cryptographic hash algorithm resulted in collision attacks faster than the birthday attack.

4.2

Properties and Observations

4.2.1 Combining Functions The design of compression functions has been extensively studied [110,35], possibly more than any other aspect of cryptographic hash algorithms, largely because of their similarity to block ciphers, which are far better understood. It is not our intention to break new ground here in the design and analysis of combining functions. Instead, we discuss in a general sense the properties which make a combining function suitable for use in a secure cryptographic hash algorithm. For most of the rest of this paper, though, the combining functions are treated as black boxes in our four component framework. Clearly, a combining function plays a large role in establishing the resistances of a hash algorithm, as a weak combiner may expose the algorithm to single-block attacks for which the other components might not be able to compensate without undue

45

mi

d bits

Si-1

wδ bits

δ

δ

δ

δ

δ

Si

δ* Figure 4.1 A “supercombiner” transforming five iterations of δ into a single iteration of δ ∗

performance degradation. Most algorithms’ security is therefore defined in terms of the security of their combining functions and how the other components support this security, as discussed further in Section 4.5. When evaluating combining functions, it is helpful to expand their boundaries beyond the structures described in algorithm specifications. We investigated this briefly in Chapter 3 when we encountered error-correcting codes as preprocessors and their near-equivalence with combining functions’ message expansion steps. Transformations within combining functions are often executed iteratively a certain number of times, with each iteration called a round. We might reuse this notion of internal iteration by joining multiple combining functions into what we call a “supercombiner” as depicted in Figure 4.1. Each combiner within the supercombiner has the same access to the message input, but each in turn passes its modified internal state to the next. This provides a convenient structure to evaluate the supercombiner as a whole object rather than a few separate objects which individually might not have the desired properties. We can use this idea to show how structures like sponge-based hash algorithms can be massaged into a more familiar construction. Theorem 4.2.1 A sponge-style hash algorithm (as defined previously) can be instantiated as a wide-pipe Merkle-Damg˚ ard iterated algorithm with a finalizaton function. Proof In the sponge construction, each message block is combined with the state in

46 two distinct steps. The first step, which we will call “injection” and denote by δINJ : {0, 1}d × {0, 1}d → {0, 1}d where the first argument is the d message bits, and the second is the d-bit portion of the state with which the message bits will be combined. As defined in [21], this combining function applies the XOR operator to the two inputs to return the updated portion of the state. The second step is denoted by δPERM : {0, 1}w → {0, 1}w which applies a permutation to the state that has had the message block injected. Note that this combining function requires no input message bits. We then define the supercombiner δSPONGE to be the sequential application of first δINJ to a d-bit portion of the state, and then the application of δPERM to the entire state. If we then define I as the Merkle-Damg˚ ard construction, and ∆ = {δSPONGE }, I∆ → Sˆ is a representation of the iterative structure of a sponge function. Since sponge functions have a much more elaborate output function than MD (which in fact has no output processing in narrow-pipe constructions), we define a separate finalization function ˆ →h ΦSQUEEZE (S) This function wraps the entire “squeezing” phase of the sponge construction to abstract away the details of how the state is transformed into the output hash value.

4.2.1.1

Basic Properties

Combining functions have inherited several important basic requirements from block ciphers, including completeness, the Strict Avalanche Criterion (SAC), and balance, all of which are discussed thoroughly in Chapter 8 of [110]. A function is complete when every output bit depends on every input bit [77]. The SAC takes this a

47

S

δ

PERM

S

δ

PERM

lsb

S

S

lsb

lsb

h

h

ΦSQUEEZE

Figure 4.2 The squeezing phase of a sponge function as a single finalization function ΦSQUEEZE

step further and requires that, when a single input bit is complemented, every output bit should change with a probability that approaches one-half [140]. A function’s balance is measured by evaluating the Hamming weight of all possible outputs. A balanced function is one whose outputs are made up of equal numbers of ones and zeroes. In a balanced function f : {0, 1}n → {0, 1}n , we expect that HW(f ) = 2n−1 . Combining functions that do not produce a high level of diffusion (dependency of the output bits on the input bits) and therefore do not possess these three properties will often fall victim to differential attacks, as described in the next section.

4.2.1.2

Differential Properties

Differential attacks are attempts to determine collisions in specific combining functions based upon how the combiner produces output bits from its input bits. We first define a delta (not to be confused with δ as used above to describe a combining function) as the bitwise difference between two binary strings, usually by applying the exclusive-OR (XOR) operator. Differential attacks seek predictable correlations between deltas in input bits and deltas in output bits. These correlations are referred to as characteristics [23] and are expressed in terms of the probability that a desired output delta will be effected by an input delta. Combining functions which have characteristics of high probability are at risk of attack by differential attacks.

48 Since most combining functions operate as a few fast operations repeated for a certain number of rounds, differential attacks tend to be focused upon the changes that occur in the intermediate outputs between rounds. Inputs which tend to produce the same intermediate values after a certain number of rounds may be used as a starting point for finding inputs which result in collisions in the final hash value. In combining functions based on block ciphers or having the same sort of internal iterations, increasing the number of rounds often increases the difficulty of differential attacks, as more and more dependencies are introduced between the input and output bits. The task of finding optimal characteristics has been shown to be equivalent to finding minimal weight codewords in a linear code [94], which is known to be an NPComplete problem, so cryptanalysts often search for shortcuts like so-called neutral bits which do not affect the differences in the intermediate states for a certain number of initial rounds. By removing these bits from consideration in the characteristic search, the probability of finding a good characteristic is increased. Using this neutral bit technique, Biham and Chen produced a near-collision attack on SHA-0 in 240 operations [24]. De Canniere et al. [32] improved upon this and offered a technique for the automated search for differential paths.

4.2.1.3

Algebraic Structures and Homomorphisms

Homomorphic cryptographic hash algorithms are those algorithms which satisfy the following for some function f : H(f (M0 , M1 , ..., Mq )) = f (H(M0 ), H(M1 ), ..., H(Mq )) As previously mentioned, the provably collision resistant hash function VSH1 is susceptible to attack due to its homomorphic structure [119]. Suppose M0 , M1 , and 1 VSH is collision resistant if one assumes that finding nontrivial modular square roots of a very smooth number is a difficult problem

49 M2 are input messages of equal length, M0 ∧ M1 = M2 , and M2 is a string of all zeroes, then: H(M2 )H(M0 ∨ M1 ) ≡ H(M0 )H(M1 ) (mod n) Functions like this are clearly not f -resistant for a reasonably simple f . By mixing operations using different algebraic structures, such as modular arithmetic, exclusiveOR operations, and nonlinear AND and OR operations, the combining function may be shielded against all but the most complex attacks which exploit homomorphisms.

4.2.2 Iterative Structure The iterative structure defines the algorithm’s state and how the set of functions represented by ∆ operates upon processed message bits to modify the state. The long message attack (as outlined in Chapter 1) that produces second preimages is a good example of an attack that can be foiled by the proper choice of iterative structure. Since the attack relies on finding state collisions between combining function iterations, the iterative structure must contain a state that is sufficiently large so that brute force attacks against these intermediate states are of higher complexity than 2n so they cannot be effectively used to attack the algorithm as a whole. The ordering of the message block bits and the state bits is also part of the operation of the iterative structure. We might, for example, effectively create a new combining function from an existing one by rearranging the input bits or might provide domain separation by using either a different set of constants or a separate state in a combining function iteration. This is precisely the technique that the “Enveloped Merkle-Damg˚ ard” and “Enveloped Shoup Transform” use to create a different final combining function from the primary combiner. On the other hand, we might use the previous supercombiner construction to simplify the iterative structure’s internal function calls.

50

4.3

Iterative Structure Schemes

Many proposed iterative schemes require a certain preprocessor or finalization function, but when these schemes are stripped of these other components, the number of unique schemes is reduced dramatically. Proposed schemes can be divided roughly into linear and tree-based schemes and then further distinguished by the presence of other modifying features.

4.3.1 Linear Iterative Schemes Linear iterative schemes are the original and most common iterative structures, and involve applying combining functions to sequential blocks of the message (usually all of the same size). The typical linear scheme, the MD Construction, is well-known to preserve the collision resistance of its combining function, though it does not guarantee the preservation of preimage resistance or second preimage resistance [7]. Linear constructions which do preserve these properties are briefly discussed below and described in more detail in [16,7,6].

4.3.1.1

Narrow-pipe Schemes

We differentiate the basic linear schemes by the size of their states. Narrow-pipe constructions (like MD, depicted in Figure 4.3) have a state that is roughly equal to longest possible hash length. Schemes that maintain a large state where w ≥ 2n are usually called wide-pipe [90]. Designers of cryptographic hash algorithms have traditionally assumed that the length of the chaining value (what we describe here as the state) was the same length as the final hash value. This may have been due to the fact that block ciphers were used as compression functions and it was easier to prove properties on structures where w = n. These narrow-pipe hashes have some significant drawbacks because of the small size of their internal states. The length of the state, w, sets an upper bound on the security of the algorithm against attacks

51

m0

m1

d bits

S0

n bits

δ

mk-1

δ

S1

S2

Sk-1

δ

S*

Figure 4.3 A Narrow-pipe Linear Iterative Structure

such as multicollisions and second-preimages in long messages which require about w

w

min{2 2 , 2n(K−1)/K } and min{2 2 , K2n } operations, respectively.

4.3.1.2

Wide-pipe Schemes

To maintain the same basic Merkle-Damg˚ ard iterative structure while increasing the size of the state and subsequently the resistance against these attacks, wide-pipe and `-pipe constructions were developed. Wide-pipe constructions utilize a combining function (or supercombiner made of up of several combining functions) which operate upon the entire w-bit state, where w > n, and a final compression reduces the final state/chaining value to the requisite n bits. (Finalization functions such as this final compression step are discussed in detail in Chapter 5.) `-pipe constructions are a special case of the wide-pipe design which uses the original n-bit combining functions, but arrange them in ` parallel levels to accommodate a state where w = `n, as depicted in Figure 4.4. In the double-pipe (` = 2) example given in [90], the final message block and the two parallel chaining values are passed to a single combining function, and the final n-bit hash is returned. If the iterative structure organizes its state and combining functions as two parallel but separate “tracks” (unlike the `-pipe construction), the final “cascaded” hash value becomes vulnerable to multicollision attacks on the output as if there were two distinct hashes [75]. This illustrates the importance of not keeping these sub-states separate and instead ensuring that there

52

m0

m1

mk-1

d bits

S0

n bits

δ

S1

δ

δ

S' 1

δ

n bits

Sk-1 S' k-1

δ

S*

d bits n bits

S' 0

n bits

Figure 4.4 An `-pipe Linear Iterative Structure for ` = 2

is a fair amount of data crossing over between the different portions of the state. In iterative structures that have w = 2n, and are part of “double block length” algorithms [69], the double block combining function may be made up of a pair of block cipher-based combining functions each defined as the mapping δBC : {0, 1}d × {0, 1}n → {0, 1}n (where d = 2n) Several constructions fit this double block length model, including Tandem-DaviesMeyer, Abreast-Davies-Meyer [88], and MDC-2 [1]. It has been recently proven that the Tandem-Davies-Meyer construction (using AES as the underlying block cipher) provides a lower bound on the complexity of collision attacks at 2120.4 operations, which is reasonably close to the upper (birthday) bound of 2128 [56]. The approach in [56] (based upon [128]) to evaluating lower bounds on attacks against this sort of construction is an endorsement for reserving the cryptographic hash algorithm’s complexity in the combining function, rather than in the iterative structure, and also provides a strong endorsement for constructions of this type.

53

Si-1

δ

+

Si

+

S' i

Mi

δ

S' i-1

Figure 4.5 Tandem Davies-Meyer

4.3.1.3

Sponge Construction

Structures based on the sponge construction are also linear but use injection followed by permutation combiners with a large internal state instead of compression functions (see Theorem 4.2.1). The sponge construction is usually described as producing its output by “squeezing” as described in Chapter 2, but that falls into the discussion on finalization functions, where it is covered in more detail. These constructions can be very efficient given their large states, though they may be vulnerable to “slide attacks” [62] without other countermeasures in place.

4.3.2 Tree Iterative Schemes A simple “hash tree” was originally described by Merkle in [100] to capitalize on its parallelizable structure for speed while maintaining collision resistance preservation. The Merkle hash tree arranges message blocks at the leaves of a binary tree where they are hashed. The results are then concatenated and are passed as input to the parent nodes, as shown in Figure 4.6. Andreeva et al. [7] improved upon this and

54

m0

d bits

δ δ

m1

d bits

δ δ

m2

d bits

S*

δ δ

m3

d bits

δ Figure 4.6 A Merkle Hash Tree

offer a different (keyed) construction, the “modified XOR tree”, which preserves both collision resistance and (second) preimage resistance. Though these proposals use binary trees exclusively, there is no reason to restrict the discussion to only binary tree structures. Indeed, SHA-3 proposals MD6 [112] and SANDstorm [131] introduce 4-ary and 10-ary trees, respectively, which result in shallower and wider trees, and improve their parallelizability.

4.3.3 Modifying Features 4.3.3.1

Multiple Combiners

Most algorithms use just a single combining function, but some use more. In the Enveloped MD structure, for example, a different combining function (actually, the same combiner is used, but the inputs are shifted) is used for the final block to produce iterative structures which are pseudorandom oracle-preserving [16]. Other iterative structures might choose to use sequences of different and unique combining functions so that the two functions disrupt one another’s algebraic structures.

55

m0 +

δ

L

m1 +

key0

δ

+

S*

m2 +

δ

key1

key2

m3

Figure 4.7 A Modified XOR Hash Tree

4.3.3.2

Periodic Permutations

State permutations may improve mixing in algorithms that use combiners on fixed subsets of state bits. Sponge functions, for example, can be described as applying a combining function, then a periodic permutation between the combining function invocations. Note that the difference between this permutation and the block permutation preprocessor in Section 3.4.9 is that here the entire state is subject to permutation rather than just a portion of the message in the preprocessor.

4.3.4 Proposed Structures Given the above schemes, we now discuss some of the iterative structures discussed in [17] and [39] to see how they break down on this framework. (We do not analyze the actual algorithms until Chapter 6)

4.3.4.1

Strengthened Merkle-Damg˚ ard

This structure is simply a narrow-pipe linear construction with a Merkle-Damg˚ ardStrengthening preprocessor applied to the input.

56

m0

m1

mk-1

d-n bits

d bits

IV1

n bits

δ

S1

δ

Sk-1 S2 IV2

n bits

δ

S*

Figure 4.8 The Enveloped Merkle-Damg˚ ard Iterative Structure

4.3.4.2

Prefix-free Merkle-Damg˚ ard

This scheme uses the Merkle-Damg˚ ard iterative structure and applies a prefix-free encoding preprocessor to the message.

4.3.4.3

Enveloped Merkle-Damg˚ ard

As depicted in Figure 4.8, this is otherwise identical to Strengthened MD, but the final combining function is different from those iterated through all the previous message blocks [16].

4.3.4.4

Shoup’s Transform

This is also a linear construction, but a fixed number of keys (or a expanded and scheduled single key) are XORed with the state before every compression function iteration. (See Figure 4.9) If we cast this XOR operation as an easily-invertible permutation (given the keys), this scheme can be described as an otherwise typical Merkle-Damg˚ ard-style iterative struction with a parameter-driven periodic permutation before every combining function iteration. A variation is the “Enveloped Shoup Transform” which takes the same basic structure and adds a different final combiner, as described in Enveloped MD above. Andreeva et al. [5] produced a generic second preimage attack on Shoup-style constructions and other similar “dithered” hash function structures. A later proposal

57

m0

m1

δ

S0

+

κ0

+

mk-1

δ

+

κ1

δ

S*

κk-1

Figure 4.9 Shoup’s Transform

from Andreeva et al. [8], relying more heavily on keying and using message bits from multiple blocks in single combining function invocations, is immune to such attacks.

4.3.4.5

NMAC and HMAC

The NMAC construction, as described in [39], is simply the MD construction with a finalization function. The HMAC construction, discussed in the same work, adds a full block worth of zeroes to the message during preprocessing, then applies the MD iterative structure, then applies a finalization using the same combining function as in the iterative structure.

4.3.4.6

3C

In the 3C structure, for example, one half of its 2n state is equal to k M

δ(mi , Si )

i=0

as shown in Figure 4.10. Once the final message block is processed, this “checksum block” is processed as an additional message block. It has been shown, however, that adding XOR-based (or other linear) checksums in this way adds virtually no additional security [58]. Attacks similar to finding multicollisions can be launched on these constructions to determine colliding values for the entire algorithm in approximately

58

m0

m1

mk-1 Finalization

d bits

S0 S' 0

δ

n bits

n bits

δ

S1

⊕

S' 1

S2

⊕

δ

Sk-1

⊕

δ

h

S' k-1

Figure 4.10 The 3C Hash’s Iterative Structure

as many operations as a collision attack against the state without the additional checksum bits. We note that the GOST hash algorithm uses a similar structure as 3C, though since it replaces the intermediate chaining values with its message blocks, its checksum can be categorized as preprocessing, as described in Section 3.4.12:

4.4

Combining Function Schemes

These are by no means exhaustive lists. Of all of the four components, combining functions are the most difficult to define and enumerate exhaustively, since the majority of the algorithm’s complexity resides here. We expect combining functions to behave as FIL versions of the entire algorithm. With that in mind, combining functions tend to have their own set of parameters, structure, and styles. Combining functions might vary in the number of rounds they execute, the size of their input block (d), the strategy they employ to produce their output, or the specific internal components they use.

4.4.1 Execution Strategy Traditional cryptographic hash algorithms have reused block ciphers as compression functions, but that is just one of several execution strategies that algorithm designers have proposed. Here are several.

59

4.4.1.1

Block Cipher-Based Combiners

Block cipher-based combiners use their underlying cipher to encrypt the algorithm’s state with each message block. Preneel notes in [110] that combining functions based upon block ciphers may be vulnerabile to attacks afflicting the block cipher. Since block ciphers are bijections (with a fixed key) and any party with access to the message blocks (like an attacker seeking second preimages) can apply the decryption operation with the original key, it is fairly easy for an attacker to “backtrack” through the message from the hash value unless other countermeasures are in place. These vulnerabilities may be exploited to produce collisions, preimages, or f -attacks in less time than brute force. A detailed analysis of the most secure block cipher-based constructions to avoid attacks such as this can be found in [111].

4.4.1.2

Inject-Permute Combiners

Where block ciphers combine the message block and state in a single step, combiners in sponge functions operate in two distinct steps: injection, where the message is typically exclusively-ORed with some equal-sized portion of the state followed by permutation, where the newly-updated state is permuted to diffuse the inserted message data.

4.4.1.3

Concatenate-Permute-Truncate Combiners

This style of combiner found in both the Snefru [101] and Grindahl [84] algorithms closely resembles the sponge-style combiners. They differe in that they operate with three phases: concatenation, permutation, and truncation. The message block is first concatenated to the internal state, the result is permuted, and a fixed-size portion of the result of the permutation is returned as the next state.

60

Li

Ri

f

k i (m)

+ Li+1

Ri+1

Figure 4.11 A Balanced Feistel Network

4.4.1.4

Belt-And-Mill Combiners

These combiners separate the state into two substates - the belt and the mill [20]. After the message bits are added, the belt portion of the state undergoes a linear transformation, while the mill undergoes a nonlinear transformation. During each iteration of these transformations, bits from each of these substates are exchanged so that they are tightly coupled and not subject to the same sort of multicollision-style attacks previously mentioned.

4.4.2 Subcomponents Within each combiner, designers tend to use a handful of tried-and-true subcomponents, most of which were enumerated in Fleischmann et al’s evaluation of the SHA-3 candidates [55]. We briefly discuss each of them below.

4.4.2.1

Balanced and Unbalanced Feistel Network

Commonly used in block ciphers like DES, balanced Feistel networks [123] divide the state into two halves, each of which is successively transformed via a keyed non-

61 linear function f , then the two are combined and the halves are swapped, as depicted in Figure 4.11. A typical round might be described as Si → Li ||Ri (fki (m) (Ri ) ⊕ Li )||Li → Si+1 This continues for a certain number of successive rounds until the state is sufficiently combined with the message, which acts as the key when Feistel networks are used in hash algorithms. Unbalanced Feistel networks differ in that the input is divided into two parts of unequal length.

4.4.2.2

Message Expansion

Message expansion is a technique used in the compression functions of algorithms like SHA-1 and MD5 to improve the diffusion of the message bits across the internal rounds. It is desirable for this message expansion to be an encoding with a high minimum distance (as briefly mentioned in Chapter 3). Though finding the minimum distance of a linear code is NP-hard [133], codes suitable for use in message expansion with good performance and high minimum distances have been described in [76].

4.4.2.3

Maximum Distance Separable Matrix

Maximum Distance Separable (MDS) matrices [117] provide a linear substitution and are used to produce a high level of diffusion in the output bits. This technique was popularized in the Rijndael algorithm [44] which was chosen as the Advanced Encryption Standard, and applied to cryptographic hash algorithms in the Whirlpool algorithm [13].

4.4.2.4

S-boxes

The design and analysis of the Data Encryption Standard block cipher spurred many to evaluate DES’ method for introducing nonlinearity: substitution boxes. For performance considerations, they are usually implemented as lookup tables, where

62 the number of input bits exceeds the number of output bits. In this way, they can be seen as lightweight non-collision-resistant compression functions.

4.4.2.5

Feedback Shift Registers

The output of each iteration of a feedback shift register (FSR) is determined by a transformation of the input and the previous state. In an nonlinear FSR, this transform function is nonlinear, and in a linear FSR, the transform is linear [122,86].

4.4.2.6

Transforms With Provable Properties

By leveraging known-hard problems, several proposals for combining functions have included proofs of security (often for collision resistance) based on the assumption that the problems are in fact computationally infeasible to solve. Specific problems like the knapsack problem, discrete logarithms over elliptic curves, finding short vectors in lattices, finding isogenies between elliptic curves, and decoding random linear codes have been propsed in [99], [31], [9], [37], and [11], respectively.

4.5

Conclusion

In order to describe the security of a cryptographic hash algorithm’s iterative structure and combining functions, we must analyze the security of how the components interoperate. A common method for evaluating the overall security of a domain extension of combining functions is to describe its capability for property preservation [39,16,7,17]. Proofs of property preservation usually begin with the assumption that the underlying combining function possesses some ideal security property, say collision resistance, then show that the overall algorithm possesses the same security property when a particular iterative scheme is used. Let δ be an X-resistant combining function. If I is an X-resistance preserving iterative structure/domain extension that uses δ, then Iδ must be collision resistant. It follows then, that to design a cryptographic hash algorithm with a set of prop-

63 erties X0 , ..., Xq , we should first strive for a combining function that provides these properties (or nearly provides these properties, if it is possible to correct the behavior with preprocessing or finalization). Once we have this combining function possessing X0 , ..., Xq , we should then build an iterative structure, preprocessor, and finalization function that provably preserves (or reinstates) each of these properties. We have described previously how the choice of preprocessor can reinforce the security of the “core” of the iterative structure and combining functions. The first and arguably most important of the properties to preserve is collision resistance, though that property seems to be preserved only by choosing an appropriate preprocessor. While the MD structure with MD-Strengthening preserves the collision resistance of a compression function, it does not necessarily maintain its preimage resistance or balance [15]—structures such as the aforementioned `-pipe hash, on the other hand, do help to enforce the algorithm’s preimage resistance. The iterative structure itself has the capability to preserve at least one specific property of its combining functions: indifferentiability from a random oracle [95]. Random oracles are often used as idealized (and not possible to instantiate in reality) hash algorithms when described as variable-input-length random oracles (VILROs) and as idealized combining functions when described as a fixed-input-length random oracles (FIL-ROs). A random oracle is a theoretical construction which, used as a public function and given an input, produces an infinitely-long output of random bits that is truncated as necessary. Should the random oracle be provided with the same input as one it has previously operated upon, it will produce the same infinitely-long string of bits since its state is maintained across all queries. Though random oracles cannot be instantiated in the real world, we hope to be able to create combining functions which simulate all of their important behaviors. We say that such a function is indifferentiable from a random oracle, and call such functions “pseudorandom” oracles. We say that an iterative structure that preserves this prop-

64 erty is pseudorandom oracle-preserving (PRO-Pr). While finalization functions like truncation (see Section 5.3.1) can produce a PRO-Pr algorithm, the iterative structure can also be PRO-Pr by using a different final combiner, as seen in the EMD and Enveloped Shoup constructions. An important distinction to make is that though a combining function that is a pseudorandom oracle is also collision resistant, an iterative structure that is PRO-Pr is not necessarily collision resistance preserving [16].

65

Chapter 5 Finalization

5.1

Introduction

Finalization is the process of converting the state produced by the iterative structure into the final hash value h. It is the simplest component, as its security requirements are less stringent and it has known fixed-length inputs and outputs. In fact, most algorithms in use today (MD5 and SHA-1, for example) do not even include a finalization component. The length of the final hash value, n, is an extremely important parameter of cryptographic hash algorithms, and in a similar way as the size of the state, the upper bound on the security of any hash algorithm is based upon n. When n = w, finalization is not necessary, but if n < w, one or more of the techniques described in Section 5.3 are required. A finalization function operates only upon the semi-final state Sˆ and any parameters and produces the final hash value h: ˆ →h Φ(S) Any finalization function that requires part of the message is not by our definition a finalization and belongs in the iterative structure.

5.2

Properties and Observations

Intuitively, it seems that it would be beneficial for the final state to be as securely generated as the output of any other FIL compression function. The reality, though, is that the simplest construction, truncation, can produce an indifferentiable algorithm with a compression function simulating a random oracle and the typical MerkleDamg˚ ard iterative structure [51]. In the case where the underlying functions are not pseudorandom oracles, however, other more complicated finalization functions which

66 have better security properties (and which may be pseudorandom oracles themselves) may supplement the properties of the other three components. As described in [30], the goals of finalization functions can be split into introducing “discontinuity” into Sˆ and producing the output in the form of a hash value of the appropriate size. The advantages provided by having a state larger than the hash value, namely improved resistance to generic multicollision and preimage attacks, necessitate finalization to reduce the large state to the desired n-bit hash value h. Finalization does not positively affect the collision resistance of the algorithm if colliding inputs producing the same Sˆ can be found, since the finalization will produce the same h. Finalization can, however, mitigate the risk of near-collisions in the state. Even if it is possible to produce a collision in some of the state bits, applying a transformation resembling a compression function dramatically reduces the probability that the near-collision in Sˆ will result in a near (or full) collision in h. On the other hand, the security of some iterative structure/combining function combinations can be compromised by finalization such as truncation. [83] shows that, in certain double block length constructions, truncation can reduce the difficulty of collision and preimage attacks to 2m/3 and 2m/2 , respectively, where m represents the number of bits that ˆ are retained from S.

5.3

Finalization Schemes

5.3.1 Subset-of-State In those algorithms that produce a state which is larger than the desired hash value, a simple way of producing the hash is by returning some subset of the state bits unmodified. This finalization’s most straightforward variant is truncation (sometimes called the “chop construction”) which returns the most or least significant n bits of Sˆ as h. Other algorithms might select a more complicated subset of state bits such as choosing every xth byte of the state. In [40], the authors explain that finalization by truncation can defeat attacks on

67 MACs which are based on length extension by hiding a portion of the final hash. Returning only a portion of the state bits in this way illustrates the importance of having combining functions that also produce outputs which are collision resistant in the specific bits that are retained after truncation.

5.3.2 Final Permutation and Final Compression Applying a permutation to the state introduces discontinuity, which may be useful in disrupting algebraic attacks as described in Section 1.2.2. Since permutations of the state do not actually reduce its size, other finalization functions that produce the proper length output are required. Applying a final compression is one way to do that, and in addition to reusing combining functions from the iterative structure, other simpler options like XORing portions of Sˆ with itself may also be fast and effective. Final compressions, like truncation, can also produce property-preserving behavior. For example, the HMAC construction applies the internal combiner as a final compression, with the state as the message and the initial state S0 as the state to produce a pseudorandom oracle-preserving algorithm. Another benefit of a final compression is that it makes an attacker’s job of working backwards from h to determine a candidate Sˆ as difficult as inverting the compression function, which is a property useful in cryptographic hash algorithms being used as MACs, as mentioned in the previous section.

5.3.3 No-Input Iterations No-input iterations are a special case of a final permutation that applies combining functions without message input a certain number of times. Since it is a permutation, its sole purpose is to separate the hash value as much as possible from the semi-final state. Variants of these no-input iterations include “blank rounds”, which substitute all-zero blocks for the message inputs, “constant rounds” which, like blank rounds, use a fixed value other than zero, and “incremented rounds” which use a counter value

68 that increments after every block. The benefit of using iterations of this sort as a permutation is that a dedicated permutation function does not need to be developed and existing combining functions can be reused.

5.3.4 Pumping Rounds Pumping rounds (also known as “squeezing rounds” in sponge functions, as depicted in Figure 2.3) are otherwise identical to the above no-input iterations except some subset of the intermediate state is output as part of the hash value after each iteration. The algorithm typically calls only enough pumping rounds to produce a hash value of the desired length.

5.3.5 Pseudorandom Number Generators Pseudorandom number generators (PRNGs) may not possess security properties that would allow them to be used as hash functions themselves, but they may be effective in translating Sˆ into h. A PRNG uses Sˆ as a seed, then produces as many output bits as are desired, in a manner resembling pumping rounds.

5.4

Conclusion

Finalization can be seen as a variation of combining functions, since they operate on inputs and outputs of a fixed size. They do not, however, require all of the same properties as a combining function. Truncation is known to be sufficient for indifferentiability, and a final compression (provided that it is a pseudorandom oracle itself) is also sufficient for PRO-Pr. Looking at the ratio of bits in Sˆ compared to the number of bits in h leads to an interesting question. How much should the semi-final state be reduced to produce the hash value? It seems fairly clear that having w ≥ 2n is a good starting point, since generic multicollisions can be of ideal difficulty, and the chop construction’s requirements for having a “super-logarithmic” [51] number of the state bits is also satisfied. Whether that number is too high (or too low) remains

69 an open question. We might ask whether it is better to calculate a large internal state and truncate it or generate as much output as needed using pumping rounds. These produce outputs with the same maximum security, as a semi-final state of a particular size is still required for a secure h. We note that finalization techniques like pumping rounds can ˆ but any additional bits beyond the first w do produce an h which is larger than S, not factor into the security of the output. Put another way, the algorithm’s security is bound above by the smaller of w and n.

70

Chapter 6 Analysis and Conclusions

We have detailed some specific techniques that each component can use to contribute to the overall security of the algorithm. Table 6.1 shows some of the requirements to preserve certain properties in the combining functions. This approach to decomposing cryptographic hash algorithms into their component parts allows us to present high-level comparisons of very different algorithms and can allow us to quickly pick out those that do not have certain desirable features.

6.1

Negative Features

It is worthwhile to look at the other side of the coin and also examine in which circumstances a component could reduce the security of an algorithm to such a degree that the other components may not be able to compensate. A preprocessor can damage an algorithm by removing any bit from the input in the production of its output. The iterative structure can damage an algorithm by making any single bit of the input not affect the state or by insecurely calling combining functions. Combining functions can affect security negatively by not possessing the required security properties, or by making any output bit independent from any input bit. The finalization function can harm the algorithm by producing a hash value with bits that are not ˆ dependent on S.

6.2

Algorithm Analysis

We show now that all SHA-3 candidates and a handful of other well-known algorithms can be expressed in this generalized framework. Note the following about the tables:

71 Property to be Preserved Requirements Collision Resistance Preprocessor: MD-Strengthening or Bits So Far Preimage Resistance Finalization: Truncation Second Preimage Resistance Iterative Structure: w ≥ 2n Indifferentiability (PRO-Pr) Preprocessor: Prefix-free encoding or Iterative Structure: Different Final Combiner or Finalization: Truncation or Final Compression Table 6.1 Requirements for Property Preservation in the Four Component Framework

Constants and their derivations are not mentioned, even when parameters are used. We consider constants to be part of their respective components, but otherwise outside of the scope of these tables. An initial state S0 , for example, is derived as part of the iterative structure. The size of the states and specific output lengths are not specified, except in general terms. Only adjectives like “narrow pipe” and “wide pipe” are used to indicate the size of the state compared to the length of the final hash value. The hash value’s length is not indicated at all, except when explicitly part of the algorithm’s name (e.g. RIPEMD-160, which produces a 160-bit output). Note, though, that all SHA-3 candidates support output lengths of 224, 256, 384, and 512 bits. The exact divisions that are made here may not exactly match those specified by the algorithm designers. For example, functions described as “finalization” in several candidates (as a different final combiner) are included in the iterative structure component by virtue of their use of message bits and updating of the state. A true ˆ finalization similar to this would be “final compression”, which is only applied to S. Another potentially confusing distinction arises in SHABAL’s three final permutations that occur after the message is fully processed [30]. In this four component model, the permutations are categorized as part of the preprocessor since part of the input is used, then later passed to a typical iteration of the combining function. If

72 Abbreviation Pad MDS S BSF Wh MSI BPos Pe Pa ECC

Preprocessing Technique Padding Merkle-Damg˚ ard Strengthening (or equivalent) Salt Bits/Blocks So Far Whitening Message Self-Interleaving/Repeated Final Block Block Position Periodic Permutation Hash Length/Parameters Checksum/Error Correcting and Detecting Codes

Table 6.2 Legend for Preprocessor Abbreviations

the inputs to these functions were instead fixed or not based on the message, the finalizations would be named appropriately.

6.2.1 Preprocessing We begin with evaluating the preprocessing techniques in use by over 60 different cryptographic hash algorithms. Traditional algorithms like MD5 use only padding and MD-Strengthening, but many algorithms go beyond these two techniques. Table 6.2 provides an explanation to the abbreviations used in all of the following tables describing the algorithms’ preprocessors.

73 Algorithm MD5 [114] SHA-1 [103] SHA-224 [125] SHA-256 [125] SHA-384 [125] SHA-512 [125]

Pad X X X X X X

MDS X X X X X X

S

BSF

Wh

MSI

BPos

Pe

Pa

ECC

Pa

ECC

Table 6.3 Preprocessing Techniques in Common Algorithms Algorithm RIPEMD-128 [50] RIPEMD-160 [50] RIPEMD-256 [50] RIPEMD-320 [50] Whirlpool [13] Grindahl [84] MD4 [113] HAVAL [145] PANAMA [42] RadioGat´ un [20] Tiger [4]

Pad X X X X X X X X X X X

MDS X X X X X X X X

S

BSF

Wh

MSI

BPos

Pe

X

X

Table 6.4 Preprocessing Techniques in Other Hash Algorithms Algorithm Abacus [126] ARIRANG [34] AURORA [74] BLAKE [12] Blender [29] Blue Midnight Wish [60] Boole [116] Cheetah [81] CHI [68] CRUNCH [63] CubeHash [19] DCH [141] Dynamic SHA [143] Dynamic SHA2 [144] ECHO [18] ECOH [31] Edon-R [61] EnRUPT [107] ESSENCE [92] FSB [11] Fugue [64] Grøstl [59] Hamsi [108] JH [142] Keccak [22] Khichidi-1 [135] LANE [73] Lesamnta [70] Luffa [33] LUX [104] MCSSHA-3 [93]

Pad X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

MDS X X X X X X X X X X

S X

BSF

Wh

MSI

BPos

Pe

Pa X

X X

X X X X

X

X X X X

X

X

X

X X

X X X X X X X X X X X

ECC

X X

X

X X

X

X X

Table 6.5 Preprocessing Techniques in SHA-3 Candidates, Part 1

74

Algorithm MD6 [112] MeshHash [52] NaSHA [91] SANDstorm [131] Sarmal [134] Sg` ail [96] Shabal [30] SHAMATA [10] SHAvite-3 [26] SIMD [89] Skein [53] Spectral Hash [120] StreamHash [132] SWIFFTX [9] Tangle [2] TIB3 [109] Twister [54] Vortex [85] WaMM [139] Waterfall [67]

Pad X X X X X X X X X X X X X X X X X X X

MDS X X X X

X X X

S

BSF

X

X

X X X X X X X

X X X

MSI

BPos X

Pe

X

X X X X

Wh

X X X X

X X X

X X

X

X

X

X X

Pa X X

X

Table 6.6 Preprocessing Techniques in SHA-3 Candidates, Part 2

ECC

75

6.2.2 Iterative Structure The typical iterative structure in use is Merkle-Damg˚ ard, a linear, narrow-pipe construction. Exactly half of the proposed SHA-3 algorithms that are linear are narrow-pipe, with the other half wide-pipe. Only a small handful use the tree construction, multiple combiners, or periodic permutation. Moreso than any other component, the SHA-3 candidates are fairly homogeneous in their iterative structures.

76 Algorithm MD5[114] SHA-1[103] SHA-224[125] SHA-256[125] SHA-384[125] SHA-512[125]

Lin. X X X X X X

Tree

N. Pipe X X X X X X

W. Pipe

> 1 Comb.

Permutations

Table 6.7 Iterative Structure Techniques in Common Algorithms Algorithm RIPEMD-128[50] RIPEMD-160[50] RIPEMD-256[50] RIPEMD-320[50] Whirlpool[13] Grindahl[84] MD4[113] HAVAL[145] PANAMA[42] RadioGat´ un[20] Tiger[4]

Lin. X X X X X X X X X X X

Tree

N. Pipe X X X X X

W. Pipe

> 1 Comb.

Permutations

X X X X X X

Table 6.8 Iterative Structure Techniques in Other Algorithms Algorithm Abacus[126] ARIRANG[34] AURORA[74] BLAKE[12] Blender[29] Blue Midnight Wish[60] Boole[116] Cheetah[81] CHI[68] CRUNCH[63] CubeHash[19] DCH[141] Dynamic SHA[143] Dynamic SHA2[144] ECHO[18] ECOH[31] Edon-R[61] EnRUPT[107] ESSENCE[92] FSB[11] Fugue[64] Grøstl[59] Hamsi[108] JH[142] Keccak[22] Khichidi-1[135] LANE[73] Lesamnta[70] Luffa[33] LUX[104] MCSSHA-3[93]

Lin. X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

Tree

N. Pipe

W. Pipe X

> 1 Comb.

Permutations

X X

X X

X X X X X

X

X X X X X X X X X X

X X X X X X X X X X

X X X

X

Table 6.9 Iterative Structure Techniques in SHA-3 Candidates, Part 1

77

Algorithm MD6[112] MeshHash[52] NaSHA[91] SANDstorm[131] Sarmal[134] Sg` ail[96] Shabal[30] SHAMATA[10] SHAvite-3[26] SIMD[89] Skein[53] Spectral Hash[120] StreamHash[132] SWIFFTX[9] Tangle[2] TIB3[109] Twister[54] Vortex[85] WaMM[139] Waterfall[67]

Lin.

Tree X

X X

N. Pipe

> 1 Comb.

Permutations

X X

X X X X X X X X X X X X X X X X

W. Pipe X X X

X X X X X X X X X X X X

X X

X X

X X

Table 6.10 Iterative Structure Techniques in SHA-3 Candidates, Part 2

78 Abbreviation BC IP CPT BFN UFN ME MDS SB FSR Pr

Combining Function Technique Block cipher-based Inject-Permute (sponge style) Concatenate-Permute-Truncate Balanced Feistel network Unbalanced Feistel network Message expansion Maximum distance separable matrix Substitution box (Non)Linear feedback shift register Combining function with provable properties

Table 6.11 Legend for Combining Function Abbreviations

6.2.3 Combining Function Table 6.11 provides an explanation to the abbreviations used in all of the following tables describing the algorithms’ combining functions. The majority of the content in these tables (in fact, columns 4 through 9) were originally found in [55]. The first three “execution style” columns and the last “provable” column are novel. The vast majority of proposed algorithms are block-cipher-based, and most also use s-boxes and message expansion. MDS matrices are also popular, with 17 algorithms using structures resembling (or directly using) AES’ matrices.

79 Algorithm MD5[114] SHA-1[103] SHA-224[125] SHA-256[125] SHA-384[125] SHA-512[125]

BC X X X X X X

IP

CPT

BFN

UFN X X X X X X

ME X X X X X X

MDS

SB

FSR

Pr

FSR

Pr

FSR

Pr

Table 6.12 Combining Function Techniques in Common Algorithms Algorithm RIPEMD-128[50] RIPEMD-160[50] RIPEMD-256[50] RIPEMD-320[50] Whirlpool[13] Grindahl[84] MD4[113] HAVAL[145] PANAMA[42] RadioGat´ un[20] Tiger[4]

BC X X X X X

IP

CPT

BFN

UFN X X X X

ME X X X X X

X X X

X X

X

X

X

MDS

SB

X X

X

X X X

X

Table 6.13 Combining Function Techniques in Other Algorithms Algorithm Abacus[126] ARIRANG[34] AURORA[74] BLAKE[12] Blender[29] Blue Midnight Wish[60] Boole[116] Cheetah[81] CHI[68] CRUNCH[63] CubeHash[19] DCH[141] Dynamic SHA[143] Dynamic SHA2[144] ECHO[18] ECOH[31] Edon-R[61] EnRUPT[107] ESSENCE[92] FSB[11] Fugue[64] Grøstl[59] Hamsi[108] JH[142] Keccak[22] Khichidi-1[135] LANE[73] Lesamnta[70] Luffa[33] LUX[104] MCSSHA-3[93]

BC

IP X

CPT

BFN

X X X X X

UFN X X X

ME X X X X

MDS X X

SB

X

X X

X X X X X X X X X X X X X

X X

X X X

X X

X X X

X

X X X X

X X X

X X

X

X X X

X X

X X

X X X

X

X X X

X

X

X X X

X X

X X

X

X X X X

X X X X X

Table 6.14 Combining Function Techniques in SHA-3 Candidates, Part 1

80

Algorithm MD6[112] MeshHash[52] NaSHA[91] SANDstorm[131] Sarmal[134] Sg` ail[96] Shabal[30] SHAMATA[10] SHAvite-3[26] SIMD[89] Skein[53] Spectral Hash[120] StreamHash[132] SWIFFTX[9] Tangle[2] TIB3[109] Twister[54] Vortex[85] WaMM[139] Waterfall[67]

BC X

IP

CPT

BFN

UFN

ME

MDS

SB

X X

X X X X X

X X X X X X X X X X X X X X X X X X

X X X

X X X

X

X X X X X X

Pr

X

X X X X

X X X X

X

FSR X

X X

X X X X X X X X X

Table 6.15 Combining Function Techniques in SHA-3 Candidates, Part 2

X

81 Abbreviation Trunc SoS FP FC NII PR

Finalization Function Technique Truncation Subset of state (not truncation) Final permutation Final compression No-input iteration Pumping rounds

Table 6.16 Legend for Finalization Function Abbreviations

6.2.4 Finalization Table 6.16 provides an explanation to the abbreviations used in all of the following tables describing the algorithms’ finalization functions. Finalization is not usually necessary for traditional cryptographic hash algorithms, since they all operate in the narrow-pipe model and their internal state was returned unmodified as the final hash value. The most common finalization of the SHA-3 candidates is truncation, with 38 algorithms using it in some capacity.

82 Algorithm MD5[114] SHA-1[103] SHA-224[125] SHA-256[125] SHA-384[125] SHA-512[125]

Trunc

SoS

FP

FC

NII

PR

X X

Table 6.17 Finalization Techniques in Common Algorithms Algorithm RIPEMD-128[50] RIPEMD-160[50] RIPEMD-256[50] RIPEMD-320[50] Whirlpool[13] Grindahl[84] MD4[113] HAVAL[145] PANAMA[42] RadioGat´ un[20] Tiger[4]

Trunc

SoS

X X

X

FP

FC

NII

PR

X X X X

X X

X

Table 6.18 Finalization Techniques in Other Algorithms Algorithm Abacus[126] ARIRANG[34] AURORA[74] BLAKE[12] Blender[29] Blue Midnight Wish[60] Boole[116] Cheetah[81] CHI[68] CRUNCH[63] CubeHash[19] DCH[141] Dynamic SHA[143] Dynamic SHA2[144] ECHO[18] ECOH[31] Edon-R[61] EnRUPT[107] ESSENCE[92] FSB[11] Fugue[64] Grøstl[59] Hamsi[108] JH[142] Keccak[22] Khichidi-1[135] LANE[73] Lesamnta[70] Luffa[33] LUX[104] MCSSHA-3[93]

Trunc X X X

SoS

FP

FC

NII X

PR X

X

X

X

X X X X X X X X X X X X X X X X X X

X

X X X X

X X X X

X X

X X

X

Table 6.19 Finalization Techniques in SHA-3 Candidates, Part 1

83

Algorithm MD6[112] MeshHash[52] NaSHA[91] SANDstorm[131] Sarmal[134] Sg` ail[96] Shabal[30] SHAMATA[10] SHAvite-3[26] SIMD[89] Skein[53] Spectral Hash[120] StreamHash[132] SWIFFTX[9] Tangle[2] TIB3[109] Twister[54] Vortex[85] WaMM[139] Waterfall[67]

Trunc X

SoS

FP

FC

NII

PR X

X X X X X X X X X X X X X X X

X

X X

X

Table 6.20 Finalization Techniques in SHA-3 Candidates, Part 2

84

6.3

Algorithm Analysis: SANDstorm

The SANDStorm algorithm [131] is a submission to the SHA-3 competition by Sandia National Labs, and serves here as a good example of a candidate which, while complex and very different from traditional algorithms, still fits in well within this four component framework. While we describe the components of SANDstorm in more detail than the other SHA-3 candidates, we do not discuss all of the specifics of the algorithm’s operation, but instead opt to explain its components generally.

6.3.1 Preprocessing SANDstorm uses the usual MD-Strengthening of padding followed by appending the message length. It also includes a counter on blocks that increments from 1 to 4 which, while it is described in the algorithm specification as being part of the chaining value, is classified as a very short “Blocks So Far” field in this framework.

6.3.2 Iterative Structure The structure is a 10-ary tree, whose depth is expanded to a maximum of 4, which only occurs when the message is longer than 1000 blocks (of 512 or 1024 bits, depending on the desired output length). This structure not only improves parallelizability, but also offers the ability to do fast incremental hashing [14] should the hashes of large inputs which differ in only a few blocks be required. The drawback to this structure, like other tree structures, is the additional storage overhead required to track the intermediate states in parent nodes.

6.3.3 Combining Function The combining function used in SANDstorm is a fairly straightforward block cipher-based function that uses message expansion and s-boxes.

85

6.3.4 Finalization At the root of the tree, the accumulated state bits have a final compression applied to them to generate a hash value of the desired size.

6.4

Recommendations

We envision two purposes for the classification: as a suggestion for an algorithm construction technique and to focus cryptanalyses on components rather than on monolithic algorithms, particularly when so many share structural components. As they seek the best candidate in the SHA-3 competition, our recommendation to NIST is that they choose the best combining function(s) from all of the existing candidates, then select the other three components a la carte. This will allow a candidate with a strong combining function but weak components to lend its strength to an overall stronger algorithm with an iteration style, preprocessor, or finalization quite different from its own proposal.

6.4.1 Recommended Construction We suggest the following very loose sequence for designing iterated cryptographic hash algorithms: 1. Determine the security and usability properties desired based on the applications’ requirements. 2. Determine the hash length(s) and length of the internal state based on the desired properties of the algorithm. 3. Choose or design an iterative structure which supports the chosen state size and preserves the relevant security properties 4. Choose or design combining functions which provide the relevant security properties and can operate on the portions of the state determined by the selection of the iterative structure.

86 5. Choose or design preprocessing steps (that interoperate with the iterative structure, if necessary) that preserve the relevant security properties. 6. Choose or design finalization steps which operate upon the semi-final output, provide the required hash length, and preserve the relevant security properties.

6.5

Conclusion

This paper describes a modular model for cryptographic hash algorithms which allows the algorithm to be evaluated for security properties by comparing their individual components: preprocessor, iterative structure, combining function, and finalization. The structure allows designers to design the algorithm with “defense in depth” in mind, so that individual components found to be weak do not necessarily compromise the entire algorithm. It also allows for drop-in replacement of any piece in a way that should not severely impact applications already using the algorithm. Cryptanalysis also benefits from the division into four components since reducedstrength constructions become much more straightforward for any algorithm. Theorem 6.5.1 According to the definitions in Chapter 1, the assertion of [97] that: ...a compression function secure against preimage, 2nd-preimage, and collision (fixed IV) attacks is necessary and sometimes, but not always sufficient for a secure iterated hash. is false. Proof We show that we can construct a secure iterated hash which uses a compression function that is insecure against collision attacks. Suppose a compression function that takes an input of d bits insecurely combines the first two bits of the input such that when the two bits are equal, they cancel each other out and do not affect the output. The function otherwise behaves as a pseudorandom oracle. Collisions become trivial, since any inputs 1 1 || {0, 1}d−2 and 0 0 || {0, 1}d−2 where the final d − 2 bits are the same produce collisions when the combining function is applied. By introducing a preprocessor which inserts bits such that the first bit of every

87 message block is a zero, the second is a one, and the remaining bits are unchanged, the overall algorithm can be collision resistant while the combining function is not.

6.5.1 Future Directions This framework does not account for specific interdependencies between components, and determining a generalized way for components to securely interconnect is an interesting future research topic. Simple interdependencies like the block length or input size are easier to decouple and fit within the framework, but the RadioGat´ un [20] hash algorithm, for example, tightly couples its iterative structure to its combining functions which can complicate analysis. The fact that the components are distinct and logically separate in this paper should not be understood to mean that we believe that all hash algorithm designs must follow this pattern. We do believe, though, that this divide-and-conquer approach will allow cryptographers to construct strong cryptographic hash algorithms from vetted and well-known building blocks. When a component is found to be weak or unsuitable in a particular application, it can be replaced by one which is known to be strong. That is also not to say that there are any guarantees that creating a new hash algorithm from known strong components will yield a strong overall algorithm, but best practices should be assembled and researched to determine the best methods for constructing algorithms piecewise. More detailed research into proofs for the specific security properties provided by each choice of component is another ongoing future research direction.

88

References [1] Accredited Standards Committee X9. Working Draft: American National Standard X9.31-1992: Public Key Cryptography Using Reversible Algorithms for the Financial Services Industry: Part 2: The MDC-2 Hash Algorithm, June 4, 1993. [2] Rafael Alvarez, Gary McGuire, and Antonio Zamora. The Tangle Hash Function. Submitted to NIST, 2008. [3] Ross Anderson. The classification of hash functions. In IMA Conference in Cryptography and Coding, 1993. [4] Ross J. Anderson and Eli Biham. TIGER: A fast new hash function. In Dieter Gollmann, editor, FSE, volume 1039 of Lecture Notes in Computer Science, pages 89–97. Springer, 1996. [5] Elena Andreeva, Charles Bouillaguet, Pierre-Alain Fouque, Jonathan J. Hoch, John Kelsey, Adi Shamir, and S´ebastien Zimmer. Second preimage attacks on dithered hash functions. In Nigel P. Smart, editor, EUROCRYPT, volume 4965 of Lecture Notes in Computer Science, pages 270–288. Springer, 2008. [6] Elena Andreeva, Gregory Neven, Bart Preneel, and Thomas Shrimpton. Sevenproperty-preserving iterated hashing: ROX. In ASIACRYPT, pages 130–146, 2007. [7] Elena Andreeva, Gregory Neven, Bart Preneel, and Thomas Shrimpton. Threeproperty-preserving iterations of keyless compression functions. In ECRYPT Workshop on Hash Functions, 2007. [8] Elena Andreeva and Bart Preneel. A new three-property-secure hash function. In Selected Areas in Cryptography, 2008. [9] Yuriy Arbitman, Gil Dogon, Vadim Lyubashevsky, Daniele Micciancio, Chris Peikert, and Alon Rosen. SWIFFTX: A Proposal for the SHA-3 Standard. Submitted to NIST, 2008. [10] Adem Atalay, Orhun Kara, Ferhat Karakoc, and Cevat Manap. SHAMATA HASH FUNCTION ALGORITHM SPECIFICATIONS. Submitted to NIST, 2008. [11] Daniel Augot, Matthieu Finiasz, Philippe Gaborit, St´ephane Manuel, and Nicolas Sendrier. SHA-3 proposal: FSB. Submitted to NIST, 2008. [12] Jean-Phillipe Aumasson, Luca Henzen, Willi Meier, and Raphael C.-W. Phan. SHA-3 proposal BLAKE. Submitted to NIST, 2008.

89 [13] Paulo S. L. M. Barreto and Vincent Rijmen. The WHIRLPOOL hash function. World-Wide Web document, 2001. [14] Mihir Bellare, Oded Goldreich, and Shafi Goldwasser. Incremental cryptography with application to virus protection. In Proc. 27th ACM Symp. on Theory of Computing, pages 45–56, Las Vegas, 1995. ACM. [15] Mihir Bellare and Tadayoshi Kohno. Hash function balance and its impact on birthday attacks. In Christian Cachin and Jan Camenisch, editors, EUROCRYPT, volume 3027 of Lecture Notes in Computer Science, pages 401–418. Springer, 2004. [16] Mihir Bellare and Thomas Ristenpart. Multi-property-preserving hash domain extension and the EMD transform. In Lai and Chen [87], pages 299–314. [17] Mihir Bellare and Thomas Ristenpart. Hash functions in the dedicated-key setting: Design choices and MPP transforms. Cryptology ePrint Archive, Report 2007/271, 2007. http://eprint.iacr.org/. [18] Ryad Benadjila, Olivier Billet, Henri Gilbert, Gilles Macario-Rat, Thomas Peyrin, Matt Robshaw, and Yannick Seurin. SHA-3 Proposal: ECHO. Submitted to NIST, 2008. [19] Dan Bernstein. CubeHash Specification (2.B.1). Submitted to NIST, 2008. [20] Guido Bertoni, Joan Daemen, Gilles Van Assche, and Micha¨el Peeters. Radiogat´ un, a belt-and-mill hash function. NIST - Second Cryptographic Hash Workshop, August 24-25, 2006. [21] Guido Bertoni, Joan Daemen, Micha¨el Peeters, and Gilles Van Assche. Sponge functions. In ECRYPT Workshop on Hash Functions, 2007. [22] Guido Bertoni, Joan Daemen, Micha¨el Peeters, and Gilles Van Assche. Keccak specifications. Submitted to NIST, 2008. [23] E. Biham and A. Shamir. Differential cryptanalysis of Snefru, Khafre, REDOCII, LOKI and Lucifer. In J. Feigenbaum, editor, CRYPTO ’91: Proceedings of the 11th Annual International Cryptology Conference on Advances in Cryptology, pages 156–171. Springer, 1992. Lecture Notes in Computer Science No. 576. [24] Eli Biham and Rafi Chen. Near-collisions of sha-0. In Franklin [57], pages 290–305. [25] Eli Biham and Orr Dunkelman. A framework for iterative hash functions: HAIFA. In NIST’s Second Cryptographic Hash Workshop, 2006. [26] Eli Biham and Orr Dunkelman. The SHAvite-3 Hash Function. Submitted to NIST, 2008.

90 [27] Dan Boneh and Xavier Boyen. On the impossibility of efficiently combining collision resistant hash functions. In CRYPTO, pages 570–583, 2006. [28] Charles Bouillaguet, Orr Dunkelman, Pierre-Alain Fouque, and Antoine Joux. On the security of iterated hashing based on forgery-resistant compression functions. Cryptology ePrint Archive, Report 2009/077, 2009. http://eprint. iacr.org/. [29] Colin Bradbury. BLENDER: A Proposed New Family of Cryptographic Hash Algorithms. Submitted to NIST, 2008. [30] Emmanuel Bresson, Anne Canteaut, Benoˆıt Chevallier-Mames, Christophe Clavier, Thomas Fuhr, Aline Gouget, Thomas Icart, Jean-Fran¸cois Misarsky, Mar`ıa Naya-Plasencia, Pascal Paillier, Thomas Pornin, Jean Ren´e Reinhard, C´eline Thuillet, and Marion Videau. Shabal, a Submission to NIST’s Cryptographic Hash Algorithm Competition. Submitted to NIST, 2008. [31] Daniel R. L. Brown, Adrian Antipa, Matt Campagna, and Rene Struik. ECOH: the Elliptic Curve Only Hash. Submitted to NIST, 2008. [32] Christophe De Canni`ere and Christian Rechberger. Finding sha-1 characteristics: General results and applications. In Lai and Chen [87], pages 1–20. [33] Christophe De Canniere, Hisayoshi Sato, and Dai Watanabe. Hash Function Luffa: Specification. Submitted to NIST, 2008. [34] Donghoon Chang, Seokhie Hong, Changheon Kang, Jinkeon Kang, Jongsung Kim, Changhoon Lee, Jesang Lee, Jongtae Lee, Sangjin Lee, Yuseop Lee, Jongin Lim, and Jaechul Sung. ARIRANG. Submitted to NIST, 2008. [35] Donghoon Chang, Mridul Nandi, Jesang Lee, Jaechul Sung, and Seokhie Hong. Compression function design principles supporting variable output lengths from a single small function. Cryptology ePrint Archive, Report 2007/416, 2007. http://eprint.iacr.org/. [36] Donghoon Chang and Moti Yung. Do we need to vary the constants? (methodological investigation of block-cipher based hash functions). Cryptology ePrint Archive, Report 2006/467, 2006. http://eprint.iacr.org/. [37] Denis Charles, Eyal Goren, and Kristen Lauter. Cryptographic hash functions from expander graphs. Cryptology ePrint Archive, Report 2006/021, 2006. http://eprint.iacr.org/. [38] Scott Contini, Arjen K. Lenstra, and Ron Steinfeld. VSH, an efficient and provable collision-resistant hash function. In Serge Vaudenay, editor, EUROCRYPT, volume 4004 of Lecture Notes in Computer Science, pages 165–182. Springer, 2006.

91 [39] Jean-S´ebastien Coron, Yevgeniy Dodis, C´ecile Malinaud, and Prashant Puniya. Merkle-Damg˚ ard revisited: How to construct a hash function. In Victor Shoup, editor, CRYPTO, volume 3621 of Lecture Notes in Computer Science, pages 430–448. Springer, 2005. [40] Jean-S´ebastien Coron, Yevgeniy Dodis, C´ecile Malinaud, and Prashant Puniya. A new design criteria for hash-functions, 2005. [41] Joan Daemen. Cipher and Hash Function Design, Strategies based on linear and differential cryptanalysis. PhD thesis, Katholieke Universiteit Leuven, 1995. [42] Joan Daemen. Fast hashing and stream encryption with PANAMA. In Fast Software Encryption, LNCS 1372, pages 60–74, 1998. [43] Joan Daemen and Vincent Rijmen. The wide trail design strategy. In Bahram Honary, editor, IMA Int. Conf., volume 2260 of Lecture Notes in Computer Science, pages 222–238. Springer, 2001. [44] Joan Daemen and Vincent Rijmen. The Design of Rijndael. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2002. [45] I. Damg˚ ard and Ivan Bjerre. A design principle for hash functions. In G. Brassard, editor, Advances in Cryptology–Crypto’89. LNCS, vol.435, pages 416–427. Springer, 1990. Lecture Notes in Computer Science No. 435. [46] George I. Davida. Chosen signature cryptanalysis of the RSA public key cryptosystem. Technical Report TR-CS-82-2, Dept of EECS, University of Wisconsin, Milwaukee, 1982. [47] George I. Davida and Jeremy A. Hansen. A four-component framework for designing and analyzing cryptographic hash algorithms. Cryptology ePrint Archive, Report 2007/240, 2007. http://eprint.iacr.org/. [48] George I. Davida and Jeremy A. Hansen. A structural comparison of SHA-3 candidates. In 12th Information Security Conference (ISC 2009), submitted, 2009. [49] Richard Drews Dean. Formal aspects of mobile code security. PhD thesis, Princeton University, 1999. Adviser-Andrew Appel. [50] Hans Dobbertin, Antoon Bosselaers, and Bart Preneel. Ripemd-160: A strengthened version of ripemd. In Dieter Gollmann, editor, FSE, volume 1039 of LNCS, pages 71–82. Springer, 1996. [51] Yevgeniy Dodis and Prashant Puniya. Getting the best out of existing hash functions; or what if we are stuck with SHA? In Steven M. Bellovin, Rosario Gennaro, Angelos D. Keromytis, and Moti Yung, editors, ACNS, volume 5037 of Lecture Notes in Computer Science, pages 156–173, 2008.

92 [52] Bj¨orn Fay. MeshHash. Submitted to NIST, 2008. [53] Niels Ferguson, Stefan Lucks, Bruce Schneier, Doug Whiting, Mihir Bellare, Tadayoshi Kohno, Jon Callas, and Jesse Walker. The Skein Hash Function Family. Submitted to NIST, 2008. [54] Ewan Fleischmann, Christian Forler, and Michael Gorski. The Twister Hash Family. Submitted to NIST, 2008. [55] Ewan Fleischmann, Christian Forler, and Michael Gorski. Classification of the SHA-3 Candidates, 2009. http://www.uni-weimar.de/cms/fileadmin/ medien/medsicherheit/Research/SHA3/Classification_of_the_SHA-3_ Candidates.pdf. [56] Ewan Fleischmann, Michael Gorski, and Stefan Lucks. On the security of Tandem-DM. Cryptology ePrint Archive, Report 2009/054, 2009. http: //eprint.iacr.org/. [57] Matthew K. Franklin, editor. Advances in Cryptology - CRYPTO 2004, 24th Annual International CryptologyConference, Santa Barbara, California, USA, August 15-19, 2004, Proceedings, volume 3152 of Lecture Notes in Computer Science. Springer, 2004. [58] Praveen Gauravaram and John Kelsey. Linear-xor and additive checksums don’t protect damg˚ ard-merkle hashes from generic attacks. In Tal Malkin, editor, CTRSA, volume 4964 of Lecture Notes in Computer Science, pages 36–51. Springer, 2008. [59] Praveen Gauravaram, Lars R. Knudsen, Krystian Matusiewicz, Florian Mendel, Christian Rechberger, Martin Schl¨affer, and Søren S. Thomsen. Grøstl – a SHA3 candidate. Submitted to NIST, 2008. [60] Danilo Gligoroski, Vlastimil Klima, Svein Johan Knapskog, Mohamed ElHadedy, Jørn Amundsen, and Stig Frode Mjølsnes. Cryptographic Hash Function BLUE MIDNIGHT WISH. Submitted to NIST, 2008. [61] Danilo Gligoroski, Rune Steinsmo Ødeg˚ ard, Marija Mihova, Svein Johan Knapskog, Ljupco Kocarev, and Ale˘s Dr´aal. Cryptographic Hash Function EDON-R. Submitted to NIST, 2008. [62] Michael Gorski, Stefan Lucks, and Thomas Peyrin. Slide attacks on a class of hash functions. In Josef Pieprzyk, editor, ASIACRYPT, volume 5350 of Lecture Notes in Computer Science, pages 143–160. Springer, 2008. [63] Louis Goubin, Mickael Ivascot, William Jalby, Olivier Ly, Valerie Nachef, Jacques Patarin, Joana Treger, and Emmanuel Volte. CRUNCH. Submitted to NIST, 2008.

93 [64] Shai Halevi, William E. Hall, and Charanjit S. Jutla. The Hash Function Fugue. Submitted to NIST, 2008. [65] Shai Halevi and Hugo Krawczyk. Strengthening digital signatures via randomized hashing. In Cynthia Dwork, editor, CRYPTO, volume 4117 of Lecture Notes in Computer Science, pages 41–59. Springer, 2006. [66] Jeremy A. Hansen. Cryptographic authentication of processes and files to protect a trusted base. Master’s thesis, University of Wisconsin - Milwaukee, May 2005. [67] Bob Hattersley. Waterfall Hash - Algorithm Specification and Analysis. Submitted to NIST, 2008. [68] Phil Hawkes and Cameron McDonald. Submission to the SHA-3 Competition: The CHI Family of Cryptographic Hash Algorithms. Submitted to NIST, 2008. [69] Shoichi Hirose. Provably secure double-block-length hash functions in a blackbox model. In Choonsik Park and Seongtaek Chee, editors, ICISC, volume 3506 of Lecture Notes in Computer Science, pages 330–342. Springer, 2004. [70] Shoichi Hirose, Hidenori Kuwakado, and Hirotaka Yoshida. SHA-3 Proposal: Lesamnta. Submitted to NIST, 2008. [71] P. Hoffman and B. Schneier. RFC 4270: Attacks on Cryptographic Hashes in Internet Protocols. Network Working Group, November 2005. [72] Paul Hoffman. Notes from the hash futures panel. In NIST’s Second Cryptographic Hash Workshop, 2006. [73] Sebastiaan Indesteege, Elena Andreeva, Christophe De Canni`ere, Orr Dunkelman, Emilia Ksper, Svetla Nikova, Bart Preneel, and Elmar Tischhauser. The LANE hash function. Submitted to NIST, 2008. [74] Tetsu Iwata, Kyoji Shibutani, Taizo Shirai, Shiho Moriai, and Toru Akishita. AURORA: A Cryptographic Hash Algorithm Family. Submitted to NIST, 2008. [75] Antoine Joux. Multicollisions in iterated hash functions. application to cascaded constructions. In Franklin [57], pages 306–316. [76] Charanjit S. Jutla and Anindya C. Patthak. Provably good codes for hash function design. In Eli Biham and Amr M. Youssef, editors, Selected Areas in Cryptography, volume 4356 of Lecture Notes in Computer Science, pages 376–393. Springer, 2006. [77] J.B. Kam and G.I. Davida. Structured design of substitution-permutation encryption networks. IEEE Transactions on Computers, 28(10):747–753, 1979. [78] Neil Kauer, Tony Suarez, and Yuliang Zheng. Enhancing the MD-strengthening and designing scalable families of one-way hash algorithms, 2005.

94 [79] John Kelsey and Tadayoshi Kohno. tradamus attack, 2005.

Herding hash functions and the nos-

[80] John Kelsey and Bruce Schneier. Second preimages on n-bit hash functions for much less than 2n work. In Ronald Cramer, editor, EUROCRYPT, volume 3494 of LNCS, pages 474–490. Springer, 2005. [81] Dmitry Khovratovich, Alex Biryukov, and Ivica Nicoli´c. The Hash Function Cheetah: Specification and Supporting Documentation. Submitted to NIST, 2008. [82] G. H. Kim and E. H. Spafford. The design and implementation of tripwire: A file system integrity checker. In 2ACMConfSec, 1994. [83] Lars R. Knudsen and Fr´ed´eric Muller. Some attacks against a double length hash proposal. In Roy [118], pages 462–473. [84] Lars R. Knudsen, Christian Rechberger, and Søren S. Thomsen. The grindahl hash functions. In Alex Biryukov, editor, FSE, volume 4593 of Lecture Notes in Computer Science, pages 39–57. Springer, 2007. [85] Michael Kounavis and Shay Gueron. Vortex: A New Family of One Way Functions based on Rijndael Rounds and Carry-less Multiplication. Submitted to NIST, 2008. [86] Hugo Krawczyk. LFSR-based hashing and authentication. In Yvo G. Desmedt, editor, CRYPTO ’94: Proceedings of the 14th Annual International Cryptology Conference on Advances in Cryptology, pages 129–139. Springer, 1994. Lecture Notes in Computer Science No. 839. [87] Xuejia Lai and Kefei Chen, editors. Advances in Cryptology - ASIACRYPT 2006, 12th International Conference on the Theory and Application of Cryptology and Information Security, Shanghai, China, December 3-7, 2006, Proceedings, volume 4284 of Lecture Notes in Computer Science. Springer, 2006. [88] Xuejia Lai and James L. Massey. Hash functions based on block ciphers. Lecture Notes in Computer Science, 658:55–70, 1993. [89] Ga¨etan Leurent, Charles Bauillaguet, and Pierre-Alain Fouque. SIMD Is a Message Digest. Submitted to NIST, 2008. [90] Stefan Lucks. A failure-friendly design principle for hash functions. In Roy [118], pages 474–494. [91] Smile Markovski and Aleksandra Mileva. (NaSHA). Submitted to NIST, 2008.

2.B.1 Algorithm Specification

[92] Jason Worth Martin. ESSENCE: A Candidate Hashing Algorithm for the NIST Competition. Submitted to NIST, 2008.

95 [93] Mikhail Maslennikov. SECURE HASH ALGORITHM MCSSHA-3. Submitted to NIST, 2008. [94] Krystian Matusiewicz and Josef Pieprzyk. Finding good differential patterns for attacks on sha-1. In Øyvind Ytrehus, editor, WCC, volume 3969 of Lecture Notes in Computer Science, pages 164–177. Springer, 2005. [95] Ueli Maurer, Renato Renner, and Clemens Holenstein. Indifferentiability, impossibility results on reductions, and applications to the random oracle methodology. In Theory of Cryptography - TCC 2004, Lecture Notes in Computer Science, pages 21–39. Springer-Verlag, 2004. [96] Peter Maxwell. The Sg`ail Cryptographic Hash Function. Submitted to NIST, 2008. [97] Alfred J. Menezes, Paul C. van Oorschot, and Scott A. Vanstone. Handbook of Applied Cryptography. CRC Press, 1997. http://www.cacr.math.uwaterloo. ca/hac/. [98] Ralph Merkle. One-way hash functions and DES. In Advances in Cryptology– Crypto’89. LNCS, vol.435, pages 428–446. Springer-Verlag, 1990. [99] Ralph Merkle and Martin Hellman. Hiding information and signatures in trapdoor knapsacks. In IEEE Trans. Inform. Theory, volume IT-24, pages 525–530, Sep 1978. [100] Ralph C. Merkle. Protocols for public key cryptosystems. In IEEE Symposium on Security and Privacy, pages 122–134, 1980. [101] Ralph C. Merkle. Fast software encryption functions. In CRYPTO ’90: Proceedings of the 10th Annual International Cryptology Conference on Advances in Cryptology, pages 476–501, London, UK, 1991. Springer-Verlag. [102] Ralph Charles Merkle. Secrecy, authentication, and public key systems. PhD thesis, Stanford University, Stanford, CA, USA, 1979. [103] National Institute of Standards and Technology (NIST). FIPS Publication 180: Secure Hash Standard (SHS), May 11, 1993. [104] Ivica Nikoli´c, Alex Biryukov, and Dmitry Khovratovich. Hash family LUX Algorithm Specifications and Supporting Documentation. Submitted to NIST, 2008. [105] NIST. ANSI C Cryptographic API Profile for SHA-3 Candidate Algorithm Submissions, 2008. [106] Philippe Oechslin. Making a faster cryptanalytic time-memory trade-off. In CRYPTO, pages 617–630. Springer, 2003.

96 [107] Sean O’Neil, Karsten Nohl, and Luca Henzen. EnRUPT Hash Function Specification. Submitted to NIST, 2008. ¨ ul K¨ [108] Ozg¨ uc¨ uk. The Hash Function Hamsi. Submitted to NIST, 2008. [109] Daniel Penazzi and Miguel Montes. The TIB3 Hash. Submitted to NIST, 2008. [110] Bart Preneel. Analysis and Design of Cryptographic Hash Functions. PhD thesis, Katholieke Universiteit Leuven, 1993. [111] Bart Preneel, Ren´e Govaerts, and Joos Vandewalle. Hash functions based on block ciphers: a synthetic approach. In Douglas R. Stinson, editor, CRYPTO ’93: Proceedings of the 13th Annual International Cryptology Conference on Advances in Cryptology, pages 368–378. Springer, 1994. Lecture Notes in Computer Science No. 773. [112] Ron Rivest, Benjamin Agre, Daniel V. Bailey, Christopher Crutchfield, Yevgeniy Dodis, Kermin Elliott Fleming, Asif Khan, Jayant Krishnamurthy, Yuncheng Lin, Leo Reyzin, Emily Shen, Jim Sukha, Drew Sutherland, Eran Tromer, and Yiqun Lisa Yin. The MD6 hash function – A proposal to NIST for SHA-3. Submitted to NIST, 2008. [113] Ronald L. Rivest. The MD4 message digest algorithm. Technical Report MIT/LCS/TM-434, MIT Laboratory for Computer Science, October 1990. [114] Ronald L. Rivest. The MD5 message-digest algorithm. Internet Request for Comments, April 1992. RFC 1321. [115] Phillip Rogaway. Formalizing human ignorance: Collision-resistant hashing without the keys. Cryptology ePrint Archive, Report 2006/281, 2006. http: //eprint.iacr.org/. [116] Greg G. Rose. Design and Primitive Specification for Boole. Submitted to NIST, 2008. [117] Joachim Rosenthal and Roxana Smarandache. Maximum distance separable convolutional codes. In Appl. Algebra Engrg. Comm. Comput, volume 10, pages 15–32, 1999. [118] Bimal K. Roy, editor. Advances in Cryptology - ASIACRYPT 2005, 11th International Conference on the Theory and Application of Cryptology and Information Security, Chennai, India, December 4-8, 2005, Proceedings, volume 3788 of Lecture Notes in Computer Science. Springer, 2005. [119] Markku-Juhani O. Saarinen. Security of VSH in the real world, 2006. [120] Gokay Saldamlı, Cevahir Demirkıran, Megan Maguire, Carl Minden, Jacob Topper, Alex Troesch, Cody Walker, and C ¸ etin Kaya Ko¸c. Spectral Hash. Submitted to NIST, 2008.

97 [121] Yu Sasaki, Go Yamamoto, and Kazumaro Aoki. Practical password recovery on an md5 challenge and response. Cryptology ePrint Archive, Report 2007/101, 2007. http://eprint.iacr.org/. [122] B. Schneier. Applied Cryptography: Protocols, Algorithms, and Source Code in C. John Wiley & Sons, New York, 1993. [123] Bruce Schneier and John Kelsey. Unbalanced feistel networks and block cipher design. In Proceedings of the Third International Workshop on Fast Software Encryption, pages 121–144, London, UK, 1996. Springer-Verlag. [124] Julian Seward. bzip2 and libbzip2, version 1.0.5: A program and library for data compression, 2009. [125] Secure Hash Standard. National Institute of Standards and Technology, Washington, 2002. Federal Information Processing Standard 180-2. [126] Neil Sholer. Abacus: A Candidate for SHA-3. Submitted to NIST, 2008. [127] W. Simpson. RFC 1996: PPP Challenge Handshake Authentication Protocol (CHAP). Network Working Group, August 1996. [128] John P. Steinberger. The collision intractability of mdc-2 in the ideal-cipher model. In Moni Naor, editor, EUROCRYPT, volume 4515 of Lecture Notes in Computer Science, pages 34–51. Springer, 2007. [129] Kazuhiro Suzuki and Kaoru Kurosawa. How to find many collisions of 3-pass HAVAL, 2007. [130] Michael Szydlo and Yiqun Lisa Yin. Collision-Resistant Usage of MD5 and SHA-1 Via Message Preprocessing. In David Pointcheval, editor, CT-RSA, volume 3860 of Lecture Notes in Computer Science, pages 99–114. Springer, 2006. [131] Mark Torgerson, Richard Schroeppel, Tim Draelos, Nathan Dautenhahn, Sean Malone, Andrea Walker, Michael Collins, and Hilarie Orman. The SANDstorm Hash. Submitted to NIST, 2008. [132] Michal Trojnara. StreamHash Algorithm Specifications and Supporting Documentation. Submitted to NIST, 2008. [133] Alexander Vardy. The intractability of computing the minimum distance of a code. IEEE Transactions on Information Theory, 43(6):1757–1766, 1997. ¨ [134] Kerem Varıcı, Onur Ozen, and C ¸ elebi Kocair. Sarmal: SHA-3 Proposal. Submitted to NIST, 2008. [135] Natarajan Vijayarangan. A NEW HASH ALGORITHM: Khichidi-1. Submitted to NIST, 2008.

98 [136] Lei Wang, Kazuo Ohta, and Noboru Kunihiro. Password recovery attack on authentication protocol md4(password——challenge). In ASIACCS ’08: Proceedings of the 2008 ACM symposium on Information, computer and communications security, pages 3–9, New York, NY, USA, 2008. ACM. [137] Xiaoyun Wang, Yiqun Lisa Yin, and Hongbo Yu. Finding collisions in the full sha-1. In CRYPTO, pages 17–36, 2005. [138] Xiaoyun Wang and Hongbo Yu. How to break md5 and other hash functions. In Eurocrypt 2005, volume 3494, pages 19–35. Lecture Notes in Computer Science, May 2005. [139] John Washburn. WAMM: A CANDIDATE ALGORITHM FOR THE SHA-3 COMPETITION. Submitted to NIST, 2008. [140] A. F. Webster and S. E. Tavares. On the design of S-boxes. In H. C. Williams, editor, Advances in Cryptology — Crypto ’85, pages 523–534, New York, 1986. Springer-Verlag. [141] David A. Wilson. The DCH Hash Function. Submitted to NIST, 2008. [142] Hongjun Wu. The Hash Function JH. Submitted to NIST, 2008. [143] Zijie Xu. Dynamic SHA. Submitted to NIST, 2008. [144] Zijie Xu. Dynamic SHA2. Submitted to NIST, 2008. [145] Y. Zheng, J. Pieprzyk, and J. Seberry. HAVAL - A one-way hashing algorithm with variable length of output. In J. Seberry and Y. Zheng, editors, Advances in Cryptology — Auscrypt ’92, pages 83–104, Berlin, 1993. Springer-Verlag.

99

Appendix: Application Programming Interface Introduction Aside from the demonstration of the applicability on individual algorithms found in Chapter 6, we describe here how this four-component framework easily fits NIST’s required Application Programming Interface (API) for all SHA-3 candidates. NIST broke down the operation of proposed algorithms into the following three basic functions: HashReturn Init(hashState *state, int hashbitlen); HashReturn Update(hashState *state, const BitSequence *data, DataLength databitlen); HashReturn Final(hashState *state, BitSequence *hashval);

Init() The purpose of this function is to initialize the internal state and any algorithm constants based only on algorithm parameters, with the length of the final hash value as the only required parameter. In our framework, this function will do exactly that, setting necessary initial values in each component as necessary. The state S and any required “global” variables like bits hashed so far are stored directly in the state structure.

Update() This function is called to introduce new message data (in the data variable) and update the state appropriately. Three of the four components are represented in this function. The preprocessor operates on the message data and any parameters, then the usual iterated structure updates S. We can represent this in C-style code as follows, with the functions IS and PP representing the iterative structure and preprocessor, respectively:

100 DataLength PP(hashState *state, const BitSequence *data, DataLength databitlen, BitSequence *ppData, bool final); This function returns the length of the preprocessed message that it allocates at ppData, and applies global preprocessors if final is set to true. HashReturn IS(hashState *state, const BitSequence *data, DataLength databitlen); The IS function invokes the combining functions “behind the scenes” as appropriate. Effectively a clone of the Init function, this is meant to be called with the output from the PP function as follows: HashReturn Update(hashState *state, const BitSequence *data, DataLength databitlen) { BitSequence *ppData; DataLength ppLen = PP(state, data, databitlen, ppData, false); HashReturn retVal = IS(state, ppData, ppLen); free(ppData); return retVal; }

Final() This function is responsible for any preprocessors operating upon the entire message, like padding or MD-strengthening, processing any remaining message bits, and applying the finalization function FF. The FF function applies its finalization transformations to the state and places the hash value into hashval. HashReturn FF(hashState *state, BitSequence *hashval); The Final function might look like the following: HashReturn Final(hashState *state, BitSequence *hashval) { BitSequence *ppData; DataLength ppLen = PP(state, NULL, 0, ppData, true); HashReturn retVal = IS(state, ppData, ppLen); free(ppData); if (retVal == SUCCESS)

101 return FF(state, hashval); else return retVal; }

102

CURRICULUM VITAE

Jeremy A. Hansen Place of Birth: Manitowoc, Wisconsin, United States Education M.S., University of Wisconsin - Milwaukee, May 2005 Major: Computer Science B.S., University of Wisconsin - Madison, 1998 Major: Computer Science Dissertation Title: A Four Component Cryptographic Hash Framework Conference Publications “A Four-Component Framework for Cryptographic Hash Algorithms,” (with George I. Davida) 16th USENIX Security Symposium (Poster), August 2007. “A Preliminary Exploration of Striped Hashing: A Probabilistic Scheme to Speed Up Existing Hash Algorithms,” (with George I. Davida) Second International Conference on e-Business and Telecommunication Networks (Poster), October 2005. “A Structural Comparison of SHA-3 Candidates,” (with George I. Davida) 12th Information Security Conference (ISC 2009) (Submitted), September 2009. Awards Chancellor’s Graduate Student Award (2006-2007) University Service UW-Milwaukee Graduate Faculty Committee (2007-2008) UW-Milwaukee Academic Policy Committee (2007-2008)

Major Professor

Date