Find algorithms to a hash value using Jacksum

 If you need to find the algorithm to a CRC, checksum or hash value you can use Jacksum.

For those who don't know: Jacksum is a free and cross platform data integrity software tool. For more information go to https://jacksum.net

Let's keep things simple and let's pretend that you know your algorithm returns a message digest of 16 bits, and the the message digest is d893 in hex, lowercase. Input was 050000 in hex, lowercase. You can call jacksum with the following options:

• Option -a unknown:16 means you don't know the algorithm, but at least you know it returns 16 bits
• Option -E hex means you want a hexadecimal encoding for the message digest
• Option -q hex:050000 means you want to calcualate the message digest from the hex input 050000 quickly
• Option -e d893 means an expected hash value of d893, expressed as hex
  

jacksum -a unknown:16 -q hex:050000 -E hex -e d893 

produces the following output:

Trying 13 algorithms with a width of 16 bits that are supported by Jacksum 3.0.0 ...

Trying 30 CRC algorithms with a width of 16 bits by testing against well known CRCs ...
crc:16,1021,FFFF,false,false,FFFF
    --> CRC-16/GENIBUS

Trying all CRC algorithms with a width of 16 bits by brute force (be patient!) ...
crc:16,1021,FFFF,false,false,FFFF
crc:16,37D2,FFFF,true,false,FFFF
crc:16,3E2D,0000,true,false,FFFF
crc:16,4175,FFFF,true,false,FFFF
crc:16,4A5B,FFFF,true,true,0000
crc:16,5A41,FFFF,true,false,FFFF
crc:16,5C63,FFFF,true,true,0000
crc:16,6287,FFFF,true,true,0000
crc:16,649C,0000,false,true,FFFF
crc:16,6D55,FFFF,true,true,0000
crc:16,75AC,FFFF,true,false,FFFF
crc:16,7D64,FFFF,false,false,FFFF
crc:16,81A6,FFFF,true,false,FFFF
crc:16,B9F9,FFFF,true,true,0000
crc:16,C3D6,FFFF,false,false,FFFF
crc:16,D436,0000,true,false,FFFF
crc:16,D6D2,0000,false,true,FFFF
crc:16,DA9C,FFFF,true,false,FFFF
crc:16,E03E,FFFF,false,false,FFFF
crc:16,F701,FFFF,true,false,FFFF


Jacksum: algorithms tested: 1048620
Jacksum: algorithms found: 21

Jacksum: elapsed time: 6 s, 460 ms

Means Jacksum has tested more than one million algorithms in about 7 seconds and it found 21 matching algorithms. Each of those returns the same CRC value. Test with more input/output sequences and/or longer input sequences in order to find the right algorithm. The most likely algorithm is printed with a name if it is a well known CRC. In this example it has been identified as the CRC-16/GENIBUS.

Once you have identified the correct algorithm, you can calculate your own input data using the CRC definitions that have been found:

jacksum -a crc:16,1021,FFFF,false,false,FFFF -E hex -q hex:050000
d893 3

The output "d893 3" means that 3 bytes have been read (050000) in order to produce the 16 bit (2 byte) hexadecimal value d893 using the algorithm as defined by -a.

Mission completed.

Cheers,
Johann

Be very careful with echo on Windows

Recently one of my users of Jacksum reported a strange behavior if echo is used on Microsoft Windows.

For those who don't know: Jacksum is a data integrity tool which can (not only) calculate and verify hash values. For more information go to https://jacksum.net

So the report (which was sent privately to me) was this:

jacksum -q "txt:Hello World!"
d0e47486bbf4c16acac26f8b653592973c1362909f90262877089f9c8a4536af

that is correct, but if Jacksum reads from stdin and echo on Windows is used, it returns a completely different hash:

echo "Hello World!" | jacksum -
61ab266cecda9b9885aedbbb5b3d9914738cec74930301ec7b68312b6b436b8b <stdin>

Is Jacksum right?

Yes, Jacksum is right. The problem is with the echo command on Windows!

I started Jacksum with "--verbose summary" which gives us additional information:

echo "Hello World!" | jacksum --verbose summary -
61ab266cecda9b9885aedbbb5b3d9914738cec74930301ec7b68312b6b436b8b <stdin>

Jacksum: files read successfully: 1
Jacksum: files read with errors: 0
Jacksum: total bytes read: 17
Jacksum: total bytes read (human readable): 17 bytes

Jacksum: elapsed time: 139 ms

As you can see, Jacksum have read 17 bytes from stdin (the 3rd line of the summary). "Hello World!" (without quotes) are just 12 characters, what are the other 5 characters?

I pasted the output of echo to a file ...

echo "Hello World!" > hello.txt

and loaded the file in a hex editor (https://hexed.it will work perfectly well for that task)

As you can see, the Windows echo

- does not only transfers the Hello World! to the pipe (Hello World! => 12 bytes), but also ...
- it appends the Windows carriage return, line feed chars (0x0D, 0x0A => 2 bytes) which is expected on Windows
- it doesn't strip the quotes (2x" => 2 bytes) - which is probably not known to everybody, and
- ATTENTION!: it doesn't strip the blank that is between the 2nd quote (") and the redirection sign (>) => 1 byte

That means a 

echo "Hello World!"        > hello2.txt

would even add more blanks to the output.

And even if we would do this:

echo Hello World!> hello3.txt

the CR LF will always be there, because there is no -n option (which GNU/Linux and macOS have for example).

So the workaround on Microsoft Windows 10 we found is this:

echo | set /p="Hello World!" | jacksum -
d0e47486bbf4c16acac26f8b653592973c1362909f90262877089f9c8a4536af <stdin>

which produces the expected hash.

Mission completed.

BTW, on GNU/Linux and macOS you can still use echo as expected, but you need to use single quotes rather than double quotes, because the ! has a special meaning on many *nix-shells:

$ echo -n 'Hello World!' | jacksum -
d0e47486bbf4c16acac26f8b653592973c1362909f90262877089f9c8a4536af <stdin>

Cheers,
Johann

Jacksum 3 is on the web!

I am pleased to announce that Jacksum 3 has been released!

Jacksum 3 is a major release. I call it the "good things will take time" release ;-)

See also https://jacksum.net

Detailed release notes at https://jacksum.net/en/release-notes.html

Here is an eagle view comparison chart with Jacksum 1.7.0. There was never an official Jacksum 2 release and I lifted the version number immediately to 3 due to the feature richness of Jacksum 3.

 

	Jacksum 1.7.0	Jacksum 3.0.0
Algorithm support
Supported Algorithms (including different lengths)	58	470
Default algorithm	SHA-1	SHA-3-256
Full SHA-3 family support	no	yes
Full SHA-2 family support	no*1	yes
Full BLAKE-family support	no	almost*2
Full FNV-familiy support*3	no	yes
Customizable CRCs	yes	yes
Modern non-US modern national standards*4	no	yes
Older non-US national standards*5	yes	yes
All algos from round three of the NIST SHA-3 competition*6	no	yes
proposals from the NIST crypto workshops*7	no	yes
eXtendable Output Functions (XOF) as cryptographic hash functions*8	no	yes
Multi core/processor support
multiple algorithms calculation	yes*9	yes
multiple algorithms calculation simultanously	no	yes
parallel hash calculation for files	no	yes
multi processor support for checking files	no	yes
Other highlights
Finding an algorithm/CRC by knowing both data and hash	no	yes
Select algorithms that match a bit width or a search string	no	yes
producing the output in a format you want	yes	yes
checking an output produced by a foreign software using parser definitions	no	yes
License	GPLv2+	GPLv3+

 

*1 SHA-512/224, SHA-512/256 were missing, because those algos were introduced in March 2012.

*2 BLAKE-[224,384,256,512], BLAKE2s-[8..256/8], BLAKE2b-[8..512/8], and BLAKE3 are supported; BLAKE2sp and BLAKE2bp are not yet supported

*3 including FNV-0, FNV-1, and FNV-1a for all bit lengths [32,64,128,256,512,1024]

*4 Streebog-[256,512] (Russia, GOST R 34.11-2012); SM3 (China); Kupyna[256,384,512] (Ukraine, DSTU 7564:2014); LSH-256-[224,256], LSH-512-[224,256,384,512] (South Korea, KS X 3262)

*5 GOST, GOST Crypto-Pro (Russia, GOST R 34.11-94); HAS-160 (KISA, South Korea) 

 *6 all five candidates from round 3 the NIST SHA-3 competition were BLAKE, Groestl, JH, Keccak, and  Skein

 *7 before the SHA-3 competition they were proposals from the NIST crypto workshops called FORK-256, DHA-256, and VSH-1024

*8 SHAKE128, SHAKE256, KangarooTwelve, MarsupilamiFourteen

*9 while a file is read only once if multiple algorithms have been selected, the actual hash calculation occurred sequentially and not in parallel