Jacksum: How to skip files during hash calculation or the verification process

You could skip files during the hash calcualation process so that you do not create hash values at all for particular files and/or you could skip files during the verification process.

Skip files during hash calculation

To keep the example short, let's say we don't want to hash program libraries. Those files usually end with .dll on Windows, and with .so on GNU/Linux.

1. Create a list of files

Let's say we want to list all files in the current working directory (.) and below.

On GNU/Linux and macOS:

> find . -type f > files.list

On Windows:

> dir /A-D /B /S . > files.list

Using Jacksum:

> jacksum --style files-only . > files.list

2. Modify the file list as required

The -v option for grep and findstr lists files only that do not match the criteria given. You can use regular expressions. The $ at the end marks the end of a string.

On GNU/Linux or macOS:

> grep -v ".so$" files.list > files-filtered.list

On Windows:

> findstr /V ".dll$" files.list > files-filtered.list

3. Hash the file list

We can hash that list using the Jacksum option -L (or --file-list):

> jacksum -a sha256 -L files-filtered.list > hashes.sha256

Skip files during the verification process

If you want to ignore particular paths from a hash file you can do it in a similar way.

1. Modify the hash file as required

On GNU/Linux or macOS:

> grep -v ".so$" hashes.sha256 > hashes-filtered.sha256

On Windows:

> findstr /V ".dll$" hashes.sha256 > hashes-filtered.sha256

2. Start the integrity verification process

> jacksum -a sha256 -c hashes-filtered.sha256 .

Note: the dot at the end of the command says: verify not only the integrity of the paths stored in the hashes.sha256, but also read the current working directory (.) and all files below (-r max is set implicitly). With that you will also find new files that have been added after the file called hashes.sha256 was produced. Without the dot you will verify the files only that have a trace in hashes-filtered.sha256.

How to create unique and secure passwords for websites with a master password and Jacksum

Jacksum 3.7.0 introduced a new feature: read text from the console without echoing, hash that string using your preferred hash-function and encode the hash-value using your preferred encoding. In other words, you can treat Jacksum as a password generator that generates uniq and strong passwords.

You only have to remember ONE secure password. Yet, you can still use different, strong passwords for all your accounts.

No password manager is required, because nothing is stored on disk, the master password will be in your brain only. Even the generated passwords are not stored somewhere, they will be regenerated on demand.

In the following examples I use "do-not-use-this-password" as the master password for demonstration purposes only. Note that this password is not secure (since it is public here and easy to remember) and shouldn't be used. Actually you should select a secure password. To learn how to create secure passwords go to the recommendation of the BSI.

Let's see how we can generate uniq strong passwords using Jacksum ...

Simple approach with sha256 and base64-nopadding encoding (a-z, A-Z, 0-9, two special chars: +/ ):

> jacksum -a sha256 -8 -q password -E base64-nopadding
Password: facebook.com do-not-use-this-password

IMY5mnvgt44sWLNZZxOusnwepHP0mAJjMH4Q0rE2AF8

 

Better approach with HMAC:sha256 and base64-nopadding-encoding (a-z, A-Z, 0-9, two special chars: +/ ):

Hashing using a a crytpographic hash algorithm is a one-way action, and in theory it is impossible to calculate the master password if someone knows the hashed password. However, there is still the risk of precalculated rainbow tables that increses the speed of an attack in contrast to a brute-force attack. Therefore you can make it even more secure if you use an HMAC. The advantage with the HMAC is that you can individualize the actual hashing with another secret which makes precalculated rainbow tables useless for attack purposes which increases the security again. Since we need to keep the HMAC key and the master password as secret, we can simply use that token for both. Otherwise we would have to remember two passwords.

After entering the command, you will be asked twice. The 1st password prompt is the key for the HMAC, the 2nd password prompt is your actual message that you want to hash. You can use the same master password at both prompts.

> jacksum -a hmac:sha256 -8 -k password -q password -E base64-nopadding
Password: do-not-use-this-password
Password: facebook.com do-not-use-this-password

5kAL95XIPnB/yDFZcMgWDOo72kFnvDeMvnzgju+8xuM

 

Strong approach with HMAC:sha3-512:240 and base64-encoding (a-z, A-Z, 0-9, two special chars: +/ ):

In order to avoid padding for the base64 encoding and not to use base64-nopadding, the number of encoded bytes (the hash) must be a multiple of 3. So we can truncate the HMAC by specifying HMAC:sha256:240 which will result in a 30 byte hash. Since it increases the security further if we don't store the entire hash to the website, but only a fraction of it - with the knowledge of only a fraction of the hash it is impossible to precalculate a 1:1 relationship between the hash and the password, because more than 50% of the hash is not being transferred, we could also use the more modern SHA3-512 and truncate it after 240 bits. This will result in a 40 character password with at least 3 different character categories (upper- and lowercase characters, numbers, and sometimes even a special character).

> jacksum -a hmac:sha3-512:240 -8 -k password -q password -E base64
Password: do-not-use-this-password
Password: facebook.com do-not-use-this-password

ms2WOeYyYShCs9txS6jc5UtagHTviGJtIrZYRT+e

The examples above show how to read a password or passphrase from the console, not including any line-termination characters. You can use that mode to generate both unique and strong passwords for websites. The returned hash values will be stronger than anything a normal human brain could remember, and nothing needs to be stored on disk. If the password is compromised, the master password will still remain secret. Oh, please do not use the password from the example above, because now it is known to all people who read this article.

Some notes and recommendations:

• Combine a master password with website-specific information, such as the domain name, to get unique passwords. In the example above I have used facebook.com
  
• For the master password you should use a strong password that is at least 8 characters long; the longer, the better, and you should be able to remember it easily. Again, please do not use the password from the example above.
  
• You should use a non-broken, strong cryptographic hash algorithm for the task. For the examples above I have used sha256, and sha3-512.
• Set the character set explicitly if you use multiple different operating systems or environments that do not use UTF-8 for the console by default, and make sure to remember the character set as well. In the example above I have used option -8 which sets UTF-8 for both stdout and stderr. See also `jacksum -h -8`.
  
• You can use all of the available characters for the password.
• You can copy and paste the password to the prompt, but it is better if you remember it and type it, as this trains your brain to remember the password, and your brain will be the only place where the master password resides.
• Dependent on the allowed characters of the website login you can select one of the encodings that Jacksum supports. See also `jacksum -h -E`. In the example above I have used base64 resp. base64-nopadding which perfectly works for the password complexity requirements of most websites. Alternatively I can also recommend base32hex-nopadding or z85.
  
• To increase the security further you can use a HMAC. Note that not all hash algorithms work as a HMAC. Or, use a truncated HMAC which is even more secure, because we do not reveal the entire hash value to the website's password store.
  

Due to security reasons the following limits apply:

• You won't see the password that you enter; in other words, echoing is disabled to prevent shoulder-surfing attacks.
• Only the hash is printed by default; it is not possible to print the password in clear text, even if particular format options such as -F or --style are set. Use option "-q readline" if you prefer echoing.
• Operating system piping is not possible, because we want to make sure that the password is coming from a keyboard and not transfer passwords in clear text between processes via piping.
• Operating system redirection is not possible, because a console is required to enter the password. Use the options -o/-O to save the hash to a file if you do no wish to see the hash value in the console.
• To minimize the lifetime of sensitive data in memory, the password is cleared from memory after processing. Java's String interning is not used for the password.

Stay safe!

Regards,
Johann