Raymii.org

Table of Contents

• Get the public key
• Generate the random password file
• Encrypt the file with the random key
• Encrypt the random key with the public keyfile
• Decrypt the random key with our private key file
• Decrypt the large file with the random key

This small tutorial will show you how to use the openssl command line to encrypt and decrypt a file using a public key. We will first generate a random key, encrypt that random key against the public key of the other person and use that random key to encrypt the actual file with using symmetric encryption.

Recently I removed all Google Ads from this site due to their invasive tracking, as well as Google Analytics. Please, if you found this content useful, consider a small donation using any of the options below:

I'm developing an open source monitoring app called Leaf Node Monitoring, for windows, linux & android. Go check it out!

Consider sponsoring me on Github. It means the world to me if you show your appreciation and you'll help pay the server costs.

You can also sponsor me by getting a Digital Ocean VPS. With this referral link you'll get $200 credit for 60 days. Spend $25 after your credit expires and I'll get $25!

Because of how the RSA algorithm works it is not possible to encrypt large files. If you create a key of n bits, then the file you want to encrypt must not larger than (n minus 11) bits. The most effective use of RSA crypto is to encrypt a random generated password, then encrypt the file with the password using symmetric crypto. If the file is larger then the key size the encryption command will fail:

RSA operation error:  020:error:0406D06E:rsa routines:RSA_padding_add_PKCS1_type_2:data too large for key size:.\crypto\rsa\rsa_pk1.c:151:

We generate a random file and use that as the key to encrypt the large file with symmetric crypto. That random file acts as the password so to say. We encrypt the large file with the small password file as password. Then we send the encrypted file and the encrypted key to the other party and then can decrypt the key with their public key, the use that key to decrypt the large file.

The following commands are relevant when you work with RSA keys:

• openssl genrsa: Generates an RSA private keys.
• openssl rsa: Manage RSA private keys (includes generating a public key from it).
• openssl rsautl: Encrypt and decrypt files with RSA keys.

The key is just a string of random bytes. We use a base64 encoded string of 128 bytes, which is 175 characters. Since 175 characters is 1400 bits, even a small RSA key will be able to encrypt it.

Get the public key

Let the other party send you a certificate or their public key. If they send to a certificate you can extract the public key using this command:

openssl rsa -in certificate.pem -out publickey.pem -outform PEM -pubout

Generate the random password file

Use the following command to generate the random key:

openssl rand -hex 64 -out key.bin

Do this every time you encrypt a file. Use a new key every time!

Update 25-10-2018

The key format is HEX because the base64 format adds newlines. The -pass argument later on only takes the first line of the file, so the full key is not used. (Thanks Ken Larson for pointing this to me)

Encrypt the file with the random key

Use the following command to encrypt the large file with the random key:

openssl enc -aes-256-cbc -salt -in largefile.pdf -out largefile.pdf.enc -pass file:./bin.key

The file size doesn't grows that much:

  $ ls -larth
  -rw-r--r-- 1 user group  40M Nov  9 21:14 Linux-Voice-Issue-020.pdf
  -rw-r--r-- 1 user group  40M Nov  9 22:03 Linux-Voice-Issue-020.pdf.enc

It's encrypted however:

$ file Linux-Voice-Issue-020.pdf
Linux-Voice-Issue-020.pdf: PDF document, version 1.4

$ file Linux-Voice-Issue-020.pdf.enc 
Linux-Voice-Issue-020.pdf.enc: data

Encrypt the random key with the public keyfile

Use the following command to encrypt the random keyfile with the other persons public key:

openssl rsautl -encrypt -inkey publickey.pem -pubin -in key.bin -out key.bin.enc

You can safely send the key.bin.enc and the largefile.pdf.enc to the other party.

You might want to sign the two files with your public key as well.

Decrypt the random key with our private key file

If you want to decrypt a file encrypted with this setup, use the following command with your privte key (beloning to the pubkey the random key was crypted to) to decrypt the random key:

openssl rsautl -decrypt -inkey privatekey.pem -in key.bin.enc -out key.bin

This will result in the decrypted random key we encrypted the file in.

Decrypt the large file with the random key

Once you have the random key, you can decrypt the encrypted file with the decrypted key:

openssl enc -d -aes-256-cbc -in largefile.pdf.enc -out largefile.pdf -pass file:./bin.key

This will result in the decrypted large file.

Tags: ca , certificate , decrypt , encrypt , openssl , pki , ssl , tls , tutorials