Build, Debug, and Experiment with Master Branch of OpenSSL on Ubuntu Linux

Overview

OpenSSL is the de facto, open source cryptography software package for Linux and other operating systems. It consists of both a command line tool and libraries for cryptography. It has cutting edge support of cryptographic algorithms and an experienced team of developers. However, there seems to be a dearth of recent documentation and examples regarding developing with OpenSSL, especially with the latest API. Therefore, this article is a brief summary on getting started with OpenSSL development.

The openssl executable itself is a command line tool that can be used interactively. A very simple example of using the openssl executable packaged with Ubuntu Linux is shown below:

$ which openssl
/usr/bin/openssl

$ openssl
OpenSSL> version
OpenSSL 1.1.1f  31 Mar 2020

The version numbering system for OpenSSL can be confusing and is explained in Release Strategy. In this article, we work with the master branch, which tracks 3.0 development.

Note: Version 1.1.1 will be supported by OpenSSL until 2023-09-11 (LTS)

OpenSSL Components

OpenSSL relies on two important libraries that are part of the OpenSSL project:

• libssl provides the client and server-side implementations for SSLv3 and TLS.
• libcrypto provides general cryptographic and X.509 support needed by SSL/TLS but not logically part of it

We have found that the best way to learn how to work with the OpenSSL libraries is to look at how the openssl application makes use of them itself, and this is shown by example below.

Build Master Branch

Below we show steps to clone, build, and use the master branch of the OpenSSL repository. We install it to /opt/openssl and make use of it in such a way that it does not collide with the version installed & supported by Ubuntu. You may wish to build and install your own version of OpenSSL in a virtual environment or container.

$ cd /build/  # /build is where we build our code

$ git clone https://github.com/openssl/openssl.git
$ cd /build/openssl

$ find . -name tls1.h | xargs grep TLS1_3_RFC
# define TLS1_3_RFC_AES_128_GCM_SHA256                   "TLS_AES_128_GCM_SHA256"
# define TLS1_3_RFC_AES_256_GCM_SHA384                   "TLS_AES_256_GCM_SHA384"
# define TLS1_3_RFC_CHACHA20_POLY1305_SHA256             "TLS_CHACHA20_POLY1305_SHA256"
# define TLS1_3_RFC_AES_128_CCM_SHA256                   "TLS_AES_128_CCM_SHA256"
# define TLS1_3_RFC_AES_128_CCM_8_SHA256                 "TLS_AES_128_CCM_8_SHA256"

$ mkdir build; cd /build/openssl/build

$../config -v --prefix=/opt/openssl --openssldir=$HOME/openssl --debug
C compiler: gcc
C compiler vendor: gnu
C compiler version: 700
Configuring OpenSSL version 3.4.0-dev for target linux-x86_64
Using os-specific seed configuration
Created configdata.pm
Running configdata.pm
Created Makefile.in
Created Makefile
Created include/openssl/configuration.h

**********************************************************************
***                                                                ***
***   OpenSSL has been successfully configured                     ***
***                                                                ***
***   If you encounter a problem while building, please open an    ***
***   issue on GitHub   ***
***   and include the output from the following command:           ***
***                                                                ***
***       perl configdata.pm --dump                                ***
***                                                                ***
***   (If you are new to OpenSSL, you might want to consult the    ***
***   'Troubleshooting' section in the INSTALL.md file first)      ***
***                                                                ***
**********************************************************************

config options used (see INSTALL for extensive list of supported options):

• --prefix: top of the installation directory tree
• --openssldir: directory for OpenSSL configuration files, and also the default certificate and key store.
• --debug: build with debugging symbols

Note that OpenSSL doesn't use autotools for its build system but instead uses perl (scripts). The following is from the OpenSSL FAQ: "For OpenSSL 1.1, we decided to base our build system on perl, information files and build file (Makefile) templates, thereby covering all the systems we support. Perl was the base language of choice because we already use it in diverse scripts, and it's one of the most widely spread scripting languages."

Let's look a little further into how we configured the build. First, let's make sure we configured properly for debugging by looking at CFLAGS and CXXFLAGS

$ perl configdata.pm -m

perl configdata.pm -m

Makevars:

    AR              = ar
    ARFLAGS         = qc
    ASFLAGS         = 
    CC              = gcc
    CFLAGS          = -Wall -O0 -g
    CPPDEFINES      = 
    CPPFLAGS        = 
    CPPINCLUDES     = 
    CXX             = g++
    CXXFLAGS        = -Wall -O0 -g
    HASHBANGPERL    = /usr/bin/env perl
    LDFLAGS         = 
    LDLIBS          = 
    PERL            = /usr/bin/perl
    RANLIB          = ranlib
    RC              = windres
    RCFLAGS         = 
...

Let's also look at what's enabled and disabled:

$ perl configdata.pm -o

Enabled features:

    afalgeng
    apps
    argon2
    aria
    asm
    async
    atexit
    autoalginit
    autoerrinit
    autoload-config
    bf
    blake2
    bulk
    cached-fetch
    camellia
    capieng
    cast
    chacha
    cmac
    cmp
    cms
    comp
    ct
    default-thread-pool
    deprecated
    des
    dgram
    dh
    docs
    dsa
    dso
    dtls
    dynamic-engine
    ec
    ec2m
    ecdh
    ecdsa
    ecx
    engine
    err
    filenames
    gost
    http
    idea
    legacy
    loadereng
    makedepend
    md4
    mdc2
    module
    multiblock
    nextprotoneg
    ocb
    ocsp
    padlockeng
    pic
    pinshared
    poly1305
    posix-io
    psk
    quic
    unstable-qlog
    rc2
    rc4
    rdrand
    rfc3779
    rmd160
    scrypt
    secure-memory
    seed
    shared
    siphash
    siv
    sm2
    sm2-precomp
    sm3
    sm4
    sock
    srp
    srtp
    sse2
    ssl
    ssl-trace
    static-engine
    stdio
    tests
    thread-pool
    threads
    tls
    ts
    ui-console
    whirlpool
    tls1
    tls1-method
    tls1_1
    tls1_1-method
    tls1_2
    tls1_2-method
    tls1_3
    dtls1
    dtls1-method
    dtls1_2
    dtls1_2-method
...

Let's build it.

$ make

$ make install

$ ls ~/openssl
certs  ct_log_list.cnf  ct_log_list.cnf.dist  misc  openssl.cnf  openssl.cnf.dist  private
	
$ export PATH=/opt/openssl/bin:$PATH
$ export LD_LIBRARY_PATH=/opt/openssl/lib64/

$ which openssl
/opt/openssl/bin/openssl

$ openssl version
OpenSSL 3.4.0-dev  (Library: OpenSSL 3.4.0-dev )

Note that the two exports above are temporary. It is probably most conveinent to add these two lines to a file (e.g., /opt/openssl/env-openssl) that you would then source:

$ which openssl
/usr/bin/openssl

$ source /opt/openssl/env-openssl

$ which openssl
/opt/openssl/bin/openssl

$ ldd /opt/openssl/bin/openssl 
linux-vdso.so.1 (0x00007ffced345000)
	libssl.so.3 => /opt/openssl/lib64/libssl.so.3 (0x00007f66bfece000)
	libcrypto.so.3 => /opt/openssl/lib64/libcrypto.so.3 (0x00007f66bf6b3000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f66bf494000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f66bf0a3000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f66bee9f000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f66c054e000)

Now you're set up to experiment with the master branch of OpenSSL in your current shell.

Debugging genrsa

Generating an RSA private key is a common and interesting function for the openssl application. Below we'll set up a breakpoint in an RSA crypto library function, which will be called when we invoke genrasa:

$ gdb --args openssl genrsa
...
Reading symbols from openssl...done.

(gdb) break main
Breakpoint 1 at 0x765fe: file ../apps/openssl.c, line 236.

(gdb) run
Starting program: /opt/openssl/bin/openssl genrsa
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, main (argc=2, argv=0x7fffffffde58) at ../apps/openssl.c:236
236	{

(gdb) break rsa_keygen
Breakpoint 2 at 0x7ffff75d33e1: file ../crypto/rsa/rsa_gen.c, line 606.

(gdb) cont
Continuing.

Breakpoint 2, rsa_keygen (libctx=0x7ffff7a7dce0 , rsa=0x5555558ca700, bits=2048, primes=2, e_value=0x5555558a5bf0, cb=0x5555558a7400, pairwise_test=0)
    at ../crypto/rsa/rsa_gen.c:606
606	{

(gdb) backtrace
#0  rsa_keygen (libctx=0x7ffff7a7dce0 , rsa=0x5555558ca700, bits=2048, primes=2, e_value=0x5555558a5bf0, cb=0x5555558a7400, pairwise_test=0)
    at ../crypto/rsa/rsa_gen.c:606
#1  0x00007ffff75d1a12 in RSA_generate_multi_prime_key (rsa=0x5555558ca700, bits=2048, primes=2, e_value=0x5555558a5bf0, cb=0x5555558a7400) at ../crypto/rsa/rsa_gen.c:71
#2  0x00007ffff76cd9aa in rsa_gen (genctx=0x5555558a5b90, osslcb=0x7ffff749fed1 , cbarg=0x55555589ba10)
    at ../providers/implementations/keymgmt/rsa_kmgmt.c:619
...

(gdb) list
601	}
602	#endif /* FIPS_MODULE */
603	
604	static int rsa_keygen(OSSL_LIB_CTX *libctx, RSA *rsa, int bits, int primes,
605	                      BIGNUM *e_value, BN_GENCB *cb, int pairwise_test)
606	{
607	    int ok = 0;
608	
609	#ifdef FIPS_MODULE
610	    ok = ossl_rsa_sp800_56b_generate_key(rsa, bits, e_value, cb);

Now at this point, you can step through the RSA key generation routine, print variables, dump memory, etc. If you need help with GDB, then consult its documentation page.

Troubleshooting

Keep in mind that OpenSSL includes a test suite:

$ cd /build/openssl/build/
$ make tests
...
01-test_abort.t .................... ok   
01-test_fipsmodule_cnf.t ........... skipped: Test only supported in a fips build
01-test_sanity.t ................... ok   
01-test_symbol_presence.t .......... ok   
01-test_test.t ..................... ok   
02-test_errstr.t ................... ok     
02-test_internal_context.t ......... ok   
02-test_internal_ctype.t ........... ok   
02-test_internal_keymgmt.t ......... ok   
02-test_internal_provider.t ........ ok   
02-test_lhash.t .................... ok   
02-test_ordinals.t ................. ok   
02-test_sparse_array.t ............. ok   
02-test_stack.t .................... ok   
03-test_exdata.t ................... ok   
03-test_fipsinstall.t .............. skipped: Test only supported in a fips build
03-test_internal_asn1.t ............ ok   
03-test_internal_asn1_dsa.t ........ ok   
03-test_internal_bn.t .............. ok   
03-test_internal_chacha.t .......... ok   
03-test_internal_curve448.t ........ ok   
03-test_internal_ec.t .............. ok   
03-test_internal_ffc.t ............. ok   
03-test_internal_mdc2.t ............ ok   
03-test_internal_modes.t ........... ok   
03-test_internal_namemap.t ......... ok   
03-test_internal_poly1305.t ........ ok   
03-test_internal_rsa_sp800_56b.t ... ok   
03-test_internal_siphash.t ......... ok   
03-test_internal_sm2.t ............. ok   
03-test_internal_sm3.t ............. ok   
03-test_internal_sm4.t ............. ok   
03-test_internal_ssl_cert_table.t .. ok   
03-test_internal_x509.t ............ ok   
03-test_params_api.t ............... ok   
03-test_property.t ................. ok   
03-test_ui.t ....................... ok   
04-test_asn1_decode.t .............. ok   
04-test_asn1_encode.t .............. ok   
04-test_asn1_string_table.t ........ ok   
04-test_bio_callback.t ............. ok   
04-test_bio_core.t ................. ok   
04-test_bioprint.t ................. ok   
04-test_conf.t ..................... ok   
04-test_encoder_decoder.t .......... ok   
04-test_encoder_decoder_legacy.t ... ok   
04-test_err.t ...................... ok   
04-test_hexstring.t ................ ok   
04-test_param_build.t .............. ok   
04-test_params.t ................... ok   
04-test_params_conversion.t ........ ok   
04-test_pem_read_depr.t ............ ok   
04-test_pem_reading.t .............. ok    
04-test_provfetch.t ................ ok   
04-test_provider.t ................. ok   
04-test_provider_fallback.t ........ ok   
04-test_upcalls.t .................. ok   
05-test_bf.t ....................... ok   
05-test_cast.t ..................... ok   
05-test_cmac.t ..................... ok   
05-test_des.t ...................... ok   
05-test_hmac.t ..................... ok   
05-test_idea.t ..................... ok   
05-test_pbe.t ...................... ok   
05-test_rand.t ..................... ok   
05-test_rc2.t ...................... ok   
05-test_rc4.t ...................... ok   
05-test_rc5.t ...................... skipped: rc5 is not supported by this OpenSSL build
06-test_algorithmid.t .............. ok    
06-test_rdrand_sanity.t ............ ok   
10-test_bn.t ....................... ok   
10-test_exp.t ...................... ok   
15-test_dh.t ....................... ok   
15-test_dsa.t ...................... ok   
...

Simple TLS Server: SSLEcho

The OpenSSL wiki provides an older implementation of a minimal TLS server. For the latest implementation , we find it under demos in sslecho.

$ cd /build/openssl/demos/sslecho
$ ls
A-SSL-Docs.txt  build.info  cert.pem  key.pem  main.c  Makefile  README.md

$ source /opt/openssl/env-openssl

$ gcc -O0 -g main.c -o sslecho -I/opt/openssl/include  -L/opt/openssl/lib64 -lcrypto -lssl

$ ldd sslecho 
	linux-vdso.so.1 (0x00007ffca92bd000)
	libcrypto.so.3 => /opt/openssl/lib64/libcrypto.so.3 (0x00007f8a6053a000)
	libssl.so.3 => /opt/openssl/lib64/libssl.so.3 (0x00007f8a601e8000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8a5fdf7000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f8a5fbf3000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8a5f9d4000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f8a60f59000)


./sslecho s
sslecho : Simple Echo Client/Server : May  1 2024 : 23:37:35

We are the server on port: 4433

And from a different shell on the same local machine:

$ curl -vv  --insecure https://localhost:4433
* Rebuilt URL to: https://localhost:4433/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 4433 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Unknown (8):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Client hello (1):
* TLSv1.3 (OUT), TLS Unknown, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=AU; ST=Some-State; O=Internet Widgits Pty Ltd; CN=localhost
*  start date: Feb  2 14:38:36 2022 GMT
*  expire date: Jan 31 14:38:36 2032 GMT
*  issuer: C=AU; ST=Some-State; O=Internet Widgits Pty Ltd; CN=localhost
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
> GET / HTTP/1.1
> Host: localhost:4433
> User-Agent: curl/7.58.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
GET / HTTP/1.1
Host: localhost:4433
User-Agent: curl/7.58.0
Accept: */*

In the other shell, we see:

	
Client TCP connection accepted
Client SSL connection accepted

Received: GET / HTTP/1.1
Host: localhost:4433
User-Agent: curl/7.58.0
Accept: */*

Note that we pass curl the --insecure option since we are using a self signed certificate.

Another good thing to do at this point is to run either Wireshark or tcpdump and watch the exchange of the TLS handshake.

References

• OpenSSL Home Page
• OpenSSL Documentation: Manpages for master
• OpenSSL Community
• OpenSSL Vulnerabilities
• OpenSSL Wiki
• OpenSSL on Github
• OpenSSL developers and users email archive
• OpenSSL Frequently Asked Questions
• Wiki: Libcrypto API

Terms / Acronyms

• ALPN: Application-Layer Protocol Negotiation
• CMVP: Cryptographic Module Validation Program
• FIPS: Federal Information Processing Standards
• PEM: Privacy-enhanced Electronic Mail is the origin of this acronym, but it doesn't convey that it is a file format for OpenSSL
• RSA: Popular Asymmetric / Private Key Crypto Algorithm, named after its inventors: Rivest–Shamir–Adleman
• SSL: Secure Socket Layer
• TLS: Transport Layer Security