How to debug openssl command line

Openssl provides command line tool to let user operate the security functions conveniently in terminal. This official site describes openssl command in details. As we know, openssl is an open source project so we can dig into the concrete internal code of a command. This page describes how to configure the openssl source code project in VSCode to debug openssl command.

Code structure

In this page, I take openssl 3.0.8 as example. Download the code from https://github.com/openssl/openssl . Let’s see the root folder below. Here we only focus on the crypto and apps folder.

The crypto folder is the core of openssl. It contains the implementation of encrypted algorithm and other security features. The apps folder is just the command line entry point. We should first build crypto then let apps link to crypto whose library name is libcrypto.a.

How to build libcrypto.a

In the openssl root folder, run:

1

./config -d

This is to make a debug configuration. “-d” means the ultimate library can be stepped into to see the code for debugger.

Then build, run:

1
2

make clean
make

“make clean” is to remove any intermediate generated files from previous build. If it is the first time, “make clean” can be neglected.

“make” is just to build the library.

After successful build, we can find the libcrypto.a in the root folder.

How to debug openssl command

Now we can debug openssl command. The whole thought is to directly run code in “apps” + libcrypto.a + header files.

We use VSCode IDE to set up the environment.
Here I don’t mention how to add C/C++ plugin/extension in VSCode and how to create tasks.json/launch.json template initially because these can be easily found on the Internet.
The file paths(/Users/weilin/…) mentioned afterwards are paths in my machine. You can easily replace them with your paths. The steps are:

1. Copy apps folder out to a new place and open apps in VSCode.

2. Add .vscode/tasks.json as below. This is just the completed gcc build command. Those code files after -g is just the C code to be built. (I used * wildcard in order not to type too many files here.)The “-I” part is additional header file search path. And we also specify the libcrypto.a.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

{
    "tasks": [
        {
            "type": "cppbuild",
            "label": "C/C++: gcc build active file",
            "command": "/usr/bin/gcc",
            "args": [
                "-fdiagnostics-color=always",
                "-g",
                "/Users/weilin/openssl/src/openssl/ssl/*.c", 
                "/Users/weilin/openssl/src/openssl/ssl/record/*.c",
                "/Users/weilin/openssl/src/apps/lib/app*.c",
                "/Users/weilin/openssl/src/apps/lib/fmt.c", "/Users/weilin/openssl/src/apps/*.c", 
                "/Users/weilin/openssl/src/apps/lib/c*.c", "/Users/weilin/openssl/src/openssl/ssl/statem/*.c",
                "/Users/weilin/openssl/src/apps/lib/n*.c", "/Users/weilin/openssl/src/apps/lib/e*.c", 
                "/Users/weilin/openssl/src/apps/lib/s*.c", "/Users/weilin/openssl/src/apps/lib/h*.c", 
                "/Users/weilin/openssl/src/apps/lib/t*.c", "/Users/weilin/openssl/src/apps/lib/v*.c",
                "/Users/weilin/openssl/src/apps/lib/o*.c",
                "-o",
                "${fileDirname}/${fileBasenameNoExtension}",
                "-I",
                "/Users/weilin/openssl/src/openssl/include",
                "-I",
                "/Users/weilin/openssl/src/apps/include",
                "-I",
                "/Users/weilin/openssl/src/openssl",
                "/Users/weilin/openssl/src/openssl/libcrypto.a",
                "-ldl",
                "-lpthread"
            ],
            "options": {
                "cwd": "${fileDirname}"
            },
            "problemMatcher": [
                "$gcc"
            ],
            "group": {
                "kind": "build",
                "isDefault": true
            },
            "detail": "Task generated by Debugger."
        }
    ],
    "version": "2.0.0"
}

3. Open openssl.c file. This is just the entry point of openssl command. You can find the beginning main function “int main(…)” in it. Tap the running button on top right, it will build and run the code. Because we don’t specify any command line arguments, the output is the help information. To confirm it, we can type “openssl” in terminal. The helper output are the same!

4. Now we almost reach our goal. The only thing we should do is to specify the arguments. Suppose we need to debug this command:

1
2

openssl pkcs12 -export -in cert_data -inkey myKey.pem -out mobileapp.p12 \
-passout pass:123456 -provider-path providers

We should add the launch.json as below. The args part are just command line arguments.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "(lldb) Launch",
            "type": "cppdbg",
            "request": "launch",
            "program": "${fileDirname}/openssl",
            "args": ["pkcs12", "-export", "-in", 
            "cert_data", "-inkey", 
            "myKey.pem", "-out", 
            "mobileapp.p12", "-passout", "pass:123456", "-legacy",
            "-provider-path", "providers"],
            "stopAtEntry": false,
            "cwd": "${fileDirname}",
            "environment": [{"name":"OPENSSL_CONF", "value":"/Users/weilin/openssl/src/apps/openssl.cnf"}],
            "externalConsole": false,
            "MIMode": "lldb"
        }
    ]
}

Set the breakpoint. We can debug the code now! Have fun~

Donate

• ← Previous Post
• Next Post →