Using CMocka for unit testing C code

CMake logo

Writing unit tests along with the source code (or even before the code itself – see TDD) is currently very popular among programmers writing in languages like Java or C#. For C code, however it is a bit different. There are only a few frameworks enabling the possibility to write unit tests. One of them is quite special – it allows to mock functions. And its name is CMocka. Unfortunately there are not many resources that describes the process of setting up cmocka, especially together with cmake to allow programmers add new executables, tests and mocks without unnecessary overhead. But before showing how to do it, let’s go back to basics (if you already know them, you can skip next heading).

What is mocking?

Mocking is a mechanism that allows to substitute object, we do not want to test, with empty implementation, which we can further configure to do whatever we like, like simulate errors. Usually objects are mocked, because we import them from some external library and this is not purpose of unit testing to test these external dependencies. This is how they work in object-oriented languages like the ones mentioned at the beginning. In C, it is only a bit different in that, instead of mocking the object (which does not exist in C), we mock functions. Similarly to mocking object, this allows us to control behavior of external function and i.e. test reaction of our code to errors.

To give very short impression about the possibilities, it opens let’s say we want to test function that accepts connection to a socket. How could we test such a function without touching the accept() function? Probably we would need another program that performs connect(). In such a simple case it requires us to write second program to cover just one case. Then, what if we want to check reaction for failed connection establishment? Manpage tells us, ECONNABORTED is returned in that case. So, how to force our test program to break the connection before second side gets return from accept() function? Do you see how complex it gets?

But, what if we could link our test program (the one that calls function using accept()) to our own accept()? Then at first we could write our fake function so it pretends that real socket had been created and return some positive integer. In second case, we could just set errno to ECONNABORTED and return -1. That’s it!

This, and by the way, much more is possible with CMocka. Let’s see how to configure simplest possible project to use cmocka with cmake.

Hey cmake, do my unit tests!

For purpose of this tutorial, let’s say we have quite simple project. We just started it, so it’s perfect time to enable unit testing for TDD approach. It consists of one console (CLI) application split into two modules. First one – program is just main() function and another function it calls. Second one – module provide just one function that is also called by main(). For configuration, we have two extremely short CMakeLists.txt files:

cmake_minimum_required (VERSION 3.0) project (cmocka_template) add_subdirectory(src)
add_executable(program program.c module.c)

You can view the code itself on Github. With this configuration it is now possible to make the project with cmake. To enable testing, at first we have to append following script to main CMakeLists.txt:

29 # cmocka
30 option(ENABLE_TESTS "Perform unit tests after build" OFF)
32   find_package(CMocka CONFIG REQUIRED)
33   include(AddCMockaTest)
34   include(AddMockedTest)
35   add_subdirectory(test)
36   enable_testing()
37 endif(ENABLE_TESTS)

In line 27, we tell cmake to look for additional cmake scripts to include in cmake/Modules directory. This would be required by includes in lines 33 and 34.

Then, we define option to enable/disable tests. This will allow users not willing to run unit tests, build the project without having to satisfy dependency to CMocka in line 32.

In that line FindCmocka will be called and will setup few variables. Of our interest would be ${CMOCKA_LIBRARIES}. This points to CMocka library, we have to link with all our test programs.

Includes in lines 33 and 34, provides functions with the analogous names, first one is part of CMocka sources, the other define simple wrapper on the first one, so user do not have to type all the parameters that usually stays the same from test to test.

At last we include another build script from subdirectory and enable testing.

With help of the mentioned add_mocked_test wrapper in simplest case, where we do not use any functions external to the module, all we have to do is call add_mocked_test(module). However, if we do call some external functions, we have to call it like below and provide sources with these functions:

add_mocked_test(program SOURCES ${CMAKE_SOURCE_DIR}/src/module.c)

Alternatively, we can also join many sources into library and pass it like that:

add_mocked_test(module LINK_LIBRARIES mylib)

This finally becomes -lmylib in ld.

Simplest test

As we should have complete build script, we can now write simple test:

24 #include <stdarg.h> 25 #include <stddef.h> 26 #include <setjmp.h> 27 #include <cmocka.h> 28 29 #define main __real_main 30 #include "program.c" 31 #undef main 32 33 typedef struct {int a; int b; int expected;} vector_t; 34 35 const vector_t vectors[] = { 36 {0,1,0}, 37 {1,0,0}, 38 {1,1,1}, 39 {2,3,6}, 40 }; 41 42 static void test_internal(void **state) 43 { 44 int actual; 45 int i; 46 47 for (i = 0; i < sizeof(vectors)/sizeof(vector_t); i++) 48 { 49 /* get i-th inputs and expected values as vector */ 50 const vector_t *vector = &vectors[i]; 51 52 /* call function under test */ 53 actual = internal(vector->a, vector->b); 54 55 /* assert result */ 56 assert_int_equal(vector->expected, actual); 57 } 58 } 59 60 int main() 61 { 62 const struct CMUnitTest tests[] = { 63 cmocka_unit_test(test_internal), 64 }; 65 66 return cmocka_run_group_tests(tests, NULL, NULL); 67 }

At first we have to include some headers. Then in lines 29 and 31, preprocessor hack is done. This is because we already have a main() function used to define test suite. So to not have it redeclared, we temporarily change its name to __real_main, so this is how our program.c/main() will be called, when included in line 30. For tests with modules not containing main() function this is superfluous.

In lines 33-40, we define sets of data to feed into our function with expected results. This is useful if we have many such vectors. For single inputs/outputs pair this is unnecessary. In line 50 one vector is extracted from that array.

Then in line 53 function under test is called and in line 56 its value is asserted with assert_int_equal.

Finally in main() function we have to define list of tests in this suite and call cmocka_run_group_tests to do rest of the job for us. Done.

add_mocked_test internals

For better understanding of the process, let’s have a short look into add_mocked_test() function. It’s source is as simple as:

32 function(add_mocked_test name) 33 # parse arguments passed to the function 34 set(options ) 35 set(oneValueArgs ) 36 set(multiValueArgs SOURCES COMPILE_OPTIONS LINK_LIBRARIES LINK_OPTIONS) 37 cmake_parse_arguments(ADD_MOCKED_TEST "${options}" "${oneValueArgs}" 38 "${multiValueArgs}" ${ARGN} ) 39 40 # define test 41 add_cmocka_test(test_${name} 42 SOURCES test_${name}.c ${ADD_MOCKED_TEST_SOURCES} 43 COMPILE_OPTIONS ${DEFAULT_C_COMPILE_FLAGS} 44 ${ADD_MOCKED_TEST_COMPILE_OPTIONS} 45 LINK_LIBRARIES ${CMOCKA_LIBRARIES} 46 ${ADD_MOCKED_TEST_LINK_LIBRARIES} 47 LINK_OPTIONS ${ADD_MOCKED_TEST_LINK_OPTIONS}) 48 49 # allow using includes from src/ directory 50 target_include_directories(test_${name} PRIVATE ${CMAKE_SOURCE_DIR}/src) 51 endfunction(add_mocked_test)

At the beginning, arguments are parsed, which is outside of the scope of this article (those interested could read the official documentation). What is important in this part is that SOURCES from the function call becomes ADD_MOCKED_TEST_SOURCES, COMPILE_OPTIONS are ADD_MOCKED_TEST_COMPILE_OPTIONS and so on.

Then in line 41 add_cmocka_test function is called. This functions does some of the job for us. For example, we do not have to worry about defining executable, linking libraries to it and making it a test called by CTest.

Then in line 42 source file list is passed to it, so it can link them into final executable. We can also pass our own compiler and linker flags, so they are routed to it in lines 43-44 and 47.

Last thing it does is to link this test executable with all required libraries and the only requirements for CMocka to work is to pass its shared library using CMOCKA_LIBRARIES variable, which is available thanks to finding CMocka package in CMakeLists.txt.

Beside that, we make C headers in src/ directory visible to our test program in line 50. That’s it.

Enabling mocks

Killer feature of CMocka however is its API for creating mocked functions. To use it, on CMake side all we have to do, beside what we already did is add flags for linker (to be precise -Wl,--wrap=function_name for every function to be mocked). As can be seen in add_mocked_test source, we could use LINK_OPTIONS argument for that purpose. However it is timesaving to have this integrated into the function’s interface. All we have to do is add a loop for creating argument list:

40 # create link flags for mocks 41 set(link_flags "") 42 foreach (mock ${ADD_MOCKED_TEST_MOCKS}) 43 set(link_flags "${link_flags} -Wl,--wrap=${mock}") 44 endforeach(mock)

Then add new argument and modify LINK_OPTIONS of add_cmocka_test to pass that list:

53                   LINK_OPTIONS ${link_flags} ${ADD_MOCKED_TEST_LINK_OPTIONS})

Now, it should work. Then on the source side we can create new test for existing test_program test suite:

79 static void test_main(void **state)
80 {
81   int expected = 0;
82   int actual;
84   /* expect parameters to printf call */
85   expect_string(__wrap_printf, format, "%d\n");
86   expect_value(__wrap_printf, param1, 60);
88   /* printf should return 3 */
89   will_return(__wrap_printf, 3);
91   /* call __real_main as this is main() from program.c */
92   actual = __real_main(0, NULL);
94   /* assert that main return success */
95   assert_int_equal(expected, actual);
96 }

And write mocked object:

43 int __wrap_printf (const char *format, ...)
44 {
45   int param1;
47   /* extract result from vargs ('printf("%d\n", result)') */
48   va_list args;
49   va_start(args, format);
50   param1 = va_arg(args, int);
51   va_end(args);
53   /* ensure that parameters match expecteds in expect_*() calls  */
54   check_expected_ptr(format);
55   check_expected(param1);
57   /* get mocked return value from will_return() call */
58   return mock();
59 }

Now let’s take a look at what is happening here. At first in function test_main, in lines 85-86 we are telling CMocka that printf function is expected to be called with parameter format of a certain content and param1 should have value 60. param1 here is first variadic argument for printf function and its name is completely arbitrary. The key to working mock is to use this same name inside of mocked function.

In line 89, we say mocked printf to return 3 (this is number of bytes written to console and will then be ignored by code in __real_main, but for completeness it is set to proper value here).

Finally, we call the function under test and assert its result, the same way as in the example without mocks above.

On the other side is the mocked printf implementation. In languages like Java it is common to have this mock automatically generated based on the rules provided by user. In C with CMocka it is not so easy, we have to write the mock ourselves. However it is not very hard.

As we chosen variadic function for our mock and we want to check if these variadic arguments are as expected, we have to extract them at first, which is done in lines 48-51. As a result we have param1 variable (notice that it is local variable). Remember that this must match exactly what we declared in test body using one of expect_* functions. This name is how function-parameter pair is extracted from internal dictionaries.

Then in lines 54-55 arguments are checked for having expected contents. Note that there is no difference in how we treat positional and variadic parameters in this step.

Finally in line 58, we extract value that we want to be returned from printf (declared in line 89). This may seem superfluous in such a simple case, however if we want use this mocked printf in more than one test, it is really useful feature. Have also in mind that linker does not give us any interface to have two or more mocks for one function. So this way we have chance to write one universal mock.

Final word

This tutorial was made during my work on template for CMocka+CMake projects. So now, as you understand how it is done, you can just go to my Github and clone the template to make it part of your project. The template is licensed under MIT license, so I don’t care about what you do with it, as long as you leave original copyright.

USB to serial converter drivers for Android revisited


Few years ago I compiled kernel drivers of cheap USB-to-serial converter for my previous Android phone. It took few years of using new phone, without single custom-compiled kernel module. Now it is time to change it. By the way, I am going to describe what changed and what hacks have to be made to make the process work on stock ROM, provided by Sony.

kernel is the key

First of all, we need kernel. To be precise, kernel sources. Without that, it is really hard to be successful (I don’t want to tell it is impossible, but really hard, believe me). Because Sony is very liberal in terms of cooperation with community, they provide anything required to tinker with the device (obviously together with caution message about warranty loss, but who cares, right? πŸ™‚ ).

First of all, we need to know, which firmware version the device uses. To be found in Android settings, as compilation number, or something like that. For me, it is 23.5.A.0.575. Then, we have to visit Open Devices downloads section and find our firmware. For me, it was a lot of scrolling, as I have no updates available for quite some time. Inside the package, there should be kernel directory, with complete kernel sources.

Where is my .config?

Next thing we need to know is, which defconfig to use. Full list should be in arch/arm/configs. Now, in case of Sony phones, there is slight problem, as they traditionally use codenames for devices. In case of Xperia Pro, I compiled for before, it was iyokan. For Xperia Z3 Compact, I use now, it is Aries and the only official source of those codenames, I know, is their Github profile. Of course it would be too easy to find some mapping and searching for z3 gives no result. Fortunately, I know my device’s codename.

$ find . -name *aries*

As we can see, there is only one config, related to aries: shinano_aries_defconfig (Sony’s Github profile explains that Shinano is platform name). Then, we can safely use this in defconfig phase.

Compilation (and hacking)

Once we have all the sources and kernel configuration, we can start compilation. Or actually, we cannot (probably).


Let’s see vermagic of random module already installed on a device:

# modinfo zl10353.ko                            
filename:       zl10353.ko
license:        GPL
author:         Chris Pascoe
description:    Zarlink ZL10353 DVB-T demodulator driver
parm:           debug_regs:Turn on/off frontend register dumps (default:off).
parmtype:       debug_regs:int
parm:           debug:Turn on/off frontend debugging (default:off).
parmtype:       debug:int
intree:         Y
vermagic:       3.4.0-perf-g43ea728 SMP preempt mod_unload modversions ARMv7

We can see at least two things that will cause troubles:

  1. -perf-g<sha-1>
  2. modversions

In the first case, git commit id is appended to kernel version. Unfortunately, we do not have their repository and after module compilation, we will end up with just 3.4.0. To fix the problem, we have to edit makefile and set EXTRAVERSION to the missing part, so it should look like:

EXTRAVERSION = -perf-g43ea728
NAME = Saber-toothed Squirrel

The second detail, forces us to compile whole kernel. Otherwise, Android kernel will try to check if the module is compatible with current kernel (using CRC checksums) and will fail on missing module_layout symbol CRC.

Bad hacking

In case of very simple drivers, there is a way to omit kernel compilation. However, it is not a safest way to go and serves as permanent --force for modprobe/insmod. I advice to skip this section, unless you are really desperate (and you are not, before trying the proper way).

Go back to some random driver, like the one, we used for vermagic check, pull it to PC and issue:

$ modprobe --dump-modversions zl10353.ko 
0x2067c442      module_layout
0x15692c87      param_ops_int
0xe6b3b90a      arm_delay_ops
0x59e5070d      __do_div64
0x5f754e5a      memset
0x0fc539b8      kmalloc_caches
0x9d669763      memcpy
0x52ac1d50      kmem_cache_alloc_trace
0x27e1a049      printk
0xfbc76af9      i2c_transfer
0x037a0cba      kfree
0xefd6cf06      __aeabi_unwind_cpp_pr0

From my experience, I know that module_layout is most troublesome. So why not add it to modversions of our module? Just run (change the CRC to match modprobe output!):

echo -e '0x2067c442\tmodule_layout\tvmlinux\tEXPORT_SYMBOL_GPL' >> Module.symvers

And you should cheat kernel to trust the symbol, even if in fact it would be different in kernel compiled by you. Then, after insmodding the module, built using the same shortcut as in my previous tutorial, you should possibly see a lot of errors on your dmesg. You can hunt for the symbols, from there and chances are it will work. Haven’t tested personally and I discourage, unless you really know what you are doing. It is wiser choice to wait those few minutes for kernel to compile.

Device specific hacking

During my compilation, I had to do some more hacks, as I had problems with missing headers. This will possibly be only relevant to the specific kernel version and device pair, but just in case, I am writing it down. You can safely skip to compilation and only go back in case of problems with framebuffer for MSM processors.

Following should fix the problem:

ln -s ../../drivers/video/msm/mdss/mdss_mdp_trace.h include/trace/mdss_mdp_trace.h
ln -s ../../drivers/video/msm/mdss/mdss_mdp.h include/trace/mdss_mdp.h


Now, it should be fairly easy, though time consuming. Type following commands, one after another:

ARCH=arm CROSS_COMPILE=arm-unknown-eabi- make shinano_aries_defconfig
ARCH=arm CROSS_COMPILE=arm-unknown-eabi- make
ARCH=arm CROSS_COMPILE=arm-unknown-eabi- make modules M=drivers/usb/serial CONFIG_USB_SERIAL=m CONFIG_USB_SERIAL_CP210X=m

If there is no unexpected error during the compilation, you should now be able to insmod your fresh module into the kernel. In case of CP2102 driver, I compiled, there are in fact two drivers: usbserial and cp210x. cp210x depends on usbserial, so usbserial have to be inserted first. Afterwards, if you connect the device, you should see in dmesg, it succeeded, and in case of cp210x, there should be a name of USB tty device (most likely /dev/ttyUSB0, as there should be no USB ttys before).

Just to prove it, that it works, below are photos of of Xperia, running Termux and minicom to connect to Cubieboard2:

Sony Z3C - UART-USB plugged
Phone sniffing on Linux boot
Sony Z3C plugged to Cubieboard2 via UART
Phone connected to Cubieboard2

Setting up new v3 Hidden Service with ultimate security: Part 4: Installing client certificates to Firefox for Android

Firefox logo

This post is a part of Tor v3 tutorial. Other parts are:

  1. Hidden Service setup
  2. PKI and TLS
  3. Client Authentication
  4. Installing client certificates to Firefox for Android

As we now have Hidden Service, requiring clients to authenticate themselves with proper certificate, it would be great to be able to use Android device to access the service. As I shown before, on desktop Firefox it was quite trivial. Unfortunately, things are different on Android. Mobile Firefox does not have any interface for adding any certificates. Furthermore, unlike Chrome, it does not use default Android certificate vault, providing it own instead. On the other hand, under the hood it is more or less the same Firefox, so the support itself is present. Therefore, we need to hack into Firefox internal databases and add the certificate there. In this part, I will show, how to do that.

Caution: similarly to desktop browser, you should not add any random certificates to your main browser. It is even worse idea to do the same with Orfox, as it might allow attackers to reveal your identity. Newer Androids have ability to create user accounts, furthermore Firefox has profiles features, just like on desktop, but harder to use. If you want to do, what is described here, separating this configuration from any other is first thing to do.

Installing CA certificate

Before we do that with user certificate, let’s start with CA. It is way easier, as Firefox has convenient feature allowing to install certificates by browsing them. All we need to provide is a valid MIME type – application/x-x509-ca-cert. So, all we need is some webserver, which we will configure to treat files with extension .crt to be treated as mentioned type. Just after opening certificate file, Firefox should ask if you are sure about adding the certificate and allow you to choose for what purpose it will be used. It also allows to view the certificate to make sure, it is the one we intended to add.

Firefox - certificate details
At first, check the certificate
Firefox - PEM download
Then use it only for website identification

In theory there is very similar MIME for user certs – application/x-x509-user-cert, but for some reason, what Firefox says after opening this type of file is:

“Couldn’t install because the certificate file couldn’t be read”

And the same effect is, no matter if the file is password protected or not.

Installing client certificate

  1. Go to /data/data/org.mozilla.firefox/files/mozilla on Android device (root required)
  2. Locate default Firefox profile. If there is only one directory in format [bloat].profile, this is it. If not, file profiles.ini should contain only one profile with Default=1. This is what we are looking for
  3. Download files cert9.db and key4.db to Linux machine
  4. Use pk12util to insert certificate into database:
$ pk12util -i [filename].p12 -d.
Enter password for PKCS12 file:
pk12util: no nickname for cert in PKCS12 file.
pk12util: using nickname: [email] - r4pt0r Test Systems
  1. Upload files back to Android. Make sure Firefox is not running
  2. Test it by opening your hidden service with Firefox. You should see messages similar to these:
Firefox Mobile - User Identification Request
Request for identification
Firefox - Certificate details
Certificate details
cgit via Tor hidden service
Finally, working cgit via tor!

Setting up new v3 Hidden Service with ultimate security: Part 3: Client Authentication

Secure Card icon

This post is a part of Tor v3 tutorial. Other parts are:

  1. Hidden Service setup
  2. PKI and TLS
  3. Client Authentication
  4. Installing client certificates to Firefox for Android

As we now have working Public Key Infrastructure, we are ready to use it for more than encrypting traffic (which is already encrypted by Tor). We can very easily turn on client verification on our server. This will prevent anybody not having valid certificate issued by us from visiting our hidden webpage – just in case hiding domain name in hidden services version 3 leaks the name somehow (which should not happen anymore in v3). In this part we will issue client certificate (the procedure is almost identical to server certificate), then configure httpd to require client identification and finally configure Firefox to try sending the certificate. Let’s go!

Issuing user certificate

In my case tmp directory emulated client machine and ca is my Cerificate Authority, which issues certificates. We start by creating request on client side, then sign it on CA side.

$ mkdir tmp
$ cd tmp
$ openssl genrsa -out 4096
Generating RSA private key, 4096 bit long modulus
e is 65537 (0x010001)
$ openssl req -config ../ca/intermediate/openssl.cnf -key -new -sha256 -out
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
Country Name (2 letter code) [GB]:PL
State or Province Name [England]:lodzkie
Locality Name []:
Organization Name [Alice Ltd]:r4pt0r Test Systems
Organizational Unit Name []:
Common Name []
Email Address []
$ chmod 400*.pem
$ cp ../ca/intermediate/csr/
$ cd ../ca
$ openssl ca -config intermediate/openssl.cnf -extensions usr_cert -days 375 \
> -notext -md sha256 -in intermediate/csr/ \
> -out intermediate/certs/
Using configuration from intermediate/openssl.cnf
Enter pass phrase for /home/r4pt0r/Research/cubie/newtor/ca/intermediate/private/intermediate.key.pem:
Check that the request matches the signature
Signature ok
Certificate Details:
        Serial Number: 4097 (0x1001)
            Not Before: Feb 27 17:14:40 2018 GMT
            Not After : Mar  9 17:14:40 2019 GMT
            countryName               = PL
            stateOrProvinceName       = lodzkie
            organizationName          = r4pt0r Test Systems
            commonName                =
            emailAddress              =
        X509v3 extensions:
            X509v3 Basic Constraints:
            Netscape Cert Type:
                SSL Client, S/MIME
            Netscape Comment:
                OpenSSL Generated Client Certificate
            X509v3 Subject Key Identifier:
            X509v3 Authority Key Identifier:

            X509v3 Key Usage: critical
                Digital Signature, Non Repudiation, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Client Authentication, E-mail Protection
Certificate is to be certified until Mar  9 17:14:40 2019 GMT (375 days)
Sign the certificate? [y/n]:y

1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
$ cd ../tmp
$ cp ../ca/intermediate/certs/ ./
$ openssl pkcs12 -export -inkey -in -out
Enter Export Password:
Verifying - Enter Export Password:

Last step was packaging certificate and key into PKCS#12 container. That is for securing key (we can encrypt it with password), and is a form required by Firefox. After creation of .p12 (and verifying it is fine), we can (and SHOULD) delete source files, as they are not protected in any way.

Configuring httpd to require user certificate

To enforce client verification, following lines must be added to virtual host configuration, in our case it might go just after SSL certificate file paths.

    SSLVerifyClient require
    SSLVerifyDepth 2

We have to reload httpd for changes to take effect.

Installing certificate to Firefox

At last, to start using newly generated certificate, we should install it to Firefox. The procedure is similar to the one with CA certificate. We need to open Certificate Manager window. Then, instead of going to Authorities, we go to Your Certificates. Then we click on Import and select .p12 file.

Firefox Certificate Manager
Certificate Manager / Your Certificates

If the file has password, Firefox will ask for it and after successfully reading the content. If everything went well, you should see your certificate on the list. Now we can try connecting to our hidden service. We should see the window like this:

Firefox - User Identification Request
Server asks for client’s identity

Finally, after confirmation, you should see your hidden service content. Congrats!

Setting up new v3 Hidden Service with ultimate security: Part 2: PKI and TLS

KGPG icon

This post is a part of Tor v3 tutorial. Other parts are:

  1. Hidden Service setup
  2. PKI and TLS
  3. Client Authentication
  4. Installing client certificates to Firefox for Android

After setting up working Tor hidden service, the next step to ultimate security is having properly implemented Public Key Infrastructure (PKI). For this step, there are a lot of tutorials already existing and there is not much that needs to be added to them. Personally, I was using tutorial available here for the second time now and I find it very well-written. Because I am going to follow this tutorial, I will just post commands that have to be executed.

Before starting, I have to add one important remark. To make our PKI really secure one, it is crucial to have root CA air-gapped, that is device, on which it will be generated should be disconnected permanently from the internet. Good candidate for such a device might be some old laptop or Raspberry Pi Zero, as it lacks Ethernet port and anything reasonable to connect to internet. It is also important to store generated certificate in a safe place and secure it with strong non-dictionary password, which will be saved only in our mind.

If the requirements are fulfilled, we can start the setup. Below are commands to type as well as output from them, for easier determination of whether the commands were successful or not.


At first, we need to create following directory structure:

β”œβ”€β”€ [drwxr-xr-x]  certs
β”œβ”€β”€ [drwxr-xr-x]  crl
β”œβ”€β”€ [-rw-r--r--]  index.txt
β”œβ”€β”€ [drwxr-xr-x]  intermediate
β”‚Β Β  β”œβ”€β”€ [drwxr-xr-x]  certs
β”‚Β Β  β”œβ”€β”€ [drwxr-xr-x]  crl
β”‚Β Β  β”œβ”€β”€ [drwxr-xr-x]  csr
β”‚Β Β  β”œβ”€β”€ [-rw-r--r--]  index.txt
β”‚Β Β  β”œβ”€β”€ [drwxr-xr-x]  newcerts
β”‚Β Β  β”œβ”€β”€ [drwx------]  private
β”‚Β Β  └── [-rw-r--r--]  serial
β”œβ”€β”€ [drwxr-xr-x]  newcerts
β”œβ”€β”€ [drwx------]  private
└── [-rw-r--r--]  serial

And file content is (enclosed between pipe symbols: |):

./index.txt: ||
./intermediate/index.txt: ||
./intermediate/serial: |1000
./serial: |1000

Then, we need to save this file into root/openssl.cnf and this file into root/intermediate/openssl.cnf. Inside them, the only thing that have to be changed is dir property in CA_default section. Use absolute path to your directory.

Root CA

Note: when giving values for certain fields, better give some country, state (I have just checked it’s necessary), ON, most importantly, Common Name and e-mail. Just in case some program will check if they exists.

$ openssl genrsa -aes256 -out private/ca.key.pem 8192
Generating RSA private key, 8192 bit long modulus
e is 65537 (0x010001)
Enter pass phrase for private/ca.key.pem:
Verifying - Enter pass phrase for private/ca.key.pem:
$ chmod 400 private/ca.key.pem
$ openssl req -config openssl.cnf -key private/ca.key.pem -new -x509 -days 7300 \
> -sha256 -extensions v3_ca -out certs/ca.cert.pem
Enter pass phrase for private/ca.key.pem:
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
Country Name (2 letter code) [GB]:PL
State or Province Name [England]:lodzkie
Locality Name []:
Organization Name [Alice Ltd]:r4pt0r Test Systems
Organizational Unit Name []:
Common Name []:r4pt0r Root CA
Email Address []
$ chmod 444 certs/ca.cert.pem
$ openssl x509 -noout -text -in certs/ca.cert.pem
        Version: 3 (0x2)
        Serial Number:
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = PL, ST = lodzkie, O = r4pt0r Test Systems, CN = r4pt0r Root CA, emailAddress =
            Not Before: Feb 20 17:22:27 2018 GMT
            Not After : Feb 15 17:22:27 2038 GMT
        Subject: C = PL, ST = lodzkie, O = r4pt0r Test Systems, CN = r4pt0r Root CA, emailAddress =
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (8192 bit)
<quite a lot of data>
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Key Identifier:
            X509v3 Authority Key Identifier:

            X509v3 Basic Constraints: critical
            X509v3 Key Usage: critical
                Digital Signature, Certificate Sign, CRL Sign
    Signature Algorithm: sha256WithRSAEncryption
<quite a lot of data>

Intermediate CA

$ openssl genrsa -aes256 -out intermediate/private/intermediate.key.pem 8192
Generating RSA private key, 8192 bit long modulus
e is 65537 (0x010001)
Enter pass phrase for intermediate/private/intermediate.key.pem:
Verifying - Enter pass phrase for intermediate/private/intermediate.key.pem:
$ chmod 400 intermediate/private/intermediate.key.pem
$ openssl req -config intermediate/openssl.cnf -new -sha256 \
> -key intermediate/private/intermediate.key.pem -out intermediate/csr/intermediate.csr.pem
Enter pass phrase for intermediate/private/intermediate.key.pem:
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
Country Name (2 letter code) [GB]:PL
State or Province Name [England]:lodzkie
Locality Name []:
Organization Name [Alice Ltd]:r4pt0r Test Systems
Organizational Unit Name []:
Common Name []:r4pt0r Intermediate CA
Email Address []
$ openssl ca -config openssl.cnf -extensions v3_intermediate_ca -days 3650 \
> -notext -md sha256 -in intermediate/csr/intermediate.csr.pem -out intermediate/certs/intermediate.cert.pem
Using configuration from openssl.cnf
Enter pass phrase for ca/private/ca.key.pem:
Can't open ca/index.txt.attr for reading, No such file or directory
140341269315520:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:74:fopen('ca/index.txt.attr','r')
140341269315520:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:81:
Check that the request matches the signature
Signature ok
Certificate Details:
        Serial Number: 4096 (0x1000)
            Not Before: Feb 20 17:35:09 2018 GMT
            Not After : Feb 18 17:35:09 2028 GMT
            countryName               = PL
            stateOrProvinceName       = lodzkie
            organizationName          = r4pt0r Test Systems
            commonName                = r4pt0r Intermediate CA
            emailAddress              =
        X509v3 extensions:
            X509v3 Subject Key Identifier:
            X509v3 Authority Key Identifier:

            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:0
            X509v3 Key Usage: critical
                Digital Signature, Certificate Sign, CRL Sign
Certificate is to be certified until Feb 18 17:35:09 2028 GMT (3650 days)
Sign the certificate? [y/n]:y

1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
$ openssl x509 -noout -text -in intermediate/certs/intermediate.cert.pem
        Version: 3 (0x2)
        Serial Number: 4096 (0x1000)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = PL, ST = lodzkie, O = r4pt0r Test Systems, CN = r4pt0r Root CA, emailAddress =
            Not Before: Feb 20 17:35:09 2018 GMT
            Not After : Feb 18 17:35:09 2028 GMT
        Subject: C = PL, ST = lodzkie, O = r4pt0r Test Systems, CN = r4pt0r Intermediate CA, emailAddress =
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (8192 bit)
<quite a lot of data>
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Key Identifier:
            X509v3 Authority Key Identifier:

            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:0
            X509v3 Key Usage: critical
                Digital Signature, Certificate Sign, CRL Sign
    Signature Algorithm: sha256WithRSAEncryption
<quite a lot of data>
$ openssl verify -CAfile certs/ca.cert.pem intermediate/certs/intermediate.cert.pem
intermediate/certs/intermediate.cert.pem: OK
$ cat intermediate/certs/intermediate.cert.pem certs/ca.cert.pem > intermediate/certs/ca-chain.cert.pem
$ chmod 444 intermediate/certs/ca-chain.cert.pem

Server certificate

In the following parts, wherever [domain] appears, it should be changed to hostname of our hidden service.

At first, we need to generate certificate request (CSR) on our server:

$ openssl genrsa -out [domain].onion.key.pem 4096
Generating RSA private key, 4096 bit long modulus
e is 65537 (0x010001)
$ chmod 400 [domain].onion.key.pem
$ openssl req -config ca/intermediate/openssl.cnf \
> -key [domain].onion.key.pem -new -sha256 -out [domain].onion.csr.pem
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
Country Name (2 letter code) [GB]:PL
State or Province Name [England]:lodzkie
Locality Name []:
Organization Name [Alice Ltd]:r4pt0r Test Systems
Organizational Unit Name []:
Common Name []:[domain].onion
Email Address []:admin@[domain].onion

Then, we will sign the request with intermediate CA private key, thus issuing the certificate. But first of all, we need to receive the CSR from the server, to intermediate/csr/ directory.

$ openssl ca -config intermediate/openssl.cnf -extensions server_cert -days 375 \
> -notext -md sha256 -in intermediate/csr/[domain].onion.csr.pem -out intermediate/certs/[domain].onion.cert.pem
Using configuration from intermediate/openssl.cnf
Enter pass phrase for ca/intermediate/private/intermediate.key.pem:
Can't open ca/intermediate/index.txt.attr for reading, No such file or directory
139810167087040:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:74:fopen('ca/intermediate/index.txt.attr','r')
139810167087040:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:81:
Check that the request matches the signature
Signature ok
Certificate Details:
        Serial Number: 4096 (0x1000)
            Not Before: Feb 20 17:52:13 2018 GMT
            Not After : Mar  2 17:52:13 2019 GMT
            countryName               = PL
            stateOrProvinceName       = lodzkie
            organizationName          = r4pt0r Test Systems
            commonName                = [domain].onion
            emailAddress              = admin@[domain].onion
        X509v3 extensions:
            X509v3 Basic Constraints:
            Netscape Cert Type:
                SSL Server
            Netscape Comment:
                OpenSSL Generated Server Certificate
            X509v3 Subject Key Identifier:
            X509v3 Authority Key Identifier:
                DirName:/C=PL/ST=lodzkie/O=r4pt0r Test Systems/CN=r4pt0r Root CA/

            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Server Authentication
Certificate is to be certified until Mar  2 17:52:13 2019 GMT (375 days)
Sign the certificate? [y/n]:y

1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
$ openssl x509 -noout -text -in intermediate/certs/[domain].onion.cert.pem
        Version: 3 (0x2)
        Serial Number: 4096 (0x1000)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = PL, ST = lodzkie, O = r4pt0r Test Systems, CN = r4pt0r Intermediate CA, emailAddress =
            Not Before: Feb 20 17:52:13 2018 GMT
            Not After : Mar  2 17:52:13 2019 GMT
        Subject: C = PL, ST = lodzkie, O = r4pt0r Test Systems, CN = [domain].onion, emailAddress = admin@[domain].onion
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (4096 bit)
<quite a lof of data>
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints:
            Netscape Cert Type:
                SSL Server
            Netscape Comment:
                OpenSSL Generated Server Certificate
            X509v3 Subject Key Identifier:
            X509v3 Authority Key Identifier:
                DirName:/C=PL/ST=lodzkie/O=r4pt0r Test Systems/CN=r4pt0r Root CA/

            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Server Authentication
    Signature Algorithm: sha256WithRSAEncryption
<quite a lot of data>
$ openssl verify -CAfile intermediate/certs/ca-chain.cert.pem intermediate/certs/[domain].onion.cert.pem
intermediate/certs/[domain].onion.cert.pem: OK

httpd configuration

Finally, we can use generated files to set up HTTPS encryption on webserver. For this, I am using httpd as it is the most common webserver in use. We need following files:

  1. [domain].onion.key.pem – this is private key, that will be used to set up TLS session
  2. [domain].onion.cert.pem – this is certificate that will prove our identity, so web browser will not display any warnings as long as we will have CA certificate installed
  3. ca-chain.cert.pem – this is chain of certificates we created together with intermediate CA, that consists of both CAs – root and intermediate

Below is httpd configuration file, after enabling TLS:

Listen 666

<VirtualHost *:666>
    DocumentRoot "/home/r4pt0r/tor/hs/public_html"
    ErrorLog "[path]/tor/hs/error_log"
    CustomLog "[path]/tor/hs/access_log" common
    ScriptAlias /cgit/ "/usr/lib/cgit/cgit.cgi/"
    Alias /cgit-css "/usr/share/webapps/cgit/"
    SSLEngine on
    SSLCertificateFile "[path]/tor/hs/tls/[domain].onion.cert.pem"
    SSLCertificateKeyFile "[path]/tor/hs/tls/[domain].onion.key.pem"
    SSLCACertificateFile "[path]/tor/hs/tls/ca-chain.cert.pem"

As can be seen above, all necessary files had been moved to tls directory of our hidden service main directory.

Afterwards, one slight change is needed in torrc file:

HiddenServicePort 443

From now on, we need to use https://[domain].onion to visit our site, as it is now TLS-encrypted and using port 443, which is default for HTTPS. For convenience, we can set up another httpd vhost on different port, that will redirect all HTTP traffic through HTTPS and link it to port 80, so remembering about https in address will not be necessary. But, it is only optional, so I will leave it as an exercise to the reader.


From this point it is useful to have Firefox that is not constantly reminding about insecure connection. To prevent this, we should install CA certificate into Firefox. One remark here: as we are going to hack Firefox to trust our certificate, now our whole browsing through that instance of Firefox relies on our CAs private key. So, it is best to not use the same instance for anything else unless you are really sure, the private keys for both root and intermediate are perfectly secure.

To install the certificate, follow the screenshots below:

Firefox privacy preferences
On preferences page, got to security and scroll all the way down to View Certificates button

Firefox CM - Authorities

Firefox - Downloading Certificate
Confirm that this certificate will be able to identify websites
Firefox - page secure
Finally, we are secure and no exclamation mark appears!

Setting up new v3 Hidden Service with ultimate security: Part 1: Hidden Service setup

This post is a part of Tor v3 tutorial. Other parts are:

  1. Hidden Service setup
  2. PKI and TLS
  3. Client Authentication
  4. Installing client certificates to Firefox for Android

As a student I was lucky to have unlimited private Git repositories on Github, since they introduced that to their first paid plan. Unfortunately, I don’t have access to educational e-mail anymore, so I won’t be able to renew the service. This leads to a need to have that feature migrated to somewhere else. Some time ago, I installed cgit and gitolite on my single board computer (SBC). But, because of Github, there was no need to use that. Now it seems like a good replacement to Github’s Developer plan.

Few weeks ago, there was interesting event – Tor Project introduced new version of their Hidden Services – v3, which changes length of hidden service address in .onion domain and disables “feature” enabling some nodes in the network to index all existing service addresses. This seems like a good moment to give it a try and check, how fast (or rather how slow) will be the solution providing git through Tor on few-year-old SBC. By the way, I will show, how to configure things with maximum security in mind.

Disclaimer: I am not a person with deep knowledge of inner workings of Tor network, so I strongly encourage you to read thing or two, about how to use it safely. This article might contain errors that might reveal your identity, especially when used together with not-self-owned hidden services.


Let’s start with summary of what we will need to make Tor v3 work:

  • tor in version or higher
  • alternatively Tor Browser 7.5 or higher
  • for Android: Orbot and Orfox (at the moment of writing this, there is no support in current version of Orbot, so custom compilation is required – I am using Termux to provide tor binary)
  • httpd or any other HTTP server, able to provide service with only one vhost on separate TCP port

Because of the way, I am planning to configure hidden service in future, it might be a good idea to set up separate Tor browser at this moment, dedicated to this service, if it is going to be production configuration. If this is just an experiment, this advice could safely be ignored. However it is good to know, how to undo any modifications to the browser that will be done in the next parts.


What we need to do is to listen on localhost, on some random TCP port. Then we will set up httpd to provide only one virtual host on this custom port. It would be perfect to disable any other vhosts as our hidden service will work also as non-hidden service for local users, so if other service is buggy and allows to connect to other local services (see e.g. DNS rebinding), at least address of our hidden service will be compromised.

I have following configuration:

Listen 666

<VirtualHost *:666>
    ServerAdmin [email]@[domain]
    DocumentRoot "[path]/public_html"
    ServerName [domain].onion
    ErrorLog "[path]/error_log"
    CustomLog "[path]/access_log" common

<Directory "[path]/public_html">
    DirectoryIndex index.html index.php index.txt
    AllowOverride All
    Options FollowSymlinks
    Require all granted

Furthermore, httpd must be able to traverse to public_html directory, so every directory from public_html up to root must have execute privilege for http user and directory itself as well as its content must be available (or better owned) by http.

After that and after starting httpd, it should be possible to visit http://localhost:666 via web browser and see content of public_html directory. If this is true, we can move on to tor configuration.


SocksPort auto

HiddenServiceDir /etc/tor/hsv3
HiddenServiceVersion 3
HiddenServicePort 80

SafeLogging 0
Log notice stdout
Log notice file /etc/tor/hsv3/hs.log
Log info file /etc/tor/hsv3/hsinfo.log

Now, on the first startup of tor,Β  it should create keys for our new hidden service. We can look into /etc/tor/hsv3/hostname to see the .onion address. It is good idea to set key files and hostname file as readable as only user running tor service. In case of service started by systemd, this will probably be tor by default.

After starting tor service (systemctl start tor in case of systemd), we can check if everything works properly by visiting our hidden service with tor-enabled browser (using tor or higher). That’s it.

Firefox for Android

At the time of writing this article there is still no upgrade for Orbot app, providing GUI interface for tor. Because of that, it might be required to use ordinary Firefox to use tor as a proxy, which is generally bad idea for connecting to any hidden services, because of privacy and anonymity. Fortunately, we can live with revealing our identity to ourselves πŸ™‚ so we can do it only this single time.

What we need to change are following configuration options, available under about:config page:

  • network.proxy.socks to localhost
  • network.proxy.socks_port to 9050
  • network.proxy.socks_remote_dns to true
  • network.proxy.socks_version to 5, if any other (should be default)
  • network.proxy.type to 1 (0 means no proxy, 5 is system proxy)


Now we are ready to use our hidden service, from both desktop and mobile. Still, we use only HTTP protocol, which is not a big problem, as tor already provides encryption. Neverheless our next goal would be to configure HTTPS. And then we will configure client authentication for ultimate security of our hidden service.

LKV373A: porting objdump

This article is part of series about reverse-engineering LKV373A HDMI extender. Other parts are available at:

After part number four, we already have ELF file, storing all the data we found in firmware image, described in a way that should make our analysis easier. Moreover, we have ability to define new symbols inside our ELF file. The next step is to add support for our custom architecture into objdump and this is what I want to show in this tutorial.

Finding best architecture to copy

If we want to set up new architecture in objdump code, we need to learn interfaces that need to be implemented. It would be easier if we can use some existing code to do so. After some looking into the binutils’ code I learned that what is of special interest are bfd and opcodes libraries. They contain code dedicated to particular architectures. The first one seem to be related to object file handling (which in our case is ELF), so we should not tinker with it too much. Second one is related to disassembling binary programs, so is what we are looking for.

I did some quick examination of source code related to popular architectures and it seems not to be easy to adjust to our needs. Architecture I found to be best suitable for modification is Microblaze. Its source seem to be quite well-written, clean and short. Also from my research of architecture name for LKV373A (part 2, failed by the way) I also remember it is quite similar to the one present in LKV373A, so it is even better decision to use it.

Compiling objdump for target architecture

At first it is useful to learn how to compile objdump, so it will be able to disassemble program written for our target. Microblaze is not really a mainstream architecture, so there aren’t many programs compiled for it available online after typing 'microblaze program elf' into usual search engine. However, I was able to find 2 of them, so I was able to verify that compilation worked. If you can’t find any, I uploaded these to MEGA, so they can serve as test cases. First one is minimal valid file, the other one is quite huge.

Compilation is very easy. The only thing that needs to be done beside usual ./configure && make && make install is adding target option to configure script. So, the script looks as follows:

./configure --target=microblaze-elf

Of course, install step can safely be skipped as well as compilation of other tools, beside objdump. objdump itself seem to be built using make binutils/objdump. However it can’t be build successfully using that shortcut, so whole binutils package must be configured the way, everything not buildable is excluded from the build.

Setting up own architecture

Next step is to add support for our brand new, custom architecture to binutils’ configuration files and copy microblaze sources, so they will simulate our architecture, until we will write our own implementation. Then it should be possible to test objdump again, against our sample microblaze programs and disassembly should still work.

Even without any modification to binutils’ source or configs, it should be possible to configure it for any random architecture. The only constraint is format of the target string: ARCH-OS-FORMAT, where FORMAT is most likely to be elf. So, if we pass lkv373a-unknown-elf as target, it will work. -unknown part is usually skipped and this will not work. If we need it to work, config.sub must be modified. config.sub is used to convert any string, passed to configure into canonical form, so in our case lkv373a-unknown-elf. If it detects, that it is already in canonical form, it does nothing.

Final configure command will be slightly more complex, as we have to disable some parts, that are not of our interest and requires additional effort to work:

./configure --target=lkv373a-unknown-elf --disable-gas --disable-ld --disable-gdb

Although passing something random as target option works on configure stage, it will obviously fail on make stage. What make is doing at first is configuring all the sublibraries. What is of our interest is bfd and opcodes. And the first one fails. So this is the first problem, we need to get rid of.


The purpose of this file is to set some environment variables depending on target architecture. If it does not know the architecture, it returns error to caller, which is probably bfd’s configure script, called by make. According to documentation in file header, it sets following variables:

  1. targ_defvec – default vector. This links target to list of objects that will provide support for ELF file built for specific architecture (stored in bfd/
  2. targ_selvecs – list of other selected vectors. Useful e.g. when we need support for both 32- and 64-bit ELFs. Not needed here.
  3. targ64_selvecs – 64-bit related stuff. Used when target can be both 32- and 64-bit, meaningless in our case.
  4. targ_archs – name of the symbol storing bfd_arch_info_type structure. It provides description of architecture to support.
  5. targ_cflags – looks like some hack to add extra CFLAGS to compiler. We don’t care.
  6. targ_underscore – not sure what it is, should have no impact on our goals (possible values are yes or no)

To sum up, what we need to do on this step is to define default vector, we will later add to and set name of architecture description structure. The structure itself will be defined later. Finally, I ended up with the following patch:

@@ -173,6 +173,7 @@ hppa*)     targ_archs=bfd_hppa_arch ;;
 i[3-7]86)   targ_archs=bfd_i386_arch ;;
 i370)     targ_archs=bfd_i370_arch ;;
 ia16)     targ_archs=bfd_i386_arch ;;
+lkv373a)  targ_archs=bfd_lkv373a_arch ;;
 lm32)           targ_archs=bfd_lm32_arch ;;
 m6811*|m68hc11*) targ_archs="bfd_m68hc11_arch bfd_m68hc12_arch bfd_m9s12x_arch bfd_m9s12xg_arch" ;;
 m6812*|m68hc12*) targ_archs="bfd_m68hc12_arch bfd_m68hc11_arch bfd_m9s12x_arch bfd_m9s12xg_arch" ;;
@@ -924,6 +925,10 @@ case "${targ}" in

+  lkv373a*-*)
+    targ_defvec=lkv373a_elf32_vec
+    ;;
   lm32-*-elf | lm32-*-rtems*)


Now we need to define vector, we just declared to use for lkv373a architecture.

505     k1om_elf64_fbsd_vec)         tb="$tb elf64-x86-64.lo elfxx-x86.lo elf-ifunc.lo elf-nacl.lo elf64.lo $elf"; target_size=64 ;;
506     lkv373a_elf32_vec)           tb="$tb elf32-lkv373a.lo elf32.lo $elf" ;;
507     l1om_elf64_vec)              tb="$tb elf64-x86-64.lo elfxx-x86.lo elf-ifunc.lo elf-nacl.lo elf64.lo $elf"; target_size=64 ;;

Unfortunately, as we did modifications to .ac script, we now need to rebuild our configure. From my experience, any tinkering with autohell, after solving one problem, creates 5 more. We need to get into bfd directory and reconfigure project:

cd bfd

Now, if it worked for you, you should definitely go, play some lottery πŸ™‚ . For me it said that I need exactly same version of autoconf as used by binutils’ developers. Because autoconf is so great, probably what I will show now is completely useless for anyone, but hacks I needed to do are at first to add:

20 m4_define([_GCC_AUTOCONF_VERSION], [2.69])

to the beginning of file. Then bfd/doc/ contains removed cygnus option at the beginning, in AUTOMAKE_OPTIONS, so we need to remove it. After that doing automake --add-missing, as autoreconf suggests, and then again autoreconf should solve the problem. But, as I said, this will probably not work for you. I can only wish you good luck.

(if were following the steps, you might have noticed that autoconf complained about not being in version 2.64 and we overridden version from 2.69 to 2.69 and it worked πŸ™‚ , don’t ask me, why, please!)

After this step, compilation should start (and obviously will fail miserably on bfd as it misses few symbols). Now its time to make bfd compilable.


This file is meant to provide support for custom features of ELF file. As we don’t have any, we can safely do nothing here. Good template of such file is elf32-m88k.c as it does exactly this.

One thing that seem to be important here is EM value of architecture described. EM is an enum used in ELF file to define target architecture, so it might be required to adjust in our new elf32-lkv373a.c file. By the way definition of this value have to be added to include/elf/common.h:

433 /* LKV373A architecture */
434 #define EM_LKV373A              0x373a

It might also be a good idea to add it to elfcpp/elfcpp.h. To make the file compile, it is necessary to add following to bfd/bfd-in2.h as value of bfd_architecture enum:

2398   bfd_arch_lkv373a,    /* LKV373A */


As we declared bfd_lkv373a_arch as symbol with CPU description structure, we now need to add this declaration to archures.c, as this is the file, where it will be used. We have to add:

611 extern const bfd_arch_info_type bfd_l1om_arch;
612 extern const bfd_arch_info_type bfd_lkv373a_arch;
613 extern const bfd_arch_info_type bfd_lm32_arch;


Similar situation is in targets.c file. Here we have to provide declaration of our vector as bfd_target. This will be another structure, which seem to be generated automatically, so we should not care about it.

704 extern const bfd_target l1om_elf64_fbsd_vec;
705 extern const bfd_target lkv373a_elf32_vec;
706 extern const bfd_target lm32_elf32_vec;


This last file, we need in bfd, provides bfd_arch_info_type structure and… that’s it! Can be easily borrowed from cpu-microblaze.c with only slight modifications. One thing that needs explanation here isΒ section_align_power. As far as I understand it, it is power of two to which the beginning of the section in memory must be aligned. It should be safe to put 0 here, as we are not going to load our ELF into memory.

This should close the bfd part of initialization. As you can see, there was no development at all to be done here. Let’s now go to opcodes library.


At first we need to define objects to build for LKV373A architecture in opcodes library. This is quite similar to what we had to do in of bfd library.

282         bfd_iq2000_arch)        ta="$ta iq2000-asm.lo iq2000-desc.lo iq2000-dis.lo iq2000-ibld.lo iq2000-opc.lo" using_cgen=yes ;;
283         bfd_lkv373a_arch)       ta="$ta lkv373a-dis.lo" ;;
284         bfd_lm32_arch)          ta="$ta lm32-asm.lo lm32-desc.lo lm32-dis.lo lm32-ibld.lo lm32-opc.lo lm32-opinst.lo" using_cgen=yes ;;

Hopefully, -dis file will be enough to be implemented. I’ve made a copy from microblaze configuration. The same way we will copy whole source file and any related headers in the next step.

Now, similarly to bfd’s, we have to reconfigure it. And again, nobody knows what errors we will encounter.


The only thing that have to be done here is to set pointer of disassemble function. For this following snippets should be added:

53 #define ARCH_lkv373a
255 #ifdef ARCH_lkv373a
256     case bfd_arch_lkv373a:
257       disassemble = print_insn_lkv373a;
258       break;
259 #endif

And to disassemble.h:

62 extern int print_insn_lkv373a           (bfd_vma, disassemble_info *);


This is, where real stuff will happen. As our goal, for now, is not to make implementation of LKV373A architecture, but rather set everything up, so objdump will build, we can copy source file from microblaze-dis.c. It is also required to copy headers, related to MicroBlaze, used by this file, so:

  • opcodes/microblaze-dis.h
  • opcodes/microblaze-opc.h
  • opcodes/microblaze-opcm.h

And change include directives in them to link to lkv373a file, rather than microblaze ones.

Now, optionally we could change names of any symbols referring to name microblaze, but this should not be required, as original microblaze files should not be included in the build. The only change than need to be done is print_insn_microblaze into print_insn_lkv373a, as this is what we added to disassemble.c.

You should now be able to compile working objdump with LKV373A support (of course with wrong implementation, for now). We can now verify that everything works on slightly modified ELF file for MicroBlaze architecture (EM field must point to LKV373A – value must be 0x373a). Well done!

NOTE: all the steps, done till now are available on tutorial-setup tag in repository on Github.

Functions to implement

Now, finally the real fun starts. Bindings between opcodes library and objdump itself, require at leastΒ print_insn_lkv373a to be implemented.

What should happen inside this function is quite simple and can be described in following steps:

  1. Gets bfd_vma and struct disassemble_info (called info below) as parameters
  2. Read raw data containing instructions using info->read_memory_func
  3. Call info->memory_error_func in case of any errors
  4. Use info->fprintf_func to print disassembled instruction into info->stream
  5. Optionally use info->symbol_at_address_func to determine if there is any symbol declared at address decoded from instructions
  6. If symbol exists, call info->print_address_func
  7. Return number of bytes consumed

Following is some documentation, I wrote for easier implementation (mostly translated inline comments), of functions to be called:

   * \brief Function used to get bytes to disassemble
   * \param memaddr Address of the current instruction
   * \param myaddr Buffer, where the bytes will be stored
   * \param length Number of bytes to read
   * \param dinfo Pointer to info structure
   * \return errno value or 0 for success
  int (*read_memory_func)
    (bfd_vma memaddr, bfd_byte *myaddr, unsigned int length,
     struct disassemble_info *dinfo);
   * \brief Call if unrecoverable error occurred
   * \param status errno from read_memory_func
   * \param memaddr Address of current instruction
   * \param dinfo Pointer to info structure
  void (*memory_error_func)
    (int status, bfd_vma memaddr, struct disassemble_info *dinfo);
   * \brief Pointer to fprintf
   * \param stream Pass info->stream here
   * \param char Format string
   * \param ... vargs
   * \return Number of characters printed
  typedef int (*fprintf_ftype) (void *, const char*, ...) ATTRIBUTE_FPTR_PRINTF_2;
   * \brief Determines if there is a symbol at the given ADDR
   * \param addr Address to check
   * \param dinfo Pointer to info structure
   * \return If there is returns 1, otherwise returns 0
   * \retval 1 If there is any symbol at ADDR
   * \retval 0 If there is no symbol at ADDR
  int (* symbol_at_address_func)
    (bfd_vma addr, struct disassemble_info *dinfo);
   * \brief Print symbol name at ADDR
   * \param addr Address at which symbol exists
   * \param dinfo Pointer to info structure
  /* Function called to print ADDR.  */
  void (*print_address_func)
    (bfd_vma addr, struct disassemble_info *dinfo);

For easier start of development, this commit can be used as template. You can find effects of implementation according to this description on lkv373a branch of my binutils fork on Github. After this step, you should have working objdump, able to disassemble architecture of your choice.

Alternative way

According to binutils’ documentation, porting to new architectures should be done using different approach. Instead of copying sources from other architectures, developers should write CPU description files (cpu/ directory) and then use CGEN to generate all necessary files. However, I found these files way too complicated comparing to goal, I wanted to achieve, therefore I used the shortcut. In reality, however, this might be a better way, as the final result should be the support for new architecture not only in objdump, but also in e.g. GAS (GNU assembler). If you want to go that way, another useful resource might be description of CPU description language.

Plans for the future

As I am now able to speed up reverse engineering of both instruction set and LKV373A firmware, I am planning to create public repository of my progress and guess operations done by some more opcodes as I already know only few of them. So, I will probably push some more commits to binutils repo as well. I hope this will enable me to gain some more knowledge about LKV373A and allow, me or someone else, to reverse engineer second part of the firmware, which seem to be way more interesting that the one, I was reverse engineering till now.

LKV373A: crafting ELF

This article is part of series about reverse-engineering LKV373A HDMI extender. Other parts are available at:

As we should now be able to follow any jump present in the code, it is now time to make analysis more automatic. My target tool for that purpose will be objdump. However, we still have firmware image as raw dump of memory. To be able to use objdump easily, we need to pack our firmware into some container understandable by objdump. Most obvious choice is ELF (Executable and Linkable Format) and this is what I am going to use.

For the purpose of packing data into ELF, I’ve made Python library that makes it easier. For now, it is able to split firmware image into sections, like .text or .data, so objdump will be able to disassembly only the parts of firmware that are in fact a code. Moreover, it can define symbols inside the binary, so it is possible to store information, where certain functions starts and ends, same for any variables, like strings. As of now, there is no CLI interface for the program. If it turns out that such interface is necessary (like for addition of many symbols), it will be added.

Library code can be downloaded from Github. Currently, any LKV373A-specific modifications to this library is stored on branch lkv373a, to not rubbish main – master branch. Throughout this tutorial, I assume, we are using code on this branch, so there might be some LKV373A-specifics, especially regarding enum types (i.e. processor architecture enum).

At this point, I need to warn, that I am not going to describe internal structure of ELF file, nor any features that might be visible from outside, like sections concept, so if you are not familiar with them, it is good time to learn about them, as it might be very difficult to understand, what I am writing about. There are many good resources explaining them. Ones I was using are: this blog post and this documentation.

Creating new ELF

Example code that creates brand new ELF file is as easy as:

 1 #!/usr/bin/python3
 2 # demo script for creating ELF
 3 import os
 4 from elf import *
 6 elf = ELF(e_machine=EM.EM_LKV373A)
 8 fp ='lkv373a.fw.elf',os.O_CREAT|os.O_WRONLY)
 9 os.write(fp, bytes(elf))
10 os.close(fp)

This, at first does all necessary imports, then creates new ELF object in line 6, and, finally, converts it to bytes object and immediately writes to file descriptor. That’s it!

After this, you should get valid, empty ELF file for architecture called lkv373a, which, obviously does not exists and no other program know how to handle, but we are going to change that in future.

While creating ELF object, few things can be defined, in addition to architecture id. They are all described in documentation, I will mention near the end of this tutorial. You are also free to dig in structure of ELF object. There is no encapsulation in it and structure validation is very permissive, so even completely broken ELFs could be produced, if needed.

Adding section

Next step is to add some sections to our ELF file.

fw ='LKV373A_TX_V3.0c_d_20161116_bin.bin',os.O_RDONLY)
fw_blob =, 0xffffffff)

irq_blob = fw_blob[:0x1000]
text_blob = fw_blob[0x7d100:0x7d100+0x0b53c0]
data_blob = fw_blob[0x0b53c0:0x0b53c0+0x102060]
smedia_blob = fw_blob[0x200000:0x200000+0x283105]

irq_id = elf.append_section('.irq',irq_blob,0)
txt_id = elf.append_section('.text',text_blob,0x7d100)
data_id = elf.append_section('.data',data_blob,0x0b53c0)
smedia_id = elf.append_section('.smedia',smedia_blob, 0x200000)

At first, I am extracting them from firmware image and then inserting them to ELF object. append_section is a handy wrapper to low-level modifications that must be done on ELF structure, hidden under what we can see as ELF instance (these low-level structures are, however still available to the user as ELF.Elf member).

Modifying section attributes

Ok, so now we have sections in our ELF file, ready to save to disk. Before that, one thing can yet be done: setting proper attributes. They tell readers, if program is able to write or execute sections of memory, among other features, I am going to ignore here. This might be useful, as some readers might be confused about what is code (text) and what is data. In our case, we have two text sections (.irq and .text), so we are going to set them executable flag (SHF_EXECINSTR). Furthermore, we will set SHF_ALLOC flag for any section that is going to be loaded into memory (so all of them).

This can be done with:

elf.Elf.Shdr_table[irq_id].sh_flags = SHF.SHF_ALLOC | SHF.SHF_EXECINSTR
elf.Elf.Shdr_table[txt_id].sh_flags = SHF.SHF_ALLOC | SHF.SHF_EXECINSTR
elf.Elf.Shdr_table[data_id].sh_flags = SHF.SHF_ALLOC
elf.Elf.Shdr_table[smedia_id].sh_flags = SHF.SHF_ALLOC

Adding segment

Segments are another concept, existing beside sections. They are stored in program header of ELF file and are somehow linked to section data. They allow to define another set of attributes to areas in memory. I don’t think they will be required to define, to perform analysis in objdump, but since at least one such program header, defining segment must exist in ELF file of type executable, there is interface similar to this for sections.

To define new segment, based on .text section, you can issue:

elf.append_segment(txt_id, flags='rx')

This also marks the segment as read and executable, but not writable.

Loading existing ELF

Loading existing ELF can be easily done from file with:

newelf, b = ELF.from_file('lkv373a.fw.elf')

Alternatively, it can also be loaded from bytes object:

fd ='some.elf', os.O_RDONLY)
b =, 0xffff)
manualelf, b = Elf32.from_bytes(b)

In latter case, I assumed that os library is already imported into python.

Adding a symbol

This is very useful for making analysis of code. New symbol can be added using calls like:

elf.append_symbol('irq0', irq_id, 0, 0x44, STB.STB_GLOBAL, STT.STT_FUNC)
elf.append_symbol('sprintf', txt_id, 0x9b9f8-0x7d100, 0x78, STB.STB_GLOBAL,
elf.append_symbol('thread_c_path', data_id, 0xba78a-0x0b53c0, 0xba7bb-0xba78a,

First call defines function of length 0x44 in .irq section. To do this, ID of .irq section must be known. Luckily, we want to add symbol at the beginning of the section, so as offset, 0 was provided.

In the second case, we also want to define a function, but now we only know absolute address of the function (0x9b9f8), but what we need to pass is offset in .text section. To achieve this, we need to subtract address of the start of .text section (0x7d100).

In the last example, we define a string as an object of certain address and length. Both address and length are computed by subtracting absolute addresses. This symbol will be marked as local, which is default behavior for append_symbol function.

Library documentation

There are many more things possible to do using makeelf library. What I showed here is mostly, what is possible using high-level wrappers, doing many things under the hood. But as there is also low-level interface, virtually anything is possible.

To make exploring interfaces easier, I’ve made doxygen documentation for most of the library. It can be found on my server, here. Feel free to use the library for anything you want.


The library presented here should allow us make one step further to easy to use reverse engineering environment. It will by the way allow to store new findings in easily-modifiable Python scripts.

What I showed in examples to library interfaces split LKV373A firmware image into 4 sections. At this moment I already know that there are at least 6 sections, where code and data are in two parts (forming ICDCDS layout, where I-irq, C-code and so on). Also there should be some more symbols possible to place at this moment.

If I succeed in porting objdump, or any other tool able to disassemble ELF file, next step would be to publish Python script, utilizing library presented here, that annotates LKV373A firmware. So stay tuned, I hope there will be many further interesting findings throughout this reversing process!

LKV373A: reverse engineering instruction set architecture

This article is part of series about reverse-engineering LKV373A HDMI extender. Other parts are available at:

As I wrote in previous part, my choice is in fact reverse engineering instruction set. The goal of this post is not to reverse-engineer whole instruction set, because even in RISC architectures some of the instructions might be quite rarely used.

Before starting the actual reverse engineering, place where such analysis would be easiest should be identified. Then it might be possible to use often repeating patterns to guess instructions that the pattern consists of. The result of this tutorial will be description of opcodes related to jumping and few other often used opcodes, mostly related to memory operations.

Identifying target

LKV373A Encoder firmware - data region
Data region

As written above, first step is to find good target. As was shown in previous article, it is possible to find references to constants mixed into code.

In the picture on the right one especially interesting information can be seen – it seems that as an operating system FreeRTOS was used. FreeRTOS is open source project, so its source code can be downloaded.

This information could be later possibly used to link compiled code with FreeRTOS source code. Let’s look into source code to find some useful location. As we already know the encoding of opcode for an operation on strings, finding some string in code might work. I was able to identify one such place in tasks.c file. It is shown on snippet below:

4110           if( ulStatsAsPercentage > 0UL )
4111           {
4112             #ifdef portLU_PRINTF_SPECIFIER_REQUIRED
4113             {
4114               sprintf( pcWriteBuffer, "\t%lu\t\t%lu%%\r\n", pxTaskStatusArray[ x ].ulRunTimeCounter, ulStatsAsPercentage );
4115             }
4116             #else
4117             {
4118               /* sizeof( int ) == sizeof( long ) so a smaller
4119               printf() library can be used. */
4120               sprintf( pcWriteBuffer, "\t%u\t\t%u%%\r\n", ( unsigned int ) pxTaskStatusArray[ x ].ulRunTimeCounter, ( unsigned int ) ulStatsAsPercentage );
4121             }
4122             #endif
4123           }
4124           else
4125           {
4126             /* If the percentage is zero here then the task has
4127             consumed less than 1% of the total run time. */
4128             #ifdef portLU_PRINTF_SPECIFIER_REQUIRED
4129             {
4130               sprintf( pcWriteBuffer, "\t%lu\t\t<1%%\r\n", pxTaskStatusArray[ x ].ulRunTimeCounter );
4131             }
4132             #else
4133             {
4134               /* sizeof( int ) == sizeof( long ) so a smaller
4135               printf() library can be used. */
4136               sprintf( pcWriteBuffer, "\t%u\t\t<1%%\r\n", ( unsigned int ) pxTaskStatusArray[ x ].ulRunTimeCounter );
4137             }
4138             #endif
4139           }
4141           pcWriteBuffer += strlen( pcWriteBuffer );
vTaskGetRunTimeStats printf format
vTaskGetRunTimeStats formatting strings

If we look at code before the snippet, we can see that the function is surrounded with ifdefs and is meant to be turned on only for demo purposes. Also searching for complete formatting string from sprintf function above fails on LKV373A firmware. Fortunately it is present in code and happens to have some modifications. One of them is the formatting string we were searching for. I was able to identify them starting at offset 0xbab2f. You can see it on hexdump. What is a bit surprising is that there are four such strings, while we expected only two in whole code. But "IDLE" string after them is confirming that it must be tasks.c module.

Now we can use method shown on previous tutorial about processor identification to find references to these strings. Finally I found usages of offsets 0xbab5c and 0xbab73 (marked in green and blue) near offset 0x91fd4.

At this moment, we have machine code and source code that is very likely to be compiled one to one into this machine code. We can also see here very useful side-effect of open source popularity: we have a system that has quite unusual function and is using open source software. So we can conclude that we can be almost sure that any random part of code also has open source software in itself.

Code patterns

As we already have quite reliable anchor for our analysis, we could try identifying more opcodes. But, to make things easier, I want to go the pattern matching way. Whichever architecture you analyze, you see some patterns that are same or almost the same on any architecture. This is especially true on RISC architectures, as they have very limited set of functions, so compiler have to join two or more instructions to get desired high level functionality. It this section, I will describe some of such patterns, I was able to identify and decode in LKV373A firmware. They are:

  • Function call
  • Function prologue and epilogue
  • Read-only memory access
  • Compare and jump
  • Variadic functions

Following is short description of the above patterns.

Function call

This is main element of ABI (Application Binary Interface) from the point of view of a programmer. Therefore it should also be well-known, even to people not involved in assembly programming or reverse-engineering. It is all about the method of passing arguments, before a call to a function.

Let’s see how such a call looks on MIPS architecture:

lw      gp,24(sp)
lw      s0,-32736(gp)
addiu   s0,s0,6872
lhu     s0,0(s0)
addiu   a0,sp,34        # a0 = sp + 34
move    a1,zero         # a1 = 0
li      a2,16           # a2 = 16
move    s7,v0
sh      s0,32(sp)
lw      t9,(get_sr_name_ptr - 0x1000BE50)(gp)
jalr    t9

As we can see in case of MIPS, arguments are passed in registers named a0, a1, a2 and so on. Then address of function to call is loaded to t9 and jalr (jump and link register) is performed. Usually, in case of ABI, where arguments are passed in registers, when number of arguments is greater than number of such registers, they are passed through memory (i.e. stack).

Function prologue and epilogue

When performing a call to different function, state of the processor have to be preserved, so after the call it is again the same as before (with exception of few registers, used e.g. for return value passing). This operation may be done by caller or callee. Let’s look again at MIPS code to see how it works there:

sw      ra,5328(sp) 
sw      s8,5324(sp) 
sw      s7,5316(sp) 
sw      s6,5312(sp) 
sw      s1,5292(sp) 
sw      s0,5288(sp) 
sw      gp,5320(sp) 
sw      s4,5304(sp) 
sw      s3,5300(sp) 
sw      s2,5296(sp) 

I think there is nothing special to comment here. The opposite happens on function epilogue and additionally, immediately after that return instruction should appear.

Read-only memory access

This one is already partially analyzed on previous part of this series. It is usually appearing where some strings need to be used in code. Strings written in code as literals are stored in part of the memory where they shouldn’t be modified. Then theirs addresses are computed using some base register, or directly if code is not relocatable. This is how it works on MIPS:

lw      t9,-30404(gp)

But, as we can see on previous post, we have something missing on our mysterious architecture. There, address was computed as $3=$3+0xbadadd. So, we should expect that register used should be set to something before that.

Compare and jump

This is after calling conventions, another extremely popular scheme. It usually consists of two opcodes. At first two numbers are compared, and some flags in processor are set. Then based on the state of one of the flags, jump is performed, or processing is continued if flag has value different than we expect. This time, let’s see how it works on x86 platform, as MIPS uses a bit different philosophy:

cmp     [ebp+arg_4], eax
jnz     loc_404CEB

On x86 cmp instruction causes the subtraction of two parameters, without actually storing the result, but only updating the FLAGS register, so it is known if the value is zero, or the overflow happened, and so on. Then based on flag value (in this case if zero flag is not set), jump occurs, or not.

Variadic function

Variadic function is function that can get variable number of parameters. The most popular example of such function is printf. It accepts format string and parameters, which number depend on format string. On system, where parameters are passed through registers, I expect it to get format through register and rest of parameters through stack or dedicated structure, so generally memory. Once we know how constants are accessed, it should be quite easy to identify, as it most likely will get format string as one of the first parameters, and somewhere close to that parameters should be stored, one after another.

Identifying patterns

Now, as we know what pattern we will look for, it is time to find them in code and guess functions of particular opcodes.

Read-only memory access

Let’s look at area near reference to our format string:

00091FB8                 .word 0x1000000A         # 04: ? 0x0A
00091FBC                 .word 0xD401D00C         # 35: ? $0, $1, $26, 0x0c
00091FC0                 .word 0x18600010         # 06: ? $3, $0+0x10
00091FC4                 .word 0x1880000B         # 06: ? $4, $0+0x0b
00091FC8                 .word 0xD4012810         # 35: ? $0, $1, $5, 0x10
00091FCC                 .word 0xA8638608         # 2A: la $3, $3+0x8608
00091FD0                 .word 0x400268A          # 01: ? 0x268a
00091FD4                 .word 0xA884AB5C         # 2A: la $4, $4+0xab5c
00091FD8                 .word 9                  # 00: ? 0x09
00091FDC                 .word 0xBC160000         # 2F: ? $0, $22
00091FE0                 .word 0x18600010         # 06: ? $3, $0+0x10
00091FE4                 .word 0x1880000B         # 06: ? $4, $0+0x0b
00091FE8                 .word 0xA8638608         # 2A: la $3, $3+0x8608
00091FEC                 .word 0xA884AB73         # 2A: la $4, $4+0xab73
00091FF0                 .word 0x4002682          # 01: ? 0x2682
00091FF4                 .word 0x15000000         # 05: ?

After splitting instruction words into parts and decoding opcode, register and immediate values, we can see that string’s address is based on register $4 and stored also in register $4. If we look few lines upwards, we can see that register $4 is computed based on register $0 and offset 0x0b (marked in orange). Register $0 is often used to always store value 0. Now, if we look at original offset of string in firmware, we can see that it is 0xbab5c! So that instruction must store immediate value in register’s higher half. Therefore we just guessed function of opcode 06. Later this opcode will be described as lh (load high).

By the way we also discovered that almost surely firmware image is mapped to address 0 after loading to operational memory or more likely mapping EEPROM to address space.

Compare and jump

In the snippet above, another one thing is quite interesting. And weird at the same time. There are few instructions that seem to not contain any register encoded and have weird uneven offsets, often with quite low values. At this moment my theory is that shorter ones are some kind of jumps (like 0x91fd8) and longer ones are function calls (like 0x91fd0). Then it is time to try to find compare and jump pattern.

If we are right, then opcodes like 00, 04 are jumps and 01 means a call. After this section we should also tell conditional and unconditional jumps apart.

Ok, so we now need to go back to source code and find some good candidate for conditional jump. It should be as close to format string as possible. If we look at the snippet from Identifying target section, we can see one such check on line 4110. It checks for value being greater than zero. Going upwards a little bit, we encounter one 04 opcode, immediately preceded by 2F instruction:

00091FB0                 .word 0xD4011808         # 35: ? $0, $1, $3, 0x08
00091FB4                 .word 0xBC050000         # 2F: ? $0, $5
00091FB8                 .word 0x1000000A         # 04: ? 0x0A
00091FBC                 .word 0xD401D00C         # 35: ? $0, $1, $26, 0x0c
00091FC0                 .word 0x18600010         # 06: ? $3, $0+0x10
00091FC4                 .word 0x1880000B         # 06: ? $4, $0+0x0b

Now, if we look at occurrences of 2F opcode, we can spot that it is appearing usually near 04 opcode. However we cannot tell that they appear in exactly this order which is quite weird. On the other hand if we look at register this particular occurrence uses, it is quite likely it is compare opcode.

If we assume that 2F is cmp (compare) and 04 is jg (jump greater), we see that this more or less matches behavior we expect from the code immediately preceding sprintf from FreeRTOS source code.

However we still miss one information: what does the offset mean. If it is jump instruction, then we cannot jump 10 bytes ahead, because we would land in the middle of instruction. If we look again at source code, we can see that our jump should not go very far forward, so value should also not be too high. We can also exclude usage of register as address, because it would be register $10, which is not set anywhere near jump.

Having no other idea, I did an experiment. I multiplied jump value by 4, because length of instruction is always 4 and added next instruction address to result. Then I checked what is there and… bingo! It jumped above the sprintf call and ended up immediately after it. Some time later, I discovered that it is not completely truth. It happens that real formula is:

addr = imm * 4 + PC

Where imm is instruction argument and PC is program counter before executing the instruction (so address of jump opcode).

The question still is how more sophisticated compares are performed, because every one I’ve seen is just telling which value is greater. As there does not seem to be any flag in instruction, maybe there is no other option and to do that some arithmetic operation must be done to bring them to greater than operation?

Function call

Another thing, we can learn near the sprintf function is how parameters are passed to function. Signature for sprintf is:

int sprintf(char *str, const char *format, ...);

So after analyzing its call, we should know how at least two first parameters are passed. Let’s see how it looks like in machine code:

00091FC0  .word 0x18600010  # 06: lh $3, $0+0x10
00091FC4  .word 0x1880000B  # 06: lh $4, $0+0x0b
00091FC8  .word 0xD4012810  # 35: ? $0, $1, $5, 0x10
00091FCC  .word 0xA8638608  # 2A: la $3, $3+0x8608   # 0x108608 = pcWriteBuffer?
00091FD0  .word 0x400268A   # 01: call +0x268a       # 0x9b9fc = sprintf
00091FD4  .word 0xA884AB5C  # 2A: la $4, $4+0xab5c   # 0xbab5c = "%-16s %c %7u%s\t%2u%%\r\n"

Here, another interesting detail appears: last instruction setting the registers appears after the actual call. This is perfectly normal and can also be found on MIPS architecture. Its purpose is to allow concurrent execution of the two instructions.

Variadic functions

Now, if we scroll a bit upwards, we can see some interesting bunch of 35 opcodes. We know, that our call should have more than 2 parameters and thanks to format string we can tell that there should be exactly 5 extra parameters. Now if we count number of 35 opcodes, we see that these numbers match.

00091FA8  .word 0xD4012000  # 35: ? $0, $1, $4, 0x00
00091FAC  .word 0xD401A004  # 35: ? $0, $1, $20, 0x04
00091FB0  .word 0xD4011808  # 35: ? $0, $1, $3, 0x08
00091FB4  .word 0xBC050000  # 2F: cmp $0, $5
00091FB8  .word 0x1000000A  # 04: jg 0x91fe0
00091FBC  .word 0xD401D00C  # 35: ? $0, $1, $26, 0x0c
00091FC0  .word 0x18600010  # 06: lh $3, $0+0x10
00091FC4  .word 0x1880000B  # 06: lh $4, $0+0x0b
00091FC8  .word 0xD4012810  # 35: ? $0, $1, $5, 0x10
00091FCC  .word 0xA8638608  # 2A: la $3, $3+0x8608
00091FD0  .word 0x400268A   # 01: call 0x9b9fc
00091FD0                    #   sprintf(pcWriteBuffer, "%-16s %c %7u%s\t%2u%%\r\n",
00091FD0                    #     $4, $20, $3, $26, $5)
00091FD4  .word 0xA884AB5C  # 2A: la $4, $4+0xab5c

So, we can tell almost for sure that opcode 35 gets value of third register parameter and stores it at address computed as follows:

addr = reg1 + reg2 + imm

So e.g.

*($0 + $1 + 0x0c) = $26

Unfortunately with only that information, we can only guess the order of parameters, i.e. if $4 is first parameter or last one.

For future, we will denote opcode 35 as sw (store word).

Function prologue and epilogue

As we know exactly at which address the call will land, we can try to decode function prologue and epilogue, so what happens just after the call and just before returning back. Let’s see how such part of code looks, for example of sprintf function:

0009B9F0           .word 0x44004800  # 11: ret
0009B9F4           .word 0x8441FFF8  # 21: ? $2, $1, -0x08
0009B9F8 sprintf:  .word 0xA9030000  # 2A: la $8, $3
0009B9FC           .word 0x18600010  # 06: lh $3, $0+0x10
0009BA00           .word 0xD7E14FFC  # 35: sw $31, $1, $9, -0x04
0009BA04           .word 0xD7E117F8  # 35: sw $31, $1, $2, -0x08
0009BA08           .word 0xD7E10FF4  # 35: sw $31, $1, $1, -0x0c

And to confirm it is the reverse at the end, let’s see epilogue:

0009BA60           .word 0x8521FFFC  # 21: ? $9, $1, $31, -0x04
0009BA64           .word 0x8421FFF4  # 21: ? $1, $1, $31, -0x0c
0009BA68           .word 0x44004800  # 11: ret
0009BA6C           .word 0x8441FFF8  # 21: ? $2, $1, -0x08

By the way immediately after sprintf there is strlen function, that is also called by function we are analyzing. So we see that registers stored with sw instructions are then recovered by opcode 21. Then we can safely assume it is reverse and denote it as lw (load word).

And we see that last jump in function is done by opcode 11, so we can denote it as ret (return). I still don’t know what is the meaning of its parameters. If we use standard decoding, it would be:

ret $0, $0, $9, 0x00

But I have no proof that it is the real meaning. I only see that sometimes this third hypothetical register have different value, but usually it is $9.

From my experience register saving, we see here is done to stack. If in this case it is also true, then we have two options for stack pointer: $1 and $31. Some more investigation must be done to tell which one is SP.

Other methods

We can also try to find constants other than strings. Then we have a chance, that there will be some arithmetic operation going on with them. Personally, I haven’t tried that approach, so I can’t show any example.

Another method might be finding references to some known structures. We can see one such structure in the function, we analyzed (TaskStatus_t). This is also left as an exercise to the reader.


Main focus of the analysis was on branching. As shown, we know quite a lot about not only branching, but also whole ABI. Now it should be possible, as soon as main entry of the system is found, to discover complete flow of the program.

We now know that first parameters are passed in registers $3, $4 and possibly so on. After analysis of function prologue and epilogue, we also know that here callee is responsible for preserving register values.

To sum up, we already know following instructions:

  • 00: jmp off
  • 01: call off
  • 03: j? off
  • 04: jg off
  • 06: lh $r1, $r2, imm
  • 11: ret
  • 21: lw $r1, $r2, $r3, imm
  • 2A: la $r1, $r2, imm
  • 2F: cmp $r1, $r2
  • 35: sw $r1, $r2, $r3, imm

We also know that in this architecture, there is mechanism of slots, identical with that in MIPS. Together with fixed-sized instructions and opcode and register field lengths, it is really similar to MIPS. Unfortunately it is not exactly the same, so reverse engineering of ISA have to be continued.

Unfortunately, after doing the research described here, I see that tools I used are not enough to do more reversing efficiently. So, before doing one step forward, I have to find a way to introduce more automation to the process. As soon as I succeed with this, I will write next part, so stay tuned!

LKV373A: identifying processor architecture

This article is part of series about reverse-engineering LKV373A HDMI extender. Other parts are available at:

In the first part, I identified two main problems for further development. First one is unidentified checksum, appended to the end of firmware image. Second one is unknown LZSS-like compression algorithm, used to compress machine code of application processor’s firmware.

Encoder firmware

The thing that till now was more or less unexplored is encoder firmware. LKV373A consists of two processors – application processor using the firmware analyzed in first part and encoder, which reversing I am going to push forward with this article.

Target frmware that I am working on is called LKV373A_TX_V3.0c_d_20161116_bin.bin and is obtainable from danman’s firmware collection.

At first, encoder firmware looks completely different than ITEPKG. The latter was completely structured. This one starts with some data fields, then there is big block of randomly looking data, interlaced with some strings. At the end is familiar SMEDIA02 structure. And this block of “random” data is our target.

First idea I had was running binwalk with -A switch to look for some known opcodes. With no luck. But if we look closer, it seems that this in fact is some kind of machine code.

Finding target

Data region

Ok, what we know now is that in firmware image there is a region with many strings written one after another, like on the first hexdump. In another place there is quite a lot data where some of the words are similar to the other. One such fragment can be seen on another hexdump.

Presumable code region

So, we can guess, there is code region around 0x81680 and code region near 0xbbc70. Now the question is: how to prove it.

Before proving our hypothesis, one thing is worth noting. If we are right and bytes here are really machine code, then we can be almost sure that instructions are always (or almost always) encoded into 32-bit numbers. That fact is very useful, when we will try to find candidate architectures to test against our characteristics.

How to prove it?

Fortunately, we can see one interesting candidate string. There is some debug message marked in red on data region. If our guess is valid, we should find some reference to it and moreover we can expect there would be only one reference to that particular string.

But now, there is another problem. How to find reference to string, saved somewhere in memory, at offset we have no chance to guess? Now the experience with any machine code might be useful. Usually assembly mnemonics are translated more or less in a form they are written by programmer. So, if we have hypothetical instruction:

add $1,$2+0x1234

Chances are it will be translated to something like:

0x21 0x01 0x02 0x12 0x34

Where lengths of any of these fields and maximum offset possible to write depends on particular architecture.

We can make another assumption. Usually if persistent memory (like EPROM) finally is mapped to operational memory, usually nobody designs device the way, where start of some section is not at address, padded to i.e. page size. So, if we are lucky final address of string in memory should have same least significant bits as our firmware image.

Then, connecting the dots, we can try searching firmware for let’s say two least significant bytes of string offset (0xbcc8). One more remark here: we still don’t know endianness of the processor. And what is worse, after reading firmware format description we don’t know it even more. So we have to check both variants.

Of course, it might happen that in case of different firmware for completely different architecture, all that might fail. There is only one advice to succeed with such analysis: be creative!

Hit found!

The first impression is that I was wrong. There are in fact 5 hits. But… If we look at general firmware structure, we can see that most of the hits are inside SMEDIA container, which as I described in previous post about reverse engineering LKV373A consists of mostly compressed data, and unlikely has any uncompressed code. So, success! We have only one hit inside area we suspected to be code.

By the way we know more than the area is in fact code section. We see that numbers in machine code are stored as big endian. And as we can, we can use 16 bits as offset here. That further narrows number of possible architectures. For instance, popular ARM architecture is little endian, so it is not likely to be used in this case.

I’ll leave as an exercise verification that we are right on another strings present in this firmware.

Further guesswork

Next thing that might ease things a bit is finding out what is a whole format of instruction, that is:

  • how many bits are used for opcode?
  • how many operands we can use?
  • how many bits encode an operand?

When I was doing the work I was not experienced enough to see it from plain word we already have. Then my approach was to guess meaning of 0xA8 opcode we see. As we know that it points to string in possibly write-protected memory, we should not expect it to perform any arithmetic or logic operations. It is rather likely to get pointer of string and store somewhere. Since architecture gives us only 32 bits to use per instruction it is not likely to copy memory somewhere. For me most probable operation is computation of string address, based on some base register and storage in another register. Therefore, I will call the operation: “Load Address” (abbreviated “la“).

Because that information alone does not give us any more hints, but can be used to match our mysterious architecture to one of the known ones, it is high time to do even more guesswork.

Specification matching

Wikipedia has quite useful page, that might help us. But, at first it is good to check some popular ones to see if they match.

As an example, I will use MIPS architecture. It is often used in embedded systems and was quite popular in routers, especially in past, when they were not running Linux. Furthermore it is the most popular architecture that might be big endian, so seems like perfect first check.

Now, we can find some hints on Wikibooks. On MIPS there is instruction format that might match our characteristics.

I Format
opcode rs rt imm
6 bits 5 bits 5 bits 16 bits

After search in MIPS manual (its name is “MIPS32β„’ Architecture For Programmers Volume II: The MIPS32β„’ Instruction Set” and is probably no more available on original source, but rather some random sites, so no link here), we can see that instruction that has first 6 bits of a word matching our target (0b101010) is “Store Word Left” (swl), so it is not really what we expected.

Now, we have to move on and check another architecture, and another, and another, until we decide we can’t find anything matching. Or finally we will find something. In my case the result is fail. I checked:

  • MIPS
  • ARM
  • ARC
  • RISC-V
  • PowerPC
  • MicroBlaze
  • Xtensa
  • IBM S/390
  • Motorola 68k
  • DLX
  • Mico32
  • LEON
  • OpenRISC
  • m32r

And nothing matched. So architecture here is something really uncommon. Because the device is probably performing HDMI signal processing beside being normal processor, it is likely it is in fact soft processor, programmed into some FPGA. Therefore it is likely, finding the architecture documentation will be impossible.

Decoding instruction format

Then we can try another approach. Let’s play a little bit and try to decode rest of instruction using above MIPS I format. First 6 bits means instruction opcode (0x2A or 0b101010), next are two 5-bit fields meaning destination and source register, so we have register 3 for destination and register 3 for source. That definitely makes sense!

So, our hypothesis is that we have 6-bit opcode, 5-bit operands and 16-bit offset encoding. Here, again proving that fact is left as an exercise for the reader.

Going back to our target instruction, we now know that it is executing code like that:

$3 = $3 + 0xbcc8

We will denote this instruction as:

la $3,$3+0xbcc8

What we know?

Now, to sum things up, we learned following things about device’s architecture:

  • 32-bit (4-byte) static instruction length
  • big endian
  • 6-bit opcodes (maximum number of opcodes is 0x40)
  • 5-bit operands (32 general purpose registers available)
  • indirect addressing of up to 65535 bytes (0xffff)

What next?

As we know quite a lot information about the architecture, we can choose one of the two ways: try to find out what is the name of the architecture and find its documentation (not likely to succeed) or reverse engineer as much instruction as we can. My choice probably will be the latter and if I will succeed in pushing knowledge forward, I will try to describe the process in next article in series.