LKV373A: identifying processor architecture

This article is part of series about reverse-engineering LKV373A HDMI extender. Other parts are available at:

In the first part, I identified two main problems for further development. First one is unidentified checksum, appended to the end of firmware image. Second one is unknown LZSS-like compression algorithm, used to compress machine code of application processor’s firmware.

Encoder firmware

The thing that till now was more or less unexplored is encoder firmware. LKV373A consists of two processors – application processor using the firmware analyzed in first part and encoder, which reversing I am going to push forward with this article.

Target frmware that I am working on is called LKV373A_TX_V3.0c_d_20161116_bin.bin and is obtainable from danman’s firmware collection.

At first, encoder firmware looks completely different than ITEPKG. The latter was completely structured. This one starts with some data fields, then there is big block of randomly looking data, interlaced with some strings. At the end is familiar SMEDIA02 structure. And this block of “random” data is our target.

First idea I had was running binwalk with -A switch to look for some known opcodes. With no luck. But if we look closer, it seems that this in fact is some kind of machine code.

Finding target

Data region

Ok, what we know now is that in firmware image there is a region with many strings written one after another, like on the first hexdump. In another place there is quite a lot data where some of the words are similar to the other. One such fragment can be seen on another hexdump.

Presumable code region

So, we can guess, there is code region around 0x81680 and code region near 0xbbc70. Now the question is: how to prove it.

Before proving our hypothesis, one thing is worth noting. If we are right and bytes here are really machine code, then we can be almost sure that instructions are always (or almost always) encoded into 32-bit numbers. That fact is very useful, when we will try to find candidate architectures to test against our characteristics.

How to prove it?

Fortunately, we can see one interesting candidate string. There is some debug message marked in red on data region. If our guess is valid, we should find some reference to it and moreover we can expect there would be only one reference to that particular string.

But now, there is another problem. How to find reference to string, saved somewhere in memory, at offset we have no chance to guess? Now the experience with any machine code might be useful. Usually assembly mnemonics are translated more or less in a form they are written by programmer. So, if we have hypothetical instruction:

add $1,$2+0x1234

Chances are it will be translated to something like:

0x21 0x01 0x02 0x12 0x34

Where lengths of any of these fields and maximum offset possible to write depends on particular architecture.

We can make another assumption. Usually if persistent memory (like EPROM) finally is mapped to operational memory, usually nobody designs device the way, where start of some section is not at address, padded to i.e. page size. So, if we are lucky final address of string in memory should have same least significant bits as our firmware image.

Then, connecting the dots, we can try searching firmware for let’s say two least significant bytes of string offset (0xbcc8). One more remark here: we still don’t know endianness of the processor. And what is worse, after reading firmware format description we don’t know it even more. So we have to check both variants.

Of course, it might happen that in case of different firmware for completely different architecture, all that might fail. There is only one advice to succeed with such analysis: be creative!

Hit found!

The first impression is that I was wrong. There are in fact 5 hits. But… If we look at general firmware structure, we can see that most of the hits are inside SMEDIA container, which as I described in previous post about reverse engineering LKV373A consists of mostly compressed data, and unlikely has any uncompressed code. So, success! We have only one hit inside area we suspected to be code.

By the way we know more than the area is in fact code section. We see that numbers in machine code are stored as big endian. And as we can, we can use 16 bits as offset here. That further narrows number of possible architectures. For instance, popular ARM architecture is little endian, so it is not likely to be used in this case.

I’ll leave as an exercise verification that we are right on another strings present in this firmware.

Further guesswork

Next thing that might ease things a bit is finding out what is a whole format of instruction, that is:

  • how many bits are used for opcode?
  • how many operands we can use?
  • how many bits encode an operand?

When I was doing the work I was not experienced enough to see it from plain word we already have. Then my approach was to guess meaning of 0xA8 opcode we see. As we know that it points to string in possibly write-protected memory, we should not expect it to perform any arithmetic or logic operations. It is rather likely to get pointer of string and store somewhere. Since architecture gives us only 32 bits to use per instruction it is not likely to copy memory somewhere. For me most probable operation is computation of string address, based on some base register and storage in another register. Therefore, I will call the operation: “Load Address” (abbreviated “la“).

Because that information alone does not give us any more hints, but can be used to match our mysterious architecture to one of the known ones, it is high time to do even more guesswork.

Specification matching

Wikipedia has quite useful page, that might help us. But, at first it is good to check some popular ones to see if they match.

As an example, I will use MIPS architecture. It is often used in embedded systems and was quite popular in routers, especially in past, when they were not running Linux. Furthermore it is the most popular architecture that might be big endian, so seems like perfect first check.

Now, we can find some hints on Wikibooks. On MIPS there is instruction format that might match our characteristics.

I Format
opcode rs rt imm
6 bits 5 bits 5 bits 16 bits

After search in MIPS manual (its name is “MIPS32β„’ Architecture For Programmers Volume II: The MIPS32β„’ Instruction Set” and is probably no more available on original source, but rather some random sites, so no link here), we can see that instruction that has first 6 bits of a word matching our target (0b101010) is “Store Word Left” (swl), so it is not really what we expected.

Now, we have to move on and check another architecture, and another, and another, until we decide we can’t find anything matching. Or finally we will find something. In my case the result is fail. I checked:

  • MIPS
  • ARM
  • ARC
  • RISC-V
  • PowerPC
  • SPARC
  • MicroBlaze
  • Xtensa
  • IBM S/390
  • Motorola 68k
  • DLX
  • Mico32
  • LEON
  • OpenRISC
  • NIOS II
  • m32r

And nothing matched. So architecture here is something really uncommon. Because the device is probably performing HDMI signal processing beside being normal processor, it is likely it is in fact soft processor, programmed into some FPGA. Therefore it is likely, finding the architecture documentation will be impossible.

Decoding instruction format

Then we can try another approach. Let’s play a little bit and try to decode rest of instruction using above MIPS I format. First 6 bits means instruction opcode (0x2A or 0b101010), next are two 5-bit fields meaning destination and source register, so we have register 3 for destination and register 3 for source. That definitely makes sense!

So, our hypothesis is that we have 6-bit opcode, 5-bit operands and 16-bit offset encoding. Here, again proving that fact is left as an exercise for the reader.

Going back to our target instruction, we now know that it is executing code like that:

$3 = $3 + 0xbcc8

We will denote this instruction as:

la $3,$3+0xbcc8

What we know?

Now, to sum things up, we learned following things about device’s architecture:

  • 32-bit (4-byte) static instruction length
  • big endian
  • 6-bit opcodes (maximum number of opcodes is 0x40)
  • 5-bit operands (32 general purpose registers available)
  • indirect addressing of up to 65535 bytes (0xffff)

What next?

As we know quite a lot information about the architecture, we can choose one of the two ways: try to find out what is the name of the architecture and find its documentation (not likely to succeed) or reverse engineer as much instruction as we can. My choice probably will be the latter and if I will succeed in pushing knowledge forward, I will try to describe the process in next article in series.

[Import]LKV373A HDMI to Ethernet converter: firmware image format

NOTE: This post was imported from my previous blog – v3l0c1r4pt0r.tk. It was originally published on 19th August 2017.

This article is part of series about reverse-engineering LKV373A HDMI extender. Other parts are available at:

Recently, I bought LKV373A which is advertised as HDMI extender through Cat5e/Cat6 cable. In fact it is quite cheap HDMI to UDP converter. Unfortunately its inner workings are still more or less unknown. Moreover by default it is transmitting 720p video and does not do HDCP unpacking, which is a pity, because it is not possible to capture signal from devices like cable/satellite TV STB devices. That is why I started some preparations to reverse engineer the thing.

Fortunately a few people were interested by the topic before (especially danman, who discovered second purpose for the device). To make things easier, I am gathering everything what is already known about the device. For that purpose I created project on Github, which is to be served as device’s wiki. Meanwhile I was also able to learn, how more or less firmware container is constructed. This should allow everyone to create custom firmware images as soon as one or two unknowns will be solved.

First one is method for creation of suspected checksum at the very end of firmware image. This would allow to make modifications to filesystem. Other thing is compression algorithm used to compress the program. For now, it should be possible to dissect the firmware into few separate fragments. Below I will describe what I already know about the firmware format.

ITEPKG

ITEPKG format (container content discarded)

Whole image starts with magic bytes ITEPKG, so this is how I call outer container of the image. It allows to store data of few different formats. Most important is denoted by 0x03 type. It stores another data container, that is almost certainly storing machine code for bootloader, and another entity of same type that stores main OS code. This type is also probably storing memory address at which content will be stored after uploading to device. Second important entity is denoted by type 0x06 and means regular file. It is then stored internally on FAT12 partition on SPI flash. There is also directory entry (0x05), that together with files creates complete partition.

SMEDIA

SMEDIA container (header truncated)

Another data container mentioned on previous section is identifiable by magic SMEDIA. It consists of two main parts. Their lengths are stored at the very beginning of the header. First one is some kind of header and contains unknown data. Good news is that it is uncompressed. Second one is another container. Now the bad news is that it contains compressed data chunks.

SMAZ

SMAZ container

This container’s function is to split data into chunks. One chunk has probably maximum length of 0x40000 bytes (uncompressed). Unfortunately after splitting, they are compressed using unknown algorithm, behaving similarly to LZSS and I have some previous experience with variant of LZSS, so if I say so it is very likely that it is true πŸ™‚ . As for now, I reached the wall, but I hope, I’m gonna break it some time soon. Stay tuned!

[Import]Understanding JCOP: pre-personalization

NOTE: This post was imported from my previous blog – v3l0c1r4pt0r.tk. It was originally published on 25th July 2017.

As I promised some time ago, now I am going to describe process of pre-personalization of a JCOP card. JCOP is one of the easier to get JavaCard-compatible cards. However they cost a bit. The problem with the ones available from eBay sellers is lack of pre-personalization. Ok, there are some advantages of buying not pre-personalizaed card, like ability to set most of its parameters, but by the way it is quite easy to make such card unusable.

Online resources, as I mentioned in the first part of the tutorial, are not very descriptive. They say that there is such thing like pre-personalization and it has to be done before using the card, flashing applets, using them and so on. There is only one source that helps a little bit. Someone has written script for the process. However there are two problem with the script. The first one is that it is written in some custom language and internet does not know about the interpreter, it is probably something provided by NXP – manufacturer of JCOP for its customers and neither me nor (probably) you, reader, are their customers. The consequence is that we can have script in custom language, with commands like ‘/select’ or ‘/send’. Fortunately, documentation of ISO 7816 (smart card connection), allows to decipher this. So this problem could be finally solved. Another problem is lack of command values and addresses in memory, so we do not know where and how to read/write/execute anything. After really deep search in Google, finally, I was able to find out all the missing values, so this tutorial could be written.

Process overview

Ok, after this way too long historical introduction, let’s see what will be needed. I assume, you are already able to communicate with your card using raw PDUs. If you don’t, up to this point there are quite a few resources to learn from, so I will not describe this. The most important thing here is to have so called transport key (KT). If you do not have it, go get it now. Seller should provide it to you, and if he did not, you are stuck, since the first step requires this key.

So, basically steps will be as follows:

  1. Select root applet with Transport Key
  2. Boot the card
  3. Read/write some data
  4. Protect the card
  5. Fuse it

Easy? Easy. But only if you know some hex numbers. Ok, here, one big WARNING: the last step is irreversible and can be done by mistake quite easily, so think twice before sending anything, and if you are sure, that you are done, think twice again, before issuing it.

Pre-personalization, step by step

At first, we use Transport Key to select proper applet. Format of SELECT command is as below:

CLA=00 INS=A4 P1=04 P2=00 Lc=10 (...)

Where CLA is always zero, INS means SELECT, P1, according to ISO7816 means selection by DF name and Lc is length of KT. After that, key have to be appended. Of course, whole APDU is to be given to communications program as binary values or hex values only.

What now follows is specific to NXP cards only and is mostly undocumented publicly. First of such commands is BOOT command. Its format is as follows:

CLA=00 INS=F0 P1=00 P2=00

Now double care have to taken, because FUSE command should be available after this point and its APDU consists only of zeros, so every mistake might make the card unusable, since security keys are generated randomly for each card.

Reading memory

Now the most important values to read are called CM_KEY and GPIN in memory dump, I shared in the previous post on the topic. First one starts at offsets: 0xc00305, 0xc00321 and 0xc0033d and are 0x10 bytes long. The other one can be found at offset 0xc00412 and by default should be 5 bytes long. However maximum length is also 0x10, so it is better to make sure the length is really 5 by reading byte at offset 0xc00407. To sum up following commands need to be issued and results be saved for future use:

CLA=C0 INS=B0 P1=03 P2=05 Lc=10
C0 B0 03 21 10
C0 B0 03 3D 10
C0 B0 04 07 01
C0 B0 04 12 xx

Where CLA + P1 + P2 is concatenated address of memory area to read, INS=B0 is read command and Lc contains length of data to read.

Writing data

Alternatively, it is possible to write custom values to these buffers. This is especially encouraged for users who want to use the card not only for testing. Overwriting the values could be done with following:

CLA=C0 INS=D6 P1=03 P2=05 Lc=10 (...)
C0 D6 03 21 10 (...)
C0 D6 03 3D 10 (...)
C0 D6 04 12 05 (...)

Where user data is filled with some random data of length in Lc field.

Required values

Beside securing keys, it is required to set CM_LIFECYCLE value to 0x01 and make sure all fields related to keys and PIN have proper values. Here, my memory dump can be used as reference, since I initialized the card before dumping the memory.

Finishing

After setting all the fields to desired values, there are two more steps to do. First one is issuing PROTECT command. It looks as below:

CLA=00 INS=10 P1=00 P2=00

And finally, sending FUSE command with:

CLA=00 INS=00 P1=00 P2=00

Here again, remember, that this command cannot be undone!

Well done! Your card should now be pre-personalized and ready to use, even in production environment. At the end, one remark: probably FUSE command does not need to be issued at all. However, if it is not issued, the card is completely insecure and should not be used in production.

Previous part of this tutorial can found under this link.

[Import]Hacking Vasco translator through binary SMS

NOTE: This post was imported from my previous blog – v3l0c1r4pt0r.tk. It was originally published on 18th February 2017.

Recently I was asked to configure internet browser on a thing called Vasco Translator Premium 7″. The device looks exactly like many of the low-end Android tablets from China. And it happens to be one. The problem is that is was locked to used the only allowed application which is the translator. It has some minor functions like camera and it seems that was a mistake of its authors. They used default Android app as Camera and Gallery applications (and forgot to lock send message button in the latter πŸ™‚ ).

At first I have to highlight the fact, this is not the full unlock or root of the device, but the fact of ease of the process allows me to suppose that rooting the thing should not be too difficult, too. Our goal here is to open up the web browser. Because as it seems the software below the shitty overlay is ordinary Android with all the apps on its place. And how useful might be the tablet without internet browser?

Prerequisites

The tool we will use to do the trick is good ol’ WAP Push protocol. For some reason in newer Android devices this dinosaur has been revived and supported out of the box. The goal is to send such message to our locked device, and since the gallery app mentioned above allows to escape to Messaging app, read it on it. This is probably the hardest part of the process. And possibly may require to buy some additional hardware (if you have access to any service, you know is sending WAP Push, you can use it and skip this part).

And that hardware is a GSM modem. It is highly possible that you already have one that can be used. The thing we will need is the possibility to send SMS through AT commands. Many Android phones allows that, at least if they are rooted, probably LTE/3G modems can do that too (not checked that personally). Ok, since the procedure to get access to AT interface is completely different for any device you can get, I have to leave you alone with getting used to that. After some time, you will probably end up in minicom or some similar program and parameters like 115200/8N1 or 9600/8N1. In my case (Android with Qualcomm processor) it is /dev/smd11 and params are 115200/8N1. Now you could type AT to check if the device you found is really an AT modem (should respond with OK) and AT+SCA? to check your SMS center address. You should be able to recognize it or Google it to check if it really belongs to your mobile operator.

Crafting SMS

Now, since we have all the tools, we can start crafting SMS. I will omit many details here since just general description of PDU format would take whole article and complete one is more than 100 pages long. The only part you need to know about is destination address. This will be the phone number of your device. Trailer of the message is WAP Push payload, which to be described will need another 100 or so pages, so skip it. As a remark, there is some program called wbxml2xml/xml2wbxml that allows to read/write the message. In our case, we want to enforce the device to visit Google.com, so this will be the address of WAP bookmark.

 

Ok, so on the picture above, thing we are interested in are [dest_addr] and [dest_len]. The first encodes telephone number “+37201234567” (note lack of ‘+’ sign), the second its length (as number of digits, 0x0b == 11). The number of your device should be placed here and you could move on to next section.

Or you can try customizing the payload. The important thing here is marked as [WBXML] and can be crafted with program mentioned before. After changing this, adjustment of [ud_len] value to number of bytes in payload (those after the length) is required.

Sending SMS

Since we already have modem, we need to type AT command to initiate message sending. But before that, we need to ensure that we are in binary mode. Type AT+CMGF? and, if value is other than zero, AT+CMGF=0. Now start sending with:

AT+CMGS=55

Where 55 is length of payload in bytes, but without SMSC header (one byte at the beginning). Modem should respond with > prompt, where SMS could be typed.

0041000b917302214365f700042a0605040b8423f0ea0601ae03056a0045c60c03676f6f676c652e636f6d00080103476f6f676c65000101

And after that press CTRL-Z (^Z) in your terminal. This should send SUB (substitute) to modem. It is important not to use any characters in between, like spaces and ENTER. After about a second, you should see that sending was successful and no error was returned.

Receiving and opening

Now, if you have your translator turned on, you should hear that new message was received, but nothing appeared on screen. That is ok. The rest of the procedure is shown on video below:

Postscriptum

After another few minutes of playing with the device I found another method of opening the browser and it is way faster than what was described below. But the first one was much more entertaining for me and is showing one of the many places where serious bugs could be found – forgotten technologies, still being implemented, possibly used, but with lack of knowledge about details in general public.

You can see the other method on video below, and possibly it is the one you want to use.

[Import]Understanding JCOP: memory dump

NOTE: This post was imported from my previous blog – v3l0c1r4pt0r.tk. It was originally published on 8th February 2017.

Some time ago I was struggling with JCOP smart card. The one I received as it turned out was not pre-personalized, which means some interesting features (like setting encryption keys and PIN) was still unlocked. Because documentation and all the usual helpers (StackOverflow) were not very useful (well, ok, there was no publicly available documentation at all), I started very deep search on Google, which finished with full success. I was able to make dump of whole memory available during pre-personalization.

Since it is not something that could be found online, here you have screenshot of it, colored a bit with help of my hdcb program. Without documentation it might not be very useful, but in some emergency situation, maybe somebody will need it.

JCOP memory dump made at the very beginning of pre-personalization

Small explanation: first address, I was able to read was 0xC000F0, first address with read error after configuration area was 0xC09600. I know that, despite of lack of privileges some data is placed there.

There are three configurations: cold start (0xc00123-0xc00145), warm start (0xc00146-0xc00168) and contactless (0xc00169-at least 0xc0016f). Description of coding of the individual fields is outside of the scope of this article. I hope, I will describe them in future.

Next time, I will try to describe the process of pre-personalization, that is making not pre-personalized card, easy to get from usual sources of cheap electronics, able to receive and run applets.

Update: Next part of this tutorial can be found under this link.

[Import]Wget with SSL/TLS support for Android

NOTE: This post was imported from my previous blog – v3l0c1r4pt0r.tk. It was originally published on 12th September 2016.

wget dependency tree

Lately I have tried to download some file from a website to my Android smartphone. Simple thing, yeah? Well, not really. Unfortunately mobile browser developers removed many features from their mobile distributions. One of them is a possibility of downloading random page to disk as is. Instead (this is the case at least with Mozilla’s product) they are forcing “Download as PDF” feature. I had a bit of luck, because the file I was trying to download was MP4 movie, which is downloadable, maybe not in an intuitive way, but it is. But before I have found that feature hidden in a player’s context menu, I tried another solution – wget. Since I am great fan of terminals, I have busybox installed on my phone. Those of you, who know what exactly is busybox should know that this is set of lightweight variants of most standard UNIX tools. So, if they are lightweight, they had to cut some part of tool functionality, right? And in case of my busybox’s wget, they cut HTTPS support. And today, it is more likely to encounter site which is only HTTPS than one that is only HTTP, at least when talking about popular sites. So I had to get my own distribution of wget, that will not be such constrained one.

Not to get you bored too much, here you can find binary distribution of what I achieved to compile. It was compiled for ARMv7 platform using NDKr12b and API level 24 (Nougat), so it will probably not work on most of current Android phones, but if you read later, it is probably working on your device or even is outdated. If you are interested in recompiling binaries yourself, you can find detailed how-to in the next part of this article.

Dependencies

Before compiling wget itself, you have to have whole bunch of its dependencies. But at first, you of course need Android compiler. It is distributed as part of NDK and I won’t describe its installation here. Sources of every program compiled here can be grabbed from its official sites (list at the end of this post). The only exception is libtasn1, which required few hacks to be done to make it compile with Android bionic libc. Its source, ported to Android can be get from my github repo.

Let’s start with programs that does not depend on anything. For all projects, the procedure is more or less the same and can be described with simplified bash script:

tar -zxvf program-1.00.tar.gz
mkdir build
mkdir install
cd build
CC=arm-linux-androideabi-gcc AR=arm-linux-androideabi-ar RANLIB=arm-linux-androideabi-ranlib CFLAGS=-pie \
    ../program-1.00/configure --host=arm-linux --prefix=/data/local/root
make
make install DESTDIR=$(dirname `pwd`)/install/
cd ../install
tar -zcvf program.tar.gz *

gmp, libidn and libffi

For these three programs, the procedure above should work without any modification.

nettle

Since nettle depends on gmp, it has to be configured with paths to gmp binaries and headers in its CFLAGS and LDFLAGS variables. They should look like this:

CFLAGS="-pie -I`pwd`/../../gmp/install/data/local/root/include"
LDFLAGS="-L`pwd`/../../gmp/install/data/local/root/lib"

when invoking configure script.

libtasn1

This was the hardest part for me, but should go smoothly now. Script below should do the work correctly:

git clone git@github.com:v3l0c1r4pt0r/android_external_libtasn1.git
mkdir build
mkdir install
cd build
CC=arm-linux-androideabi-gcc AR=arm-linux-androideabi-ar RANLIB=arm-linux-androideabi-ranlib CFLAGS=-pie \
    ../libtasn1/configure --host=arm-linux --prefix=/data/local/root --disable-doc
make
make install DESTDIR=$(dirname `pwd`)/install/
cd ../install
tar -zcvf libtasn1.tar.gz

p11-kit

This is the last dependency of gnutls which is the only, but very important dependency of wget. Just embedding libtasn1 and libffi should do the job well.

CFLAGS="-pie -I`pwd`/../../libtasn1/install/data/local/root/include"
LDFLAGS="-L`pwd`/../../libtasn1/install/data/local/root/lib -L`pwd`/../../libffi/install/data/local/root/lib"

Notice that libffi has no headers, so we add it just to CFLAGS here!

gnutls

This one was more complicated than the rest. As I mentioned above, it is very important to wget functionality. However wget’s dependency on it could probably be turned off, we would not have TLS support then. When compiling it I had some problems that seemed to be serious. There were a few errors while making it, so I had to call make twice and even though it failed. Despite that it seem to work after make install, which obviously failed too. In my case following script did the job:

mkdir build
mkdir install
cd build
CC=arm-linux-androideabi-gcc AR=arm-linux-androideabi-ar RANLIB=arm-linux-androideabi-ranlib \
    CFLAGS="-pie -I`pwd`/../../gmp/install/data/local/root/include -I`pwd`/../../nettle/install/data/local/root/include -I`pwd`/../../libtasn1/install/data/local/root/include -I`pwd`/../../libidn/install/data/local/root/include -I`pwd`/../../p11-kit/install/data/local/root/include" \
    LDFLAGS="-L`pwd`/../../gmp/install/data/local/root/lib -L`pwd`/../../nettle/install/data/local/root/lib -L`pwd`/../../libtasn1/install/data/local/root/lib -L`pwd`/../../libidn/install/data/local/root/lib -L`pwd`/../../p11-kit/install/data/local/root/lib" \
    ../gnutls-3.4.9/configure --host=arm-linux --prefix=/data/local/root --disable-cxx --disable-tools
make || make
make install DESTDIR=$(dirname `pwd`)/install/ || true
cd ../install
tar -zcvf file.tar.gz *

Compilation

Since we should now have all dependencies compiled, we can try compiling wget itself. The procedure here is the same as with dependencies. We just have to pass path to gnutls. And then standard configure, make, make install should work. However if your NDK installation is fairly new and you were not hacking it before, you most likely don’t have <sys/fcntl.h> header and make should complain about that. Luckily Android itself have this header present, but for reason unknown it is kept in include directory directly. To make wget, and any other program that uses it, compile you can just point “sys/” instance to <fcntl.h> with symlink or do something like that:

echo "#include <fcntl.h>" > $TOOLCHAIN/sysroot/usr/include/sys/fcntl.h

where $TOOLCHAIN/sysroot is path at which you have your headers placed. Depending on tutorial you were using for making it work it may have different structure.

Installation

All commands I presented above implies that you have your custom-compiled binaries in “/data/local/root”. I made it that way to have clear separation between default and busybox binaries. If you want to have them somewhere else, you should pass it to configure scripts of all programs you are compiling. After successful compilation of all tools, I have made single tarball containing all compilation output (this file’s link was placed above). Its content can be installed into Android by typing

tar -zxvf wget-with-deps.tar.gz -C/

using adb shell or terminal emulator.

Sources

Below you can find links to sources of all programs nedded to follow this tutorial.

[Import]HDCB – new way of analysing binary files under Linux

NOTE: This post was imported from my previous blog – v3l0c1r4pt0r.tk. It was originally published on 10th February 2016.

As any observer of my projects spotted, most of the biggest projects I do involves binary file analysis. Currently I am working on another one that requires such analysis.
Unfortunately such analysis is not an easy task and anything that will ease this or speed it up is appreciated. Of course there are some tools that will make it a bit easier. One of them is hexdump. Even IDA Pro can make it easier a bit. Despite them I always felt that something is missing here. When creating xSDM and delz utils, I was using hexdump output with LibreOffice document to mark different structure members with different colors. It worked, but selecting 100-byte buffer line by line was just wasting precious time.

Solution

SDC file analyzed by HDCB script

So why not automate whole process? What we really need here is just hexdump output and terminal emulator with color support. And that’s why I’ve made HDCB – HexDump Coloring Book. Basically it is just extension to bash scripting language. Goal was to create simple script that will hide as much of its internals from end-user and allow user to just start it will his shell using old good ./scriptname.ext and that’s it. HDCB is masked as if it was standalone scripting language. It uses shebang, known from bash or python scripts to let user shell know what interpreter to use (#!/usr/bin/env hdcb). Those who are python programmers should recognize usage of env binary.

In fact it is just simple extension to bash language, so users are still able to utilize any features available in bash. Main extensions are two commands: one (define) for defining variables and the other (use) for defining field or array of that defined type. Such scripts should be started with just one argument – file that is meant to be hexdumped and analyzed.

Internals

Bash scripts are just some kind of a cover of real program. Main core of the program is colour utility. It gets unlimited number of parameters grouped in groups of four. They are in order: offset of byte being colored, length of the field, background and foreground colors. As standard input, hexdump output (in fact only hexdump -C or hexdump -Cv are supported) is provided. Program colors the hexdump with rules provided as arguments. This architecture allows clever hacker to build that cover mentioned in virtually any programming language.

Downloads, documentation and more

As usual, program is available on my Github profile. Sources are provided on GPLv3 license so you are free to contribute to the project and you are strongly encouraged to do so or make proposals of a new functions. Program is meant to be expanded according to my future needs, but I will try to implement any good idea. Whole documentation, installation instructions and the like are also available on Github.

[Import]SDC file format description – Errata

NOTE: This post was imported from my previous blog – v3l0c1r4pt0r.tk. It was originally published on 6th November 2015.

Last year, I published a program for Microsoft Dreamspark’s SDC file decryption. Soon after that I wrote article about SDC file format and its analysis. Now it’s time to complete the description with newest data.

This article wouldn’t be written if not the contribution of GitHub’s user @halorhhr who spotted multi-file SDC container and let me know on project’s page. Thanks!

When writing that post year ago, I had no idea what multi-file container really looks like. Any suspicions could not then be confirmed, because it seemed that these files simply where not used in the wild. A days ago situation changed. I got a working sample of multi-file container so I was able to start its analysis.

Real container format

SDC files with different signatures

After quick analysis, I knew that I was wrong with my suspicions. Filename length and encrypted filename strings are not part of a file description. In fact they are placed after them and filename is concatenated string of all filenames (including trailing null-byte). So to sum up filename of n-th element starts at file[n].filename_offset and ends just like any other c-style string.

Whole header structure is like on the sample header on the right. Note that all headers beside 0xb3 one has been already decrypted for readability. In real header the only unencrypted field is header size at the very beginning of the file. 0xb3 sample has unencrypted header and header size is not present in a file. However file name is encrypted in some way, I haven’t figured out as of now. Encryption method is blowfish-compat (the difference between this and blowfish is ciphertext endiannes). Filenames are encrypted once again.

After header, all other data is XORed using key from EDV string and then deflated, so before reading them, you have to inflate and XOR again. Format of data in 0xb3 version is still unknown, however analysis of compressed and file size hints that it may be stored the same way. It is important to note that depending on file signature different configuration of deflater may be needed. It is now known that files older than 0xd1 header, which appears to be newest (because only this one supports files greater than 4 GiB) need to have deflater initialized with

inflateInit2_(&stream,-15,ZLIB_VERSION,(int)sizeof(z_stream));

or equivalent.

Unknowns

This errata does not contain all information needed to support all variations of SDC files. Beside unknowns I mentioned above, there is another variation that uses 0xc4 signature and which I had no sample of. The only trace of its existence is condition in SDM code. Because of that I cannot write support for that type of file. There is also possibility of multi-file containers having 0xb5 or 0xb3 signatures existence. That type of files seems to appear only lately, but it is probable that it existed in the past. Because of having no samples of them I cannot verify that xSDM properly handles them.

So if you have sample of any of variations mentioned here, just send them to me at my email address: v3l0c1r4pt0r at gmail dot com or if you suppose it may be illegal in your country, just send me SDX link or any other hint for me how can I find them.

Other way?

Few days ago, after I started writing this post Github user @adiantek let me know in issues that there is a method to obey SDM in Dreamspark download process. To download plain, unencrypted file you just have to replace ‘dfc=1‘ to ‘sdm=0‘ in a link Dreamspark provided in SDX file. If it true that it works in every file Dreamspark provides, my xSDM project would be obsolete now. However, because Microsoft’s intentions when creating this backdoor (it seems to be created just for debugging) are unknown, I will continue to support the project and fix any future bugs I will be aware of. But now it seems that this project will start to be just proof of concept for curious hackers and will start to slowly die.

Nevertheless, if you have something that might help me or anyone who may be interested in SDC format in future, just let me know somehow, so it will be available somewhere on the internet.

[Import]Decoding Aztec code from polish vehicle registration certificate

NOTE: This post was imported from my previous blog – v3l0c1r4pt0r.tk. It was originally published on 1st August 2015.

About a year ago I interested in mysterious 2D code placed in my car’s registration certificate. After quick research on Google it turned out to be even more mysterious because nobody knew how to decode it. There was even no official document like act or regulation that describes the code somehow. People knew that the code is Aztec code and that’s it. Some companies shared web and Android apps to decode this. And all of them was sending base64 to some server and receive decoded data.

Of course for me it wasn’t rewarding so I started my research on it. After initially scanning the code I’ve seen long string that I immediately recognized as base64. The real fun started after that, because stream I’ve got after that was so strange that at first I had no idea what to do. Upon closer examination it was clear that this data is not damaged but encoded in somewhat strange way. Few days later I was almost sure that this is not encoding but rather compression, because some unique parts of stream was easily readable by human. About a month of learning about compression, looking for even most exotic decompression tools and I was left with almost nothing. I had only weak guess on how decompression parameters could be encoded. I gave up…

Polish vehicle registration certificate (source: pwpw.pl)

About a year later I tried one more time. This time I was a bit more lucky. I found a program that decodes the code. Again. But this time was different. I shut down my network connection to make sure. And it worked! So now a bit of reverse engineering and it’s done. I will skip any details because I do not want to piss off the company which created this, even though I was right and I HAD right to do this.

As usual the source code is available on my Github profile. There is also a bit more information about whole scanning/decoding process. If you like to know more technical details about the algorithm or how to decode the data, everything can be found in README file in the repo.

[Import]CP210x (USB to TTL adapter) for Android devices

NOTE: This post was imported from my previous blog – v3l0c1r4pt0r.tk. It was originally published on 22nd August 2014.

Few days ago, while searching for something interesting I can do with my Android phone I found out that Sony is publishing kernel source of all their (and old SE ones too) ROMs. Beside hacking the kernel itself I was wondering if it is possible to compile additional kernel modules (stock ROM provides only internal WiFi module’s drivers). Well.. it was. The first one I tried was driver of my cheap, Chinese USB to serial adapter based on CP2102 chip.

Prerequisites

  • kernel source – it is important to be the same kernel as the one working on your device. Otherwise it would probably be necessary to boot your system with kernel compiled with source you have and it will not be described in this tutorial. The reason it is important is that between kernel version compatibility is not guaranteed. Different configuration could mess with functionality too (but not tried myself). I have Sony Ericsson device so I downloaded from its official repository.
  • cross-compiler – while compiling programs for Android you definitely should get special version of a compiler because Android is different than usual Linux box so compiler options are different too. The easiest way is to download official NDK which has built-in compilers and easy-to-use script to make standalone toolchain out of them. Same toolchain should also work as compiler for kernel but I haven’t tested it myself. In case it didn’t there is very powerful tool for making your own toolchain for any platform supported by GCC so in practice any you can imagine called crosstool-ng (on Arch available on AUR).
  • uucp source code

Kernel module

We will start with a kernel module. First of all we will unpack our kernel. In case of SE kernel need to be uncompressed and then unpacked so I did:

bzcat 4_1_B_0_431_tar.bz2 | tar -xv
cd kernel

And changed dir to kernel. Next thing I had to do was patching Makefile, because it complained about unused variables. If you are using Sony or SE kernel you probably need to do it too. If so create file named Makefile.patch with your favorite editor, i.e.

vim Makefile.patch

and paste following content:

--- Makefile  2012-05-25 12:07:05.000000000 +0200
+++ Makefile.new  2014-08-20 21:16:50.642703198 +0200
@@ -342,7 +342,6 @@
 KBUILD_CPPFLAGS := -D__KERNEL__

 KBUILD_CFLAGS   := -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs \
-		   -Werror \
 		   -fno-strict-aliasing -fno-common \
 		    -Werror-implicit-function-declaration \
 		    -Wno-format-security \

Now you can patch it with:

patch Makefile < Makefile.patch

Then we can start the compilation process which should end in a second. I assume you have working toolchain in your $PATH and it is prefixed with arm-unknown-eabi- (arm-unknown-eabi-gcc, etc.). If its name is different change all occurrences of arm-unknown-eabi- below.

make mrproper
ARCH=arm CROSS_COMPILE=arm-unknown-eabi- make semc_iyokan_defconfig
ARCH=arm CROSS_COMPILE=arm-unknown-eabi- make prepare
ARCH=arm CROSS_COMPILE=arm-unknown-eabi- make modules_prepare
ARCH=arm CROSS_COMPILE=arm-unknown-eabi- make modules SUBDIRS=drivers/usb/serial CONFIG_USB_SERIAL=m CONFIG_USB_SERIAL_CP210X=m

If everything worked you should have your module compiled in drivers/usb/serial directory. It is worth to note you could compile any other module the same way. It is only important to change SUBDIRS to your module location and ensure it is configured to be built by issuing make menuconfig or setting appropriate CONFIG-* variable to ‘m’.

Now you can copy modules to your devices. With adb it will be:

adb push drivers/usb/serial/usbserial.ko /mnt/sdcard/others
adb push drivers/usb/serial/cp210x.ko /mnt/sdcard/others

We have two modules to copy here since cp210x (and any other serial device driver) depends on usbserial and without it being already in kernel it would be impossible to insert cp210x and furthermore it will give you strange error message (tested :). So now, again with help of ADB, we can insert our modules.

adb shell
su
cd /mnt/sdcard/others
insmod ./usbserial.ko
insmod ./cp210x.ko

After that you can connect your device and device you compiled module for and test if it works. Serial adapters should create new device file in /dev and, at least with CP210x, it uses ttyUSB* name so you can list it with:

ls -l /dev/ttyUSB*

You can also read from your port as simply as by typing

cat /dev/ttyUSB0

Now after you connect some device talking UART you should see its output.

Program to easily control our port – cu and the rest of uucp package

Warning: in the following steps I assume that you have directories like /data/local/bin and /data/local/etc already on your device. As far as I remember there is only /data/local by default.

At first we need to unpack our source and create some useful directories. Then we will configure our script in build-uucp so we won’t get garbage in source directory and could easily clean things when something goes wrong.

tar -zxvf uucp-1.07.tar.gz
mkdir build-uucp
mkdir install-uucp
cd build-uucp/

Then we will patch our program so it will work on Android out of the box. If we omitted following patch it would be necessary to create configuration file, configure program to read it before work and whole bunch of similar fun. So it is easier to let him know where he could store his files now. You can obviously use the one you want to. If you stay with my config you won’t be able to start cu as normal user which is no problem because by default you won’t have permission to use serial terminal.

Now the procedure is the same as with previous patch. I assume you used policy.h.patch name and the content is:

--- policy.h  2003-05-29 08:08:45.000000000 +0200
+++ policy.h.new  2014-08-20 12:03:45.595405893 +0200
@@ -297,7 +297,7 @@
    systems the lock files are placed in /etc/locks.  On some they are
    placed in /usr/spool/locks.  On the NeXT they are placed in
    /usr/spool/uucp/LCK.  */
-/* #define LOCKDIR "/usr/spool/uucp" */
+#define LOCKDIR "/data/local/etc/spool/uucp"
 /* #define LOCKDIR "/etc/locks" */
 /* #define LOCKDIR "/usr/spool/locks" */
 /* #define LOCKDIR "/usr/spool/uucp/LCK" */
@@ -572,7 +572,7 @@
 /* The name of the default spool directory.  If HAVE_TAYLOR_CONFIG is
    set to 1, this may be overridden by the ``spool'' command in the
    configuration file.  */
-#define SPOOLDIR "/usr/spool/uucp"
+#define SPOOLDIR "/data/local/etc/spool/uucp"
 /* #define SPOOLDIR "/var/spool/uucp" */

 /* The name of the default public directory.  If HAVE_TAYLOR_CONFIG is
@@ -580,7 +580,7 @@
    configuration file.  Also, a particular system may be given a
    specific public directory by using the ``pubdir'' command in the
    system file.  */
-#define PUBDIR "/usr/spool/uucppublic"
+#define PUBDIR "/data/local/etc/spool/uucppublic"
 /* #define PUBDIR "/var/spool/uucppublic" */

 /* The default command path.  This is a space separated list of
@@ -644,21 +644,21 @@
 /* The default log file when using HAVE_TAYLOR_LOGGING.  When using
    HAVE_TAYLOR_CONFIG, this may be overridden by the ``logfile''
    command in the configuration file.  */
-#define LOGFILE "/usr/spool/uucp/Log"
+#define LOGFILE "/data/local/etc/spool/uucp/Log"
 /* #define LOGFILE "/var/spool/uucp/Log" */
 /* #define LOGFILE "/var/log/uucp/Log" */

 /* The default statistics file when using HAVE_TAYLOR_LOGGING.  When
    using HAVE_TAYLOR_CONFIG, this may be overridden by the
    ``statfile'' command in the configuration file.  */
-#define STATFILE "/usr/spool/uucp/Stats"
+#define STATFILE "/data/local/etc/spool/uucp/Stats"
 /* #define STATFILE "/var/spool/uucp/Stats" */
 /* #define STATFILE "/var/log/uucp/Stats" */

 /* The default debugging file when using HAVE_TAYLOR_LOGGING.  When
    using HAVE_TAYLOR_CONFIG, this may be overridden by the
    ``debugfile'' command in the configuration file.  */
-#define DEBUGFILE "/usr/spool/uucp/Debug"
+#define DEBUGFILE "/data/local/etc/spool/uucp/Debug"
 /* #define DEBUGFILE "/var/spool/uucp/Debug" */
 /* #define DEBUGFILE "/var/log/uucp/Debug" */

@@ -669,17 +669,17 @@
 /* The default log file when using HAVE_V2_LOGGING.  When using
    HAVE_TAYLOR_CONFIG, this may be overridden by the ``logfile''
    command in the configuration file.  */
-#define LOGFILE "/usr/spool/uucp/LOGFILE"
+#define LOGFILE "/data/local/etc/spool/uucp/LOGFILE"

 /* The default statistics file when using HAVE_V2_LOGGING.  When using
    HAVE_TAYLOR_CONFIG, this may be overridden by the ``statfile''
    command in the configuration file.  */
-#define STATFILE "/usr/spool/uucp/SYSLOG"
+#define STATFILE "/data/local/etc/spool/uucp/SYSLOG"

 /* The default debugging file when using HAVE_V2_LOGGING.  When using
    HAVE_TAYLOR_CONFIG, this may be overridden by the ``debugfile''
    command in the configuration file.  */
-#define DEBUGFILE "/usr/spool/uucp/DEBUG"
+#define DEBUGFILE "/data/local/etc/spool/uucp/DEBUG"

 #endif /* HAVE_V2_LOGGING */

@@ -692,16 +692,16 @@
    be replaced by the system name (if there is no appropriate system,
    "ANY" will be used).  No other '%' character may appear in the
    string.  */
-#define LOGFILE "/usr/spool/uucp/.Log/%s/%s"
+#define LOGFILE "/data/local/etc/spool/uucp/.Log/%s/%s"

 /* The default statistics file when using HAVE_HDB_LOGGING.  When using
    HAVE_TAYLOR_CONFIG, this may be overridden by the ``statfile''
    command in the configuration file.  */
-#define STATFILE "/usr/spool/uucp/.Admin/xferstats"
+#define STATFILE "/data/local/etc/spool/uucp/.Admin/xferstats"

 /* The default debugging file when using HAVE_HDB_LOGGING.  When using
    HAVE_TAYLOR_CONFIG, this may be overridden by the ``debugfile''
    command in the configuration file.  */
-#define DEBUGFILE "/usr/spool/uucp/.Admin/audit.local"
+#define DEBUGFILE "/data/local/etc/spool/uucp/.Admin/audit.local"

 #endif /* HAVE_HDB_LOGGING */

If you prefer I have it on my gist so you can just issue one command and get it. Then we are patching as usually:

patch ../uucp-1.07/policy.h < policy.h.patch

As mentioned above you can change path of uucp’s files by issuing the following (remember to escape every occurrence of slash with backslash, otherwise it will fail):

sed -i "s/\/data\/local\/etc/[your-path]/" ../uucp-1.07/policy.h

Now you are ready to compile. It can be done with following commands. Your compiler should have same name (at least if you use NDK’s compiler). It is important to note that I had to switch off HAVE_SYSCONF flag since it was causing ugly errors. In my case makescript couldn’t also find a rule to make ftw.o so I had to make it myself. If you have no trouble here, just omit the line after make.

CC=arm-linux-androideabi-gcc AR=arm-linux-androideabi-ar RANLIB=arm-linux-androideabi-ranlib \
../uucp-1.07/configure --prefix=`pwd`/../install-uucp/ --host=arm-linux-androideabi
sed -i "s/#define HAVE_SYSCONF 1/#define HAVE_SYSCONF 0/" config.h
make
cd unix; make ftw.o; cd ..; make
make install
adb push ../install-uucp/bin/cu /mnt/sdcard/others
adb shell
su
cp /mnt/sdcard/others/cu /data/local/bin/

Finally you can test the program with the following and you should be able to talk RS232 with just a phone/tablet!

cu -lttyUSB0 -s115200

where 115200 is the speed the device you connect to transmits.

BTW: uucp have few other tools and by following this tutorial you compiled them all so you can explore them on your own.