Isolated execution, or why open CPUs are so important.

With all that recent hysteria associated with NSA and all different sorts of spy trojans it’s a good time to think about truly secure computation. Is it feasible at all? Of course you can use a variety of fancy anti-malware software and even run some sort of isolated OS, like popular Joanna Rutkowska’s Qubes OS. But does it give you a real security?

One of the biggest trends going now is isolated execution. The idea is very simple, you can run some payload code (normally executed as a user-level application) with somewhat strong level of isolation from other applications, or even from supervisor software, running at more privileged level. Sounds great, right? There is a number of problems associated with this approach, like if your application is isolated from the OS, how would it get any data coming from OS in response of a system call, etc. These problems have been more or less solved in research papers.

Many of these solutions still rely on some critical software, normally hypervisors. To prove their point, they usually make an argument that the hypervisor can theoretically be small enough to be formally verified, or at target system can use measured boot mechanism, provided by a TPM. Unfortunately no one has shown working formally proved hypervisor yet. And why are you supposed to trust your TPM chip fabricated somewhere in china by some third-party random performing secure boot for you? Moreover, successful attacks against TPM have been demonstrated.

Looks like we need to do security in hardware. So, reasonably would be to limit trusted computing base (TCB) to the most essential hardware, like the CPU itself and possibly memory, the bus, etc. However, newest research papers tend to exclude even the RAM out of TCB. Summarizing, we have decided we need a hardware security (isolation) features and we need them to be implemented in hardware. That’s where a new big thing from Intel, called SGX coming to. It is a hardware isolated execution mechanism, it provides your applications with an ability to run completely isolated from the rest of the system, even your OS! It also provides you with a mechanism to share some data between trusted (isolated) and untrusted application. Unfortunately, Intel SGX is not available yet. Rumors say it will appear on first chips shipped starting from December 2014. We’ll see.

Intel SGX is a great giant step to secure computation. But there is also a downside… Why are you supposed to trust Intel? Or any other chip vendor. With current level of complexity in modern processors and the number of transistors in them, there could likely be some security holes, or even intentionally left backdoors. One very interesting research paper shows how it’s easy to design a hardware backdoor and build it into a processor and the malicious mechanism is not something limited, targeting some very specific algorithm or whatnot. It is a flexible, configurable execution mode. You can outline some basic hardware backdoor design even yourself. Like, some exact instruction sequence triggers a mechanism in the processor that switches currently executed program to supervisor mode. Having such a simple tiny backdoor could be enough to do a lot of bad things with a carefully constructed attack.

So, what’s the possible solution? How can you check that there is no backdoor in your CPU? In software world it’s usually done by source code audit. Opening the source code of an application makes it possible to get the application audited by thousands of people. Researches show that in general, open-source has less bugs than comparing to proprietary software. Obviously it’s very hard to embed a backdoor into a widely-spread open-source project.

How can we apply that to the hardware world? Many people will be surprised to find out that there exist open-source processors. Several of them. First, the most heard one is OpenSPARC, but it seems like the project is dead after Oracle bought Sun Microsystems. Meanwhile, there is a very nice project called OpenCores. They have all kinds of open-source hardware cores, including general-purpose CPUs and specialized cores, like encryption cores, etc. The most noticeable core is OpenRISC processor core, a full featured RISC processor. Yes, you can really do things on it. For example, on this video the guy plays a game on this processor, running linux. 

It’s so cool that you can download a processor source code written verilog, modify it, then put it on a FPGA chip and run it! And what does it give us in terms of security? A great framework! Just imagine how can you implement all possible security hardware features and be sure that no one puts a backdoor on your chip. There is no underlying layer where attacker can hide. Or at least, you can download a processor source code and audit it, to make sure that there is no any bad code in it. I think it should become our future. We need to run open-source programs on open-source hardware, everything available for audit. It’s the only way to the real security.

How to translate virtual to physical addresses through /proc/pid/pagemap

I currently work on a project where I need to make translations for virtual addresses of user-level application to physical addresses in Linux. I implemented my own system call to do that, but had hard times with verifying the results I'm getting.

Later I found out that in newer kernels there is a really nice virtual file in the /proc file system to get this information. I tried to cat it, doing cat /proc/self/pagemap, and got terrible binary output in my console.

So, it looks like working with this file is not such a pleasant experience. It's a binary file with all that it implies. I found couple of scripts that access this file and provide you with a nice text result, but unfortunately those were written in perl and ruby, and I needed to run it on very minimalistic embedded system. I needed something that fits into a single binary.

Long story short, I decided to bite the bullet and write a tool in C. My contribution might be helpful for someone, that's why I'm sharing this code.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <assert.h>
#include <errno.h>
#include <stdint.h>

#define PAGEMAP_ENTRY 8
#define GET_BIT(X,Y) (X & ((uint64_t)1<<Y)) >> Y
#define GET_PFN(X) X & 0x7FFFFFFFFFFFFF

const int __endian_bit = 1;
#define is_bigendian() ( (*(char*)&__endian_bit) == 0 )

int i, c, pid, status;
unsigned long virt_addr; 
uint64_t read_val, file_offset;
char path_buf [0x100] = {};
FILE * f;
char *end;

int read_pagemap(char * path_buf, unsigned long virt_addr);

int main(int argc, char ** argv){
   //printf("%lu\n", GET_BIT(0xA680000000000000, 63));
   //return 0;
   if(argc!=3){
      printf("Argument number is not correct!\n pagemap PID VIRTUAL_ADDRESS\n");
      return -1;
   }
   if(!memcmp(argv[1],"self",sizeof("self"))){
      sprintf(path_buf, "/proc/self/pagemap");
      pid = -1;
   }
   else{
         pid = strtol(argv[1],&end, 10);
         if (end == argv[1] || *end != '\0' || pid<=0){ 
            printf("PID must be a positive number or 'self'\n");
            return -1;
            }
       }
   virt_addr = strtol(argv[2], NULL, 16);
   if(pid!=-1)
      sprintf(path_buf, "/proc/%u/pagemap", pid);
   
   read_pagemap(path_buf, virt_addr);
   return 0;
}

int read_pagemap(char * path_buf, unsigned long virt_addr){
   printf("Big endian? %d\n", is_bigendian());
   f = fopen(path_buf, "rb");
   if(!f){
      printf("Error! Cannot open %s\n", path_buf);
      return -1;
   }
   
   //Shifting by virt-addr-offset number of bytes
   //and multiplying by the size of an address (the size of an entry in pagemap file)
   file_offset = virt_addr / getpagesize() * PAGEMAP_ENTRY;
   printf("Vaddr: 0x%lx, Page_size: %d, Entry_size: %d\n", virt_addr, getpagesize(), PAGEMAP_ENTRY);
   printf("Reading %s at 0x%llx\n", path_buf, (unsigned long long) file_offset);
   status = fseek(f, file_offset, SEEK_SET);
   if(status){
      perror("Failed to do fseek!");
      return -1;
   }
   errno = 0;
   read_val = 0;
   unsigned char c_buf[PAGEMAP_ENTRY];
   for(i=0; i < PAGEMAP_ENTRY; i++){
      c = getc(f);
      if(c==EOF){
         printf("\nReached end of the file\n");
         return 0;
      }
      if(is_bigendian())
           c_buf[i] = c;
      else
           c_buf[PAGEMAP_ENTRY - i - 1] = c;
      printf("[%d]0x%x ", i, c);
   }
   for(i=0; i < PAGEMAP_ENTRY; i++){
      //printf("%d ",c_buf[i]);
      read_val = (read_val << 8) + c_buf[i];
   }
   printf("\n");
   printf("Result: 0x%llx\n", (unsigned long long) read_val);
   //if(GET_BIT(read_val, 63))
   if(GET_BIT(read_val, 63))
      printf("PFN: 0x%llx\n",(unsigned long long) GET_PFN(read_val));
   else
      printf("Page not present\n");
   if(GET_BIT(read_val, 62))
      printf("Page swapped\n");
   fclose(f);
   return 0;
}

And now how you use it. It's very simple. Of course you need to compile it. Then you need to find out what mapping your target process does have. You can do that by reading /proc/pid/maps file. Fortunately that file is human readable.
When you know a valid virtual address, you can pass it to our tool to get actual value from pagemap, including physical frame number. Here is an example:

$ #let's find get virtual address of a page
$ cat /proc/self/maps 
00400000-0040b000 r-xp 00000000 08:02 1177367                            /bin/cat
0060a000-0060b000 r--p 0000a000 08:02 1177367                            /bin/cat
0060b000-0060c000 rw-p 0000b000 08:02 1177367                            /bin/cat
0223a000-0225b000 rw-p 00000000 00:00 0                                  [heap]
7fe7e15e1000-7fe7e1cc3000 r--p 00000000 08:02 1577390                    /usr/lib/locale/locale-archive
7fe7e1cc3000-7fe7e1e80000 r-xp 00000000 08:02 527324                     /lib/x86_64-linux-gnu/libc-2.17.so
7fe7e1e80000-7fe7e2080000 ---p 001bd000 08:02 527324                     /lib/x86_64-linux-gnu/libc-2.17.so
7fe7e2080000-7fe7e2084000 r--p 001bd000 08:02 527324                     /lib/x86_64-linux-gnu/libc-2.17.so
7fe7e2084000-7fe7e2086000 rw-p 001c1000 08:02 527324                     /lib/x86_64-linux-gnu/libc-2.17.so
7fe7e2086000-7fe7e208b000 rw-p 00000000 00:00 0 
7fe7e208b000-7fe7e20ae000 r-xp 00000000 08:02 527300                     /lib/x86_64-linux-gnu/ld-2.17.so
7fe7e228d000-7fe7e2290000 rw-p 00000000 00:00 0 
7fe7e22ab000-7fe7e22ad000 rw-p 00000000 00:00 0 
7fe7e22ad000-7fe7e22ae000 r--p 00022000 08:02 527300                     /lib/x86_64-linux-gnu/ld-2.17.so
7fe7e22ae000-7fe7e22b0000 rw-p 00023000 08:02 527300                     /lib/x86_64-linux-gnu/ld-2.17.so
7fffce6b6000-7fffce6d7000 rw-p 00000000 00:00 0                          [stack]
7fffce722000-7fffce724000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
$ #don't forget alsr, normally only /bin/cat will remain same
$ #so let's pick 0x00400000. Now we run our program. 
$ #First argument is pid, "self" is a legal option too, the second is virtual address
$ ./pagemap self 0x00400000
Reading /proc/self/pagemap at 0x2000
Result: 0a60000000008c445

We got  0x0a60000000008c445 as a result. There are some bits showing that the page is valid, along with the size of the page. You can reed more in Linux documentation: https://www.kernel.org/doc/Documentation/vm/pagemap.txt. Basically, the physical page number is 0x8c445.

Note, that in different kernel versions bits 56-60 have different meaning. In most current versions, they are forced to zero, however in kernel version 3.11.0 they represent page size.

UPDATE!
The original version of the code worked only on x86-64 machines. It used to read sizeof(unsigned int) amount of bytes from the binary file, but it must read 64 bits no matter what word size target machine has. There was another issue, on OpenRISC simulator the fread() function always failed, that's why I changed the code to use cget() instead, that way it works with all architectures I tested so far.

I plan to expand the functionality of my program, for example include support of range lookups in the /proc/*/pagemap files. If someone is interested in some functionality like that, leave a comment below.

Where to start: TPM on Linux

It is extremely hard to find any useful examples of using TPM. There is a lot of theoretical stuff, like discussion is it good or bad, but really few code examples that one can use to play with.

The last few days I was looking at the security issues with TPM, and found out that this topic is not addressed very well. Actually, I had hard times finding a good entry point to the world of trusted platforms. I thought, for someone things that I found might be useful, so here we go:

1. First go to the BIOS and enable your TPM there (if you don't have such option, probably you don't have the TPM).

2. Then, check if your TPM module is loaded:

$ lsmod | grep tpm

You can also check your dmesg, if your TPM is active, you should see something like that:

$ dmesg | grep tpm
[    9.132580] tpm_tis 00:09: 1.2 TPM (device-id 0x0, rev-id 78)

3. Here is a nice web-page showing how to use your TPM. Probably, you need to take ownership on the TPM to use it (tpm_takeownership comand).

4. But we want something really simple, right? I think, generating a random number is a nice thing to do first.

There is a package rng-tools, which allows us to use the TPM as a hardware random number generator, but for some reason it didn't work for me, you can find more details here. Then I found the other really nice page showing exactly what we want. Unfortunately, there is no description how to compile the code. So, here is what I did:

First install the trousers-dbg package to get required header files:

$ sudo apt-get install trousers-dbg

You need tspi library to compile the program. So, try compile to compile it as following:

 $ gcc -o getrand tpm-getrand.c -ltspi

If compiles without any errors run it as it was described in the original page:

$ ./getrand | hexdump -C
00000000  97 37 73 96 98 ad 07 08  f3 bc 14 90 c6 10 73 aa  |.7s...........s.|
00000010

Congratulations! These are your true-random bytes.