A hack menu for FLIR thermal imaging cameras

Intro/description

Being somewhat of a thermal imaging enthusiast, and owning a number of thermal cameras produced by FLIR, I was quite disappointed by the obvious paywalls and crippled firmware.
Their cameras are best-in-class, but often contrasted by heavy price tags. What FLIR never tell their customers is that a lot of their lineups, namely the Ex, Exx, and Txxx series, are nearly identical in hardware, and are only differentiated by firmware.
This was discovered around 2013-2014 by the brilliant people at EEVblog.com, who managed to hack the bottom-of-the-range E4 camera into the highest end model — the E8, and add even more features that were never present in that model.
Later on, the Exx series, the bigger brother of the Ex series with 30/60Hz frame rates, adjustable focus, and PC connectivity, was also hacked. With that, any camera, from E30 to E50 was hackable to the E60 model, with higher resolution and again, features never present in that lineup at all.

In early 2018, I managed to score a FLIR T440bx for myself at a reasonable price, and shortly after receiving it I had all the possible features unlocked. That, however, was not enough for me.
But what could I do? Almost all efforts were focused on unlocking features that were already present in the original firmware, which were somewhat limiting. My camera already had a factory resolution of 320x240. It already had streaming, photos, and panoramas. It already had MSX. There had to be more I could do. When I was reading the threads and browsing through the camera's file system, I couldn't keep my mind off the fact that it was running Windows CE. I kept thinking - what if I could write my own applications for it? Could it work? So I kept reading, and reading, until I found this post.

After all, someone did manage to compile their own application for the camera, and run it successfully! That was a start, but a very painful one. I tried to get the gcc cross-compiler to work, tried to use the SDK, but nothing - it either would crash or not run on my camera at all. Maybe it was because I was(and still am) not a very good programmer, or because something was missing, but I decided to go a different route. Since this appeared to be a stock ARM machine running WinCE, I downloaded VS2008, set up the Pocket PC 5.0 SDK, and tried to compile my first Hello World application. It worked. Success, I was in! Now, I had to figure out what was possible.

The hack

Turns out, without a lot of digging, not a whole lot could be done. Most of the camera's functionality was locked away deep in libraries with no debug symbols, and the entire system was built on a very strange registry-like system, with no API to speak of. First, I had to create a way to interact with my code without a PC attached. This turned out to be a nightmare, because the stock GUI would override any windows and messages I created using the standard WinCE API. Thankfully, with a bit of trickery, I was able to get around this issue. Using the FLIR-proprietary registry, it is possible to entirely disable the UI using a special keyvalue, not available on lower-end models, but still present and functional. I had a complete UI that I could interact with.
The first menu

I knew that I wanted to port this to the FLIR Ex and Exx series cameras, so I had a slight issue - while the Txxx series cameras had a touch screen, the Ex and Exx did not. I had to implement a way to navigate the menu with the joypad. That wasn't too difficult - all user controls on the camera, with the exception of the laser button, are handled via the Windows keyboard API, so a global keyboard hook via the undocumented SetWindowsHookExW function was the obvious(and only) option. Since I already had that implemented to bring up the menu in the first place, all I had to do is figure out which virtual keycodes corresponded to each button. For this, I created a separate tool, about which I have an article here.

Keyboard navigation without the touch screen

Now I had complete freedom to do what I wanted, so it was just a matter of digging through the camera's proprietary registry and figuring out what I could abuse and turn into features that myself or anyone else could need. So I came up with a basic set of features that were useful:
  • Removing the persistent FLIR logo, to have a clean UVC(webcam) output for live video recording.
  • Disabling the automatic non-uniformity correction(shutter) that could be distracting in some applications.
  • Forcing the saved JPEG still photo quality to 100%, making them useful as an end-product, rather than something to get rid of via FLIR Tools.
  • The ability to change the MSX alignment on the fly by means of an unused key combination.
  • The ability to change the zoom level, also on the fly.
  • A way to capture a live sequence of fully radiometric images at the maximum camera frame rate(sadly limited by available RAM), and save it to the internal memory or SD card.
  • On cameras with motorized focus, the ability to capture a set of images with different focus positions, to post-process into a focus stack, not unlike conventional camera focus stacking techniques.

Focus stacking

Persistency

The last issue I faced was persistency - finding a way to run the hack when the camera boots. On older versions of the device, this came quite easily - all I had to do was to add a single line to applaunch.dat - the name of my executable. But while having other people test it on their cameras with newer firmware, applaunch.dat was locked behind signature checks which were a nightmare to bypass - and for all intents and purposes, nearly impossible. At first, I thought: no way, my whole hack idea was a failure, people couldn't run it without a PC, which was useless! But then, I remembered that preset files looked suspiciously like sets of commands, not unlike those I executed to turn features on and off. I started digging, and a single resource value got my attention - ".appl.supv.exec". I wrote the hack's executable name to that variable, and sure enough - my hack was running a short while later! So I loaded a few executables and files into IDA Pro, and found a preset that was always mentioned, but never used - $preset_estimate_max_load. I created a folder, placed a custom .rsc file containing the resource entries to execute the hack, and sure enough - it was there. I selected it in the menu, and it ran. Through further testing, it was determined that the custom presets are only enabled in the Txxx-series cameras, and do not appear in Ex and Exx-series cameras, but replacing existing presets works. A compromise, but a good compromise regardless, especially considering some presets are beyond useless.

Things left to do

With the ability to create a set of 16 images in rapid succession, there arises a possibility to perform what is called superresolution, or as FLIR like to call it, UltraMax. However, my programming skills, especially in the field of image processing, are rather lacking. This is where external help would be appreciated. Right now, the application is able to capture the set of images into a single .seq file - a FLIR-proprietary image format that contains raw, 16-bit images in the .fff format(again, proprietary). There has already been some work done to reverse-engineer and convert this file format to other, more easily processable formats. Namely work done by tomas123 from EEVblog and Phil Harvey, the developer of ExifTool. This thread is a good place to start.

Now, a bit on UltraMax - while the goal of creating third-party superresolution images is nice, in my opinion, a better solution would be to convert the 16 images captured by this hack into the UltraMax format. I chose 16 as the image count precisely for this purpose, as the FLIR UltraMax format is a combination of the compressed preview JPEG(not unlike regular Ex/Exx images), and 16 fully radiometric images compressed with the odd JPEG-LS format(note: not JPEG), alongside some data, like sensor temperature, planck constants, and calibration data.
A good place to start reverse-engineering and possibly recreating this file format is another post by tomas123, located here. Another place is here.

Installation

Since this isn't purely a theoretical article describing my mischevious endeavors, I have compiled a set of versions of the current hack to run on all three camera lineups - Ex, Exx, and Txxx. To install, you will need to follow these steps:

  1. Ensure you have a complete backup of the system and can recover it via .fif files if something fails - this guide deals with files that will prevent your camera from booting if modified incorrectly.
  2. Download the package corresponding to your camera series - Ex, Exx, or Txxx.
  3. Download the persistency package corresponding to your camera's firmware version - CRC01 or CRC32 respectively.
  4. Connect to the camera via FTP.
  5. For older cameras, place hack.exe and hack.ini into /FlashFS/system. For newer cameras, place it into /FlashBFS/system.
  6. Find the DCIM directory within the filesystem structure - on cameras with SD card support, it must be on the SD card, not on internal storage.
  7. Create a folder called ImageSequences inside DCIM.
  8. Open hack.ini, and edit save_path to point towards the directory. Example: save_path = \StorageCard\DCIM\ImageSequences\

  9. For older, CRC01 cameras with modifiable applaunch.dat:
    There are two ways to establish persistency - first is automatic, by appending applaunch.dat with a line that starts the hack on boot. An example file is provided in the archive.
    Second is via an unused preset - by placing the folder named $preset_estimate_max_load into /FlashFS/system/profile.d

    For newer, CRC32 cameras with signed, unmodifiable applaunch.dat:
    Replace one of the existing presets in /FlashFS/system/profile.d with the one provided - the example uses $preset_delta_spots, but the user.rsc file should work with any other preset you deem less useful. You may need to re-sign the user.rsc file if it isn't working.

  10. Perform a cold reboot by disconnecting the power and completely turning the camera off - putting the camera into sleep mode will not apply the persistency mods.
  11. Turn the camera on. If you're using an older camera with automatic persistency, skip step 12.
  12. If you're not, navigate to the presets menu, and trigger the preset you chose to replace. The hack will take about 3 seconds to load.

Usage

For Txxx series cameras, the menu is opened by holding the joystick button for 1 second, then releasing it. Options can be navigated by either joystick or touch screen.
For Ex and Exx series cameras, the menu is opened by holding the back/menu button and tapping the joypad's down button once, then releasing the back button.
When in normal operation, if the superresolution and focus stacking(only available on Txxx) options are turned on, they can be triggered by pushing the joystick up and down, respectively. The programmable input is the combination of holding the back button, and tapping the joystick left or right, to decrease or increase the value selected in the menu, for easy adjustment. The files are output in the save_path folder you configured in step 8.

Video tutorial


Discussion

To discuss this hack, contribute features, or request new features, go to the EEVblog thread here.
If you'd like to contact me privately, go here.

Download

Legal disclaimer: By following the instructions or downloading the files provided, you acknowledge and accept that the author is not affiliated with FLIR in any way and assumes no responsibility for anything that may happen to your camera or any other property. This is done as an exercise purely for educational purposes and should not be used commercially.
A less legal disclaimer: This shouldn't break or damage your camera, and it's designed to be completely reversible and non-intrusive, but if anything breaks, it's your fault. This also voids your warranty.

Changelog
  • Version 1.0, Jun 17, 2018 - Initial release.
  • Version 1.0a, Jun 19, 2018 - Fixed a persistency file error - main binaries remain unchanged.

Main hack package:
Ex series Exx series Txxx series

Persistency:
Older FW with CRC01 Newer FW with CRC32