Adrian
Courrèges

Introducing Linux Visual Novel Reader

When it comes to self-learning Japanese, you can read books, watch Japanese TV with the JP subtitles on…

However if your level is just beginner or intermediate, you’ll often hit some kanji you can’t read, some words you don’t know.
Chances are you’ll spend 5 minutes looking up kanji by radicals in the dictionary to decipher a single page of a manga – the same page you would read under 5 seconds in its English version.

There goes your motivation…

Isn’t there a way to make the whole process more fun and efficient?

I find reading visual novels to be an interesting approach to making progress in Japanese: the game pauses after each sentence so, unlike a movie, you can take your time to understand all the words. Plus many novels are fully voiced, which is really important to learn the intonation.

All these games draw Japanese words on your screen, but unfortunately you can’t even select or copy them!
The words are here though, deeply burried in the game binary, if only we could grab them…

So here comes LiViNo Reader: it’s a small hack I wrote to live-stream the text rendered by the game directly to the Rikaichan dictionary so you can get an instant translation by just hovering a word.

It’s actually a combination of:

Check out the LiViNo Reader page for more information.
I also made all the code available on GitHub.

Deus Ex: Human Revolution - Graphics Study

2015/03/12: Back online after Reddit and Slashdot killed my bandwidth with 30,000 visits in the last hours. Followed by HN.
2015/03/11: Update with comments from Matthijs De Smedt.

The original Deus Ex is among the most critically acclaimed PC games of its time and I spent countless hours helping JC Denton fend off the conspiracies of UNATCO or the Illuminati.

I never had the chance to play the second opus “Invisible War”, but I gave a shot to “Human Revolution” when it was released. All I can say is: it lived up to the expectations!

Deus Ex: Human Revolution is a game released in 2011 by Square Enix, and developed by Eidos Montréal and Nixxes for the PC version. It uses a modified version of the Crystal engine made by Crystal Dynamics and was one of the earliest games to support DirectX 11.
It featured great graphics at the time (still looks good!), and it was as beautiful as light-weight: even low-budget video cards could run the game smoothly.

I was curious about the rendering process, so I spent a few hours reverse-engineering the game, playing with Renderdoc.
Here are the results of my investigation.

How a Frame is Rendered

Below is the scene we’ll consider. This is an actual screenshot of the game: the final image presented on the player’s monitor.

Let’s dissect this frame!

At first glance, Deus Ex HR seems to use an approach similar to the forward+ rendering technique.
Except that the game was developed years before forward+ became popular, actually it uses a precursor technique: the “light pre-pass” approach.

Normal + Depth Pre-Pass

The game renders all the visible objects, outputting only a normal map and a depth map.
Transparent objects are not rendered.
Depending on the mesh, each triangle will be rendered as a flat surface (same normal for all the fragments of the triangle), or can also be modulated by its own normal map. For example, the hand sculpture has its own normal map modulating the final normals rendered to the buffer.

Normal Map: progress 10%
Normal Map: progress 40%
Normal Map: progress 70%
Normal Map: progress 100%

Customizing IRKit Firmware: LED and Offline Mode

IRKit is a device really fun to play with.
After writing a custom initialization guide and a web solution to control the device, I decided to dig into the IRKit firmware.

There were still 2 points bugging me with the official firmware:

  • The LED is always on. It acts like a feedback, to show the user the device is working well. It’s all great and fine unless, like me, you can’t stand yet-one-more-LED lighting-up the dark room at night. I sleep much better in total darkness.
  • IRKit needs Internet access. During the setup phase, IRKit needs Internet access to validate the device key (used by the cloud control). Also the firmware polls the cloud servers every 25 seconds to check if there are some IR commands to send. I don’t need the cloud feature, I use IRKit Web Remote instead. I wanted a LAN-only mode for IRKit, it is much more secure than granting the device WAN access.

So here is my modified firmware which allows you to customize these 2 points.

The rest of this post explains how to retrieve the code, recompile the firmware and deploy it to your IRKit.

Retrieving the code

First let’s get the code from GitHub. You can clone the repository:

git clone https://github.com/acourreges/device.git

or alternatively simply download the zip archive.

Customize to your taste

Navigate to device/firmware/src/IRKit and rename the file version.template to version.c.

Open the file config.h, this is where you can configure the settings:

config.h
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
// Different modes for the LED feedback.
typedef enum {
    LED_VERBOSE    = 0, // LED almost always-on to indicate the device status.
    LED_QUIET      = 1, // LED on during setup and blinks when receiving/sending commands. Off when the device is idle.
    LED_SETUP_ONLY = 2, // LED on only during the setup, then always-off.
    LED_OFF        = 3  // /!\ This disables the LED completely.
} LedFeedbackProfile;

// Defines what kind of LED profile you want.
const LedFeedbackProfile ledFeedback = LED_QUIET;

// Enables/disables cloud-control through the "deviceapi.getirkit.com" server.
// If you use your IRKit device exclusively over your LAN, you can disable
// this option: the device won't send regular polling requests to the cloud server.
// This also lets you setup your device without internet access (no need for a valid device key).
const bool useCloudControl = true;

To modify the LED behavior simply modify the line const LedFeedbackProfile ledFeedback = LED_QUIET; and replace LED_QUIET by either LED_VERBOSE, LED_SETUP_ONLY or LED_OFF.

If you want to enable the offline mode, then set useCloudControl to false.

Set up the Arduino environment

You should install the “Arduino IDE” for your OS. Official instructions can be found here.

I’m going to give some more details concerning the setup with Windows because I encountered a few problems.

First install Arduino IDE 1.0.6, it should install the drivers for the Arduino boards also.
Then connect your IRKit to a USB port of your computer.

Let’s check if the board is recognized correctly. Open the device manager (‘Computer’, right-click, ‘Properties’, ‘Device Manager’), if the board was recognized it should appear on a COM port, in my case COM7:

If you see your device then congratulations, you can skip the following and go directly to the next step.

But chances are, your device was not recognized and just appears as a yellow question mark in the list.
No worry, we will just be more explicit and tell Windows which driver we want to use.
Still inside the Device Manager, right-click on your unrecognized device, ‘Properties’, ‘Details’ tab and in the drop-down menu select ‘Hardware Ids’.
It should display something like this:

Note down the values, we’re going to need them later.

Then navigate to C:\Program Files (x86)\Arduino\drivers and extract the archive Old_Arduino_Drivers.zip.
Edit the file Arduino Leonardo.inf and modify the Vendor and Product ID Definitions section to add the IDs you got previously from the Device Manager:

Arduino Leonardo.inf
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
...
[SourceDisksFiles]
[SourceDisksNames]
[DeviceList]
%DESCRIPTION%=DriverInstall, USB\VID_2341&PID_0036
%DESCRIPTION%=DriverInstall, USB\VID_2341&PID_8036&MI_00
%DESCRIPTION%=DriverInstall, USB\VID_1D50&PID_6085&REV_0100&MI_00
%DESCRIPTION%=DriverInstall, USB\VID_1D50&PID_6085&MI_00

[DeviceList.NTamd64]
%DESCRIPTION%=DriverInstall, USB\VID_2341&PID_0036
%DESCRIPTION%=DriverInstall, USB\VID_2341&PID_8036&MI_00
%DESCRIPTION%=DriverInstall, USB\VID_1D50&PID_6085&REV_0100&MI_00
%DESCRIPTION%=DriverInstall, USB\VID_1D50&PID_6085&MI_00
...

Save the file, go back to the Device Manager, then right-click, ‘Properties’, ‘Driver’ tab.
Click on ‘Update Driver…’ and ‘Browse my computer for driver software’. For the folder simply provide C:\Program Files (x86)\Arduino\drivers\old_drivers and let Windows install the device.

Windows should now associate the Leonardo driver with your board. Your device should appear as a COM device.

Since we’re still in the ‘Device Manager’ let’s fix one more issue you might run into.
When trying to upload the new firmware to IRKit I got the following error:

avrdude: error: buffered memory access not supported. Maybe it isn't a butterfly/AVR109 but a AVR910 device?

To prevent this problem, right-click on your device, ‘Properties’, ‘Port Settings’ tab, ‘Advanced…’:

Make sure the Use FIFO buffers checkbox is not selected.

Compile and upload the firmware

Go back to the folder with the source code of the firmware (where you edited config.h).
Double-click on IRKit.ino, this will launch Arduino and load the project.
Make sure the correct COM port is selected:

Then simply click on ‘File’, ‘Upload’: this will compile the firmware and upload the binary to the board.
And that’s it! No more bright LED. :)

Note that in some case, flashing the firmware can cause a reset of the EEPROM, meaning the board might forget the credentials to join your home WiFi network so you might have to go through the setup phase again.

An important point: after flashing this custom firmware, the device in setup-mode will always broadcast a WiFi network which password is now XXXXXXXXXX, so don’t try to use the old password you found on the small piece of paper inside the IRKit box.

Introducing IRKit Web Remote

I previously wrote about the IRKit device and how to set it up from a basic webpage.

The setup guide was handy as a proof of concept, but I was still lacking a way to organize efficiently all the IR commands I recorded from various remote controls.

So here comes IRKit Web Remote: a web-solution to control and dispatch commands from a web-browser.
I host it on a small Raspberry Pi so anybody in the family can just fire-up a browser on the phone, open the page and send commands.
Having everything centralized on the server is really useful when I need to add new commands: I simply update the web page on the server and I don’t have to care about maintaining clients one by one, clients will get the latest features upon the next page reload.

A few features of IRKit Web Remote:

  • supports sending one single command or a series of command (with custom delays between each)
  • command buffer to avoid overloading IRKit, with current queue visual feedback. Queue is also cancellable.
  • any size of screen supported thanks to the responsive design of Bootstrap
  • can be exposed to the Internet (a script forward the POST request within the LAN)
  • includes the interactive guide to do the initial setup of a new IRKit device
  • easy to tune and adapt to your needs

The code is on GitHub, feel free to grab it and play with it.

IRKit Setup Guide for Android, iOS, Linux, Mac, Windows

Recently I got my hands on an IRKit device. It’s a pretty neat tool to control several devices remotely by sending IR signals. I use it to control my TV, air-conditioning, light, sound system…

It is fairly cheap compared to the competition (65$ on Amazon Japan), and one strong point is it is completely open-source: you can play with the firmware, customize it any way you want…

What is IRKit exactly?

It’s basically an Arduino board, with a wifi module and some IR receiver and transmitter.
The device is low-consumption and is powered through a micro-USB connection, just like a smartphone.
It’s very tiny and fits anywhere: a 6cm x 6cm square, 1.5cm tall.

It connects to your wifi network and runs its own web-server listening to your commands. All the communication with IRKit is done through HTTP requests, which makes it super-easy to send commands directly from a web-browser.

The official website talks mainly about iOS clients, but don’t worry, any platform supporting a web-browser can be used: Android, Linux, Windows are perfectly fine.

The device also supports control from the cloud: you can send commands from outside your home, they will be relayed through the IRKit servers.

How do I read/send IR signals with IRKit?

You first ask your IRKit to read an IR signal. For example, you point your TV remote at IRKit and press a button, IRKit will automatically pick-up the signal.
Then you can retrieve the IR data by making a simple GET request. For example:

$ curl -i "http://192.168.11.30/messages"
HTTP/1.0 200 OK
Access-Control-Allow-Origin: *
Server: IRKit/__VERSION__
Content-Type: text/plain

{"format":"raw","freq":38,"data":[18031,8755,1190,1037,1190,1037,1190,1037,1190,1037,1190,1037,1190,1037,1190,1037,1190,1037,1190,3228,1232,3228]}

When you want to send a command, you just ask IRKit to replay the IR data.
This is done by sending a POST request, for example:

curl -i "http://192.168.11.30/messages" -d '{"format":"raw","freq":38,"data":[18031,8755,1190,1037,1190,1037,1190,1037,1190,1037,1190,1037,1190,1037,1190,1037,1190,1037,1190,3228,1232,3228]}'
HTTP/1.0 200 OK
Access-Control-Allow-Origin: *
Server: IRKit/__VERSION__
Content-Type: text/plain

The API is very simple and nice to use.
The logic is completely client-side: IRKit is just a “dumb” state-less transmitter. So you could build your own automation system that dispatch commands to automatically turn on some equipment at some time of the day.

How do I initialize it?

When you receive your IRKit, it doesn’t know about your home wifi.
There is first a phase of setup during which the device exposes its own wifi network. By connecting to the device wifi network, you can provide the credential so the device can join your home network. It’s a process a bit complex since it involves getting some API key from the IRKit servers, obtaining a security token for the device, sending your home wifi login details in some serialized format. Quite tedious by hand…

The official way (iOS)

The official website mentions iOS support only: you can download an app from the App-Store, follow the steps and it will setup IRKit for you.

What if I don’t have any iOS device?

I tried a few other alternatives:

  • IRKitJS: it’s browser-based, so multiplatform, unfortunately it failed silently during the last step for me.
  • IRKit Controller Free it actually works pretty well, but it’s only available on the latest Android versions, if you have an old terminal you’re out of luck.

Note that you need an understanding of Japanese to use both of these solutions, there’s no English version.

My solution

In the end, since I wasn’t satisfied with any of the solutions, I came up with my own initialization system: a single HTML page dispatching HTTP requests through Javascript.
It’s a step-by-step interactive guide, telling you what’s happening at each stage. And since it’s web-based it works on any platform.

Try it here:

>> IRKit English Setup Interactive Guide: Android, iOS, Linux, MacOS, Windows <<

Note: my guide is in English, but you’ll still need to get the default password for your device. It is written in Japanese on a small piece of paper included in the IRKit box:

And you don’t have to trust my server: if you have doubt you can save the HTML document and run it locally, the JS code is all available under a MIT license anyway.

What next?

Your device is operational? Great! You can now record all the IR signals you wish.
To organize them, give a try to IRKit Web Remote, a solution I came up with.