Saturday, September 19, 2020

An AppImage for the NumericalChameleon, problems and solutions

Preface

What is an AppImage?

AppImage is a format for distributing portable software on Linux without needing superuser permissions to install the application. See also https://appimage.org/

What is the NumericalChameleon?

The NumericalChameleon is a free, open source unit converter running on the desktop on Windows, macOS, and Linux. It supports more than 6000 units in more than 90 categories. It is entirely written in Java. See also http://numericalchameleon.net

The Goal

On GNU/Linux the NumericalChameleon already supports several ways for a deployment: .deb, .rpm, self extracting file and a bzip2 compressed tarball. I wanted to support .AppImage as well, but I run into problems. That article should help other developers to bypass those issues that I have been facing with.

Problems and Solutions

While I was working on the .AppImage for the NumericalChameleon two major problems arise. To say it in advance, without code modification the AppImage did not work out of the box. Fortunately the code changes can be minimized.

Problem 1: Restart of the application didn't work

There are situations where the user has to restart the app. For example, if the user selects a different language for the GUI or if the user selects an option in order to let render the frame decoration by Java rather then by the operating system. The NumericalChameleon also offers the user to force a restart explicitly by selecting the menu item called "Restart" from the program menu. Example:

If a restart is being triggered, the current implementation determines the Java class that is being called when the user double clicks on the (portable or non-portable) jarball. It passes that class (along with some program args) to the startJarApplication method. The following is a short excerpt from the source of the NumericalChameleon 3.0.0:

                Class clazz = isPortable() ? net.numericalchameleon.launchers.MainGUIPortable.class : net.numericalchameleon.launchers.MainGUI.class;
                ProcessHelper.startJarApplication(JVMoptions, clazz, args);

See also https://github.com/jonelo/n16n-desktop

The startJarApplication method basically finds both the java executable binary and the .jar file that were used to start the app. It builds a new process using ProcessBuilder and starts it so that a new process with a new PID gets created. The parent process simply exists and the new process will survive.

That approach worked perfectly fine on all supported operating systems Windows, Linux, and macOS. However it failed on Linux with the .AppImage, because it is the AppImage itself that starts its contained application. The AppImage does that by mounting its payload in read only mode and calling an AppRun script that launches the JVM which launches the .jar file. So knowning the path to the java executable isn't helpful in this case, because it will change each time the AppImage starts.

Solution to Problem 1: Check whether we are running from an .AppImage?

The solution to the problem 1 is to determine whether the app is running from an .AppImage. And if it does, we have to restart the AppImage binary rather than the .jar file. To to that we can check, whether we are on Linux, because .AppImages are Linux only and we can check whether the system proberty called java.home starts with "/tmp/.mount", because that is the mount point that AppImage uses to mount its payload. If we are runnig from an AppImage, we also need to know the path to the .AppImage file itself. This can be done by passing the $APPIMAGE environement variable as a system property to the application launcher called AppRun:

# -- snip --
# start the application in the expected working folder ../openjdk/bin/java -Dapplication.appimage="$APPIMAGE" -jar nc.jar "$@"

 And in Java land, we need to add some code to achieve that:


        if (System.getProperty("os.name").equalsIgnoreCase("Linux")
         && System.getProperty("java.home").startsWith("/tmp/.mount_")) {
        ...
        String appimage = System.getProperty("application.appimage");

Since we now know the entire path to the AppImage, it is easy to restart the AppImage using a standard ProcessBuilder.


Problem 2: Files are not writable

The NumericalChameleon comes with a lot of files, many of them can be modified by the user. For example, if the user would like to load historic or current exchange rates, the user can do that by selecting the appropriate function in the app. However, since the AppImage mounts everything in ready-only mode, persistance is disabled and the app data cannot be updated by the user. The user has to wait for an updated AppImage. Well, since I am not going to upload an updated AppImage every day, those volatile data should be continued to be updated by the user on demand. The NC stores all changable data in the folder called data.

Solution to Problem 2: let's use a symlink to /tmp

Let's see whether we can find a solution for problem 2 that doesn't require any code chages in Java land. Since all files in the AppImage are being mounted ready-only and cannot be changed, the app image needs to hardcode a path that is available on all Linux platforms and it also must be accessible by the user. The home folder is not an option here, because the user's home is unknown at the time when the AppImage is being built. The solution is to use the /tmp path, because /tmp has the sticky bit set by default and it is writable by the user. So we bake a symlink called data that points to /tmp/.NumericalChameleon/data into the AppImage. The preciding dot is to mark that folder hidden on Linux. Here is an excerpt from my build script:

# -- snip --
# prepare a pointer to /tmp which is both known and user writable
mv "${n16nDir}/data" "${n16nDir}/data.ori"
cd "${n16nDir}"
ln -s /tmp/.NumericalChameleon/data data
cd -
"./${appImageTool}" --no-appstream "${appdir}" "${outdir}/${outfile}"

If the application starts, we simply need to check whether there is an up to date data folder and if it isn't, we simply copy the actual data folder from the mounted .AppImage mount point to /tmp/.NumericalChameleon/data:

# -- snip --
if [[ $refresh -eq 1 ]]; then
    mkdir -p /tmp/.NumericalChameleon
    printf "copying data files to %s\n" "$dataFolder"
    cp -r ../data.ori/. "$dataFolder"
    printf "%s\n" "$currentVersion" > "$versionFile"
fi

# start the application in the expected working folder

../openjdk/bin/java -Dapplication.appimage="$APPIMAGE" -jar nc.jar "$@"


And now we also can update the exchange rates on the fly:

./NumericalChameleon-x86_64.AppImage --filter ebc.europa.eu --continue

Mission completed.

The next release of the NumericalChameleon will also support the AppImage :-)


Sunday, October 13, 2019

AHT Fix - restore the Apple Hardware Test on your (old) Mac

This article applies to you only if you have an old Mac that was released before June 2013. Yes, this article is about old hardware that is still out there. It is even still traded on ebay for exmaple. So I am pleased to announce AHT Fix. The tiny app restores the Apple Hardware Test on older Macs (released before June 2013) where Mac OS X, OS X or macOS has been reinstalled from scratch. Please also have a look at the "Implementation details and credits" in this article.

The Problem

„Apple Hardware Test (AHT) contains a suite of diagnostics that will test the hardware of your computer. […] If your Mac was released after June 2013, you will use Apple Diagnostics rather than Apple Hardware Test (AHT).“ See also https://support.apple.com/en-us/HT201257

If you have reinstalled an older Mac from scratch, the diagnostic tools might no longer be available. If there is content in /System/Library/CoreServices/.diagnostics you are affected. Unless you have the original disks that came with your Mac, there seems to be no way to restore the AHT.

The Solution

AHT Fix. You can grab a copy from https://github.com/jonelo/ahtfix

The application will determine the model of your Mac, query an Apple server, download the appropriate AHT binaries and restore it on your Mac. And once restored you just need to follow the instructions at https://support.apple.com/en-us/HT201257 in order to perform all hardware diagnostics on your Mac.

Note: Starting with El Capitan (OS X 10.11) there is a new feature called System Integrity Protection (SIP) that prevents the modificaiton of a number of operating sytem directories by default.
In order to use AHTFix on El Capitan and later you have to disable SIP temporarily.
See also https://www.macworld.com/article/2986118/security/how-to-modify-system-integrity-protection-in-el-capitan.html

System Requirements

  • A Mac, released before June 2013
  • Mac OS X 10.6 or later
  • SIP disabled if you run El Capitan (OS X 10.11) or later
  • An internet connection (any bandwidth)

Implementation details and credits

On https://github.com/upekkha/AppleHardwareTest you find detailed instructions from Claude Becker how to restore the AHT manually. I put those instructions (and a few more) to a bash script so that you can restore the AHT in a comfortable way.

I also would like to thank Apple for hosting the AHT binaries on their servers. That really helps users to restore the AHT on Macs that would be otherwise incomplete.



Saturday, January 19, 2019

How to create a .jar association for your OpenJDK

From Java 11, Oracle has changed the license of their JDK, so you probably may want (or even have) to move to OpenJDK, and if you have moved already to OpenJDK, you may have noticed that clicking on a .jar didn't work anymore as it did with Oracle JDK binaries. This is because most of the pre-build OpenJDK binaries are coming without an installer on Microsoft Windows. Fortunately, it is easy to fix - with jarfix.

In the following example I show you how to use jarfix in order to setup the .jar association for your OpenJDK.


1. Download the OpenJDK

Download the OpenJDK from the source of your choice. You could get it from http://jdk.java.net/ or https://adoptopenjdk.net for example. Actually there are many more sources where you can get it, but I have downloaded the latest OpenJDK from https://adoptopenjdk.net, and extracted the .zip file to C:\Applications\jdk-11.0.1+13


2. Download jarfix

Download jarfix.exe from the jarfix homepage and store the .exe on a folder of your choice on your harddrive. On my Windows instance, I have stored the jarfix.exe to C:\Applications\jarfix\




3. Create a jarfix config file

Create a jarfix.ini file to C:\Applications\jarfix\ with the following content:

[jarfix]
options=/a
action="C:\Applications\jdk-11.0.1+13\bin\javaw.exe" -jar  "%1" %*


Those entries tell jarfix to setup a .jar association with the OpenJDK in C:\Applications\jdk-11.0.1+13\ - you may want to modify that path according to the actual path that you have choosen at step 1. The line called "options=/a" is optional, it tells jarfix to create a "Open as Administrator" menu entry in order to run .jar files also under the Admin account.

Hint: you can run jarfix /x in order to extract an example jarfix.ini file which has a lot of useful examples for a Java developer on Windows.

Update January 27, 2019: from Jarfix version 3 you can recreate hijacked .jar associations out of the box (by a double click without an .ini file) not only for the Oracle JDK, but also for the OpenJDK from Azul Systems.



4. Run jarfix using the config file

Open a Windows Command Prompt or simply hit both the Windows Key and R in order to enter the following command:

C:\Applications\jarfix\jarfix.exe /c


Update January 27, 2019: from Jarfix version 3 it is not necessary anymore to pass the option /c as a parameter. If a .ini file exists with the same name as the .exe in the same directory, jarfix will read the .ini file in order to initialize itself with the values from the .ini. In other words, you can simply double click on jarfix.exe and the content of jarfix.ini will be taken into account.








5. Test it

If you have run "jarfix /x" at step 3, you can double click on the existing jartest.jar in C:\Applications\jarfix in order to test whether it works as expected:




As you can see above we have launched the OpenJDK from AdoptOpenJDK by double-clicking on a .jar file.


Sunday, June 17, 2018

NumericalChameleon is running in the web browser using Webswing

Today I run the NumericalChameleon (a Swing based app) in my web browser on my desktop by using Webswing. It really works like a charm! The NC GUI acts in just the same way as it does when running natively.

It works like a charm - in the web browser on the desktop

I downloaded and installed Webswing 2.5.4. and I did no modifications on the NumericalChameleon 2.1.0. I just copied the NC folder to the Webswing demo folder and entered some values at the configuration dialog such as Name, Icon, Home Folder, Working Directory, Class Path, etc. After I have configured it in Webswing, the app appears in the app list of Webswing:





The NC GUI acts in just the same way as it does when running natively on bare metal.

Even theme selection, language selection, restarting the app, updating exchange rates and audio was working (at least when the process is running on localhost)! The performance is surprisingly good, although a little bit slower if I compare it with native performance.

One (minor) problem I found was that opening help and opening data tables didn't work, because the help files and data tables are stored locally as HTML. In the log I found traces of "WebPaintDispatcher:notifyOpenLinkAction", so the action was recognized, but no action happened.

Normal http links were working, however.

It didn't work well on the smart phone, though

I also tried to access the above setup on my smart phone, but without having a screen keyboard, the usage of a Swing app in the browser is very limited, especially if I rely on entering numbers to a software. Also it seems not to be possible to move windows around inside the browser. At least it was pretty funny to hear the numbers spoken by NC on my desktop audio boxes when I pushed the play sound button on my smartphone ;-)


Summary

In the web browser on the desktop using Webswing, the Swing based app NumericalChameleon is working pretty well actually. It is worth to try it!


Sunday, March 11, 2018

... and all I got was this lousy T-shirt

To be honest, it is not a lousy T-shirt at all, it is great T-shirt from the Linux Foundation actually. It has a simplified Tux icon and a little riddle on its front:



Warning: don't scroll down if you want to solve the riddle by your own. Scroll down in order to see how I solved the riddle.
.
.
.
.
.
.
.
.
.
.
   2nd WARNING
   DON'T SCROLL DOWN IF YOU WANT TO SOLVE IT BY YOUR OWN !!!
.
.
.
.
.
.
.
.
.
.
   Last WARNING
   Ok, either you have solved the riddly by your own or you are simply too curious ;-)
   OK, FEEL FREE TO SCROLL DOWN IF YOU WANT TO SEE THE SOLUTION !!!
.
.
.
.
.
.
.
.
.
Actually, I was wondering whether I can solve the riddle by using the NumericalChameleon (yeah, you know I am the author of it, don't you?) and I am glad to announce that it can be used to solve the riddle as well. I entered the bitstream at the category "Positional Notations, Radix 2-36", and converted the value to hex. Since the Unicode character set is a superset of ASCII, and numbers 0 - 127 have the same meaning in ASCII as they have in the Unicode character set, it was possible to prefix each hex number by 00 in order to satisfy the expected input format at the "Unicode® Characters (Basic Multilingual Plane)" category. Below you see screenshots of both the NumericalChameleon (with the solution) and a hex editor in the background (with the solution) in order to verify the result.