Monthly archives: September, 2014

Aligning qhost output on the commandline when hostnames are too damn long

qhost is a UNIX command line tool to print the status of nodes on a Grid Engine system.

The output is normally quite readable and is sorted by columns to give information on the hostname (“HOSTNAME”), architecture (“ARCH”), no. of CPUs (“NCPU”), processor load (“LOAD”), total available memory (“MEMTOT”), current memory usage (“MEMUSE”), swap memory size (“SWAPTO”) and current swap usage (“SWAPUS”) of each node on the cluster.

Unfortunately, when the hostnames are too long, instead of truncating them to keep the columns aligned the row gets shunted along, making the output messy and much harder to read quickly. Example ( in this case you probably also get wrapping because the main frame of this WP template is too narrow, sorry!):

HOSTNAME                ARCH         NCPU  LOAD  MEMTOT  MEMUSE  SWAPTO  SWAPUS
-------------------------------------------------------------------------------
global                  -               -     -       -       -       -       -
host1.grid.place.ac.uk lx26-amd64      8  2.99   15.6G    7.0G    7.5G   74.1M
host2.grid.place.ac.uk lx26-amd64      8  1.05   11.7G    2.7G    7.5G     0.0
hostname3.grid.place.ac.uk lx26-amd64     12  0.01    7.8G  229.9M   16.0G     0.0
bigsimulationsgohere.grid.place.ac.uk lx26-amd64      4  0.51   15.5G  847.8M    7.9G   48.4M
johhnycomputer.grid.place.ac.uk lx26-amd64      8  2.88   11.7G    5.0G    7.5G  158.6M
fishing.grid.place.ac.uk lx26-amd64      8  1.99   23.5G    2.8G    7.5G     0.0
localhost               lx26-amd64      4     -    7.7G       -    7.9G       -
a0.grid.place.ac.uk lx26-amd64      8  3.57   11.7G    4.1G    7.5G  324.0K
jointjoint.grid.place.ac.uk lx26-amd64     32 20.40  125.9G   18.4G   16.0G     0.0
krillmaster7.grid.place.ac.uk lx26-amd64     32 24.80  125.9G   21.6G   16.0G     0.0

I am going to pipe the output to awk and see if I can fix it by trimming the first column while leaving the rest as they are.

First, let’s start by reviewing how to print certain columns only, using awk, To print just columns 2 and 3 from the qhost output we use.

$ qhost | awk '{print $2, $3}'

The comma makes sure that awk puts a space between the columns.

But using printf gives us more formatting options. The following command gives output with all columns aligned. The formatting part is in the “quotation marks” and the arguments given to it follow the commas. We can truncate the first column string (“s”) to 8 characters and specify tabs (“\t”) between each column and a newline (“\n”) at the end of each row:

qhost | awk '{printf("%.8s \t%s \t%s \t%s \t%s \t%s \t%s\n", $1, $2, $3, $4, $5, $6, $7)}'

And actually I don’t need the architecture column, so the following gave me the best results:

qhost | awk '{printf("%.8s \t%s \t%s \t%s \t%s \t%s\n", $1, $3, $4, $5, $6, $7)}'

There are, surely, more advanced and perhaps more elegant solutions but this is a simple way to get the job done to a satisfactory degree.



Course of Theoretical Physics available online

A collection of classic theoretical physics books, the Course of Theoretical Physics by Landau, Lifshitz, Pitaevskii and Berestetskii (English edition), is available online courtesy of the Internet archive.  It seems that the last two books in the 10 book series are not available, perhaps because their later publication dates meaning they have a longer copyright term. For a discussion on the copyright details of the books and the original source for my links see this Reddit thread. The Russian versions can apparently be found here although I have not verified this.




Class of 2014

Congratulations to our Industrial Simulation bachelor students who graduated today.

Sponsion 2014

Sponsion 2014

 



ZotFile for syncing PDF articles from Zotero to my eReader

I use Zotero to manage my literature collection, including all the associated PDF attachments. It really made my life easier when I set up the WebDAV file sync on Box. However, until now the only way to sync files to my Onyx Boox M96 eReader (image) was by connecting a USB cable and copying them manually to the device. Since Zotero stores the files in cryptically-named individual folders it is hard to do this manually in an organised manner and involves lots of clicking. Today I am going to find a better way.

This is what we want! Our technical documents on a large-screen  eInk device that allows annotation. Great for proofreading!

This is what we want! Our technical documents on a large-screen eInk device that allows annotation. Great for proofreading!

I searched for an Android version of Zotero, hoping that I could just sync to the extra device. Aside from the fact that the article files would take up too much of my internal flash storage space there is actually no official Zotero for Android; only some discontinued third-party project called Zotero Reader and a paid-option called Zandy, which had mixed reviews on the Play Store.

But then there is ZotFile, which can be installed on your PC as a Firefox add-on or as an extension to the stand-alone Zotero, and gives additional functionality. Zotfile is able to copy the article attachment (PDF) to a location on your PC or Mac, for example a Dropbox folder, that is set up to sync with an external device. It can also extract any annotationsto the PDF that you create on the device. My eReader runs Android so I have a number of options for the software that does the syncing, including DropBox, Google Drive or even BitTorrent Sync. I am jumping between Windows and Ubuntu on my main PC and I had issues doing this with BTSync in a single shared folder before, but as long as the collection of files does not grow too big I can keep separate sync folders in Win7 and Ubuntu to avoid any issues. Using BTSync means that you have a collection of devices syncing to each other but your files are not stored in the cloud on someone else’s servers. All transfers are encrypted so in principle it is secure.

OK, here are the steps I took.

1. Install ZotFile as a Firefox Add-on (to work with the Zotero Firefox Add-on).

2. Create a new folder somewhere on the PC that will be the sync folder. In Ubuntu I chose “/home/username/Documents/ZotFileSync” and in Windows I chose “C:/Users/Documents/ZotFileSync” (or whatever). The important thing is that it is not the same folder on a shared drive, something that caused problems for me in the past.

3. In Firefox, find the ZotFile preferences. In the second tab, check the box to “Use Zotfile to send and get files form tablet”. Give it the sync “Base” folder path. I also chose to create subfolders and save a copy of annotated files with the suffix “_annotated”.

Setting up ZotFile

The ZotFile preferences can be found inside Zotero by going to Tools > Add-ons > ZotFile > Options.

4. Use BTSync or dropbox or whatever you like to sync this folder to the cloud.

5. Set up BTSync or dropbox or whatever on your tablet/eReader to sync the files to/from the cloud. You might need to install a file manager app in order to create a new sync folder on your device. I used ES File Explorer, which incidentally seems to work quite well on an eInk screen.

BTSync on eInk!

BitTorrent sync in the Play store on my Onyx Boox M96 eReader.

Now, in Zotero (on your PC) you can right-click an item, Manage Attachments and Send to Tablet (see image below). It should shortly appear on your device, as long as it is connected to the internet and syncing with the cloud. Just like magic!

Sending an article to your sync folder.

Sending an article to your handheld device from Zotero is easy with ZotFile, even though it is simply copying it to a folder that is synced by separate software.

UPDATE:

There are a few (device-specific) issues with this solution.

1) BitTorrent Sync is not running when I boot the eReader. I have to manually start it. It also forgets to sync my shared folder automatically so I have to remind it every time I run the app. I hope that the option to run apps on startup will be included on a later firmware update.

2) The Onyx Boox M96 (Booxtor edition) only scans the internal storage for new books when you boot the device, with no option to manually scan from inside the OS. That means that once the synced files appear in the btsync shared folder the device library doesn’t show them until the next reboot. Again, I hope a manual “scan for new books” will be added to the library app on a later OS update. You could alternatively use a 3rd party reading app that manages its own database of books on the device.

These two issues combined make the process of transferring documents to the eReader a little less automatic than I would like. However, I have eliminated the need to attach the eReader to my PC via USB cables. Let’s see if the arbiters of the device will tweak the software for us in the future.

At the moment I don’t seem to be able to export my annotated PDF files, so I have not tested the annotation extraction feature of ZotFile yet.