Category: Guides

Moving to another country is expensive

Guides Leave a comment

In December I moved to Hannover, Germany to start a postdoc at the Albert-Einstein-Institute, a Max Planck institute dedicated to developing instrumentation and conducting data analysis for gravitational wave astrophysics. I’ve started work on the AEI 10m prototype, which is an absolutely awesome experiment, but that’s for another post.

I found the move to be relatively painless, but expensive. I knew there would be some up-front expenditure before my first pay cheque, but I wasn’t quite expecting the amount I ended up shelling out. This is a guide to approximately what I paid, which I hope will serve as a guide to others moving to this part of the world in the future.

Moving and on-going costs

Flights

Hannover Airport. Conveniently just north of the city, and only about 15 minutes from the station closest to my apartment. Ah, Germany.

Hannover is not a particularly popular tourist destination, so direct flights from Scotland don’t exist. You can instead fly via Manchester, Amsterdam or London with various airlines. I chose to fly BA via London Heathrow, costing £220, albeit a return ticket so I could come home for Christmas. The flight back to Hannover after Christmas was £110.

Apartment rent and deposit

As is normal, I had to pay a deposit to my landlord (“Vermieter”) in order to rent my apartment. The standard in Germany is equivalent to three months of rent – this is much higher than in Scotland, where the standard is equivalent to one month’s rent, with a legal maximum of one-and-a-half.

I don’t want to divulge what I pay in rent, but taking the average rent in Hannover to be €450 for a single bedroom apartment, expect to pay around €1350.

Apartment furniture and appliances

Furnished apartments are rare in Germany. Most apartments offered to tenants are free of all but the most basic fittings such as oven, hob and fridge (but not light fixtures – apartments often have free hanging electric wires waiting for a light fixture to be attached!). The walls are usually painted in neutral (read: white) colours, with any holes made for hanging pictures or furniture sealed with filler. One of the most missed items is surely the washing machine – you may quickly tire of hand-washing or using the local laundrette.

So, one of the earliest tasks for a new resident in Germany is to head to IKEA, browse the “for sale” section of the newspapers or time their move for when a colleague is leaving their own apartment and needs to sell their stuff.

I managed to borrow a lot of furniture from friends – in fact, I was extremely lucky and got almost everything on loan for free – but I did still spend around €500 in IKEA. Expect to spend significantly more if you aren’t as lucky – between a bed, mattress, sofa, kitchen table, desk, chairs, shelves, washing machine and any other white goods not provided, you will surely spend €1500 or more.

Utilities and broadband

Some apartments are provided “warm”, i.e. with heating, electricity, gas and water included in the rent. This is particularly the case if the block has some form of central heating shared between apartments, where the factors charge the landlord for usage, who passes it on to you via rent. At the end of the year, the landlord then calculates the difference between what they’ve charged and what they’ve paid, and you either pay or receive the difference. My apartment is older, and does not have shared heating, and so my rent is for a “cold” apartment; as such I had to arrange to pay electricity, gas and water myself.

The local supplier in Hannover is “enercity”. There are others, but this company is the de-facto standard. They might even be owned by the city in some form. That’s what my landlord used before I moved in, and given my inexperience with utilities I just kept them on. In all, enercity decided that €93/month is adequate for gas, electricity, fresh water and waste water supply/extraction. Whether this is too low, too high or just right remains to be seen.

You certainly don’t miss “in your face” marketing in Germany.

For my broadband connection, I wanted a supplier that didn’t lock me in to a two year contract, as seems to be the norm in both Germany and the UK. I found 1&1, and signed up for their 50Mb/s plan at €19.99/month. They actually advertise it at €16.99, but as is unfortunately standard practice, they make you pay for a new modem – similar to one you probably already own from a previous broadband contract. To add insult to injury, you not only pay a one-off fee for the modem – €49.99 – but also an additional €2.99/month on top of your plan as a “rental” fee. I already own a modem/router, but opting for the modem-only option (there is no option to take nothing) results in a one-off charge of €149.99. The extra €100 up front would require about three years of payments to eventually start saving money, and I find it unlikely I’ll stick with the same provider for so long, so I went for the €49.99 + €2.99/month one.

I also had to pay a whopping €150 installation fee because I didn’t opt for a two year contract. I imagine this will simply involve a technician turning up and twiddling a few wires, but it’s another hoop you have to jump through. Man, broadband supply in Germany, the UK and the US is crying out for a disruptive start-up company to move in and kill the monopolistic incumbents!

Bank charges

Banking in Germany is in the stone age by UK standards, with many shops not accepting debit or credit cards, and all preferring cash. Germans like it this way due to their distrust of consumer credit and debt in general (many small or medium German family businesses – termed Mittelstand – prefer to grow organically over decades instead of taking loans, allowing seemingly simple objects such as pencils to still be manufactured profitably in Germany).

As banks don’t tend to profit from overdraft charges, VISA/Mastercard kickbacks and other means of money-making from cashless transactions, they charge customers for maintaining a current account. They also sometimes charge for opening an account, especially for the most basic accounts used for holding apartment deposits (which, in keeping with the stone age theme, use paper passbooks).

As the Max Planck Society is a partner with Deutsche Bank, they offer members a free current account, so I took this offer. In general, other bank accounts charge somewhere in the region of €2-10 per month depending on features. About €4/month seems typical from asking around. Online-only bank accounts exist, often available free of charge, but the sign-up process apparently often involves a Skype conversation to verify your identity, which sounds daunting to me as a German non-speaker.

My apartment deposit account cost a one-off €20 to open.

German TV licence

German public TV and radio stations. Note that none cover the state of Niedersachsen, where Hannover is located.
From Wikipedia.

I had barely unpacked in my apartment after registering as a resident, when I received a letter from the ARD, ZDF and Deutschlandradio, demanding that I register for a TV licence.

Whether you like it or not, you are obliged as a householder to pay the German TV licence (“Rundfunkbeitrag”). Unlike in the UK, where it is required only in the case of those who watch live broadcast or, since September 2017, BBC catch-up TV, in Germany it is a required* payment whether you watch TV or not. The spirit behind this rule is that even if you yourself do not benefit from live broadcast TV, society in general does, and therefore all must pay for its upkeep.

 

There is a small but vocal minority of residents in Germany that refuse to pay this TV licence, but eventually the agency responsible for its collection will take the money owed directly from your bank account and so you should just pay up to avoid trouble, and contact a local MP to complain about it. The charge is €17.50/month.

*Apparently this is a strange legal loophole. It is not technically illegal to avoid paying the Rundfunkbeitrag, but the agency in charge of collecting the fee are legally allowed to collect the money from you. If after enough warning letters you have not paid, they can collect the money from your bank account anyway.

Health insurance

All German residents must have health insurance. The state provides coverage to certain state employees, including Max Planck Society contract holders, and anyone earning less than around €50,000 per year. The payments are standardised, with the employee and employer splitting the cost equivalent to approximately 15% of the employee’s salary. Private health insurance is available for anyone willing to pay for it, and certain employees ineligible for public coverage. Private coverage is typically better than public, offering better quality care and treatment – private rooms in hospitals, treatment from more experienced doctors, quicker appointments, etc. – at a large premium. It’s not the way it should be, but it’s the reality of treatment in Germany, the UK and most other developed nations.

Due to the nature of my contract at the Max Planck Society, I have to pay for private health insurance. The premium depends on your age, and for a late-20-something it costs an eye-watering €300/month (here is a table). I do get some of that back from my employer, and if you consider that you would be paying around £200/month in National Insurance in the UK on an average salary, then it’s not that much worse.

Total

One last thing to consider is food. Food costs are about the same as in the UK, so about €100/month depending on your tastes and frugality.

Here’s a table with my approximate up-front and ongoing costs. Consider these to be a very rough estimate; individual circumstances and the cost of living in different parts of Germany will vary things.

Expense Up-front cost Monthly cost
Flights €250 €0
Apartment rent and deposit €1350 €450
Apartment furniture and appliances €1500 €0
Utilities and broadband €200 €113
Bank charges €20 €4
German TV licence €0 €17.50
Health insurance €0 €300
Food €0 €100
Total €3320 €985

Even if you have a job lined up before you go, depending on how well organised you and your employer are, it would be wise to assume that there’s a chance you won’t get paid for a couple of months. If you miss some deadline to get the paperwork sorted out, your company’s HR department might decide to postpone your first cheque until the next month. With that information, it seems reasonable to take the up-front costs above and add twice the monthly outgoings to come up with a figure you may wish to consider having available when you come. For me, that comes to €5300. Ouch!

Of course, there’s a chance that you get paid without a hitch in month one, or you have significantly lower costs (for example, staying with a friend or in an Airbnb for the first month would potentially be better purely from a cash flow point of view, due to the lack of deposit), or your employer offers you an advance or moving bonus, or you slum it without furniture for the first month. You might not need such a high figure. Tailor everything to your circumstances.

Most importantly of all, if you are planning to move to Germany: have fun! Enjoy the welcoming environment, tasty food and fascinating culture this amazing and quirky country offers!

Passing the DVSA theory test

Guides Leave a comment

I recently passed by DVSA theory test, which is a requirement for sitting the practical driving test to legally drive on UK roads. I booked my theory test 3 days ago, grabbing a last minute slot for earlier today, having not studied before then. Normally the waiting time is around 3 weeks, but when checking for future availability I spotted a late cancellation and grabbed it, then made plans to frantically study.

The tests are split into a 50 question theory test, with multiple choice answers, and a “hazard perception” test where you are shown a driving scene where you must click on potential hazards as they arise. There are plenty of guides available for studying for the tests, but the only ones I would trust are the official DVSA ones. You don’t want to fail part of the test by following an incorrect, unofficial guide that might be out of date (for example, in 2013 the DVSA updated the hazard perception test from live video to CGI, making a lot of practice videos obsolete). They’re not that expensive, either.

Software

I was about to download the DVSA theory test PC software, but later found that they provide a mobile phone app that does the same thing. I normally try to avoid apps, because the screen is small and I don’t like giving them access to my phone, but in this case the difference in price was significant (£5 vs £15 for the same software). I also didn’t want to risk the software not working on Linux via Wine, which is what I use (badly written Windows software often can’t be properly emulated on Linux).

Theory

Almost all of the theory questions can be studied for simply by reading the Highway Code once, carefully, and using common sense; and of course doing lots of practice questions to check your understanding. You can even skip the parts of the highway code specifically for cyclists, motorcycles and trucks/heavy goods vehicles if you’re lazy, as these won’t be tested directly. Ensure you cover the road signs, as there are many and they aren’t all ones you’d have observed regularly on roads where you live.

The Highway Code took me about 3 hours to read, and I did this first before using the app. The app contains hundreds (~750) of questions, some of which will be asked in the actual test, and written guides to the Highway Code sections. Reading everything in the app, and doing the practice tests for each section, took me about 10 hours split over two days. Mock exams of 50 questions took about 15 minutes each, and I did four or five of those. My average score was 49/50.

Thinking, braking and stopping distances

Questions in the test can involve thinking distances, i.e. the distance travelled by the car between a hazard occurring and you processing it and braking, braking distances, i.e. the distance travelled by a car between you applying pressure to the brake pedal and the car coming to a halt, or both of those combined – the stopping distance. For these distances, you are supposed to just learn the numbers for speeds of 20 to 70 miles per hour. The Highway Code lists the following:

Speed (mph) Actual thinking distance (m) Actual stopping distance (m)
20 6 12
30 9 23
40 12 36
50 15 53
60 18 73
70 21 96

While I could have memorised these numbers with a bit of effort, I find it easier to remember formulae so I came up with some equations. For the thinking distance, simply multiply the speed by 0.3:

\(\text{thinking distance (m)} = 0.3 \times \text{speed (mph)}\)

This gives you the exact thinking distances. For total stopping distances, including thinking and braking, multiply the speed by 1.5, then divide by 100, then add 0.3, and multiply what you get by the speed again:

\(\text{total distance (m)} \approx \text{speed (mph)} \times \left(0.3 + \frac{1.5}{100} \times \text{speed (mph)} \right)\)

This equation is not exact, but the largest error it generates is 1.5 m, which allows you to easily select the correct answer from the multiple choice answers.

Speed (mph) Actual stopping distance (m) Calculated stopping distance (m) Error (%)
 20 12  12  0
 30 23  22.5  2.17
 40 36  36  0
 50 53  52.5  0.94
 60 73  72  1.4
 70 96  94.5  1.6

It may be possible that you are asked about just the thinking distances or braking distances. For the braking distance, calculate first the total stopping distance and subtract the thinking distance (\(0.3 \times \text{speed (mph)}\)), or leave out the \(0.3\) term in the total stopping distance equation.

Hazard perception

To get the hang of hazard perception, you not only need to train your eye to learn what potential hazards look like, but also learn how to play the game the software presents: if you want to score points, you can’t click on a hazard too late, nor too early. Each hazard has an unseen window in which you score from 5 to 0 points, with quicker response in principle giving you higher scores; however, some hazards have weirdly late scoring windows and I was often caught out when practising having clicked too early on something I knew to be a hazard, meaning I got 0 points. Ridiculous! I therefore changed my strategy to click multiple times on each hazard, in case the window had not yet started by the time of my first click. This is a risky strategy, because the software monitors your clicks and fails you if you are perceived to be gaming the system by clicking on everything that moves or in a pattern. I failed a few of the practice tests by clicking too much, which is frustrating because I was not trying to game the system but just click within the scoring window.

It turned out the app only contained 10 example hazard perception scenarios, which was woefully little. To give myself extra training on hazard perception, I paid for access to the DVSA’s online hazard perception course. It reports that it provides 100 examples, but some of these (around 40) are not actual hazard perception tests, but multiple choice questions. You are shown a similar scene to a hazard perception test, but some time in the video the scene freezes and you are asked something about what you should have observed (such as “Which lane should you be in” when an approaching motorway lane merges from the left). These are still useful, as it trains you to observe signs, but it’s not exactly what you will be tested on in the real thing.

The test

I easily passed the theory part of the test with 47/50 correct. You are not told exactly which questions you answered incorrectly, but you are given a list of their categories. I expected to get at least one wrong answer, as a question I had never seen before about an aspect I had never heard of came up – it was about “electronic stability control (ESC)”. It’s not even listed in the highway code’s index, so I’m a little miffed that it was even asked. Note that this is not ABS, which is in the highway code. One of the others I got wrong may have been about how to respond to an accident – it asked a question about what to do next after stopping and putting hazard lights on, and I think I selected “check for breathing” instead of “ensure an ambulance is called”. (To be fair, my way might have been better for the victim’s immediate health!)

I got 64/75 for my hazard perception test, but it should have been 69/75. Again, remember that you must play the DVSA’s silly game here. On one of my scenes, I clicked only 7 times in total, 5 of which were on the hazard (a car stopping abruptly to pick up a passenger in your lane), and afterwards I was told that I got 0 for that scene due to unacceptable clicking behaviour. I was totally miffed by that. In all of my practice, the only times I was given this message were when I clicked excessively on everything that moved, and at least 10 times in total. Nonetheless, 64/75 is well above the pass mark (44), and about the same as what I was scoring in my practice sessions.

One thing to note about the hazard perception test is that the videos appear to run about half as fast as the practice scenes. The car moves quite slowly during the test. I guess this is deliberate, so that you practice with the “hard” material and then ace the actual test. I managed to get 5/5 for 8 of the 14 clips, and 4/5 for 3 more because there was plenty of time to see potential hazards.

Summary

To do well, you need to study. It’s possible to pass without studying, with a bit of common sense, but some questions can stump you without practice, such as the stopping distances and the exact order in which to handle accidents. I recommend the app, and the online videos. If you’re studying for your own test, good luck!

Publishing my thesis in book form

Books, Gravitational Waves, Guides, Projects, Python, Science, TeX Leave a comment

This is a story and guide to publishing my thesis in book form. If you’re looking for the good stuff, feel free to skip to the instructions.

Last year I completed my PhD studying interferometry for the detection of gravitational waves. My thesis is now available in electronic form that anyone that wishes to read it can obtain, and the source code is on GitHub. A paper copy – following the standard fabric-bound, A4 format specified by the university court – is available courtesy of the university library: at the time of writing, my thesis is stored in the research annexe on 29 Saracen Street (interestingly, that’s not on campus, but near the Bowmore whisky bottling plant in the north of Glasgow).

The title page of my thesis. Pretty, eh?

One of the ways in which I procrastinated while writing it was to try to produce diagrams, charts and figures which were entirely made from vector graphics such that they could be scaled for display or print at any resolution without losing fidelity. This was unfortunately due to rather of an obsession I have with data integrity: why throw away information when you don’t have to? It might come in handy later. Information is lost in photographs when they are scaled up – for example in preparation for print. This is in general true of all raster graphics like the JPEGs, PNGs and GIFs used all over the web and in designs for print. Vector graphics typically look nicer when displayed in a PDF viewer at high screen resolution or printed with a high quality printer. There are downsides, and there is definitely an appropriate time and place for raster graphics, but not for most of the plots and diagrams in my thesis.

The upshot of all of this time spent making nice, scalable graphics was that it was possible to render the thesis in (theoretically) infinite fidelity for printing. My printed thesis graphics looked really nice in the A4 tome I submitted to the library, but the book design itself looked dated and boring. That’s what gave me the idea to investigate professional print services.

University of Glasgow approved book binding service from Downie Allison Downie in Partick, Glasgow. While the workmanship is good, the university-enforced standard design is boring. No thanks!

Publishing my book

My thesis is almost entirely made up of text, plots of data and diagrams. Almost all of the diagrams are vector graphics. The scant few raster graphics were of circuit board layouts (which could, in hindsight, also have been produced in vector format). I had no photographs, both because my work was mainly focused around simulations and designs for new, not-yet-built experiments, and because of my aforementioned dislike for raster graphics.

As I had written the thesis in LaTeX – a scripting language for documents – I realised it would be straightforward to resize my A4-formatted thesis into a nicer A5 size more suited for reading, and in the process lay it out the way I really wanted to instead of conforming to the boring old University of Glasgow approved format. If nothing else, I’d save on paper by decreasing the margins and text spacing to result in a book more like the ones you buy today. Don’t you hate it when you have to turn the page so frequently?

So, what follows below is a guide to how I published my thesis as a nice, custom book.

Step 0: find an online book printer

The obvious preliminary step. After a bit of searching I found InkyLittleFingers, who had good reviews, made everything at their own factory (in Gloucester) and offered what seemed like a pretty comprehensive binding service that could be arranged completely online. Importantly, they are based in the UK, which cuts the cost of postage, allows for efficient communication regarding specifications, and provides a VAT exemption due the rules in the UK regarding book tax.

One of the appealing parts of InkyLittleFingers’ offer was the level of passive support they provide in the form of guides on their website. They even have a YouTube channel with some video tutorials.

There is a comprehensive guide to designing the artwork for the book. “Artwork”, as I learned from their helpful glossary (under “Useful terms” on this page), is the term in the book binding industry for the content you want printed. That means both the case (i.e. the front and back covers and the spine) and the book block (the inside pages) – the whole thing.

Form on Inky’s website for creating hardback book orders.

The website is a little rough around the edges, and sometimes you might find yourself hitting a 404 or some other sort of error, but for the most part it works as intended. To order a book you first need to upload your artwork and ask for an automatic quote. As I wanted a hardback, I went to this quote form and filled in the details. You can guess the page count just to get a quote, but you need to know the exact page count before making a proper order. The page thickness you choose is combined with the number of pages you specify to calculate the spine thickness, which you then have to use to appropriately dimension your cover design. After some research of their FAQs, I chose the following options:

  • Quantity: 4. Enough for my two supervisors, my parents and myself. It seemed that ordering a fifth made the price higher than five times the cost per book when ordering four. Maybe they prefer to make batches of 4?
  • Size: 210x148mm A5.
  • Orientation: portrait (bound long edge).
  • Case (cover) type: custom full print colour. You can also choose a coloured cloth, but because I wanted to make a design for my cover I chose a printed form.
  • Corner protectors: no. These would look a bit silly on a full colour cover.
  • Ribbons: no. These are used for keeping your place in the book. They weren’t yet available when I ordered, but I may have taken them if had they been.
  • Headbands: no. These are little coloured ribbons that go above and below the paper block inside the cover. These would look weird alongside a glossy, printed cover.
  • Cover laminate: gloss. As noted in the guide, gloss cases are not the same as standard glossy photo paper. That stuff has a mirror-like sheen that looks pretty ugly on anything but colourful photographs. “Gloss” is the de-facto cover laminate for most new books, contrasting with “matt” which is the duller variety. This one is personal preference.
  • Hot foil cover: no. This is the gold or silver writing you put on standard theses. This only suits cloth covers.
  • Dust jacket: no. I decided it’s a bit pointless for a full colour cover to have a dust jacket: these are normally found covering cloth cases as a poor man’s full colour cover. It would only have been a few pounds more to order them, though, and you can always remove them if you don’t like them, so if you fancy it then go ahead.
  • Inner pages printed: mixture of colour & HiQ B&W. My thesis is mostly black text, but some pages have colourful graphics. If you don’t want a monochrome book, choose this, and you won’t regret the high quality black and white. When you select it, it asks for a count of the black and white and colour pages, so you’ll need to do that (just count the colour ones and subtract from the total to get black and white).
  • Paper type: 100gsm matt uncoated white. You can get thicker, thinner, shinier paper, but I went for standard and I’m very happy with it. If you’re publishing a chic novel you might want to go for cream.
  • Proofing type: PDF (online soft proof). This was the only option for me; it’s possible other options exist for higher order quantities. If you want a “real” proof to check everything looks good in print, you need to order a single copy with the same form. See the note on the PDF proof later.
  • Print, delivery, postcode service: tailor these to your requirements. I went for the (cheapest) defaults as I was not in a hurry.

My order ended up costing around £79 all-in, or just under £20 per copy. Considering that binding the single, “official” copy of my thesis for the library to the official university format cost £26 at one of the recommended companies, even having provided them with the printed pages to use, I consider that a pretty marvellous deal. That’s especially true given the creative control I was allowed.

Once you submit your order, the site works out how thick your book will be. Thicker books cost more to make and send, but more importantly you need the exact dimensions to use to design your case artwork in the event that you chose to make one yourself. The thickness of the spine is obviously determined by the number of pages and the weight of the paper, but also the type of cover you specify.

You are given an automatically generated note with your quote. In my case it said:

NOTES:
(1) Please make sure you read the online help explaining the differences between standard and high quality black and white printing.
(2) Spine width is approximately 21.8mm.
(3) Case artwork dimensions, height: 234mm, width: 345.8mm.
(4) Dust jacket (if ordered) artwork dimensions, overall height: 222mm, overall width: 491.8mm, front/rear cover width (each): 154mm, flap width: 77mm.
(5) If there is anything that you do not understand in this specification, please contact the helpdesk.
(6) Please make sure that you read the artwork preparation guide before placing your order.

So, my spine will be 21.8 mm. Given that I selected A5 paper, the case artwork needs to be equivalent to two A5 sheets next to each other, plus the spine, plus some extra bleed. For me, it was 345.8 × 234 mm (width × height). That budget breaks down as:

  • Two A5 pages: a single A5 page is 148 × 210 mm (width × height), so I need 2 × 148 mm in width, i.e. 296 × 210 mm
  • The spine: 21.8 mm, as defined above, added to the overall width, i.e. 317.8 × 210 mm
  • Extra 28 mm (width) and 24 mm (height) bleed, i.e. 345.8 × 234 mm. Note that you get grooves at the spine, where the glue that attaches to the inside paper ends, allowing for flexibility at the spine. This makes the actual A5 covers slightly wider. This adds to the extra width you need to account for.

The equations for the dimensions are, essentially:

  • Total height = book block height + 24 mm, so in my case the height is A5’s 210 mm + 24 mm = 234 mm.
  • Total width = 2 × book block width + spine width + 28 mm, so in my case the width is 2 × A5’s 148 mm + my spine’s width of 21.8 mm + 28 mm = 345.8 mm.

    The “hang over” of the cover over the book block. This is an extra 4 mm of material that helps to protect the pages of the book.

The extra 24 mm (height) and 28 mm (width) are required to allow the design to fit the cover. The budget for the extra width (28 mm) breaks down as:

  • 8 mm used to wrap the cover artwork around each edge of the book (16 mm total)
  • 6 mm used to allow the cover to “hand over” the paper, and to allow for extra material for the two “grooves” at the book’s spine (12 mm total)

The budget for the extra height (24 mm) breaks down as:

  • 8 mm used to wrap the printed design around the edges, as above (16 mm total)
  • 4 mm above and below the book block height to allow the cover to “hang over” the paper

Step 1: design the cover

If you choose to provide custom cover artwork like I did, you need to create a PDF of the correct size and upload it alongside your book block. Fortunately, Inky provide a video tutorial showing how to achieve this with the free, open source design software Scribus. Fantastic! It’s easy to set the dimensions in Scribus using guides, which help to define the area you need to provide artwork for, and show what will and will not be visible on the front, back and spine of the book. The video walks you through calculating the dimensions of each part of the cover (as I also listed above), so you’re finished with a grid of guides showing each area.

Guides for my A5 book in Scribus. In addition to the standard margins, I added guides to show the horizontal and vertical centre lines to assist with placing text and graphics.

Of course, the trickier bit is actually coming up with a cool cover design! For that, you’re on your own, except a note that you should design your cover with the back on the left, so that when folded over the book the text still reads the right way up.

Here’s what I came up with:

My thesis cover design.

I’m no design pro; I have the creativity of Status Quo when it comes to art. I included the usual stuff: the title and my name on the front cover and spine. I also added a generic picture related to my work*, prettified for the front cover with the removal of the labels, tweaking of axis limits and the use of a nice complementary colour wheel (it took a very long time to render, but that’s a story for another time). Tempted as I was, I decided against adding the thesis abstract to the back – this is not a novel. I managed to find a nice vector graphic of the university’s logo, so I added that to the front cover opposite my name. The background is a nice bold blue – my favourite colour. I’m quite happy with it!

*For anyone that is wondering, it shows a surface map of the power of the error signal from the second RF control sideband in the signal recycling cavity of ET-LF as the Schnupp asymmetry and exact cavity length are tuned (see Chapter 7).

Step 2: compile the pages in A5 format

I spent over half a year writing my thesis, formatting it for A4 paper as I went along. LaTeX, however, takes most of the control over page layout away from the user and uses algorithms to decide where to place text, figures and everything else. This generally works quite well, but occasionally has issues, especially when your text is figure-heavy. I found that changing the paper size from A4 to A5 led to some unintended blank pages at the beginning or end of chapters and sections, and small figures hogging their own page. Some of these issues were because of blank pages I had added when it was formatted for A4, which were no longer appropriate for A5, so let’s not blame LaTeX too much. Some of them, though, necessitated moving the figure definition in the source code to force LaTeX to alter its positioning behaviour, which is a bit more time consuming.

I had to go through the whole document and fix these as they occurred. To prevent wasting time in a recursive doom fixing one part and breaking another, I did this sequentially from the start. Whatever you do, don’t just fix layout issues at random as you find them – you’ll no doubt break the layout later in the document.

Naturally, going to a smaller paper size while maintaining the margin and font sizes leads to extra pages. Going from A4 to A5 in my case, the page count went from 244 to 271 – not as much a jump as I expected. As I did not alter the text size, the relatively small increase in page count is probably due to the page-spanning figures taking up less physical real estate where otherwise text could go. Put another way, the ratio of the graphic dimensions to the text dimensions was reduced.

Step 3: upload the sources and proof the full book

The PDF proof of my book generated by InkyLittleFingers. The cover is naturally larger than the rest of the book, as it has to wrap around the two cover pages and spine. The proof contains bleed marks, where the paper will be cropped after printing (to allow graphics to “bleed” over the sides of the paper, so that when it is cut the graphic is flush with the edge).

After creating the order (and paying!) you need to upload your work. The book block (the pages of the book) is uploaded as a separate document to the case artwork. In my case, the book block was a PDF compiled with LaTeX, and the cover was as PDF exported from Scribus. I thoroughly examined these documents before uploading, but the main check to make is of the automatic proof that InkyLittleFingers generate for you after uploading. It contains the case artwork and book block rendered as you will see them once printed: with the colours set, the pages appropriately sized, the fonts chosen and the images appropriately cropped. Check this proof thoroughly – this is exactly what you’ll see, minus the marked bleeds, when your printed books arrive.

In my case, the automatic proof contained a few small artefacts on some diagrams. I phoned Inky to verify where this was coming from, as I was not sure if this was just an effect of my PDF viewer. It turns out that they were real, and somehow produced by their systems: the book block I uploaded did not contain these artefacts, but only the proof generated by them. In hindsight, though, I doubt it is a bug with their own software, but rather some bug with the way that the Adobe PDF format handles embedded graphics. The worst artefact I found was in a graphic that was itself a PDF embedded within the PDF book block. This was originally generated with Inkscape using a library of SVG parts themselves made with Adobe Illustrator. Somewhere along this myriad format conversion process something happened that led to this artefact in the final version.

One of the minor artefacts in a vector graphic within the book block. The beam splitter – designated “M6” – should not have pixellated edges. Indeed, the source file I generated the graphic from does not have this artefact – it’s only when I got the proof from InkyLittleFingers that it appeared.

I never found out the culprit for the artefacts, but I did manage to get rid of almost all of them by playing with the SVG sources in each case. Flattening images from many layers to one, and merging overlapping segments were two strategies that seemed to work.

The nice thing about the Inky interface is that you can continually upload new book blocks and case artworks and generate new proofs until you are happy. I did this at least three times, gradually getting rid of all of the artefacts, until I was happy. In the end I left the worst artefact in, as I had managed to make it better than to begin with and it was very, very minor and hardly noticeable. My last hope was that it would not appear in the final print despite the proof.

Once you sign off on the final proof, Inky get to work making the book. Now the waiting begins!

 

Results

After about a week, I got the books in the post. Accompanying the delivery was a note saying that I should wait at least 12 hours before opening the pages fully; they are apparently immediately dispatched after production so there’s a chance the glue might not have fully set if the delivery was unusually quick. I was so eager to look at my creation that I did open one copy – the one I intended to keep myself – and it seems in hindsight that it didn’t do any damage.

The black and white pages were printed to very high standard.

The quality is superb. The whole book seems tough and durable. The book as a whole feels sturdy. The glue is neatly applied to the paper block, which is firmly glued to the inside of the case, over the edges of the cover which are tucked properly in place. It easily sits free-standing on its narrow side, able to sit on a shelf and support its own weight. The binding of paper and cover has been achieved flawlessly, and the book block sits exactly in the middle of the cover. The paper quality is great. The print fidelity on the black and white and colour pages is equally excellent. I am extremely pleased and impressed with the results, especially given the price I was charged.

The artefact mentioned above was, while present in the final print, hardly noticeable (see photo below). It was more than offset by the superb print quality for the diagrams and images (and indeed text!).

Highly recommended!

What’s the difference between resistor types?

Electronics, Guides 3 comments

Assorted resistors. Which type is useful when?
(Source: Wikimedia Commons)

Anyone who’s ever had to buy resistors from sites like RS, Farnell or DigiKey will know that there are myriad types to choose from: “carbon composite”, “wire-wound”, “carbon film”, “metal film”… what do these mean, and in which applications are they useful and harmful?

Carbon composite resistors

Advantages:

  • Cheap
  • Withstand high current transients

Disadvantages:

  • High temperature coefficient
  • Resistance drifts with time when used with high current
  • High tolerance

These are made from conductive carbon dust packed together with clay, with the ratio determining the resistance. They are generally available only for low power applications, but can withstand short high current bursts. The material properties are such that it is difficult for manufacturers to precisely define the resistance, so tolerances are generally poor. The temperature coefficients of carbon composite resistors are also generally poor (typically 1000 ppm/°C or greater), meaning that these resistors are useful mainly for hobby and general purpose circuits.

Carbon composite resistor. These have mostly been superseded by film resistors.
(Source: Wikipedia)

Carbon composite resistors have been largely superseded by film resistors, of which there are two main types: carbon film and metal film.

Film resistors

Advantages:

  • Low temperature coefficient
  • Low tolerance
  • Available with very high resistances

Disadvantages:

  • Tenancy to be damaged by current overloads
  • Generally only available in low power ratings

Film resistors are the bread-and-butter of most designs. They are made with a mix of ceramic and metal (“cermet”) or metal oxide deposited in a film over an insulating substrate. The resistance is determined by the deposited thickness of the film, and very high resistances can be achieved in small packages with this technique. Precise tolerances are reached during manufacturing by using a laser to etch a spiral around the film to create a longer effective path around the outside of the substrate.

Carbon film resistor with exposed film. Note the spiral etched into the film at the factory to trim to a precise resistance.
(Source: Shaddack, via Wikimedia Commons)

Film resistors have very good temperature coefficients (typically 50-1000 ppm/°C, with carbon and metal types tending to have opposite signs), providing stability for precision circuit applications. Due to the deposition of layers of metal without a clay matrix, these resistors are susceptible to damage from high currents and therefore are not generally available in high power packages.

Wire-wound resistors

Advantages:

  • Suitable for high current applications

Disadvantages:

  • High series inductance
  • Only available with relatively small resistances

Wire-wound resistors are, as the name suggests, made by tightly winding conductive wire around an insulating substrate. The thickness (“gauge”) of the wire and the number of turns made determine the resistance, but this creates a volume constraint and so these resistors are typically only available with small (less than 100 kΩ) resistances.

Available tolerances are typically fine for most applications, but precision laser trimming is not an option unlike with film resistors and so tolerances are generally not lower than 1%.

Wire-wound resistor. Note the metal packaging and ability to mount the resistor to a panel for good heat sinking.
(Source: Olli Niemitalo via Wikimedia Commons)

Given the relatively thick (thin wire is still much thicker than film) conductors used in the construction, these resistors are able to carry large currents and are available in very high power packages – 50 W or more – making them ideal as heating elements (intentional or otherwise). The downside, however, is that the wound wire creates a coil which induces an electric field in the middle, creating a phase shift in signals particularly at high frequencies. Some wire-wound resistors combat this effect by counter-winding the wire to cancel the inductance, but it is difficult to remove the inductance entirely. Use of wire-wound resistors in precision circuits – which typically don’t require high power anyway – is discouraged in favour of film resistors.

References

Fixing discarded messages in Dovecot sieve filters due to incorrect message IDs

Guides, Software Leave a comment

I like StackExchange newsletters, which send you weekly emails for each site you sign up for containing top rated discussions. I subscribe to Electronics, Physics and, for fun, Aviation.

In February, I changed the email address associated with my StackExchange account to use my own server. After that, I noticed I no longer received any StackExchange newsletters! For a while, I figured there was something wrong with my email server that caused StackExchange not to deliver the mail. I eventually found time to debug the server last week, and discovered, incidentally, that the PTR record for my server (I run my own DNS server), which is used to map IP addresses to hostnames, pointed to attackllama.com and not alpaca.attackllama.com as it should*. After fixing that, I thought I would start to receive emails from StackExchange again. After waiting for a week, though, it was clear that this was not the problem.

I finally decided to do something about it, and forced StackExchange to send me an email by clicking the “I forgot my password” button. I then looked in my mail server’s logs:

Jun 16 09:30:41 alpaca postfix/smtpd[9251]: connect from mx-out.stackexchange.com[198.252.206.125]
Jun 16 09:30:42 alpaca postfix/smtpd[9251]: Anonymous TLS connection established from mx-out.stackexchange.com[198.252.206.125]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Jun 16 09:30:42 alpaca postfix/smtpd[9251]: D943F18400CE: client=mx-out.stackexchange.com[198.252.206.125]
Jun 16 09:30:42 alpaca postfix/cleanup[9255]: D943F18400CE: message-id=<2.a00e7ff480d1dd548444@NY-WEB02>
Jun 16 09:30:43 alpaca opendkim[1413]: D943F18400CE: mx-out.stackexchange.com [198.252.206.125] not internal
Jun 16 09:30:43 alpaca opendkim[1413]: D943F18400CE: not authenticated
Jun 16 09:30:43 alpaca opendkim[1413]: D943F18400CE: DKIM verification successful
Jun 16 09:30:43 alpaca opendkim[1413]: D943F18400CE: s=dk d=stackoverflow.com SSL
Jun 16 09:30:43 alpaca postfix/qmgr[25776]: D943F18400CE: from=<do-not-reply@stackoverflow.com>, size=14102, nrcpt=1 (queue active)
Jun 16 09:30:43 alpaca dovecot: lda(stackexchange@attackllama.com): sieve: msgid=<2.a00e7ff480d1dd548444@NY-WEB02>: marked message to be discarded if not explicitly delivered (discard action)
Jun 16 09:30:43 alpaca postfix/qmgr[25776]: D943F18400CE: removed
Jun 16 09:30:43 alpaca postfix/smtpd[9251]: disconnect from mx-out.stackexchange.com[198.252.206.125] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7

The first 3 lines show the successful, secure connection to my mail server from StackExchange – good. The next line shows the message ID received, before any filtering. The next four check that StackExchange’s email server has DKIM records to prevent some forms of email spoofing. The next line shows the email entering into the processing queue, then the next shows that a sieve (filter) is discarding it because it was flagged by a filter. The next line shows the message being removed, and the last line shows the connection with StackExchange being closed.

So, it turns out one of my sieves was discarding the messages. I opened up my sieve file and found the following likely candidate:

# Trash messages with improperly formed message IDs
if not header :regex "message-id" ".*@.*\\." {
 discard;
 stop;
}

I took this from Ars Technica’s guide on setting up a mail server. It’s supposed to fight spam, by silently discarding badly formed message IDs that presumably only spammers would create. It turns out that simply checking the message ID with a regular expression like .*@.*\\. is insufficient. RFC guidelines on message ID validity are complex, leading to the need for horrendous regular expressions like ((([a-zA-Z0-9!#$%&'*+/=?^_`{|}~-]+(\.[a-zA-Z0-9!#$%&'*+/=?^_`{|}~-]+)*)|("(([\x01-\x08\x0B\x0C\x0E-\x1F\x7F]|[\x21\x23-\x5B\x5D-\x7E])|(\\[\x01-\x09\x0B\x0C\x0E-\x7F]))*"))@(([a-zA-Z0-9!#$%&'*+/=?^_`{|}~-]+(\.[a-zA-Z0-9!#$%&'*+/=?^_`{|}~-]+)*)|(\[(([\x01-\x08\x0B\x0C\x0E-\x1F\x7F]|[\x21-\x5A\x5E-\x7E])|(\\[\x01-\x09\x0B\x0C\x0E-\x7F]))*\]))) to be more correct, and even then, apparently not strictly correct.

Checking old emails from StackExchange, I found the message IDs lacked a trailing full stop (.) which caused the regular expression in the filter from Ars Technica to fail. However, as far as I can tell, having no full stop at the end of the filter is perfectly reasonable. Perhaps the filter should have had an asterisk after the full stop to ensure that there is a full stop somewhere after the @. I don’t know.

It seems to me that trying to validate the message ID in a sieve to detect spam is not easy, and should not be attempted with a regular expression, so I just disabled the sieve by commenting it out. After that, boom! I was able to receive emails from StackExchange again. I guess I’ll have to wait and see if the amount of spam I receive goes up (from zero, currently!) to check if this filter was having a positive effect before.

Isn’t running your own mail server fun!?

*For DNS nerds: servers should not be given a hostname that is the same as the full domain name that points to it. I initially set up my server with a hostname attackllama.com, which essentially meant that the server should be the provider of all services on *.attackllama.com. While that is in fact the case for me, that’s not how domains should be used: the mail server providing alpaca.attackllama.com can in principle be hosted on a different server, without the need to ever talk to the main attackllama.com server – that’s the polymorphic magic of DNS. So, I fixed this by simply changing the hostname of my server to alpaca.attackllama.com, and correctly pointing the IP address via the DNS PTR record to that. That means the IP address points to a specific server on the attackllama.com domain, as it should. And, since some email providers check the reverse PTR record to make sure the IP address points to the mail server, this fixes some issues with mail not being sent by strict sending servers.

Spectra Microvision Plus install guide for modern operating systems

Electronics, Guides, Science, Software 12 comments

The Spectra Microvision Plus is a nice residual gas analyser (RGA), allowing one to examine the elements and molecules present within a vacuum system. In the speed meter group at the Institute for Gravitational Research, we use it to test our vacuum system for cleanliness, leaks and the presence of unwanted elements before we place our delicate optical surfaces inside and shine high power lasers upon them.

Spectra Microvision Plus. This box is able to analyse the residual gas profile of a vacuum system, but it’s tricky to install the aged software on modern machines.

Despite being around 15-20 years old, this hardware is solid and still works nicely. The problem is that the software used to control the box is a similar age, and has well and truly suffered from bit rot. In fact, it came with the unit in the form of three floppy disks. These were old even when I was growing up, and CDs and later DVDs became the standard distribution channel for software. Adding insult to injury, the software is only designed to work on Windows 3.1 (!), 95, 98 or XP, meaning any modern computer is going to struggle to run it. Finally, it demands a physical RS-232 connection to the machine, since it harks from the days when USB was but a teenager with braces and oily hair, and Apple and IBM were still preventing one standard from ruling them all by pushing FireWire.

Incidentally, if you’re looking to download the Spectra RGA For Windows software that is used with the Spectra Microvision Plus, it’s unfortunately not available directly from the manufacturer. They apparently insist on users paying $2k for newer hardware. If you want the software, though, just send me an email – I’ve made images of the floppy disks for safe keeping.

Spectra RGA For Windows software running on VirtualBox on a Linux host

I managed to get the software to work by using VirtualBox to emulate Windows XP on a Linux host. I run Ubuntu on my machine, so VirtualBox is easily installable via the package manager. Fortunately, my university still has XP campus licences kicking around, so I fired up an ISO and installed XP as a virtual machine within VirtualBox. If you need to know how to install XP on VirtualBox, just google it. If you run Windows or Mac OSX, don’t dismay – VirtualBox is also available on those platforms, so this will still work provided you can obtain a Windows XP disk or image.

Interface to RS-232

Now comes the slightly more tricky part. As stated earlier, the RGA uses RS-232 for communication between the hardware and the RGA For Windows software, and modern computers don’t come with RS-232 ports. However, given that USB is, eponymously, universal, you can buy an adapter which emulates an RS-232 serial interface over USB for a few pounds:

RS-232 to USB adapter. Could it be any simpler?

I got mine from StarTech.com, but you’ll find them anywhere with a quick search. Ubuntu 16.04, but probably all recent Linux kernels, supports this adapter out of the box, giving you a virtual RS-232 port that you can hand over to the virtual machine.

If you use Ubuntu or a Debian derivative, make sure your user account is part of the “dialout” group (run usermod -a -G dialout [your-username] as root, then log out and back in again), otherwise you won’t be able to read from and write to the serial adapter.

Once you’ve got an RS-232 USB adapter, plug it in and fire up VirtualBox after having installed Windows XP. Before starting the operating system, however, you need to tell VirtualBox to pipe the RS-232 connection through to the virtual Windows XP system. In the settings for the Windows XP virtual machine, go to “Serial Ports” and enable Port 1. Then choose “COM1” for the port number and “Host Device” for the port mode, assuming that your kernel has successfully identified the adapter (run dmesg in a terminal and look at the kernel messages to check). Put /dev/ttyUSB0 as the Path/Address, but check that this is actually the path to the serial adapter you plugged in (again, use dmesg or similar to find out). Once this is done, you should be able to fire up Windows XP and look in the device manager to see a new COM port present.

Installing, configuring and running the RGA For Windows software

Given that the software was distributed on floppy disks, I first had to get a USB floppy disk drive and create images for each of the three disks (using dd in the terminal – again, Google is your friend). VirtualBox can mount these images in turn, allowing your virtual Windows XP machine to think it actually has a real floppy disk drive attached. I inserted the first “disk”, opened up “A:” in Windows and started “INSTALL.EXE”. It is very simple: after clicking next a few times and inserting the second and third virtual floppy disks when asked, it was installed.

To get the software to run, the first thing to do is to run the “Configure” program from the start menu. For me, it found the only COM port configured on the computer (COM1) and was happy. After that, fire up the “RGA” software with the RS-232 connector plugged in to the Spectra Microvision Plus unit. You should be presented with the default screen – hurrah! – and after that, you can take some measurements. Here is a screenshot of the RGA For Windows software running on a Windows XP virtual machine, itself running on Kubuntu 16.04:

The Spectra RGA For Windows software running on a virtual Windows XP machine, running on an Ubuntu host.

There we go! A virtual machine that is able to talk to the Spectra Microvision Plus RGA, future proof and fit for modern operating systems. Hopefully this allows this old but solid hardware to continue to serve a purpose in modern research labs. Let me know if you run into any issues when getting the software to work – I’d rather not let Spectra hustle thousands of dollars out of customers just by refusing to update or release 20 year old software…

Online Labbooks for Scientific Research

Guides, PHP, Science Leave a comment

Over at my workplace, the Institute of Gravitational Research at the University of Glasgow, we undertake lots of experimental research (primary physics) and produce lots of data. For the most part, this data is saved on individuals’ computers (centrally backed up but only accessible to that user), a shared hard drive on the (Windows) network, or written in paper labbooks (though these do have serial numbers for archival and accountability).

In my group, Stefan Hild’s European Research Council funded Speed-Meter experiment, we were slightly ahead of the curve. My interferometry colleagues and I for many years used a simple SVN-based labbook that involved placing a text file called “Notes.txt” and any images associated with the post into a directory with the current date. This was then parsed by a simple web interface to provide an online labbook that could be viewed by anyone on the network with the URL. It worked quite well, and was certainly better than a paper-based labbook in most regards, but it lacked a few key features:

  • Ability to link to previous labbook posts directly (via URIs/URLs): we instead had to quote the labbook title so that the user, if they were interested, could find the referenced labbook post.
  • Users: we used a single login for the SVN to add posts, all which which were marked as having been uploaded by “jifadmin”. It was only possible to tell who made a post by context.
  • Categories: the ability to group posts together in logical sets.
  • Versioning: otherwise known as “keeping track of edits”. We kind of hacked this to work with the use of the SVN, but the ability to roll-back or view a diff of two files involved the use of a separate system (command line or GUI).
  • Image/media upload: display of in-line images in posts, where they would logically go. Instead, our images always appeared at the bottom of the post and we would have to reference them by file name in the appropriate part of the post.
  • Centralised user credentials: this was part of a wider issue in the group which was gradually getting better. We would require a login for the SVN to add content to the labbook, but also a different login for other services like email, WiFi and the shared calendars.

We put up with this system for a number of years, but eventually when we started a major project we ditched it and turned to some software called ‘OSLogbook’.

Open Source Logbook

Imaginatively named ‘OSLogbook‘, this software is used in the Virgo and LIGO collaborations, which are scientific organisations that we are members of and frequently collaborate with. This brought a number of new features to the table:

  • LDAP login: centralised login system using centralised credentials, via the LDAP protocol.
  • Categories and tasks: each category can have subtasks, and each post can be assigned a single task within a category.
  • Authors: each user has their own login and can post their own content that is marked as their own.
  • HTML posts: a basic WYSIWYG editor allowed us to add headings, tables and coloured text to our entries.

We used this for a couple of years and found it useful, and a vast improvement over the SVN-based labbook. However, it still annoyed us in a few places and had some quirks which were not intuitive. For instance, posts could only be assigned to one category. In our daily work we often conduct lab work or experiments that involve multiple aspects of the same experiment, such as the organisation of power supplies to various corners of the lab or temperature and humidity changes affecting sensitive measurements.

The labbook also allowed us to assign multiple authors to each post, but in the form of a text field. That sounds useful at first, but it’s actually a pretty bad way of implementing this feature. If a person makes a labbook entry and puts their name before another collaborator, and the next day the other collaborator puts their name before the first collaborator, that author list is then a different set of characters despite containing the same names! Searching for posts by a certain author was therefore difficult because you don’t know in which order their name appear. Furthermore, it was left to the whim of the author of each post to correctly specify the names of their collaborators, meaning that occasionally a nickname or mis-spelling found its way into this field.

Perhaps the weirdest feature was that comments made on each labbook entry actually showed up as their own labbook entry. The list of labbook posts therefore contained in chronological order the comments made on each post as well as their respective posts – meaning the homepage was a cluttered mess of crossing conversations.

Not good.

The sum total of a number of small niggling issues with OSLogbook led me to investigate an alternative. We wanted our alternative to have the following features, in no particular order of importance:

  • Centralised login system: using our already-established LDAP system to authenticate users.
  • Permission system: we wanted our summer students and undergraduates to contribute to our labbook, but not be able to mischievously edit or delete other users’ posts.
  • Comments: labbook entries in an experiment with dozens of members more often than not require organised discussion. A comments system allows each user to contribute their thoughts in a coherent manner, without making a new labbook entry as a follow-up.
  • Media upload: beyond images, but also for zip files, analysis and plotting scripts, PDF documents, etc.
  • Rich text editing: we wanted to be able to add tables of data, links, coloured text, lists, special characters, centrally-aligned text… every little detail. Don’t constrain the user in making a post as clear as possible.
  • Multiple authors: the ability to assign a post to multiple authors, and allow any of those authors to further modify the content, and to have all collaborative posts an author is part of show up in a search of that author’s posts.
  • Reference pages: the ability to create static pages with reference information that does not fit in a labbook post. For instance, a list of procedures for ordering equipment or links to useful intranet/internet pages.
  • Ability to add new features: ideally, rather than a black-box monolithic software like OSLogbook, we wanted to be able to modify the software to fit our needs. There are a few programmers in the group, so with a nice API it would be able to tailor the labbook the way we want it.
  • Looks nice: people don’t read ugly websites unless they really have to. Let’s not make it difficult!

The obvious choice: WordPress!

Almost all of these features can be found on full-blown news sites, blogs and other content mills. It turned out that the popular blog software WordPress fit the bill… with the help of some plugins. Its vibrant community and extensive theme and plugin support made it a no-brainer. However, it lacked a few of the desired features that would not necessarily be useful on full-blown sites, but useful to a scientific organisation, like multiple authors and LDAP support. Before taking the plunge and moving my colleagues over to the system, I had to research whether there were plugins that could help us meet our list of requirements. In the end, the huge WordPress community delivered. There were plugins or native WordPress settings for all of our desired features!

Implementing Desirable Features in WordPress

This section is a guide to the plugins required to implement cool features, listed below, in our labbook software. The same plugins should work for anyone wishing to build a scientific labbook.

Multiple authors

As described above, we wanted to be able to assign multiple authors to individual posts, and allow these posts to show up in searches of any of the individual authors of that post. This can be achieved with the Co-Authors Plus plugin – and it does exactly what we want! After a single author finishes writing the post, they can start typing the names of their collaborators. As they type, other registered users with names matching the text spring up and can be assigned authorship. If a user is named as an author on a post, they can then edit it. This lets users create and edit any post they are named as an author on, but not necessarily edit other posts should you not wish them to.

Mathematical markup in posts

As a scientific group, we frequently share mathematical equations with one another. The plugin MathJax-LaTeX lets us type LaTeX equations in our posts and displays them in-line. It also utilises MathJax rendering, which displays in modern browsers nicely and allows the corresponding LaTeX code to be copied-and-pasted.

Email alerts for new posts and comments

We wanted to have the option of receiving an email alert when a new post or comment was made. Somewhat surprisingly, this is not a standard feature of WordPress. A Google search will probably point you towards JetPack, but this plugin is too bloated for what we require – it’s focused on commercial sites and making money, not a simple task like notifying registered users via email.

Email Alerts

Some users wanted alerts via email when new posts and comments were made on the site. This was accomplished by using the already built-in feature of WordPress, RSS (“Rich Site Summary” or “Really Simple Syndication”), which allows an external feed-reading service to download a copy of the recent posts on the site and send a digest to a user’s email address. Perfect, right? Not quite. Our labbook was to be private – scientifically sensitive measurements were under discussion – so external feed readers would not be able to access the RSS feeds by default. This hurdle was overcome by the plugin Private Feed Keys, which allows users to create a special URL which allows external readers to download RSS feeds. This doesn’t require a login, but each special URL is unique to each user and can be given to a trusted feed reading service (most of us use Blogtrottr).

 

With our plugins sorted, we still had to do something about the old labbook content. Since we had used our old labbook for two years, we had plenty of content already on it that we would not want to lose. We couldn’t just delete it, so I had to investigate a way to transfer the old content across.

Importing posts from old labbook

The number of posts was not so high (roughly 250, including comments) that it was beyond the scope of hacking together an import script. WordPress also has a nice API to allow a PHP script to directly create posts in the WordPress database – so I was able to create a script to extract the old posts from the OSLogbook database and inject them in to the new labbook. Comments were a little trickier, as were image and file attachments – but I managed to do it in the end with a mix of crafty PHP scripts and some elbow grease!

Styling the new labbook

The last job was to make the labbook look great. As I said earlier, people don’t like to use nasty looking sites with bad user interfaces. Luckily, WordPress is of course designed for ease-of-use, so we already had a great default theme. To allow the site to operate more like its intended labbook and less like a blog, we adopted the theme Simple Life and modified it slightly for our particular purposes. This led us to create a child theme (see it on GitHub) and a custom plugin or two to provide specific functionality that we wanted:

  • A recent edits widget to show any posts that were recently modified, to keep track of changes across all content (GitHub page)
  • A widget containing a list of comments and links to recent SVN commits (GitHub page)

Our awesome labbook

This is our labbook:

Our Speedmeter labbook software in use.

Our Speedmeter labbook software in use.

Pretty cool, huh?

Since we made the labbook, it has garnered the attention of other colleagues in our group who have since adopted similar blogs. The IT manager simply uses WordPress’s network administration tool to create a new site in seconds and clicks a few buttons to enable the plugins we use.

I hope this guide will be of use to the wider academic community and the science industry. You don’t have to suffer with poor labbooks!