Parsing JSON

For reasons I won't go into, I had need to write a JSON parser recently - yea, I genuinely couldn't just use Doug Crockford's json2.js. It was my first attempt at writing a parser and was an awesome learning experience. I was lucky that I had some pretty smart people on hand to help with the methodology for doing this. The process is relatively simple once you know how, but I will admit it took me a while to get my head round how to approach the problem.

The task was to write something that would take a JSON string and parse it back into a JSON object. The string might contain n depth of nested types (for example an array of objects with each object containing other objects or arrays). I initially expected to have to work out what the first tier primitive types were, then recursively pass in their contents to the parsing functions. So, I was somewhat surprised to find that the method suggested to me was simply to run through the string character by character, from left to right, keeping track of my position within the string, recursively calling the parse function each time I found a symbol which denoted a new primitive.

Functions

I wrote a few helper functions:

• ignoreTrash - which moved past trash characters in the space between primitives, such as spaces, commas and semi-colons
• MoveToNextCharacter - which moved the string position counter
• amEscaping - which checked for backslashed quotes

Ok, so that list was shorter than I recalled. The parse function itself was really just a glorified if statement, looking for the first character in the string - if it is quotation marks, call parseString; if square bracket, call parseArray and so on.

The parse primitive functions would create a new object of their type, then they contained while loops to move through the string until they found their corresponding terminator symbol (usually a regular expression). If during the parsing they found another starter character, they simply called the main parse function again.

It all sounds pretty simple and the nice thing is that the code is really readable, but at the time, it took me about a day of thinking hard, to get my head round the concept. Still it was time well spent, as I think I learned quite a lot during the process. Unfortunately, one of the things I learned here is that the browser implementation of the javascript regular expression engine is a real let down on most browsers. The language seems to support groupings using parantheses, allowing lookahead and lookbehind, but I certainly couldn't get them to work.

XULRunner - an odyssey

When I first discovered that we were using XULRunner as part of our app. I have to admit to being quite excited. I thus jumped at the chance to work on that part of the project when the opportunity arose, a chance to play with some cool technology and learn a thing or too. The project sounded pretty interesting, create a small interface in XUL and add some functionality to it. Then implement some of browsers accessibility features and open them to the web page through an API.

XUL

XUL itself is fairly nice and the MDN documentation is readable enough to make initial progress rapid enough to be fun. However, I was a bit disappointed that CSS support is not complete, in particular z-index is not supported, which immediately removes you from your confort zone when trying to make your interface cool. There are two entirely reasonable solutions to these problems, the easiest is simply to abandon ideas of web 2.0 interfaces and go with what XUL provides natively, such as overlays. The other is to add html elements into the mix. Clearly, these must be namespaced to differentiate them from XUL elements, as outlined in MDN

So far so good, XUL is pretty simple and the documentation is helpful, lets move on to adding the functionality... This is where, unless you are doing something pretty trivial, problems can occur. It's not that the way XULRunner works is difficult, or indeed illogical, it's that the documentation is sparse, confusing, confused or a heady mixture of all 3. I ended up coding initially by trial and error, which is a big problem because de-bugging straight out of the box in XULRunner is painful. After a bit of searching, I found a great resource for setting up a reasonable debugging environment.

The type attribute

It turns out that when opening an API from the chrome to the browser, the type attribute is very important. It seems a little odd to me that the default setting for type is "chrome" which carries a security warning. It also seemed odd to me that when trying to access the API with this incorrect type, there is no error, the functionality simply appears buggy, for example, when accessing the full zoom property the page zooms, but only partially requiring a screen redraw to display correctly. This lead me to try and fix the apparent buggy behaviour, rather than realise there was a security issue.

However, once I did realise there was a security issue, things fell into place and the methodology for creating an API seems sane, logical and more importantly bug free. However, I will freely admit that there was much wailing and gnashing of teeth prior to this from myself and the others I dragged in to help figure out what was going wrong. To give an idea of how hard won this knowledge was, we got most of our information from downloading plugins such as Venkman and Firebug and just trawling through the source code to see how they work. I even have a copy of the Firefox source code checked out... N.B. I was impatient to get the download, but I still feel Mercurial was pretty slow.

Progress Listeners

Once the type was set, we wanted to attach an object to the browser window which would allow us to call functions in the Chrome javascript. This turned out to be a bit more complex than I had hoped, as there are timing issues; what chrome event corresponds to when the window of the browser will exist and can have objects attached. For this, I pretty much used the code from the Progress Listeners MDN page. OnStateChange was the only event needed and when aFlag and STATE_START were both true we knew the browser window existed for the first time. As stated in that documentation, aWebProgress.DOMWindow gave me access to the browser window object and we were but one hurdle away from glory :)

Wrapped JS Object

I had missed another security hurdle and somehow skipped this section in the documentation. To make my object available to the browser, I needed to use wrappedJSObject, fortunately, an astute colleague remembered reading about this somewhere during the early days of development.

The API was now available to the browser! Using aWebProgress.DOMWindow.wrappedJSObject.myAPI = { ... } within the chrome and simply accessing myAPI within the browser, the work was complete and not a moment too soon!

Looking back

It all seems pretty straightforward looking back and the documentation all seems in place. However, what I think is missing is the continuity between the two. It's pretty difficult to find a page that talks about all of the pieces of the puzzle mentioned here in one place. Still, it was good practice solving problems by reading the source code of other plugins.

Kindle and reading

As I had a 3 month contract in Cambridge, I had little time to post, but lots of time to read. I got a kindle for Christmas, which I will admit to absolutely adoring. I've read quite a lot of the free kindle books and revisited many old favourites. I spent a lovely amount of time reading and re-reading The Raven, by Edgar Allen Poe, I also read Machiavelli's The Prince, which was an interesting read. Travelling always makes for lots of reading and over the past 3 months, I have been averaging a book a week and have been really lucky in reading quite a few belters, so I though a post might be in order.

Rodger Penrose - Shadow's of the mind

A very hard read, I found this book exceptionally challenging, but a revelation once I grasped the concepts. The book discusses how we think and the actual process of conciousness and the concept of understanding. It challenges the assumption that the human brain processes information solely in a computational manner - which has an enormous impact on the future of A.I.. I admit to already being sympathetic to the view that true A.I. is a much more complex task than people seem to think and that in fact we are a very long way away from machines that can understand the problems they solve, rather than simply perform pre defined instructions at increasing rapid rates.

This is one of those books that I would love to recommend unreservedly, but there is an awful lot (to my mind...) of maths and logic, that make it a challenging book, whose reward is only worth it if you have a definite interest in the field. If you do, this is the guy to be reading.

Philip K. Dick

I have been a big fan of Philip K. Dick for a long time. His books are often challenging, but never without a definite point to make on either the human condition or the nature of reality. I have never read a book of his that I wouldn't recommend, so I am slowly working my way through his entire works. During the last 3 months I have read Ubik, Do androids dream of electric sheep, The man in the high castle, Flow my tears, said the police man. Do androids dream of electric sheep (the book from which Blade Runner was made) stands out among the books... It differs from the film greatly, the book is a discussion about empathy, in which the androids are simply held up as humans without empathy. I found it fascinating and inspiring of many discussions with friends.

Turgenev - Short Stories

I have always had a very soft spot for Russian literature and music. Crime and Punishment being my favourite book as a teenager. War and Peace also features high on my list of favourite reads. The settings of the stories (particularly His first love) remind me of Dostoevsky's The Fool and the romantic nature and depictions of passion are just what you would expect. Well written and very touching, I was especially moved by Mumu as it involves a dog! A highly recommended read.

H. Ryder Haggard - King Solomon's Mines

These stories come from a time when the British "stiff upper lip" was a thing to be feared. They are proper "Boy's own" type stories of manly men, with adventuring to do and no time for lady folk. Well written and utterly engaging, they reveal both the good and bad sides of a time gone past. It's hard to get behind a hero who is an ivory hunter, but it's written with such an innocent view and such good intentions, that it gradually wears down your reserves.

Robert A. Heinlein - The Man From Mars

Heinlein was a fine author and a great mind. This book is a truly thought provoking look at how alien, aliens would actually be. He overcomes this issue in the story by having a human raised by aliens and then returned to earth. He depicts with a thought provoking ease, how an this alien human might pick up our language with great ease and yet still utter lack the contexts which would give words meaning.

It's quite a long book and for me, the payoff was almost all in the first half, when the alien struggles to understand (or grokk) humanity. In the second half of the book, the alien tries to shine a light on some of the issues of society from this new and innocent perspective. I didn't dislike the second half of the book, it's just that reading a book like this, some 30 years after publication, can make ideas which were shocking and new then, seem tired and overused now. Well worth the read though, especially for the first half.

John Christopher - The Death of Grass

Now here is a book I can safely recommend to all and state that people "should" read this book. It's an "end of society" book, which, even after 70 odd years is still the most elegant expression of how quickly and easily our society could fall. With it's utterly realistic plot line and character, it depicts the change in morality of a group of people travelling across the UK during the collapse of society. I can't recommend this book highly enough - I've already purchased additional copies for friends.

Thomas Hardy - Tess of the d'Urbervilles

Another of the free kindle books and one I'd been meaning to read for a while now. This was a strange read, I immediately admired the quality of the prose and how well written the characters were. I was impressed by the ending and yet, I couldn't wait to finish the book. I think with so many other books to read I was a little impatient with a story that didn't have something to teach me. Perhaps, had I read this book on holiday with nothing else in the wings to read I may have been able to enjoy it more. I genuinely believe this was more my fault than the books.

Hunter S. Thomson - The Rum Diaries

As always with Hunter Thomson, you know that whatever the topic, it will be related with an absolute fairness to all characters. The work will be subject to his astounding attention to detail and an uncanny ability to get directly to the heart of the matter. I absolutely love his style of writing and his descriptions of scenes, such details in the minutia that you could be sat in the bar with him and yet not a single word is wasted. This is a little like the Fear and loathing in las vegas for Rum addicts.