Development of SprigFern

14 March 2014
Last year I wrote about my motives for re-learning C#, and as a vehicle for this I wrote SprigFern which is a Futoshiki puzzle solver. Although I originally started the implementation using Windows Forms, for various reasons I switched to GTK#, and as a result of the whole experience was able to form an informed view-point of several technologies that came onto my radar. Since it also represents the first time in several years I have designed and implemented a piece of software that is not tied up in company contractual obligations, I can talk freely in a way that is not possible with my professional projects.

The puzzle the program solves I first came across in a New Zealand newspaper that happened to be lying around while I was waiting for someone, but rather than thinking about the puzzle itself, I thought about an algorithm that could be used to solve it. Since then I forgot the name of the puzzle, so I used the name of the pub (Sprig & Fern) as a placeholder, and in the end decided to keep the name. I am not sure exactly when this thought exercise turned into an actual intention to implement the algorithm, but it eventually happened.

Design side

While I don't exactly subscribe to Joel's Big Design viewpoint, I do see the merit behind how it quickly nails down issues. There are times when diving straight into the code is the thing to do, but for things like algorithms and basic appearance it helps a lot to rough things out on paper first, because being away from the compiler is the only sure-fire way to stop you constantly getting distracted by implementation details. Many hours bashing out and testing C# code is in large parts merely the realisation of two sides of A4 paper I spent at most a cumulative 45 minutes drawing out. In fact this sort of top-level architectural drafting is the one thing I did at my last company that I miss at my current one.

One thing I came across once or twice was what in Mythical Man Month are called mini-decisions, which are implementation details that nevertheless have longer-term consequences. In this case it was the algorithms that map between a widget's position and the associated puzzle back-end index parameters. I opted to pass an index rather than position parameters, which turned out more suitable because the lower-level primitives on the whole don't care what their location is, and it restricted all the mapping logic to the higher-levels. Unfortunately I have seen many instances at my current and previous company that amount to having chosen the pass-by-position option: The mapping algorithm becomes a can kicked down the road.

The puzzle is basically a process of elimination, where the set of possible values for each square is iteratively reduced based on constraints. For instance if a square has a definite number, it can be removed as a possibility for all other squares on the same row and column. For squares that have an inequality between them the highest possible number in the ‘higher’ square can be eliminated from the ‘lower’ square, and vice-versa for the reverse. The process of elimination continues until there is no change. Getting into a no-possible-puzzle state should be prevented by only allowing one change to be made at once, with subsequent choices being limited to calculated possible values.

Visually the template for the user interface is how the puzzle appears on paper, so the only real design part of the GUI was the mapping alorithms mentioned above. While the general idea was to use point-and-click to change square numbers and the inequalities between them, the details of whether this toggles between possibilities or bring up a pop-up menu was something that could only really be left to experimentation to see what felt most right, so could not really be thought about as a design issue. Veteran GUI programmers will disagree, but programming GUIs has never really been my primary calling.

Development environments

The programming side of the project originally started as a Visual Studio Express one while I was in Hong Kong, but due to other things getting in the way this version never really got much beyond an experimental front-end to remind myself of various Windows Forms features. By the time I had spare time to restart, I had come across MonoDevelop, and being mainly a Linux user felt that a switch to GTK# would be more useful than re-learning Windows Forms. Around the same time I also came across SharpDevelop.

SharpDevelop specifics

As a means of giving SharpDevelop a proper try-out, I used it to develop the puzzle back-end, and later copied the source code into the main MonoDevelop GUI project. SharpDevelop feels very much like how Visual Studio used to be, which I quite liked as I find Visual Studio 2012 awful compared to Visual Studio 2002-2008. Given the choice between Visual Studio 2012 Express and SharpDevelop, I think SharpDevelop blows VS Express out of the water.

The only thing that takes some getting used to with SharpDevelop is how it handles bracing. It seems the only supported bracing style is Allman, which although I can live with, is not my preferred style. In addition typing an open brace will add a close brace, which unless you are starting a new function (or left no blank lines between a new if-statement and existing code), will usually end up being places where you do not want it. However once you get used to these, SharpDevelop is very nice to use. I had considered whether to build GTK# for use with SharpDevelop, or even build an alternative Windows Forms based version of SprigFern in addition to the GTK# version, but felt that this was a bit too much scope creep.

MonoDevelop specifics

While on the whole I found MonoDevelop to feel that bit less welcoming than SharpDevelop, it also has a few nice features of its own in its favour. The Linux and Windows versions also have some differences, in part because the Windows version to some extent is an advert for Xamarin's mobile development version. As far as Linux-based development IDEs go, MonoDevelop is certainly among the best I've tried.
Embedded binary resources
MonoDevelop makes it easy to embed a binary file as a resource, and provide access to it that is much the same as reading an actual file. I very much like this, as in the past I have had headaches with building the installers for GTK-based programs that came with dozens image and Glade files.
Source code folding
For some reason code folding is turned off by default in MonoDevelop, but when it is enabled, hovering the mouse over the close-fold icon will highlight the section of code that will be folded. Nice feature, though perhaps more eye-candy than practical use, and for some people it might actually be annoying.
Code completion
There are times when code completion kicks in when it should not, which mostly happens when typing the arguments for new functions, and the new parameter happens to be the prefix of some other built-in identifier. This is in large part due to the spacebar acting as a select button, as opposed to aborting the autocomplete, and it is very annoying.
Bracing
I was able to get MonoDevelop to use my preferred bracing style, although it is a bit hit-n-miss compared to Emacs's bracing engine. It tends to defer indentation until you have explicitly typed the close brace, and unlike SharpDevelop (at least with my setup), does not auto-generate a close brace after typing an open brace. Unfortunately it breaks down when writing unit test code, probably due to the presence of decorators.
One thing that does stick out a lot is how MonoDevelop does generation of XML docstring templates, in that it make a reasonably good attempt at guessing parameter descriptions. In particular, guessing that posX means Position X:

/// <summary> /// Initializes a new instance of the <see cref="NumberFrame"/> class. /// </summary> /// <param name="tableParent">Table parent.</param> /// <param name="posX">Position x.</param> /// <param name="posY">Position y.</param> public NumberFrame(Table tableParent, uint posX, uint posY) : base()

The biggest down-side with MonoDevelop is that the pre-built versions of it available on Linux are horrendously out-dated, and building it from source on Ubuntu is bit of an ordeal. As a result I ended up running it from a Slackware virtualbox, where it actually performed quite well.

GTK# specifics

The way GTK# is presented in MonoDevelop does not seem that far off how Windows Forms is presented in SharpDevelop & Visual Studio, so the jump was not that difficult to make. In all cases the forms designer was pretty much useless for anything other than basic templating and learning the basis of the respective tool-kits, but it soon became apparent how GTK# is substantially more modern. My previous experience with GTK was using the underlying C based interface which at best is somewhat verbose to use, and my most recent dealing with it being the deploying (including recompiling GTK itself) GTK based applications has left me with a lasting impression of something that is somewhat Byzantine. GTK# in contrast has left me with a somewhat more positive attitude.

The big down-side with GTK# is that finding documentation on it is difficult at best, and there are one or two things I never worked out how to do, such as rotating text. It ought to support the latter as GTK# includes Cairo, but in the end I used icons instead. As far as GTK bindings go GTK# is definitely a second-class citizen. Most GTK tips'n'tricks use the base-level C interface which is basically useless as a reference for the C# bindings, and in practice working out how to do things requires a bit of guess-work based on the GTKmm (C++ bindings) documentation. Searching for “GTK Sharp” rather than “GTK#” helps a lot for specific questions, for some reason. Having said all that GTK# is a joy to use once you've worked out all the quirks, if you manage to work them out at all, especially with the autocompletion in MonoDevelop.

In the end programming the GTK# front-end involved writing the code three times: a completely experimental messing around with the tool-kit, a more guided attempt at writing what I wanted, and then finally a major re-factor to tidy it all up. Like web-programming, GUI programming is something that I quite like for personal projects, but I rather doubt it is my forte for a career given that there is little overlap between writing nice tight code and writing user interfaces.

Overall GTK#.NET impression

Rather than being GTK for C#.NET in the same way GTKmm is GTK for C++, it seems GTK# has been subsumed into the Mono project, which in turn is diverging from its root as a .NET reimplementation. As a result GTK#.Mono is perhaps a better name than GTK#.NET, and although it has yet to get to the stage where it has to be treated as a now-separate fork, GTK# for version of GTK later than v2.12.21 is not available standalone from Mono. To me GTK# is what cross-platform development should be, and I certainly rate it much more highly than Java Swing. GTK# is something you go to if your interest is the C# language rather than the .NET platform.

Although most of SprigFern uses hand-written GUI code, there were still some parts that were using Stetic (the GUI generator), most notably the about dialog. However, Stetic uses functions that are specific to Mono, resulting in code such as the following:

this.label = new Gtk.Label(); this.label.LabelProp = global::Mono.Unix.Catalog.GetString ("Name")

The problem is that although the GTK# .NET run-time includes the Mono.posix DLL needed for this, it seems to be a .NET v2 assembly rather than a .NET v4 one, and MonoDevelop compiled binaries only look for the latter. In contrast binaries compiled using Xamarin Studio (Windows version of MonoDevelop) look for both. To me this was the most visible way Mono is trying to escape from the shadow of .NET and be a platform in its own right.

Unit Testing

Originally unit testing was going to just be part of trying out SharpDevelop, but for some reason I ended up getting runtime error MSB3247 (found conflicts between different versions of the same dependent assembly), and couldn't work out how to get around it. In the end I added unit tests as a separate project within my MonoDevelop GUI solution, although these tests only targeted the solver backend. Although use of unit testing did help in tracking bugs, the bugs themselves were not in the code under test, and it was thinking differently about the program that was the real help.

Although the writing of the actual test code within NUnit test-cases is not really that much less effort than writing a basic test harness from scratch, I have noticed that the presence of a framework that tracks which tests have passed & failed encourages the writing of more tests than is strictly necessary. Writing unit tests also affects the way you think about a program, as you are forever thinking how it could be further broken down into independent functions, but for an iterative process such as this one I think that was a temptation best resisted. It did catch a simple error or two that I may have missed for a quite some time, but get the feeling that spending effort trying to tests sub-components in isolation is not the way to go when the complexity is in how the algorithm performs as a whole.

On the whole the experience reinforced my scepticism of test-driven development, but it also much reinforced my belief in test-driven debugging. Test-driven development is very data-centric, but for an algorithm-orientated project where data-structures are quite fluid throughout prototyping, writing test-cases up-front sets the wrong things in stone. You end up having to second-guess how internals are going to end up, and when they turn out wrong having to spend more effort hammering the test infrastructure back into shape. In contrast when it comes to debugging, interfaces and correct behaviour have become concrete, and unit-testing is a valuable tool in pinning down what is and is not working as expected. When it comes to applying fixes, it also makes sure that other things stay working. The fundamental issue is that up-front thinking will highlight some edge cases, but the really problematic corner cases only tend to show themselves through actual usage rather than thought experiments. The problem with testing is picking the point where details are sufficiently concrete that correct behaviour can be asserted in a sane way, but not leaving it so late that tests assert the implementation rather than actual intentions.

Deployment

As mentioned above one irritation was Stetic using functions that caused problems running using .NET rather than Mono on windows, which was a bit of a spanner in the works. In short it showed how deployment cannot be independent of the development process, as it meant either using Windows for release builds, or removing the functionality that was leading to the problems in the first place. In this case it amounted to using GTK's built-in about dialog rather than my own custom-made one, but I was lucky because it affected something of minimal actual importance. In most cases, going from development to deployment is a waterfall, and asking for other developers to change their ways will most likely result in them throwing a spanner at you.

I did have the potential workaround of including a DLL or two from the Mono run-time (mercifully these were X11 licensed), but I know that as soon as a program becomes non self-contained, things get messy quickly. In short as soon as you have anything other than a single self-contained program file, you have to use some form of installation infrastructure, and then you get whacked by a load of edge cases. RPM vs. DEB is bad enough, and that is not even the start of the pain. Even if you opt for a relatively agnostic installer, one issue that comes up a lot is deinstallation, particularly in cases where foreign files may have somehow creeped into the program's installation hierarchy.

The nice thing about shipping a Mono binary is that all the grubby details such as the idiosyncrasies of where a given distribution keeps support libraries are not your problem, as they are all dealt with as part of Mono's installation. That means when I dig up the binary in 5-10 years time, it will run just as I expect some program I wrote for Windows95 to still run. More immediately it means that the same binary will run on Ubuntu 12.04, Slackware 14, and whatever RedHat is called these days. I also know how nasty it is trying to compile an old GTK v1.x program on a modern system, so I don't entirely buy the argument that source distribution is protection against bit-rot.

Time commitment

From initial spark to first publication SprigFern took 7 months, but this is not too surprising given that during this time I had stayed in four different countries. Start of development that actually fed into the actual deployment was 3-4 months ago, and although hindsight estimates are inevitably approximate, my guess is that around 90 hours was spent on thinking & programming. Much of the programming effort was in sessions of around 4-6 hours, which seems to correspond with what planning at my current company assumes is a good day's effort. My gut instinct based on past commercial projects I have done is that this would translate into around a month's worth of company time.

Commercial vs. non-commercial

One thing that is noticeable with personal projects like this compared to professional ones is that while start-to-end times are naturally very long, time actually spent on the project is exceptionally productive. In part this is due to the motivation barrier that needs to be passed before a work even commences, but there are other factors also in play. One of these is impediments such as the above-mentioned mini-decisions all being short-circuited, but more importantly it is because it more directly maps onto interests and ulterior motives. My company's decision to move me from the “core” team to the networks team looks like a good bet based on what I know they know about me, but more often than not companies are not able to take advantage of these motivational factors.

The final straight

I am not sure where I came across the reference (probably id software), but in the past I have heard software development described as the first 90%, then the remaining 90%. Getting a working program is one thing, but going from this to a fully polished final product requires a surprising amount of extra effort. For SprigFern this was things like adding the dialog that allows new games of different sizes to be started. In large part it is clearing up all the loose ends that result from pulling everything together, and along with testing to find these loose ends in the first place, is something that too often is not budgeted for. For personal projects this late hurdle is often a killer.

Physical environment

While in New Zealand I had a desk of around 20" deep and I found it that bit too uncomfortable to really get into the programming mood. Like most people I prefer to lean forward rather than sit back, and shallow desks simply don't give the room for the elbows. Small desks also don't leave much real-estate once the display, keyboard, mouse, and computer itself have had their share. In contrast my current desk is 32" deep, and wide enough that the far corner has scraps that I have no idea when they were put there. Incidentally this desk is the only piece of furniture in the flat I specially bought. Mercifully most companies have latched onto this.

In brief

The main purpose of this project was to get back into C# programming, and in the process covered a lot of ground. Although it has its irritations, I feel that GTK# is a good platform to use for GUI-orientated personal projects. I left out some issues such as source releases and installers, as this article was getting a bit long as-is, but I suspect I will cover those separately before long.