Past life in outsourcing

07 June 2017
I had not intended to write an article about a short-lived job I did over a decade ago, but during drafting it became apparent that what I thought would be a short-tie in of my own experience in IT outsourcing ended up being far longer than than the originating article. This job was in a Microsoft-orientated start-up in the post-dotcom world, and what I remember of software development at the time helps explain the recent problems with legacy Windows software. The job itself I got surprisingly quickly, although being towards the back-end of the dot-com crash jobs in small start-ups such as this one were quite rare — at the time most of the graduate IT jobs were in big London companies. The company itself has changed names since I was there, and while it seems to have done quite well in the mid-2000s after a few internal changes, I have no idea if it is still trading. Last I heard it was issued with a Companies House strike-off order which it managed to see off, but at the time I had more pressing concerns than to keep track of ex-exployers.

The company products

While I was with the company, most of my time was spent implementing the PDA interface for the medical record system being developed. My personal feeling was that the product line was very well placed, because at the time most hospitals wheeled around trolley of paperwork alongside patients, but it embraced certain parts of the Microsoft ecosystem more readily than it should have. It was full-blooded Microsoft C#.NET, which at the time was a brand new technology being pushed very hard.

.NET Crippled Framework

The company product I spent most of my time on was the PDA interface, which was implemented using the .NET Compact Framework, version 1.0. This was a supposedly cut-down version of the desktop .NET framework for PDA, but given the features that were cut I think it would be better called Macheted Framework, because the way it was cut down crippled it. For instance one of the things I needed was access to the raw pixel data from a drawing panel, but the only way to access such data using just the Compact Framework API was to use getPixel or setPixel for every single pixel. This was prohibitively slow: Think like 2-3 minutes for a load or save operation to complete. Workaround was to use P/Invoke to drill down to the underlying GDI object in the native WinCE API, and access the backing raster data. But this was far from my biggest headache, which was implementing dictation facilities using waveInRead and waveOutWrite, and this was plagued by all sorts of random issues which at the time I suspected that the Garbage Collection system may have been upsetting my attempts to keep track of which blocks of unmanaged memory. Ultimately this reflected badly on myself, and at some point the job was one of zero positive stimulus.

From my point of view the amount of drilling down using P/Invoke made a complete mockery of the idea that .NET was independent of the underlying hardware, and I concluded it would be better to implement the painting and dictation as self-contained controls in native WinCE and then import them into .NET a complete units wrapped up in C# classes. In the week or so after leaving the company I implemented in Win32 API (using only functions listed as available in WinCE) a significant portion of what I had been working on, purely as an exercise in validating confidence in my abilities. A few years later when I got my Xperia X1 handset I recompiled this code for Windows Mobile 6 with only a few changes and it worked perfectly. The cruel irony is that C#.NET CF v1.0 had by this point sunk without trace, and in hindsight it was little better than a technology preview.

Changing security models

The main product was implemented as a browser-hosted application, which was quite a novel idea for the time. I think it made heavy use of ActiveX, and to get it to work required MSIE's default security settings to be loosened a bit, which even at the time I was a little suspicious of. My feeling was that such a setup would end up being more trouble than it was worth, and that it really should be deployed as a stand-alone application. It was from what I recall pretty much tied to a specific version of MSIE, let alone being Windows-only, so I did not see any real value in the browser-hosting. Maybe update deployment played a part as this was before automated software updates were a big thing.

To my knowledge the security model introduced in Windows Vista disallowed this way of doing things, because keeping it secure was too much of a whack-a-mole. I went to a one-day Microsoft roadshow where a security feature they were showing off was that web-hosted applications were only allowed 50% transparency, preventing their use as invisible input-capturing overlays. These days a web applet would not be allowed to do anything outside of the bounds of the web-browser. Even Microsoft has deprecated ActiveX, but it pretty much already had pariah status as far as everyone else was concerned.

Windows Forms

I quite liked Windows Forms, in part because it was quite a thin wrapper around the underlying Win32 GUI controls, and hence got the feel of it very quickly. However the form designer frequently broke so in practice the GUI code had to be maintained by hand. Whereas I preferred to use parametrised loops that allowed easy tweaking of GUI layout, within the company itself was a preference for cut'n'pasted code with dozens of hard-coded values. And I mean putting in the final value, rather than putting in calculations that ought to get optimised down to single constants. What struck me at the time was recommended order and way to create controls, which at the time I thought any half-assed compiler ought to work out for you.

XML and all that

In all fairness I learned lots of small tricks that still apply to modern mobile development, such as the use of
HTML panels with place-holder links that were caught using an overridden link handler. This company also gave me my first working knowledge of CSS & XML/XSLT, although admittedly it was Microsoft's not-quite-standard dialects. For a long time the keywords associated with this job such as these were the source of a lot of queries from recruitment agencies, and hence why this employment has survived as a few lines on my CV for such a long time. I think in hindsight I would have lasted longer at the company had I worked on the desktop product rather than the mobile offerings.

Non-Microsoft aspects

One interesting diversion I had at this company was evaluating non-Microsoft PDAs, most notably the PalmOS series. This included trying out both the MetroWerks IDE and the equivilent GCC-based tool-chain, although its record-based storage system would have made integrating it into the rest of the company's product line-up somewhat difficult. For all its faults PocketPC made other mobile frameworks, especially the Java-based MIDP, look extremely primitive. In hindsight PDAs did not really take off in the way tablets & smartphones did, but during my Ph.D years I did not really pay much attention to how mobile was developing.

Legacy of this job

Being my first graduate job I was going to make mistakes, including some that I really should have not done, but in hindsight this was a job doomed to not work out. For me the hardest lesson was the extent that actual programming ability did not really matter, and the real limiting factor was not knowing the entire API library inside-out, which extends to knowledge of undocumented faults. Nevertheless there are three incidents I remember very well which made sure it was just a matter of time before I walked out:
Just copy the code
I was pretty much told that stealing code off the internet was standard practice and that I should do it as much as possible. No doubt it was common practice at the time, and in hindsight the code I came across was probably OK to copy, but these days it is a very different issue. My preferred way of operating was to understand sample code and then reimplement it based on my understanding, which of course was slower.
Where is my code?
Before leaving one Friday I made sure I had working code checked into CVS. I then came in on the Monday and found out that over the weekend it had basically been rewritten. I spent several days working out where things had been moved to, and from that point I gave up planning more than half a day ahead.
Don't be so smart!
I can't remember what I had changed, but whatever it was I was on the receiving end of criticism about “unilaterally” making decisions. In itself fair enough, but on giving my reasons the retort I got was “Don't be so smart”.This inability to defend my decisions finally crushed any remaining motivation, and in my current company such an attack on personal character is considered unacceptable.
Where things went wrong was the development of a day-to-day attitude which threw out attitudes that served me well during my undergraduate degree, and at some point my innate motivation finally cracked. Once the latter had happened it was a mental push just to want to get stuff done, let alone make a good job of it, which by this point I considered impossible. Almost a decade later memories of a cracked motivation from this job leading to a downward spiral of more and more mistakes as burnout set in played a big part in me deciding to leave Bristol (and the UK).

Software backward compatibility

As pretty much everyone knows, the NHS was a turkey shoot for the WannaCrypt outbreak. This was to be expected given that some trusts spent nothing on keeping systems secure. Apparently only 5% of NHS computers run WindowsXP, but this is a moot point as it only takes one insecure system to compromise a whole network, and the majority of NHS trusts have at least a few such systems. However what interests me is why WindowsXP needs to be used at all, and this is where my almost-forgotten (and maybe best forgotten) time working in this Microsoft-orientated medical software house comes into play.

Sheer age

Windows itself which in technical terms means the underlying Win32 API, has very good backward support — Windows10 will run some programs I originally compiled back in 1998 for Windows95. However support for Windows 3.x applications seems to have been dropped after WindowsXP, as a Windows2000 VM seemed happy running WinTex v4.3 whereas a Windows7 one decided not to play ball. WinTex v4.3 was released in 1996, and was written using Visual Basic 3 which itself was released in 1993. The fact that WindowsXP is the newest version of Windows that can run this software strongly suggests that a lot of software out there was originally built for Windows 3.x back in the 1990s, which to me is not much of a stretch of the imagination as back in 2011 I was dealing with legacy systems that dated back to the early 1980s.

Because of concern about the millennium bug there was a lot of investment in IT infrastructure around the turn of the century, but come the crash all that work dried up and a lot of the smaller consultancy outfits would have disappeared, in part because a lot of the companies that did actively maintain their IT systems took the work in-house. Those small outfits that were left would have gone through a phase of living dangerously, which would not have helped the IT project failure rate. There would have been another squeeze on investment in the late-2000s as a result of the credit crunch, so the odds are that most software out there was commissioned prior to 2005.

Overall eco-systems

A lot of software requires more than just Windows to run, so backward compatibility also relies on how resilient dependent software is to bit-rot. As a result the eco-system required to run a piece of software extends far further than just the version of Windows, and .NET is a complicating factor in these eco-systems. Although .NET run-times in theory are able to run binaries built for previous releases, Microsoft itself cautions that there may be side-effects, and in any case this is predicated on having a configuration file that specifies what versions of .NET the program binaries are compatible with. If present at all it most likley will only list the version of .NET the software was developed for, so dependencies on specific versions of .NET more often than not are likley tighter than they really need to be. The introduction of program manifests by Microsoft may have helped solve DLL Hell, but the typical response results in dependencies that are time-specific rather than being function-compatible maintained. I remember once being told not to apply the .NET v1.0 to v1.1 update pack as it caused problems with the existing software, which to me illustrates the extent that software houses assume a very specific software eco-system in the solutions they offer.

Software quality

Back in the early 2000s there was talk of reliability metrics like like six-sigma (also known as five-nines) and some ISO certification, but in reality the priority was always to get things working in the most expedient way possible. Bodges were more often than not the rule rather than the exception, and on many occasions extra features were hammered in at the last moment. Rather than being shrink-wrapped, most software out there are custom-written solutions for a single client, and all too often they were knocked together by a fly-by-night software outfit that has long since sunk without trace. Needless to say this does not bode well for any sort of QA, and in practice the build quality of a lot of the software out there was never up to scratch in the first place. Newer systems typically are stricter when it comes to not-quite-correct ways of doing things, which causes problems when the software solutions were just-got-working in the first place. All my drilling down using P/Invoke mentioned above would these days be filed under nasty hacks, and I have no doubt that other software written around that time is littered with such expedient Saturday night specials. Microsoft itself was notorious for its own applications hooking directly into undocumented internals rather than going through the public APIs.

The short version

The Microsoft approach was progressive if somewhat monopolistic, but in most cases the support vanished for various reasons and all that was left was software that had no provisioning for ageing well. Back in 2000 it was a time of rapid growth where people went for the new market rather than worrying too much about upgradability, and most outfits were happy just with having moved from paper, but today we see the upgradability issue coming home to roost. As is typical in IT people ran systems until they stopped working, and in the case of the NHS just kicked the can down the road.