Thursday, January 28, 2016

Joys of the status page

If you're building web applications, no matter how many unit tests you write, there will still be bits of code that don't get test coverage. Usually that's the code that makes the actual call to the external service, or subscribes to a feed, or writes to a file, or accesses some other outside resource.

Many things can go wrong with that code. Fat fingers, missing file permissions, firewall rules. End-to-end tests will take care of some of that, but usually they involve connecting to some mock resource in a controlled test environment. Not very good for telling you why the field that's supposed to be updating live in prod parallel environment is blank.  And if you're trying to follow test-driven development, writing piece of code and then waiting for it to fail in some deployed environment is not very satisfactory.

I found the following practice helps. When writing an web application, one of the first things I make is a status page that lists every required external resource. When I write a new piece of code that accesses some resource, I also write a test (really, an external facing "unit test") that tests invokes that code and displays results, e.g. connects and authenticates with a service, and then queries it for something trivial.

It serves two purposes: (1) it makes sure that the connection and authentication code is correct - it should fail immediately in dev environment if it's not correct, and (2) given that it works in dev environment, failure in any other environment means that the environment is misconfigured, and likely contains bad credentials, or bad endpoint URL, bad something else - usually it should be clear from the error message or context.

End result: feels both reasonably test-driven, and saves time troubleshooting things later on.

Saturday, April 18, 2015

Seeking the perfect command line interface (in C#)

As a programmers, we inevitably end up building a console app at some point, whether it is to perform an extra-special pre-build step, or as command-line interface to some system, or for whatever other reason.

After doing this enough times, I think I've started converging to an approach that I like, and an implementation that I like.  So, may as well share it with the world.

The interface

It took surprisingly long time figuring out what makes a good command line interface.  I'm still not sure whether I got it right.  So far, I favor the design of the form
> program-name <verb> [options]
or, if there's a consistent set of actions applied to some targets:
> program-name <verb> <target> [options]
> program-name <verb> <source> <target> [options]

I like this approach because it's concise, intuitive, and, if verbs and targets are picked right,  options often become really optional. 

Saturday, December 3, 2011

Automating Excel with F#


Every once in a while at work I get to do things of the type "do this analysis every day for the next few weeks and email the results to these people".  Sometimes this analysis is not exceptionally complicated (update data in an Excel workbook and do some calculations) and is expected to be performed for only a few weeks, making it not worth involving developers or writing new server-side applications.

As I don't like spending too much time on these, the solution was to write simple scripts that automate data retrieval and Excel manipulation.  Unfortunately the resources on F# Excel automation tuned out to be a bit scattered around the web, so I am putting the results of my work here in a simple example application that demonstrates most of interactions involved in these "open Excel - load data - recalculate - send email" tasks.

Why F#?  Well, I needed a concise and powerful scripting language that played nice with Excel, and F# with complete access to .NET libraries (including Office Interop assemblies) seemed like a pretty good choice.

The Task

The example application I'll be presenting will be completely unrelated to my work, and will go as follows:
  • There's an excel worksheet with a table of dates/times, and several cities on it.  As it happens, the cities are only Vancouver and Toronto, but this may change later if we want to track more cities.  
  • For each date/time, the table has a temperature reading for each city, downloaded from Yahoo! Weather.
  • Every once in a while we'll want to check Yahoo! Weather, download the temperatures for the specified cities, add them in a new row to the workbook, run a macro on a worksheet and email the resulting workbook to someone.
 The complete code, together with the Excel workbook, is available on GitHub.

Saturday, February 5, 2011

A Quick Experiment with Websites in C++

After spending a couple years wondering what it would be like to build a website in C++, I finally got an excuse to do it.  As it turned out, there were no websites that would make the user guess which words were valid Scrabble words and which ones weren't; this was exactly the simple, useful website I was looking to build, and, after a couple weeks of work, it is done.  The source code is on GitHub.

The website consists of five "pages": the dynamically generated main page where only the individual words to guess vary between the requests; two statics pages (about and fine print); a "dynamic" 404 page; and the end point for Ajax requests that provide the next batch of words to guess.  In addition, the server allows downloads of various resource files (JavaScript, CSS, images).

This post will attempt to briefly explain how this was done, as well as the lessons learned.

Monday, November 29, 2010

Google AI Challenge (Planet Wars) Entry

UPDATE: I seem to have gotten second place.  Not bad, that's definitely higher than I expected.

At last, I am free of it!  Google AI Challenge (Planet Wars) is over.  No more spending all my free time thinking about how to eek out extra few wins, no more rushing home after work to start testing new bots on the TCP server.  Or, for that matter, no more postponing some pretty important things just because I wanted to implement a new idea.  However, it was fun, and quite worth it.

Without further ado, here's the description of what I did.  The code is on github.

1. Laungage, Tools

The first important choice was the language.  Of those that had starter packs, I would have been most comfortable with C++ and Python; remembering that most top bots in the previous Tron challenge were written in C++, I decided to use that for my bot.  In retrospect it was a pretty flimsy argument, as it could have easily been that the top contestants were just more comfortable with C++, or thought it would be better than other languages, creating a self-fulfilling prophecy.  I regretted the choice of language a few times as I find Python to be a bit more developer-friendly, but in either case, C++ worked out well enough and pretty quickly it was too late to back out.