Afterthoughts - Summer of Testing (and Tasting!)

Guess what? It was shady again.
In this post I'll go through some highlights of our testing themed Geek Collision. We covered things like unit and acceptance testing, automated accessibility testing and fuzzing. There was a nice little war story too. So something for everyone!

Overall it was great fun and we managed to cover a variety of topics. It definitely gave some extra perspective for those that attended.

At peak there were around fifteen people, a record of sorts I think. Thanks for attending!

Tuukka Turto - pyherc

Tuukka Turto showed us how he tests pyherc, a NetHack lookalike of his. You could say he has the thing covered. Besides unit tests you'll find at least acceptance and UI tests in the project.

Through testing make it possible for him to keep the ball rolling. To quote Tuukka it enables him to get back to flow after interruptions. He plans to write a web based user interface for the backend. Perhaps we'll see a MMORPG version of pyherc sometime in the future? By the way, pyherc comes with a vintage mode already just so you can relive those golden NetHack memories.

If you want to know more about the project, check out Tuukka's interview at Five Hour Projects. That should give you more insight on what he does, why and how.

Asko Soukka - Automated Accessibility Testing

Asko Soukka gave us an overview on how he tests user interfaces built with Plone. He demonstrated how to use Robot Framework to build tests executable in browser. Furthermore it was possible to get video output and individual screenshots. The most impressive thing, no doubt about that, was the way he had integrated the purposes of documentation with testing.

It is possible to use the screenshots given by the tooling in your user facing documentation. This way your tests and documentation will stay always in sync as the workflows change. How neat is that?

Even better he had integrated Wave Toolbar as a part of his automated testing workflow. In this case the screenshots contained additional information highlighting various issues detected by Wave Toolbar. Rather than having to run the tool yourself this automation makes it fast and easy to get the feedback as you developed.

Do keep in mind that you cannot automate accessibility testing fully. A real person is always required. The main point about this sort of tooling is that it raises the bar higher and allows developers to reach better initial results before tweaking further.

You can find more information about Asko's accessibility testing workflow at his blog.

Tero Tilus - a Tale of Two Codebases

Tero Tilus told us a story of a recent case he was involved in. It was a project with two distinct codebases for frontend and backend. Latter of these was well tested whereas the former was not before Tero got his hands on the project. Guess which part is going to receive some sort of a rewrite?

Dimensions of Testing

Tero also explored the dimensions of testing. It is very easy to have a narrow view on testing. There's more to it than just unit testing, TDD, acceptance testing, whatnot. It is simple to forget about qualitative testing. This includes aspects such as usability, user experience, robustness, code quality and performance. Some of these can be tested using hardware but some aspects require human effort.

It is possible to enforce some of these qualities by using the right development practices. You can, for instance, affect code quality by introducing peer reviews. If you have the right kind of process in place, quality will follow (or will it?).

Testing Maturity

Tero discussed Beizer's five levels of testing maturity. It is more about how view the purpose of testing. Unfortunately it is very hard to provide that code is correct. We can, however, try to minimize the effect of this through various ways. Of course we can ignore the whole thing and just "cowboy" it. Or we can be conscious about it and try to do something about it. That's where that testing maturity thing comes in.

Juho Vepsäläinen - Fuzz Testing

As I saw Tuukka speaking about Python testing I could not help but to showcase a couple of my approaches to Python testing. Namely documentation driven testing and speccer. The latter one is a DSL of sorts that "solves" the problem of unittest module. It is a module derived from Java. Let's just say it's a very verbose way to write unit tests. And I prefer brevity myself.


That is the reason why I wrote speccer. Implementation-wise it just generates unittest code and uses its test runner even. Some of the implementation details are a bit sketchy but in principle you may mix regular Python code with it. The slides below should give you a better idea.

The real beef of my bit had to do with JavaScript. It's something I've been dabbling with during the past few years. During this time I've developed some solutions of my own (talk about NIH). The first one I showcased briefly was bunit.js. It is something I developed because I didn't like the syntax of QUnit. Again, too verbose and not AMD compatible (at the time anyway).


After a while bunit.js started to feel verbose too. So I went to the absolute minimum I could think of and came up with suite.js. Its primary concept is a test suite. It executes a set of units (individual check) for the function provided to the suite. I guess the syntax might look a bit weird at first. It is extremely powerful for small scale unit testing, though. The slides below explain the basics.

As I developed suite.js I started to think about test automation. Wouldn't it be neat if it was possible to generate those units? This lead me to the world of fuzzing. I simply wrote something that generated units based on some simple generators and an invariant to be tested. The syntax wasn't that nice on retrospect but at least it was a start.


In order to overcome this syntax problem I decided to split the problem in two. I moved half of the definition to the function level and leaving the invariant part to the external test. This seems to be the sweet spot for me. In addition I get runtime checks which isn't a bad thing. As I don't want to drag this blog post further, have a look at a small fuzzing demo I have developed. It gets to the gist of it.

I'm still in the middle of wilderness with this thing but at least I feel like I'm going to the right direction. The audience raised some good points about improving the methodology by making it deterministic (not purely random like now) and checking certain border conditions to improve the test coverage. His thoughts on generating permutations based on these conditions were very interesting as well.


I think it was really nice to have a themed Geek Collision such as this. It provided a way for people to showcase some of the technology and solutions they use in practice. This way it fostered interchange of ideas and perhaps lead to some new development. I definitely would not mind participating in these sort of events in the future. I hope you think so too.

Speaking of future events, remember the Battle of Web Frameworks held at 16th of July. We plan to provide a specification people can write their participating entries against. It's not meant as a serious competition but rather as a way to learn something more about technology. So even less than perfect entries are allowed.

GC - Summer of Testing (and Tasting!) on Thursday (27.6) 18:00- at Hemingways

Sunset by Christian Senger
The people have spoken. There shall be a testing themed Geek Collision. See you at Hemingways next Thursday (27.6)!

Bring a laptop if you have something to demo. I'll have some fuzzy material available at least. And I hope you will bring something too.

Upcoming GC - Summer of Testing (and Tasting)!

Testing by pedrosimoes7
Testing is one of those topics that can bring endless amount of joy and make puny coders well up in tears. The purpose of this GC is to share some experiences, techniques, and perhaps war stories.

If you have some tools you wish to showcase, now is the chance. We'll have a projector available. In addition there might be something to drink.

In case that sounded like a good deal, go ahead and mark suitable dates at the Doodle.


Upcoming Event - Battle of Web Frameworks

You guys are in for a special treat. We will arrange a battle of web frameworks. What an awesome chance to showcase your framework-fu and to get to see what some others have to offer!

So far we've managed to attract two contestants, both from the camp Haskell. These are namely Snap and Warp (used by Yesod). They look awfully powerful based on some measurements.

The event will be held tentatively around 16th of July (Tuesday) although that may change. We'll let you know of the exact date closer to July.

To make the whole thing work, we'll need to decide upon a set of tasks. Each task will be implemented using each framework before the event. This way we'll have something to discuss and we don't need to get that physical (although that may still happen).

In order to make this event as awesome as possible, enroll or at least suggest some tasks for our contestants to implement. We can discuss the details at #geekcollision in IRCnet.

Lions by Tambako the Jaguar

Meet Sandman on Friday (17:00-) at Hemingways

It's time to continue our "meet famous or not so famous people" type of meetings. This time there will be an opportunity to meet the Sandman.

All you need to do is to appear at Hemingways around 17:00 today (Friday).

Devaamo Summit + treweb - 14-15.6 @ Uusi Tehdas, Tampere

It's Devaamo Summit time again. This nice little event covers quite a range of topics (mobile, open source, whatnot). This year they will be collaborating with treweb unconference.

Yes, there will be talks about web tech but as it is an unconference, you never know what you are going to get (to quote certain Forrest). You can get some initial idea of the program already although there are still plenty of blanks at the time of writing. At least Jolla will be there so you can ask them some tough questions.

Of course the event is free but I think the organizers will appreciate if you decide to register. Esp. the latter day (Saturday) is filled with action. There will be two tracks and if some talk is boring, just change the room.

The first day is for more social ones although the lightning talks can be quite nice. In this case you'll have to organize some sort of accommodation for yourself. There are affordable motels at Tampere although I recommend avoiding a room shared with too many. Someone always snores.

If there are people going, it will likely make sense to coordinate travel and so on. I'll set up a thread at our mailing list. Also IRC (#geekcollision@IRCnet) will work. Let's hope we get a delegation together. :)
Copyright © Geek Collision