Search results matching tags 'General Software Development' and 'Agile'

Goodbye mocks, Farewell stubs

Anonymous — Sat, 20 Sep 2008 12:45:59 GMT

About 4 years ago I was consulting for one of the largest companies in Israel (about 15,000+ employees) and helping them grok TDD and automated unit testing. I had spent there over a year and a half, on and off, and was teaching both .net, C++ and java developers about the techniques and tools they can use.

I was consulting for C++, Java and .NET developers.

The C++ devs were first.

They had plenty of objections to what I was teaching them, because the amount of code refactoring they needed to do in order to make their code testable was almost unrealistic for them. Even writing new code meant writing something on top of old-untestable code, and they had a hard time adjusting to the more interface (pure virtual) based, contract driven TDD approach for new code. C++ devs has a hard time becuase it takes more code to write good testable code, and the coding tools they used (VS on windows and VI and such on UNIX based systems) didnt’ provide a good productive way to make code “bend” easily as they refactored it or created it in the first place. every little change meant a big “search and replace”. every method refactoring meant adding it the declaration in an .h file and thus they tried to have a little to do with changing code as possible.

Looking back, today they still don’t have the right infrastructure in place or the right tooling or the willingness to do unit testing or TDD becuase it’s too hard with the current tooling to accomplish it.

I had also spent some time with the .NET devs

and thought them the same things. Becuase tooling support was better it was a bit easier for them to accept the practices, but still people had a lot of trouble grasping concepts such as dependency injection, inversion of control and designing for testability in general. I found that most people didn’t really have a big problem with the idea of exposing things they didn’t use to expose before, it was more about the concept of learning how to refactor the code in interesting OO ways to make it testable. Making a class non sealed, a method virtual, and then inheriting from that class and testing against the inherited class (“extract and override”) was very much unintuitive for them, and other concepts like containers (spring at the time) didn’t help them get into the groove either. For them it was like drinking water from a fire hose.

Consulting for the java devs

Java has lots going for it in terms of unit testing, automation and TDD

Java devs have a much longer “heritage” of tooling and unit testing than in the .NET world, and so you’d expect them to be more inclined to do these things.
JUnit is part of any decent java idea so the barrier for entry should be lower, right?
Refactoring was also in most decent java IDE's, so you’d expect them to grok that too. but that wasn’t the case.
class methods in java are virtual by default so you can override what you want as long as the class is not sealed (final)== more testable by default.
Java required a much higher OO learning curve to start with. Good OO knowledge is fundamental for testable designs.

Unfortunately, most developers I met didn’t even know how to use the refactoring tools they had available and had a hard time with the same dependecy injection principles that the C++ and .NET devs had. It was just too cumbersome to make the code testable, but eventually the infa team at the company had made the decision to take a couple of weeks and make all the infra stuff replaceable for testing purposes. they did not regret that decision.

So the java group had the most success with testable designs, but they still had a hard time doing it. in fact, only the infra team at that group took the plunge. the other (higher level) devs in the java group didn’t grok it. and don’t use it today.

Why was this adoption been such a failure? They were willing to learn, but they didn’t like what they learned. It didn’t fit their needs in the real world. Or the way they worked, or the way they knew how to do things.

Think what you will ,but 99% of the dev worlds aren’t ready for the “testability” message. and that is a fact that is hard to realize for most of us, which just can’t grasp why isn’t that any sensible human being isn’t willing to do everything much better than they are doing today.

Spending the past week running around the Microsoft Campus in Redmond had made several things clearer for me. I’ve been feeling them for a while, but this week these abstract feelings materialized into something that is more coherent that I can talk about.

Why do most developers not write unit tests, still?

Reason #1: Learning curve

One of the main reasons most developers still today don’t really take to unit testing (and TDD) is the really high learning curve that is forced upon them. That is not to say that learning about

mocks, stubs, dependency injection, IoC containers, Extract & override technique, Record\Replay, AAA and more

-- is not useful, but it is a big obstacle to get people into the habit of doing something which they know is good for them – test their code, verify their assumptions, automate, integrate, be confident, get feedback, define behavior – all those and many other benefits are being thrown away by developers all over the world because the entry fee to this world is too high, for all the wrong reasons.

It’s time to simplify. Get back to basics. Take a good hard look at the huge spiral that we are leading people into and asking whether removing some things today can lead to those things being accepted and being easier tomorrow. or maybe they weren’t needed at all.

Say Goodbye to “mock” and “stub”

“mock” and “stub” have played their part in us trying to explain to ourselves what it is we are trying to do, to start learning the minute and important differences in the patterns of our test code. They helped us define what it is we are looking to do, in terms that, for lack of anything better, are now the de-facto pattern names for something that should never have been named in the first place for the casual programmer.

That’s why in the new version of Isolator we’ve chosen to completely get rid of the words mock and stub, and get back into “fake”ing. You can fake a method or a class instance, and you can verify that something was called. You should not care whether this is a stub or a mock.

We need to clean our language: mock, stub, fake, test double: They all mean something in the context of us trying to identify patterns in our test code. But as users mature and start using the frameworks, those same words increase the entry barrier, and effectively block people out. If I’m showing a simple test that fakes some dependency that a class uses to a new dev on my team, I should not have to send them over to read fowler’s “mocks arn’t stubs” article (which is enlightening, but should not be relevant to them) because it misses the point.

Less interaction testing, more state based testing

Most crappy unit tests I’ve seen were over specified in that they asserted on internal interaction with an object way too heavily, which made the tests very fragile. Getting back to basics means making it easy to enable state based testing (assert style checks) and only enabling interaction testing where it makes sense: (when your app does one-way messaging to an external component). verifying that something was called should be the exception to the unit testing rule. The AAA style APis make that a more clear distinction.

Record-replay APIs make it too easy to make interaction based tests when you really want a state based one (getting a result vs. checking that something was called)

Use AAA style Apis

As an industry we are learning and changing all the time. The “Arrange-act-assert” pattern in the field of mock frameworks represents us trying to return to basics. Moq did that nicely by showing a way that leads there, and it is being taken gladly by Rhino and Typemock Isolator. It is taking off because we are starting to look beyond our own little echo chamber and seeing what is is that all those other 99% of devs are expecting as guidance, and in what form.

AAA is guidance because there was an outcry (a silent, vote of feet) that record-replay is not cutting it – it’s too confusing for non-experts. It’s too technical. and it does not lend itself to the way people want to work. So most people didn’t use it.

Visual Basic Language Support

Everyone screwed up on this one.most VB developers who will try to use one of the current frameworks will not succeed since they required anonymous methods or lambda expressions. Some frameworks use VB specific keywords which makes them VERY awkward to use. If we want broad acceptance of our practices we need to make sure whoever wants to can use the tooling we make. VB is huge and its amazing how we all pretend it does not exist. But I think it’s the key to getting to those people who want to be better but all they see is a high wall instead of an entrance.

Use “Isolate” and “fake” instead of “mock”

Don’t say ‘mock’ frameworks, say “isolation” frameworks. Because ‘Mock’ is an overloaded word it is confusing. Explaining to a developer that you can “isolate” something from its dependencies rather than “mock” its dependencies feels more natural.

Simplify,Simplify,Simplify

Remove APIs instead of adding to them. Have single point of entries to your frameworks because discoverability is the key to good framework use. That’s why extension methods are problematic, they mean multiple points of entry.

Be open: Realize that other tools and techniques can act as bridges to the agile world

Design for testability is just one way to get where you want to go. it is not the only way. Sometimes it’s impossible. Discouraging the use of other ways of working (using tools like Typemock Isolator or JMockit) will only make the current echo chamber more closed and feel more un-achievable for those who need to get to the same point with a very different set of constraints than what most of the “loud” voices need to face.

By “allowing” other forms of working you enable those who want to, to get a bridge to help them instead of walking by foot the whole way. Any other way would mean that we will all fail in delivering what we believe in to those who have no idea what we’re talking about.

Typemock Isolator with AAA syntax

Anonymous — Thu, 24 Jul 2008 07:34:15 GMT

We've just posted a preview of the next Isolator API coming out in august. It is AAA-style (Arrange, act, assert) and has many changes. We need your input so head on over, read it and tell us your thoughts. This is the time where you can make a huge difference. anything you don't like, say it in the comments.

A sneak Preview of Typemock Racer : Thread Deadlock Finder

Anonymous — Tue, 08 Jul 2008 15:42:28 GMT

Here's a sneak preview of the upcoming Typemock Racer product, which I'm currently working on. We should be out in private CTP in a couple of weeks I hope.

Typemock Racer tries to solve one huge problem for developers working in multi threaded environments: Detecting code that can result in deadlocks or race conditions. It does this by providing a framework API which you can use in your tests (under NUnit for example), that allows you to execrise your code under test until it finds a deadlock or times out.

Here's an example of how you'd use it. Consider the following class which has two methods, that if run by two seperate threads, can result in a deadlock condition:

At some point in time the threads could execute such that thread 1 is waiting on resource b, which is held by thread two , which is waiting on resource a, held by thread 1 - a pure deadlock.

Given this class we can write a test that looks like this:

The ThreadTest class has an "AddThreadAction" method that receives a delegate. you can use it to invoke your own code, it will be invoked in a different thread. IN this example we are creating two threads, each one running a different method on MyClassWithLocks.

We can just right click and run the test with TestDriven.NET to recieve the following results:

The output states that there is a deadlock that was found, with the exact steps that happened (with line locations) to reproduce this.

It currently only works with lock (monitor.Enter\Exit) but will support all locking constructs in .NET.

What if the methods you point to create their own threads? Like this:

Here is the output that you'd get if you run this test:

Two Faced Commits - how the ALT.NET Community is becoming more and more dogmatic

Anonymous — Mon, 19 May 2008 10:50:32 GMT

disclaimer: I work at Typemock.

There is a big trend I am starting to notice in the "Advanced" unit testing community. That trend feels more and more like hypocrisy. Take Daniel's post about mocking statics without Typemock. (Daniel is the creator of Moq, an open source mocking framework) Granted, it is important to know that you can always refactor things for testability, but just looks at what he writes:

"So, don't buy a product just because you're lazy and want to avoid creating a few wrapper classes and interfaces"

followed by

"Always use the simplest thing that could possibly work."

Which is it, community? The idea of being prgamatic is using the right tool for the job. if that tools makes your job easier than not using it "cause you're lazy otherwise" is plain old hypocrisy.

The same argument also holds for tools like FinalBuilder vs. Nant\MsBuild\Other XML Hell. Tools don't create problems if you use them correctly. The community (especially the ALT.NET community) is going in a too extreme direction. so extreme that their advice are sometimes skewed too much to be considered pragmatic, sometimes falling into "dogmatic".

View the forum thread.blog comments powered by Disqus

Project White and a cool debugging tip

Anonymous — Sun, 24 Feb 2008 20:41:18 GMT

Some cool stuff happening in the testing world:

* Project white seems like a nice takeoff on winforms testing (a la NUnitForms), from the thoughtworks Guys. Bil Simser talks about it some more.

* Gregg Miskelly came up with a brilliant debugging trick on his blog. He shows how to set a breakpoint to stop only on a specific object instance using Object Ids during Debugging. (via BugSlayer)

Trying out Team City - Looks Promising!

Anonymous — Sat, 19 Jan 2008 00:36:17 GMT

I've been looking both at CI Factory by Jay Flowers, and Team City by JetBrains lately. this has to do with the fact that at Typemock we currently have a Continuous Integration solution that is largely based on CruiseControl.NET and MSBuild files.

It sucks.

The MSBuild files are hardly maintainable and the logic for running the build, signing things, obfuscating, licensing, installation, 64 bit versions and more - it's chaos, held by some gum and a couple of strings. Not unlike many other builds I've seen.

We're in the process of moving the builds into FinalBuilder, but I'm also looking for a better CC.NET. CI Factory looks nice in that it is very simple to setup but still requires messing around with MSBuild targets and the like to make your builds work with it.

TeamCity looks very promising. I read this blog post from Elutian about their experience with Team City and wondered what it would be like to work with it against a FinalBuilder script. I installed TeamCity and ran it against SVN and FinalBuilder (like it would run at Typemock) and got it all working in less than 20 minutes. Pretty snazzy and no XML tampering needed to take place, Thank the gods.

So, I'm thinking, with team city's professional license being free for up to 20 developers and 3 build agents, that we'll give it a shot at Typemock HQ and see what happens.

The evolution of Unit Testing Syntax and Semantics

Anonymous — Thu, 17 Jan 2008 11:21:59 GMT

The semantics of how you write a unit test, the basic syntax, or Domain Specific language of how we write them, has been relatively stale for quite some years now. But under the covers, the syntax revolution seems to be brewing, as people try to come up with newer, hopefully better and more readable syntaxes for tests and specifications that become more and more complex as the current Agile community struggles to accept best practices in this arena.

I'm guilty of that too, but a recent post by Jeremy Miller prompted me to try and list all the different syntax variations I've come across in the .net and some in java world (not necessarily in chronological order). Most ofthese didn't necessarily catch on as de facto standards, but are real world attempts to make a better world or explore new horizons.

POUT (Plain Old Unit Test) Syntax

JUnit initial syntax
Attributes: NUnit with attribute syntax, removing inheritance and test naming requirements
Extensibility: MbUnit is extensible testing framework with lots of powerful attributes.
Naming conventions for unit tests seem to be something people are actively looking for.
Simplicity is back: XUnit tries to go back to basics: No setup\teardown, different attribute names.
Fluent: My implementation of NRehersal uses delegates and a more fluent interface for tests

Mocking

Mocks Vs. Stubs: The Difference is explored (here too) and makes its way into some of the mock frameworks (NMock, JMock, Rhino.Mocks)
Explicit (reflection based) mocking statements such as (New DynamicMock(typeof(Interface), mock.Expect(...))
Record-replay syntax from Rhino.Mocks, EasyMock, TypeMock and others
fluent mocking APIs into our tests (NMock2)
Auto Mocking Containers (introduced by Elutian) will use RhinoMocks and Ioc Containers under the test covers (back to base classes)
Mocking the Un-Mockable: TypeMock (and JMockIt on the java side) introduce the possibility of mocking production code that is currently untestable without needing to refactor it.

BDD, Specifications and other Domain Specific Languages

Ruby SpecUnit, RBehave and RSpec make a dent in the .NET community and stuff start showing up in our world:
Extension Methods: Scott bellware comes up with the idea of using C# 3.0 extension methods for an easier expectation syntax.
NUnit.Behave, NBehave, NSpec , NSpecify try to use DSLs and fluent interfaces to express BDD specs.
Jeremy shows off his version of a more fluent and natural spec syntax by taking the auto mocking container idea, using structuremap under the covers and creating something that tries to be much more declarative and understandable at the same time. He uses delegates a lot as well.

I'm probably forgetting a few here, so let me know in the comments.

So, we see a nice evolution from beginner style to bdd and spec style. lots of work on the syntax is done, but there is currently lots of chaos. Think Blue-Day Vs. HD-DVD, only with about 5 main competitors. The nice thing about this: The community will decide what worls, so it would be interesting to see where we are one year or a two years from now in relation to this list!

Organization Problems - Sounds Familiar?

Anonymous — Thu, 18 Oct 2007 13:08:03 GMT

I just finished a summary of a company's review about work modes and problems before we can start doing something about it. I'm posting part of it as anonymous because I think this seems to relate to many other companies out there:

"The QA Department seems to be the main gripe of most of the departments in the company. The main issues are of it being staffed with personnel who does not really understand the application aspects of what they are testing, and so don’t really know what to test in the application. Another main issue is the lack of knowledge about proper QA methods. Each QA department seems to be reinventing the wheel and creating its own methods. This also leads to the face that most tests are not automated and most tests don’t even get repeated for regressions after each release, which leads to an overall lower quality in the product.
...
The Marketing division seems to be a large source of problems. Most of these come from the fact that Marketing people also serve as Product Managers in the company (Those who should be truly representing the client’s real needs), setting feature priorities and agenda for the next release with ultimate decision power. Because there is a conflict of interest between the role of marketing and role of Product manager, many of the things that are requested from other departments such as Customer Support are considered haphazardly if at all, which leads to a product that contains features a real customer might not need, but misses features the customer really does need. Add in a true product manager role into the company can help alleviate many of these issues as well as the partial amount of time that marketing has of actually communicating with all relevant departments as a customer.
...
The X group is highly motivated in implementing agile practices such as unit testing, and are suffering from lack of automation in the group, “islands” of knowledge and lack of code reviews.
...
Management has a big role in creating the current conditions for problems. The lack of CTO, and the fact that the CEO is the one who is “acting CEO” lead to lack of real technical leadership (lack of time to handle things) and no clear “head” to turn to and get quick good answers. Management can help by supporting practices such as unit testing and automation and getting and hiring good people into the missing roles in the company (Product managers, CTO) and by allowing for real time extensions to projects that would like to implement them.

...

Generally speaking, there are various “pain points” that currently prohibit the departments from either getting the full data they need, influence the current releases so they contain the correct feature set or be more productive in their work. A main example is the big confusion about the various issue management software: which is reported to be almost unusable and there is no clear sync between that and X and Y issue management software also used in the other departments. Also the fact that developers aren’t really aware of what the application is used for in the field leads to believe that more time spent with the customer can help. Overall many “small” things can make for a large difference in the organization’s ability to create quality software and get better communication between the departments more quickly.
"

Things I learned at JAOO 2007

Anonymous — Thu, 27 Sep 2007 11:37:57 GMT

Martin Fowler may be one of the great minds of our engineering age, but his accent sounds a lot like MR. Bean. He's also not a great social figure and also sems to speak when spoken to. Eccentric.
Robert C. Martin is a good Drummer. He played the drums for me while I was doing some of my songs at the first ever JAOO Jam Session. He's also working on a new book called "Clean Code". I suggested he might want to take a look at Code Craft so that he has some differentiators from it. I still don't get why people call him uncle bob, but he says a friend of his called him that once and it stuck. From now on just call me Uncle Roy.
Jim Coplien seems to bring out the worst in me. In his session about Scrum and Architecture he publicly denounced TDD and said (quoting) "TDD will deteriorate your architecture" and spoke loudly against using it. Him outspokenness seems to have hit me where it hurts because it led to a full hour of me and him going at each other outside the lecture hall with people gathering around as if it was a boxing match. The day after that I decided to go to the Open Spaces sessions at JAOO and open up a new topic called "TDD+Scrum=Good?". about 12 people showed up and Jim was one of them. For more than an hour we all exchanged ideas and some back and forth took place (lots of banter). I'll have a whole separate post about this discussion later on. The bottom line is - TDD can be misunderstood which can lead to people saying "don't use it" which can lead to people actually listening to this very bad advice which can lead to world chaos and the eventual death of the world. So don't listen to Jim.
Mark Polack is a great guy to talk to and is the main driving force behind Spring.NET. Spring.NET is much more powerful than I initially thought. He's working hard on V 1.1 of that these days. He did a session on Spring.NET at JAOO. He doesn't blog enough. In fact, I can't even find his blog.
Hamilton Verissimo is the main force behind the Castle Project and is a very smart and shy person. he also is a good listener and thinks twice before saying anything. I would love to talk to him more but I always fear I will say the wrong thing next to him (he feels so polite!). He's the opposite of the Israeli state of mind.
The Danish crowed is just as quite and polite as the Tech-Ed Europe crowd or the DevDays Belgium Crowd.
I really like the voting system on the sessions. As you leave you get the choice of taking a yellow, green or red note. you pick one and drop it in a big bucket as you leave the room. then they count the reds greens and yellows. very agile although I suspect they may be missing some important specific feedback this way.
A Guitar that starts out with you as flight baggage will not necessarily find its way through 3 connection flights. It may arrive 3 days later as you prepare to leave back home.
There are some great people at JAOO and I got to play with some of them at the JAM session. great idea!
Free wi-fi in your hotel room will allow you to run utorrent so that when you leave back home you will have downloaded lots of nice TV shows to view on the place ride back home.
Erlang is making a comeback. It's been there 20 years mainly used by the telecom industry for parallel high volume stuff. now it's the talk of the conference as the answer to multi-core and scalable application programming. I don't remember who told me this nice sentence but it went like this: "I guess its true what they say. Everything has already been invented 20 years ago". true.

The Art Of Unit Testing - Available to Purchase Online Now

Anonymous — Wed, 26 Sep 2007 15:49:42 GMT

I'm happy to announce that you can now purchase my book The Art Of Unit Testing through the Manning Early Access Program. Currently, the first five chapters are available, but when you purchase the MEAP version you get the new chapters as they come in, before anyone else. Also, you can help make the book better and more accurate as a MEAP reader since the author forum can be used to discuss book topics you find interesting and I will be reading it at least twice a week (hopefully daily).

What's with the ugly cover art? I'm working on it. We had to have something to start putting this online. The final print will have something which I can actually stand behind.
What are the first five chapters about?

The way I see it, the book is divided into several parts. the first part (chapters 1-5) are about the basic tools you need to know to start writing unit tests in the real world. Everything from working with NUnit and the basic attributes of unit tests, and up to using fake, stub and Mock objects. There is also quite a full reference for using the Rhino.Mocks framework. The idea is that you can take someone with no unit testing experience whatsoever in .NET, and have them read those five chapters. then they have the basic knowledge to actually know what you're talking about when you say "Mocks" or "integration testing". They may even be able to write some real world tests using these chapters, even interaction tests.

Part I Getting started
1. The basics of unit testing - FREE
2. The first unit test - AVAILABLE
Part II Core techniques
3. Using Stubs to break dependencies - AVAILABLE
4. Interaction testing Using Mock Objects - AVAILABLE
5. Mock Object frameworks - AVAILABLE