Search results matching tags 'Unit Testing' and 'Art Of Unit Testing'

Comments on Corey Haines’ String Calculator TDD Kata Implementation

Anonymous — Wed, 23 Dec 2009 21:26:35 GMT

Corey Haines recreated my TDD Kata on his KataCasts blog. (click here to learn more about TDD katas)

Since Corey is an experienced TDD practitioner I was honored that he’d given it a go, and was eager to see the video. Later, corey asks for my remarks on it, which i will gladly put here. In general, Corey’s video is a great piece of TDD work to watch as an example of a master TDDer doing his thing. Everything flows gracefully and the code produced is simple, readable and of course – works.

I especially loved the large amount of merciless refactoring Corey made during the Kata. As you TDD you should not cut any corners in the refactoring area – and almost after every passing test or two, Corey does refactoring to some very simple code patterns.

Another worthy note – Corey takes care in refactoring the tests as well throughout the kata. Kudos!

A few things that stood out for me can be thought of as very little things – and are mostly questions:

Partial Test: at some point, Corey seems to be writing the tests incrementally as well, in that he writes an empty test that passes, and then fills in the test itself to see it fail (which is usually where I start). I wasn’t sure why he’d do that, but the reasoning could be that he wants to make sure the environment works for the test. since he does it all in Ruby, which is a more forgiving environment for typing errors, this might be it.
Always passing test: very early on, in the second test actually, Corey creates a test that asserts that sending “0” returns “0” – which passes and never fails. That is because by default empty strings as input return zero. I do it a bit differently – i try to start with a failing test that drives more functionality, and if i can find a way to make a test for “0” fail as well to being, i will do it (so that i can ‘test the test’ in TDD style).
A different way: Don’t write a passing test for “0” input. instead write a failing test for “1” input. Another options would have been to indeed start with a passing test for “0” input, but then to go production code and deliberately return –1 to see the test fail, then fix it to return “0”, see the test pass again, and move on to the next test.
Magic inputs: Corey sends in various seemingly random, out of the air numbers as inputs to the tests starting from the third test onwards. For example, to a reader who sees a test that says “sending in 5 returns 5” that reader might ask themselves “why 5? why is 5 so special? should I also use 5 in my tests?”.
A different way: To avoid such confusion i like to stick to the lowest common version of an input that still proves the application wrong. if sending in “1” produces the same result as sending in “5” then my all means i will send in “1”. If the input is 2 digits, I’d send in “10”, “100” and so on. If the input still does not make sense I would put it in as a variable named “SINGLE DIGIT NUMBER” to explain to the reader what is so special or not so special about this value.
The simplest Test:Corey wrote the tests for inputs with \newline characters without considering the most basic test for this kind: a simple string containing only a new line character (equivalent to an empty string). I agree that not everyone would pick up on this as an idea, since you might feel this is an invalid input into the system. nevertheless, when I teach TDD this is the way i teach the kata. It feels lime a more natural incremental evolution of the code and tests, and a good way to send the message that anything can be divided into small increments if you want to.
Refactor to un-readability: At some point Corey refactors the code that handles delimiter characters into a single line of Ruby code. while it isa form of refactoring, i found that the code after refactoring was much less readable than it was before.

A different way: I try to avoid any “immediate if”s or single line “smart” statements that feel more “smart” than “simple”. When i have to choose between length and readability, i will choose length every time. this makes sure the reader is always comfortable with the code.

Summary:

These small things aside, I think Corey’s video is a great example of TDD in practice! Go watch it.

Test driven design – Willed vs. Forced Designs

Anonymous — Thu, 12 Nov 2009 21:59:48 GMT

I’m writing this as a typemock employee, but also as someone who has sat on the other side of the line for several good years, and can argue in both ways. The following, I feel, is true no matter where I work.

There are two ways people use tests to drive design, as far as I see. one is great, and I agree with, the other is not so great and I don’t agree with it. Sadly, both of them are categorized together these days, and the baby gets thrown out with the bath water - You either use both (BAD) or you use neither (BAD!)

Here are the two usage patterns:

#1 Willed Design

By writing tests, you can observe the usability of your design from a consumer perspective, and can decide whether or not you like it, and change it accordingly

#2 Forced Design

By using a subset of the available isolation frameworks(rhino, moq, nmock) or specific techniques *manual mocks and stubs) you discover cases that are not technically “mockable” or “fakeable” and use that as a sign for design change.

I highly agree with #1, and highly disagree with #2.

#1 makes sense.

You get to decide what is a good and bad design, and the experience of using that design from the test perspective is your guideline. But you make the rules on what you like and don’t like.

#2 is problematic for several reasons:

The tool decides. Not you.

You let an automated tool (rhino mocks, Moq etc..) tell you when your design is OK or not. That point alone should go against anything ALT.NET has ever stood for, doesn’t it? If you need a tool to tell you what is good or bad design, then you are doing it wrong. You should either know good design beforehad, or you shoud pair program together to find the best design, or you should learn by a mentor who can review your design mistakes, but don’t ever let a tool tell you what is right and what isn’t, especially when the only reason that tool works for that is by chance and not on purpose (as you’ll see in the next point)

A technical limitation that grew into something else

tools like rhino and MOQ and NMOCK just happen to support some OO ideas that seem good enough for design activities because of the underlying technology they use underneath. It’s pretty simple – they all use some form of either generating code at runtime that inherits from a class or interface, and then overrides methods on it (therefore they need to be virtual methods and non sealed classes, or an interface) or they use a proxy of some kind which underneath does pretty much the same things. (typemock is an exception since it uses a profiler api which has non of these requirements) . simply that means that the reason rhino, or MOQ or NMOCK “let you know” that you should use an interface somewhere, is a technical limitation of the tools, not a choice. Ayende was asked once what he’d do if he was technically able to fake static methods in rhino mocks – would he add that feature? “in the blink of an eye” he answered. and I agree. adding more options to the tool just extends the limitations of the possible design under test, not the “goodness” of it.

Languages like Ruby, Javascript, Python etc.. have isolation frameworks (or in some cases don’t even need such frameworks) that fully support any type of behavior changing, regardless of the design, since the language is less strict. yet, somehow, proper design arises in those languages tool. perhaps those languages are just “too powerful” and should not be used because they will cause you to do bad design? see the previous point for my answer.

What happens if tomorrow, or using C# 4.0 those tools get such abilities? will you all stop using them?

of course, you don’t have to use isolation frameworks to be dissuaded by the idea of limiting your design by using something – in this case a technique: using manual mocks and stubs in an object oriented language is just as “limiting” technically as is using one of those frameworks. You’re still bound to play within the simple laws of OO and using a design that is even a little out of place (even though it might make perfect sense for your application for security, performance or other reasons) is either untestable, or a general no no.

see the previous point for what i think about that.

the point here is that you’re using a technical limitation of a tool or a technique to tell you what to do instead of thinking for yourself and learning proper design guidelines. that limitation just happens to be somewhat partially consistent with what you might currently to believe to be true for design. but technically, it is a limitation that could end soon. when it changes it’s behavior, will you just change your design guidelines? switch or won’t upgrade to a new version of the language? or actually start using your head and your peers to see what’s right and what’s not?

The Typemock Dilemma

Typemock gets a lot of flack for not inhibiting the design of the program, and I can see how people would be afraid to lose that limitation in other tools, since all they head from alpha geeks in .NET is that if it’s not “testable” then your design is wrong. worse, they hear “if you need typemock your design is wrong”.

there’s nothing a silly as absolute “fact” theories in the software world. In fact, let me go out and say that all fact theories are wrong. How that’s for irony?

The message should be, I feel, more like “here are some principles of good design as we think of it today”, but instead it is based on tool choice and not on technicques or craftsmanship.

Unfortunately, I don’t think getting rid of #2 is possible in .NET today without using a tool like typemock, and that’s a shame, since it means that, because it costs money, people in the community will still want to use the free tools, which force design, instead of allowing them to decide on it like the mature developers they are.

Maybe it’s time to have some sort of free version of Isolator so that everyone can benefit. what do you think?

Minimizing unit Test Fragility – 8 features in Typemock Isolator to help

Anonymous — Thu, 09 Jul 2009 22:52:50 GMT

in continuation to my challenge (which no one had bothered answering, lazy web!)

One of the things that inhibits unit tests in organizations is the idea of fragile tests. a Fragile test is a test that can easily break when the production code changes. That’s not to say that tests should never break as you change production code, but there are ways to minimize this effect so that when tests do break, they break for the right reason.

the right reason is when a feature in production code isn’t working.

the wrong reasons can be many and varied. Of the most common ones is the idea of over specification in tests. The more your test expects of your code’s internal implementation, the more “specified” it is, the more “expectations” it has on the internals of the code. since internal code is more prone to change, so will the test break more often. so the situation can occur that the production code changed, but still gets the job done, but since it’s doing it differently internally, the test breaks even though the end result is still OK.

One of the main ideas in the new Isolator API was to reduce the fragility of tests by default. there are several features in Typemock Isolator that remove lots of fragility from tests by default, compared to other tools:

All Fakes are non strict by default.
Recursive Fakes
Method stubs ignore parameters by default
“From now on” Method behavior as a default
Isolate.Swap.AllInstances
Isolated.NonPublic APIs will not break when refactoring from private to public
Ignore one or all static methods on a type
True Properties

All Fakes are non strict by default.

when a fake object is strict, then it will throw an exception when someone calls one of its methods that wasn’t specifically “expected” by the test (or a different number of times than expected). since internal code can change and start interacting with other code in different ways, strict fakes will almost always fail your test on the lightest nuance of change. Most frameworks used to have strict fakes by default.

Isolator has all fakes non strict by default. in fact, there is no “strict” mode at all.

var fake = Isolate.Fake.Instance<SomeType>();

//code under test can now call any method without the test needing to specify it. the test only needs to specify behavior of methods it cares about.

Recursive Fakes

This is a feature first introduced by Isolator more than a couple of years ago and then adopted by Moq and Rhino Mocks. the idea is simple : allow chained calls in your code without the test needing to specify fakes at each hierarchy level. for example, calling MyFake.SomeProperty.DoSomething() will, by default, do nothing since its root is a fake object. the “SomeProperty” will be initialized by default to a fake object as well, recursively down the line.

With other frameworks this will only work with interfaces, methods which are virtual and classes that are non sealed. Isolator can make this work with anything, including static methods and non sealed classes.

var fake = Isolate.Fake.Instance<SomeType>();

//code under test can

fake.SomeProperty.DoSomething()

//or

var propObj = fake.SomeProperty;

propObj.DoSomething();

As your internal code is refactored, or changed to call a method down the hierarchy, this should not break a test (as long as the method does not matter to the test)

Method stubs ignore parameters by default

all other frameworks except Isolator will use the values being sent to methods of fake objects during the “setup” phase, as expectations of these actual values. Isolator by default ignores values being sent in, and just returns the fake result we tell it to, no matter what was being sent in. Of course, you *could* tell it to use the exact argument values, but that is not the default behavior.

other frameworks will require that you put in special constraints on each parameter value to say that you don’t care about the actual value.

var o = Isolate.Fake.Instance<SomeType>();

const int IGNORED_VALUE=10;

object IGNORED_OBJECT= null;

Isolate.WhenCalled(()=> o.SomeMethodWithParams(“ignored string”,IGNORED_VALUE,IGNORED_OBJECT) ).WillReturn(3);

this method will return 3 no matter what parameters are being sent to it. so internal code can change parameters slightly and it will still work.

by specifying only on specify param values, we could be over specifying the test.

“From now on” Method behavior as a default

with Isolator, once you set behavior on a method (“Ignore”, WillReturn(..), WillThrowException(..) ), it will use that as its default behavior for all subsequent calls. That way, if the internal code ends up calling a method more than once, the test should not care about it (as long as the result does not hurt the test). This is great for ignore calls to “Log” objects and such, where you don’t know or care how many times it will be called. you just want to ignore them without having to worry if the code will use them or not.

for example, the method from the code above will *always* return 3, no matter how many times it will be called.

if we specified it twice with different return values, it will return the *last* behavior as the default, after it was called once with the previous behavior.

Isolate.Swap.AllInstances

This beauty will replace all existing and future instances of a specific type in the code under test with the fake type. that way you don’t care how many times some object might be created, or how many instances of it are used. for “util” type objects, this makes things a breeze to setup for a test that could be very complicated or impossible in other frameworks.

as the internal code changes with usage of these types, the test will still pass.

var fake = Isolate.Fake.Instance<SomeType>();

Isolate.WhenCalled( ()=> fake.Foo() ).IgnoreCall();

Isolate.Swap.AllInstances<SomeType>().With(fake);

//now production code can do this:

var x = new SomeType();//<—this will behave like the fake object we created

Isolated.NonPublic APIs will not break when refactoring from private to public

Isolator allows setting method behavior on non public methods as well:

var x = new SomeObject();

Isolate.NonPublic.WhenCalled(x,”SomeMethod”).WillReturn(3);

The beauty is that if you do refactor the method to make it public, the test won’t nudge you or throw an exception. It will just work with public methods as well. You can then refactor the test without being a slave to it.

Ignore one or all static methods on a type

Isolate.Fake.StaticMethods<SomeType>() will ignore all void methods on that type, and return recursive fakes on all functions. That way, if you add more methods to that type, the test won’t care about it. great for utility style classes with lots of static methods.

True Properties

“True properties” is a feature that Isolator picked up from Rhino Mocks – you can set properties on a fake object as if they were real properties.

The nice thing is that if internal code then sets these properties to values you don’t care about, the test will work just fine. it’s also very readable.

var fake = Isolate.Fake.Instance<SomeType>();

fake.SomeProperty=3; //will behave like a real property and return 3;

Art of Unit Testing on Hanselminutes

Anonymous — Tue, 07 Jul 2009 12:28:40 GMT

Scott Hanselman took me in for a 30 minute interview about Unit Testing Dos and Don’ts in his podcast, Hanselminutes. It was a pleasure, and I hope to be there once more about other topics. Maybe in TechEd Berlin..

Have a listen

Testing that an event was raised

Anonymous — Sat, 23 May 2009 18:25:20 GMT

This question keeps coming up: “How can I test that an event was actually raised from my class under test?”

actually, there is an easy way to check if an event was raised.
you subscribe to the event in your test, and in the event handler you set a boolean flag to true. then you just assert on the flag.
here is a quick example:

Code:

public void Test()
{
bool wasRaised=false;
var button = new Button;
button.Click += ()=> wasRaised=true;
button.DoSomethingThatShouldHaveTriggeredTheEvent();
Assert.IsTrue(wasRaised);
}

If you feel less comfortable using lambdas:

public void Test()
{
bool wasRaised=false;
var button = new Button;
button.Click += delegate

{

wasRaised=true

};

button.DoSomethingThatShouldHaveTriggeredTheEvent();
Assert.IsTrue(wasRaised);
}

Unfortunately, with VB.NET’s current version, doing this in a single method is next to impossible, so you are forced to register to the event with a method at the class level, and check that:

Code (VB.NET):

dim wasRaised as Boolean=false;

public sub test()

wasRaised=false
dim button = new Button()
AddHandler( button.Click , AddressOf(OnClick))

button.DoSomethingThatShouldHaveTriggeredTheEvent()
Assert.IsTrue(wasRaised);

end sub

public sub OnClick(source as object,e as EventArgs)

wasRaised=true

end sub

Art Of Unit Testing (The Samurai Book)– Get it now, it’s done.

Anonymous — Wed, 20 May 2009 21:22:14 GMT

The time has actually come. After 2.5 years, and two kids, my book is finally done and is available in full form as an EBook. at the end of the month it will be in print form. Now would be the time to get it, when it is still at a “pre-order” pricing. Get the preorder price either at Manning(along with the EBook) or at the amazon page.

I love the cover image. How about we call this “The Samurai book” from now on?

This is the book that I wished I had when I started out writing my first tests, and that combines my knowledge about unit testing from the past 5-6 years or so working with companies on real projects (and real failures).

It contains things I have not seen in other places – writing readable, maintainable and trustworthy tests, as well as guidelines on how to review someone else’s tests and what to watch out for.

I was lucky to have the foreword written by no other than Michael-Legacy-Code-Feathers himself, and with some great quotes including one from Kent Beck about the book.

Here is the Table of Contents.

Free Chapters

The book comes with two free chapters:

Chapter 1 – The basics of unit testing(PDF)

Chapter 3 – Using Stubs to Break Dependencies (PDF)

Or get the full book

Book description:

Unit testing, done right, can mean the difference between a failed project and a successful one, between a maintainable code base and a code base that no one dares touch, and between getting home at 2 AM or getting home in time for dinner, even before a release deadline.

The Art of Unit Testing builds on top of what's already been written about this important topic. It guides you step by step from simple tests to tests that are maintainable, readable, and trustworthy. It covers advanced subjects like mocks, stubs, and frameworks such as Typemock Isolator and Rhino Mocks. And you'll learn about advanced test patterns and organization, working with legacy code and even untestable code. The book discusses tools you need when testing databases and other technologies. It's written for .NET developers but others will also benefit from this book.

What’s inside:

Test Review Guidelines
How to create readable, maintainable, trustworthy tests
Stubs, mock objects, and automated frameworks
Working with .NET tools, including NUnit, Rhino Mocks and Typemock Isolator

Test Review #3 – Unity

Anonymous — Mon, 23 Mar 2009 05:58:55 GMT

Watch previous videos:

In this video I go over the tests for Microsoft Unity Application Block.

Overall the quality of the tests in Unity is pretty good! I could certainly recommend that people look at them as examples of a bunch of tests against a framework, which are mostly very readable and maintainable.

Things I walk through:

Using [Ignore]
exploration testing
magic numbers and strings
un-needed asserts
Stub hard coded behavior
handling config files in tests
Asserts hidden in a utility method
more naming conventions
Separating integration tests
“smart” code that is less readable
Meaningless “isNotNull” tests
cleaner ways of expecting exceptions

Test Review #2 – ASP.NET MVC Unit Tests

Anonymous — Sun, 22 Mar 2009 03:42:38 GMT

See other reviews:

Review #1: NerdDinner

Here’s the second video review of Unit Tests. This is another one written by Microsoft – ASP.NET MVC (source).

First, it’s important to state how surprised I was by the high quality of the tests in MVC. The tests are readable, maintainable and trustworthy, with very little issues that I could find. whatever Issues I found are rather easy to fix. In any case, if one is looking for examples of systems written in what seems almost entirely in TDD, or at the minimum with very good testing guidance, ASP.NET MVC should be a good stop to look at.

Issues discussed in this video:

Implementing RowTest with MSTest, and the importance of naming (14:00)
Verify() that is splitted from the mock expectations (17:00)
some naming conventions
Over-specification in tests (mainly more than one mock object per test)
verifying mocks when it’s not required
logic inside tests (concatenation)
test factory methods with too much logic
a very good example of when multiple asserts is really bad (11:50)

Again – I’m very pleased with the test quality. Now is the time to make sure the things above are fixed. they are still important!

Test Review #1 - NerdDinner

Anonymous — Fri, 20 Mar 2009 11:45:53 GMT

I’ve decided to start doing some test reviews of tests that I see in the wild. I figured it’s the best way to show people what I mean when I say they do not have readable, maintainable or trustworthy tests.

The first episode is the review of the tests from the NerdDinner MVC source code. It’s 30 minutes long. and it was shot at 2AM, so I’m quite cranky. But the tests sure don’t try to ease my mind, and only make me crankier.

If you have tests you’d like me to review send an email to Roy AT osherove.com with the subject “Test Review [project name]”

problems that are dealt with in this video:

Naming conventions
Magic strings in tests
Hard coded values in hand written stubs
Magic assertions (expecting a value that no one understand why it should be that value)
Lack of tests

Isolator Feature Focus: Duck Typing and Isolate.Swap

Anonymous — Sun, 02 Nov 2008 12:25:00 GMT

Typemock Isolator 5.1.1 has been released, and this release brings with it some awesome (seriously) features that are unique fro any other framework I've seen.

a good overview of features can be found here.

The Isolator Swap feature allows swapping calls between real and fake objects (kind like redirects) so that any relevant calls made against the real object will be redirected and invoked on the fake object.

Here's how you use it:

Unlike doing a standard "WhenCalled()" on some object method and telling what its custom behavior will be, "Swapping" objects redirects all relevant calls(I'll explain what "relevant" means in a second) to the fake object which you have created.

Duck Typing Awesomeness

Another cool thing about it is that "Duck" and "Dog" don't have to have a shared interface or base class. The "Swap" feature will redirect a method call if it exists on the "fake" object, but if it does not, it will invoke the original object. This is one interpretation of what's called "Duck Typing".

The "CallsOn" and "WithCallsTo" methods take an object type, so you can send in anything you want. if a method on the fake object matches the signature and name of a called method on the real object, the fake method will be invoked.