Putting on your Test Hat

I have noticed a real divide between development and test in every company I have worked in. Perhaps this is an Irish attitude? I say this because I don't see much about it coming out of the US in any blogs I follow etc. The tell-tale sign I most often see is a really poor attitude from developers towards the QA team. There is often arrogance and border-line bullying taking place. It can be summarised as "Devs are better than test" and "test should do what we say". There is a real under-appreciation for the role of QA and SDET and a belief that QA are somehow below Developers - because if they were any good they'd all be developers right? This attitude completely stinks.

I have worked in a bank, automotive diagnostics, enterprise software and consumer web and I have found this attitude present in all industries to varying degrees. "Throw it over the wall" is commonly heard in teams like these. Devs write some code, check it quickly in whatever version of whatever browser they happen to have open and throw it over to QA to test and move on to their next task. They are then surprised when there are several bugs opened against the code. I often see devs try to influence QA or weasel out of responsibilities. "It's not a bug really", "We don't have time to fix that now" etc. This is typical. There is a quite a lot of time wasted logging bugs and screenshots, communicating and tracking issues and the actual re-work to fix issues that could have been prevented if the developers cared more about testing (not just unit testing, although that would be a start...). Testing is not something that is solely the responsibility of the QA team. If you are developer and you do not test your code then I do not want to work with you.

 I found it very interesting reading the following blog post on Google Testing Blog that Google Developers perform testing to a certain standard before handing over testing to the Software Engineers in Test. They call this "testing 1.0" using tools such as WebDriver and FIO. This raises the bar on quality and reduces the amount of re-work necessary (the schedule-killer...). I love this idea. We are all one team and we are all responsible for creating the best product we can to the highest standard we can. There is no "throwing it over the wall" in a team like that. I will push to add some testing 1.0 to our criteria for committing new code. The developers will benefit from putting on their QA hats and gain an appreciation of the subtleties of testing, not to mention reducing the amount of bugs they will have to work on.

We run fast in the company I work for. We run fast and sometimes quality suffers. Sometimes bad things happen. We fix them and move on. Keep moving forward, keep shipping. The most difficult part of introducing the type of changes that would improve quality is convincing people who enjoy the fast-paced nature of the culture that they need to slow down a little. This is a tough sell to some developers and senior managers. I would say the best chance of succeeding to introduce change in an environment like this is not to criticize the current process or try to formalize things too much. Just a little incremental improvement here and there and try to shift attitudes a little. For example, write a wiki article on how you installed and configured tool-X to help with testing your code. Get your fellow devs to read the article and make it as simple as possible for them to follow you. Measure. Measure the bug count against you in the previous project against the bug count using a higher standard of pre-commit testing in a new project. Everything is easier to sell with hard data to back it up. I have thought long and hard about how to influence and introduce change from the bottom-up in companies. I at one time came to the conclusion that you simply can't. It's too hard. That it has to come from the top down. Well maybe it does. But if I can influence people even a tiny bit in my team and our little bubble improves quality maybe we can influence other teams and managers that this is an approach worth following.

Unit Testing - Is 100% Code Coverage Useful?

I recently had a short twitter debate with @unclebobmartin of clean code fame about 100% code coverage. He is of course much smarter than I as well, is vastly more experienced and enjoys a well-earned reputation in the software community that I could only dream of. However, I felt I had a point. I disagree with the 100% dogma as far as Unit Tests go. In fact, I disagree with it strongly. I argue it is more important to unit test some of your code and ship it quickly and iterate - certainly in web development. That does not mean your application is not thoroughly tested by the QA Engineers through automated and manual testing on several environments before hitting production - we are talking about Unit Testing here.

 In my experience it can be quite difficult to achieve 100% code coverage in the real world (remembering what a unit test actually is and keeping in mind when it stops becoming a unit test and starts being a system test - database, file system, network). Think of all the plumbing and infrastructure code involved, think of all the getters and setters, think of all the external interaction with databases etc. Developers, if they are tasked with 100% code coverage, will probably find a way to hit the mark. Will those tests be useful? Does 100% actually guarantee anything? Should the tests not focus on key behavior and key user scenarios at the function level? Unit tests should cover some of the code and a good company will have great test engineers who write great automated system tests and do great manual testing and a great company will have constant feedback loop with product owners demonstrating the software as it progresses.

Of all the code out there running some of the biggest sites and most popular applications in the world, does anybody really think they have 100% unit test code coverage? Of course there are some situations when perhaps 100% code coverage may make perfect sense such as extremely critical systems in finance or medicine. And of course it also can depend on the culture and size of the company - startups are going to focus less on code coverage and more on just shipping code and features whereas enterprises are going to focus on quality more.

But I question the engineering fixation on 100% and it's actual value and ROI. It is NOT something I would blindly follow or suggest for every situation.

P.S. I like Kent Beck's answer on Quora.

Delete a row from a table in Objective-C

Here's a gotcha I came across while deleting a row from a UITableView in iOS. Basically, the order you delete the record from the view and the underlying data source can be crucial.

To delete an item from a UITableView, for this example, I'm going to assume your delegate and datasource are both set to be your TableViewController. This would be pretty common anyway. To give your tableview delete/insert support is actually very straight-forward.

1. Simply implement the tableView:commitEditingStyle:forRowAtIndexPath: method (on the data source - in this case the controller). 
2. You must then send deleteRowsAtIndexPaths:withRowAnimation: or insertRowsAtIndexPaths:withRowAnimation: to the table view to direct it to adjust its presentation. 
3. And then update the corresponding data-model array by either deleting the referenced item from the array or adding an item to the array. 

This is described in the official documentation here

BUT WAIT! If you attempt to remove the item from the view first, then the data source you may run into an error. Something like this (for example if you tried to delete 1 row from a list of 10)...

Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Invalid update: invalid number of rows in section 0. The number of rows contained in an existing section after the update (10) must be equal to the number of rows contained in that section before the update (10), plus or minus the number of rows inserted or deleted from that section (0 inserted, 1 deleted).

Internally deleteRowsAtIndexPaths calls the tableView's numberOfRowsInSection. If the data source has not changed then this will return the same count as before the delete (or insert for that matter). The runtime does not like this and tells you as much in the exception - it expected something to change.

So, simply remove the item from the underlying data source first and then call deleteRowsAtIndexPaths. Which is the other way around to the way it is described in the Apple docs (which are usually excellent). Here's some code...

- (NSInteger)tableView:(UITableView *)tableView numberOfRowsInSection:(NSInteger)section
{
    // Return the number of rows in the section.
    return [recentImages count];
}

- (void)tableView:(UITableView *)tableView commitEditingStyle:(UITableViewCellEditingStyle)editingStyle forRowAtIndexPath:(NSIndexPath *)indexPath {

    // Delete from underlying data source first!
    recentImages = [recentImages removeObjectAtIndex:indexPath.row];
    
    // Then perform the action on the tableView
    if (editingStyle == UITableViewCellEditingStyleDelete)
    {   
        [tableView beginUpdates];
        [tableView deleteRowsAtIndexPaths:[NSArray arrayWithObject:indexPath]
                             withRowAnimation:UITableViewRowAnimationFade];        
        [tableView endUpdates];
    }
    
    // Finally, reload data in view
    [self.tableView reloadData];
}

A (Very) Gentle Introduction to Protocols in Objective-C

Protocols are a powerful abstraction in Objective-C OOP. This is a gentle introduction and should be treated as such. For comprehensive cover of Protocols, the Apple Developer Guide is the de facto resource.  I'm going to focus on formal protocols here and show an example of how they can be used to allow a View to call a method on its ViewController without breaking good design principles.

I come from a C# background so I think of them rather like Interfaces but with some differences. The similarities are that they both offer a contract that allow objects to make use of other objects behavior without the need for subclassing. Developers tend to favor Composition over inheritance as it can provide a more flexible way of building a system. It is a good design principle and worth understanding the benefits of to truly appreciate Protocols or Interfaces.

Protocols have some interesting features and although some of the ideas are the same as Interfaces, the terminology can take a bit of getting used to.

• The object that declares the prototype will be the object that calls methods on it. 
• Another object may "adopt" the prototype, meaning it implements the methods outlined in the prototype. 

Protocols are essentially lists of methods. Some methods are required to be implemented and some can be marked as optional. If the "adopter" of an interface (i.e. the object that implements it and provides the behavior) does not implement all required methods you will get a compiler warning.

@protocol MyProtocol
 
- (void)requiredMethod;
 
@optional
- (void)anOptionalMethod;
- (void)anotherOptionalMethod;
 
@required
- (void)anotherRequiredMethod;
 
@end

An basic example where I have started to use Protocols is when a View needs to call a method on its ViewController. A View should not "have" a ViewController directly. Sometimes a View does need to call a Controller to receive more data or updated data. You can do this through a Protocol.

• I declare Protocol in my View.h 
• I call the protocol method I need in my View.m 
• I implement the Protocol in my ViewController. 
• I set the ViewController to be the delegate for that protocol.

View.h

// Here a declaration of a View includes the declaration of a Protocol. 
// This means the view will want to call methods declared in this protocol
// and another object will have adopted the protocol and implemented the methods to provide behavior.
@class GraphView;

@protocol GraphViewDelegate
- (double)evaluateExpressionForX:(double)x;
@end

@interface GraphView : UIView { 
    id  delegate;
}
@end

View.m

// The declaring object actually goes ahead and calls the method it needs
// Note: the declaring object has no idea who implements the protocol or how they implement it. Nice.
double y = [self.delegate evaluateExpressionForX:someValue];

ViewController.h

// Declare that you adopt the protocol in your interface declaration.
@interface GraphViewController : UIViewController 

ViewController.m

- (void)viewDidLoad {
    // State that you will take care of messages from graphView (provided you have the protocol implementation!)
    self.graphView.delegate = self;
}

- (double)evaluateExpressionForX:(double)x {
    // do some calculations
    return result;
}

That's it. Now the View can message the delegate and get a response back. The View has no idea who implemented the protocol or how they went about implementing it. The Controller is in charge of declaring that it will look after the messages from the View through the Protocol. This is one short quick example of the power of Protocols.

PHP fun with numbers

An integer in PHP is from the set Z (..-2, -1, 0, 1, 2..). Typically on a 32-bit machine this will be 2^31 (signed int) which is equal to 2,147,483,648 (approx 2 billion). You can get the size of an int from the constant PHP_INT_SIZE, which on my machine = 4. There is another constant PHP_INT_MAX which tells you the max value given above (roughly 2 billion). In PHP any value above that limit is automatically a float value.

$x = PHP_INT_MAX; 
var_dump($x); // gives int(2147483647)

$y = PHP_INT_MAX + 1; 
var_dump($y); // gives float(2147483648)

If you attempt to cast a float to an integer and the value is within the bounds for an integer value then it will cast as expected (always rounded towards zero).

$x = 34.5;
$y = (int)$x;
var_dump($y); // gives int(34)

Now comes the funny business...If you attempt to cast a float to an int that is outside the boundaries for an integer value you get...well, officially an undefined output. With no warnings and no errors. In my experiments it is always zero.

$x = PHP_INT_MAX + 1;
$y = (int)$x;
var_dump($y); // gives 0. Nice...

Here's an even crazier one. The official docs recommend NEVER casting an unknown fraction to an int or you will suffer the consequences and hell shall rise through the earth and buildings will fall and all will perish. Or something similar.

$x = (0.1+0.7) * 10;
var_dump($x); // gives float(8). Okay.

$y = (int)$x;
var_dump($y); // gives int(7). WTF?!

And ALWAYS be careful when comparing floating point values. Because of the way a base10 float is stored internally on a machine in base2 you can get some quirky results.

$x = (0.1+0.7)*10;
$y = 8.0;
var_dump($x); // gives float(8)
var_dump($y); // gives float(8)

So they are equal right? Wrong.

echo($x == $y); // gives 0 (i.e. false)

$x is stored internally probably as something like 7.9999999999999991118. From php.net offical documentation

"Floating point numbers have limited precision. Although it depends on the system, PHP typically uses the IEEE 754 double precision format, which will give a maximum relative error due to rounding in the order of 1.11e-16"

The workaround is to provide an epsilon value that provides an allowable difference for two floats to still be considered equal.

if (abs($x-$y)<0.00001) 

Also, as a footnote, converting a string to an int can also result in zero if the string is not an integer value.

$x = "peter";
$y = (int)$x;
var_dump($y); // gives int(0)

:)

JavaScript variable scope and closures

I wrote a bug recently. I didn't mean to, it just happened. I thought the code looked good and two other code reviewers did too. Of course none of us are JavaScript programmers and clearly none of us understood closures and variable scope properly. This is one of the basic things about JavaScript that seems to catch people out all the time. THERE IS NO BLOCK SCOPE - ONLY FUNCTION SCOPE! I blame the C-like syntax for giving people a false sense of security. Take the following contrived example. Three buttons called One, Two and Three. Add a click event handler to each that simply displays the name of the element. The following code DOES NOT WORK. Some people (me included, not so long ago) would have expected that clicking One would show '#one', Two would show '#two' and Three would show '#three'. But they all show '#three'. Why? Because scope is function-based in JavaScript. And 'url' is declared in the document.ready function. Clicking on any of the buttons will result in the last value assigned to the url (i.e. '#three') to be displayed.

$(document).ready(function() {
    var links = ['#one', '#two', '#three'];
    for(var i=0; i<links.length; i++) {
       // oops! The scope of url is the function - not the for loop block
       var url = links[i]; 
       $(url).click(function(e) {
           alert(url); 
       }); 
    }
});

To fix this, we introduce a closure to limit the scope of that variable

$(document).ready(function() {
    var links = ['#one', '#two', '#three'];
    for(var i=0; i<links.length; i++) {
       displayId(links[i]);
    }
});

var displayId = function(url) {
    $(url).click(function(e) {
        alert(url); // now the scope of url is within this function
    });
}

So lessons learned.

• Have a good grasp of the fundamentals of a language before ploughing ahead coding in it
• Only do code reviews with people who UNDERSTAND THE LANGUAGE!
• Normal advice of declaring variables as late as possible can be BAD ADVICE in JavaScript - declare at the top of a function instead.

Modifying an item in an array while iterating over it in PHP

So you want to iterate over an array in PHP and change each item as you do.
In PHP5 there are two ways of doing this.

1) Pass by reference
Note the ampersand in the declaration of the foreach loop before the variable $i, passing each item by reference and allowing it to be changed directly.

$arr = array('a', 'b', 'c');
foreach($arr as &$i) {
    $i = 'changed';
}

2) Using the key of each item
This is the classic way of achieving the same, using the key of the item

$arr = array('a', 'b', 'c');
foreach($arr as $k=>$v) {
    $arr[$k] = 'changed';
}

Weighted Random Selection

I recently came across an interesting problem. Given an array of items where each item has a name and a weight (integer value), select a random item from the array based on the weight. So items with a larger weight value are more likely to be returned.

One solution I thought of was to create a new array where the weight determines how many times an element appears in the new array. You then just pick an item from that array at random. However this is limited due to the alarming size of the new array when the weights and the number of items in the original increase.

So here is my solution to the problem - there are many ways of solving this problem but I think this one is easy to understand.

Essentially the algorithm is
1. Add up all the weights.
2. Pick a number at random between 1 and the sum of the weights.
3. Iterate over the items, decrementing the random number by the weight of the current selection.
4. Compare the result to zero, if less than or equal to break otherwise keep iterating.

The key is that the larger the weight the more likely to be less than zero when compared to the random selection between zero and the sum of weights. Remember - that random decrements each pass through the array of items, so you will always hit zero i.e. it is guaranteed that something will be picked every time.

I will use PHP for this (now my primary language day-in-day-out) so I'm sure there are improvements to be made in this code - I'm very new to PHP still!

In this example say the array contains 4 foods and each has a weight. This code demonstrates the distribution over a 1000 iterations.

// We don't like fruit in this example, so give a low weighting
$foods = array('vegetables'=>4, 'fruit'=>1, 'chips'=>8, 'burgers'=>10);
$sumOfWeights = array_sum($foods);

// hold the results of a thousand random selections. 
$count = array('vegetables'=>0, 'fruit'=>0, 'chips'=>0, 'burgers'=>0);

for($i=0; $i < 1000; $i++) {
    // choose a random between 1 and the sum of the weights.
    $random = rand(1, $sumOfWeights); 
    foreach($foods as $name => $weighting) {
        // ***The next two lines are the heart of this algorithm***
        // decrement the random by the current weighting.
        $random -= $weighting;        
        // The larger the weighting, the more likely random is less than zero.
        if($random <= 0) {
            $count[$name]++; 
            break;  
        }
    }
}

foreach($count as $name => $result) {
   echo($name."=".$result);
}

This will give results similar to

• vegetables=210
• fruit=157
• chips=278
• burgers=355

Changing the weighting value for each item will yield the expected distributions, for example increasing the weighting for vegetables to 80 and fruit to 60 (let's be healthy) would result in something similar to

• vegetables=553
• fruit=421
• chips=19
• burgers=7