On the question of constrained versus flexible products

Apparently before his death the film director Stanley Kubrik took all his outtakes into a field and burned them. Thus leaving us with just his finished films to evaluate. Its a position I have a lot of sympathy with. Similarly for reasons such competitive advantage most companies do not reveal the missteps in how they develop products. This can lead to some wild ideas about how great products are made and myths such as the design genius whose mind just gives birth to fully formed products. But this myth also devalues the mind boggling amount of hard work that goes into making anything decent. Part of the plans for Berry Forest was to lift the lid and share some of the messy reality that goes into designing and creating an app.

When I worked at Symbian a colleague of mine by the name of Scott Jenson had many catch phrases he used to highlight  design problems. One of which was 'flexibility is the root of all evil'. I never took this to mean products should avoid flexibility, but just that people constantly misjudge how hard it is to make what people later criticize as being highly constrained. It is a common complaint of Apple products. But it is also noticeable the same people making such claims also fail to deliver anything better. Why is that?

Background

mzl.inlntbrq.480x480-75.jpg

Toca Boca are the premier digital toy producer for iOS. They recently celebrated their 27 millionth download.  No doubt at the start of the project I was full of bravado and thinking oh we can better Toca Boca. Yet as the project progressed I really grew to appreciate how good their toys are. The closest toy Toca Boca has to Berry Forest is Toca Tailor. You select a character and then chose which clothes they wear. One of the key innovations is the ability to use the camera to take pictures which then become the textures for the clothes. We live in a crazy age where after spending $400 for a device you can pick up something that will delight your kids for hours for just $2.99. If you don’t have Toca Tailor already you should be snapping it up right now. 

 

mzl.cotwvtra.480x480-75.jpg

 When you drag on the clothes to your character in Toca Tailor they always snap to a predetermined position. For example it does not matter when you drag a pair of sunglasses. Once you let go they will position themselves to a specific point on the characters face. The size of glasses is also fixed and so is the rotation. Finally you cannot have more than one accessory.

The plan for the berry builder in Berry Forest was to have full flexibility. It was to be something like Mr. Potato head. Here you can create your own berry from an assortment of pieces. It is only as the builder developed that the consequence of this change started to emerge. One problem lead to another and soon 'lets be like Mr Potato head' had enormous consequences.

A touch of magic

Each berry piece is a rectangular image. In the first revision the areas you can touch to manipulate the piece just matched the original image size. Now if you have used a lot of apps and games you may have found some of them seemed to respond poorly to your touch. Often this is due to poorly placed or sized touch areas and this is easily done as these touchable areas are invisible. We addressed this with a debug mode that visualizes the areas that can be touched as semi transparent yellow shapes. By being semi transparent you can still roughly see them if they are several that overlap each other. 

squaretouch.jpg

In this image you can now see all the parts of the builder that can be touched. The huge rectangle that covers most of the berry is just from the yellow curly hair image. Now if the hair image is above the other pieces then that touch area is blocking all the others and you cannot pick them up. You can see it above that there would be no way to grab the eyes as they are completely covered by the hair touch area. A lot of the hair image is just transparent pixels though. To a normal person it looks like you can tap and drag the eyes with ease. But attempting to do that instead results in only the hair moving. You soon have frustrated and angry kids. Now as part of a visual rendering optimization we actually have a pretty good idea what the shape of the hair within that image actually is. So we reuse that data to mask the shape of the touch area and we now get this.

masked-touch.jpg

This results in a more usable berry builder. 

More on multitouch

My own kids are of course perfectly behaved. But I have observed that other children, not my own of course, tend to fight over the same app running on an iPad. Now part of this may just be natural disagreements, but some of it can just be a lack of proper multitouch. If both kids put their fingers on the screen at the same time the app stops working and soon my kids, sorry other kids are fighting and demanding no one else touch the iPad. For Berry Forest we have our own multi-touch system lovingly crafted by Kim. You could call it a cooperative multitouch system as it happily lets multiple people use the berry builder all at the same time. You can pick up as many items as the device can distinguish indivdual fingers. Which is five for the iPhone and ten for the iPad. We then allow a pinch gesture to be used to both scale the selected piece larger and smaller and 'drum roll please' pinch to rotate the item. This opens up a world of creative options such as moustaches that become eyebrows and hair that becomes skirts. As show in this video.

The problem is how do you know if the intent is to pick up and move two separate items or to just pinch manipulate a single one? The device cannot see the shape of you hand or if the fingers were attached to more than one child. All we know is more than one finger is touching the screen. We settled on a system where we presume two touches near each other mean you are attempting to pinch scale and rotate a piece. A side effect is if you cannot move two pieces that are close to each other at the same time. Most of the time this just works as intended.

outside.jpg

This then lead to a new problem. To calculate if an item should stick to the berry we were doing a simple calculation to see how far from the center of the berry the image was. But this still allowed items to appear far outside the berry as shown in this image.

Kim then went away and reused the same data that is used to both optimize the graphics and mask the touch areas to calculate if a piece is inside or outside the berry body. You can see a test video of this working here.

distancedebug.jpg

During development we only had a single strawberry body. When the other bodies such as blueberry and cloudberry were added problems specific to those bodies started to show up. they were not the same size and shape and so items that fitted one body did not fit another. So another tweak and another debug mode allowed specific ellipse that control what size the body is and how far an item can be placed outside it.

This is finally tweaked again as some items such as hair look great outside the boundaries of the body. While others such as eyes only look good within the body.

Too big to fail.

black.jpg

In the initial versions items could be scaled to any size. My main memory is of a strange bug report. A tester had been visited by their young cousins and after playing with the toy the background to the forest was black. This persisted even if Berry Forest was closed via the recent apps list and restarted. It just seemed to be impossible. I could understand bugs that make the whole app be blank, but just the forest? The tester sent me a screen shot. What I could see is that the screen was indeed very dark, but it was not uniformly black. It was just their cousin had pinch zoomed some black hair to be about 20 times the size of the entire screen!

Each piece now has a minimum and maximum size it can be pinch scaled to. At first this was the same size for everything, but it soon became clear that each type of piece needed its own constraints. So you can make the berry hair huge, but not go over the top with eye brows. 

Its not over yet!

We also had to address issues such as changing the eye lid colors to match the selected body color. If pieces overlap which one should be at the top? For this the last item you touch is left on top of everything else. What happens if parts are covering legs and arms? In some situations it would appear an arm was sticking out an eye. But in others the body parts obscured the leg and arm animations and made it look like the berry was sliding up the beanstalk and not climbing. We would knock down one issue only to have another three pop up.

When to give up?

There is no design law that states it will be possible to solve all the problems that arise for any app. Nor is their any magic process that can guarantee success. Each new solution will have unintended consequences. You just have to hope you can solve them all to a degree that the app reaches a point where the only issues left are minor enough that it still works as a complete product. Often all that keeps you going is a hunch you can address all the issues or sheer bloody mindedness. Here in this post it might look like as soon as one problem popped up we had a solution. But often I just sat at my desk feeling rather glum and suspecting we had hit an unsolvable issue. Then someone else would come up with a solution, or after a good nights sleep an answer would just pop into my mind. But for others we just had to admit defeat, if only because even if every problem is solvable, you don’t have infinite time. You need to ship and at some point you just have to ask yourself if it is worth carrying on with some feature or just leaving it out. It would have been neat to have resizable bodies. As this image shows it can have quite an effect on the personality of the berry. But shipping Berry Forest took priority.

I will follow up with some closing thoughts in a later post. 

size.jpg

Berry Forest can be found on the App Store for iPad and iPhone here.