Why is it taboo for apps on a phone to rotate upside down?

apple-orginal-iphone.jpg

I recently saw a news story that claimed apps running upside down on a phone is taboo. It left me wondering how many people think these are just capricious and seemingly irrational choices and not instead design discussions. Ones often made after seeing the problems upside down apps can cause on a phone.

Think about your phone. At the top is a speaker so you can hear who you are speaking to and at the bottom is a microphone to pick up your own voice. If the phone is upside down your ear won't be near the speaker and the microphone will be too far away to properly pick up your voice. Today phones are mainly just a large screen. When you get a call, a quick glance won't be enough to know if the phone is the right way up. If the phone app is upside down, you can now answer the call and it might take some time to realise why the other person sounds so faint and why they cannot hear you.

You could suggest that in this case the phone app always launches the right way up. Here you get a new problem. Lets say you are writing a text message, but with the phone upside down. You get a call and now see the phone view is to you the wrong way up. You now have to rotate the whole phone to properly view it. It isn't a huge deal, but it is not pleasant. You try and avoid forcing the user to rotate the phone as they use it. This is why if an app does run in landscape, all of its views will also work in landscape. You don't want to have the user press a button in an app and then be forced to keep changing the phone between landscape and portrait. Both these issues go away if you disallow apps from running upside down.

hustle-31-600x331.jpg

Another related issue is why some apps on a phone only run in portrait. At least for the phone app itself there is a clear reason. You normally start or answer a call with the phone held in portrait. But once held to your ear, as seen above, the phone is closer to the landscape position. If you then need to check some info and find your app had now in landscape, it is disorientating. Most people don't realise the phone had been held at an angle during the call. It can be stressful enough talking to someone and trying to check some data. So this kind of confusion is something designers opt to cut out by just having the app run in portrait.

So there you have why it is advised to not allow your app to run upside down and also why its not alway preferable for an app to be usable in landscape. There are good exceptions. It is also why on a tablet without phone hardware, these problems go away and all apps do indeed run in any orientation.

Designers. Get over your fear of code and on with your lives (part 2)

Part of any craft is mastering the materials you work with. A  Chef does not just write out a menu, but is a manipulator of food. A jeweller is not someone who just sketches rings and necklaces, but a metalsmith. Just as the furniture designer is also a professional carpenter. They know how to work with materials. Work that included understanding overcoming the materials inherent properties. The material a User Experience is crafted from is software! This is why designers need to get their hands dirty with it. Yes, it may be  more abstract. Software indeed has differences to physical materials but it also shares some characteristics. 

Somehow the dangerous idea that designers don't need to get hands on with software still dominates the field. My own view is that designers who can get more involved with the implementation of the product are not just better designers but also happier in their daily work. We claim to be creatives, yet what do we create? For many it will just be PowerPoint decks, wireframe views and UI specifications. Are these really the products of creativity? Can you proudly point at the fingerprints you left on a product?

Endlessly debating over if designers code can easily get boring. Once you are working hands on it is pretty clear that a lot of the tools and technology to enable better user experiences are simply terrible. One reason is a lack of designers getting in there, trying to use them and making smart suggestions on how to improve them. What follows is some of the main arguments I have experienced trying to get designers to get more hands on.

"You need a degree in computer science to understand how to develop software."

While the more you know isn't going to hurt you a full degree is not needed. I don't have a computer science degree. Hell, some amazing developers I know don't even have one. You just need time, supportive patient colleagues and decent tools. 

"A little knowledge is a dangerous thing."

At the beginning, as designers try and get to grips with the fun world of software development and the product itself, they are going to say many dumb things. Take version control. Often designers new to it will worry that they now have the power to destroy the entire project by adding some broken code! This is not the point to laugh and call someone dumb. Just as you don't laugh at children when they start to speak. Okay maybe you do laugh, but only to encourage them. We should see the mistakes and incorrect assumptions as part of the path to proper understanding. Trust me, if you are a developer and you think designers are saying silly things, you should be a fly on the wall when only the designers talk to each other. Don't be daft and suggest designers should stay away from engineering issues, go explain how things really work.

"Knowing about development issues makes you less creative."

I often feel embarrassed at how often I hear this claim. The idea here is you need to be ignorant of the daily realities of software development to be able to push the limits. First of all when it comes to pushing the limits of what software can do no one is more active than software developers. Take the world of open source software, where designers are massively underrepresented. Here a constant stream of projects exist due to various developers frustrations with the status quo. 

There is germ of truth here. Engineers are still mostly human. Estimating how hard a problem is to solve, simply by thinking about it almost always proves wrong. What is thought to be easy can take weeks, while the impossible turns out to not just be possible, but sometimes simple. The solution here is not ignorance, but high standards combined with finding the issues worth pushing.

"You cannot stand up for the end user if you also have to worry about development issues."

It is possible to walk and chew gum. For sure anyone can get lost in the details. It runs both ways though. Just as developers can get lost in the details of implementation, designers continually get lost worrying about issues that are not that grand or important to people when the product ships. Loss of perspective is best fixed by knowing about a wider range of issues. Designers don't need to stop standing up for the end user by knowing about the implementation. 

The reality is the separation between designers and developers is what more often leads to products that don't work well for the user. A lack of hands on involvement, leads to poorer communication of what the genuinely important design issues to be solved really are. Focus goes to the wrong parts of the product and by the time the mistake is realised the chance to make significant changes to the software has passed.

There is also a flip side to this. Even the purest of designers, whatever that means, will have some knowledge of what is possible with software. No one wants to look like a crackpot by suggesting nothing but impossible ideas. Experience shows a poor understanding of what is possible just as easily leads to possible ideas never being suggested due to the designer thinking they cannot be done.

 

We live at a time where it is still fashionable to claim there is dangerous knowledge. Knowledge that can make us less creative. Knowledge that would harm us to know. Be it the well meaning manager protecting employees from information that might distract them from their daily work. As well as the bureaucrat who claims some knowledge must be kept secret to protect our safety. But we are not little children and are best able to decide for ourselves what is useful and what is not. Also we need to constantly stride to broaden our education. To open ourselves to new ideas and new skills. It saddens me to no end that so many of my colleagues have crafted a wide range of reasons to push fingers in ears, heads in the sand and cut themselves off from a world of knowledge that could transform their daily work. This is meant to be an enlightened age. It should go without saying that taking a bite from the tree of knowledge is not original sin, but how we liberate and improve both ourselves and the world around us.

 

Designers. Get over your fear of code and on with your lives (part 1)

It is time for an intervention. It is no longer clear if designers are helping or hindering the creation of great products. We have backed ourselves into a ghetto of our own making. It is both an anti-intellectual ghetto where we claim engineering and software development is a dangerous type of knowledge. A type of knowledge that will make us worse at our design jobs. It is also a real physical ghetto, or silo. We have our own separate tools that often don't play well with the ones engineers use. We even sit and talk apart from those that should be our greatest allies in great product creation, the engineers. There is an answer though. We need to learn to code. Yes I can imagine the howls of pain. I think I can hear the sound of pitchforks being sharpened. The torches have been lit and soon a lynch mob will be coming to burn me at the stake. But hold back the cries of heretic and hear me out.

But first I will throw some salt in the wound. Being able to code won't be enough. There are plenty of designers who in some way claim to know how to code. Be it based on a brief programming module at University. Or those who started out as engineers before moving to design full time. It even includes those of you who 'code' prototypes. Yes none of that is enough. Designers need to get their hands properly dirty. We need to be working on the real software that will ship as the final product. This will indeed mean craziness such as installing a software development kit. Or as we pros call it, an SDK. You will have to use and even love version control and yes *gasp* maybe even the command line.

If this is right there should be clear benefits. Designers should find themselves happier, more productive and more creative. We should see designers and developers working better and more respectfully together. We should also see a new generation of tools and technology that allow designers to happily work hands on. And critically we should see many more delightful products.

A lot of the arguments designers have is about how scary some of the tools we have to use to work directly on software are. Again this this is our own fault as instead of giving feedback to improve them, we have just run away claiming it is the engineers job to suffer their use. My own experience is that if you stereotype engineers and designers the type of tools created for them are terrible. It is just blindly accepted that engineers will put up with hard to setup and use tools, while designers will not learn to use something unless it works like Photoshop. Let us return to this later though.

Before we can really dive into this conversation though we need to fully confront the issues around the idea that learning to code can make you a worse designer. More on this soon.

Talking about icons: Between the grid and intuition

At the end of episode #90 (1:05:55) of The Critical Path, Horace Dediu raised the issue that "we don't know how to talk about design, we have a real problem with the language... and even designers have this problem. They can't justify their work because so much of design is ambiguous". It is an issue close to my heart, although specifically conversations about trying to define design quickly lead me to wanting to kill myself. Instead I would rather get back to doing some real work. All I will say here is I remain unconvinced a new vocabulary is needed and that everyone, designers included, should just strive to be clearer in what they say. That is also to say as designers we should always be alert to the every present danger of being overly pretentious and disappearing up our own arse.

Last week Neven Mrgan made an interesting critique of the iOS 7 icons, which was followed up by a dickish follow up which focused on Neven's claim that some intuition is needed to make icons look right. I don't know Neven, but I am a fan of his work and have followed him on Twitter while now. So I am going to restate some of what rung true for me.

With iOS 7, every detail warranted the same rigor toward design. Like refining the typography down to the pixel. Redrawing every icon around a new grid system
— http://www.apple.com/ios/ios7/design/

For iOS 7 there is a new grid system and Apple were so excited about this they felt it worthy enough to use in their public marketingWe see grid systems at work every day in newspapers, magazines and their online equivalent. A grid is used to consistently align the columns and rows of text and images. This grid, or variations of it are then used throughout the paper and magazine and give it a consistent personality. If you ever made an invite for a party or tried to create a newsletter and found the end results looked somehow amateurish, it was probably due to among many other things a lack of understanding of how to use a simple grid to line everything up. This is just a small taste of what grids help with.  

When you start to create software and developers discover grids they get excited as well. They see how everything should be spaced and aligned. And then something funny happens. The designers start to ask for changes that seem to outsiders to randomly break the grid. Just as it seemed those wishy washy designers had brought some rigour to their work they want to stop using it everywhere. So what is going on?

Once you have a real design using a grid system you start to see weird things happen. The grid tells you certain items are in perfect alignment and are correctly sized. But your eyes tell you they are not aligned and mis-sized. What is going on here?

 A Necker Cube

A Necker Cube

It starts with the issue our eyes are not video cameras. Our eyes have some inbuilt assumptions about reality and then our brain does a huge amount of work to reinterpret these results which we perceive as the real world. This whole process is riddled with mistakes. If the human visual system was considered hardware and software we would say it is rather buggy. One of my favourite aspects of how poorly the eyes are engineered is that the optic nerve at the back of the eye means we have a large blind spot. Other aspects of the poor engineering of the eye are the wide range of ways it can be tricked in the form of optical illusions. An example is a Necker cube. Look at the cube above and stare at the red dot. The cube appears to flip so that the red dot is sometimes inside, and sometimes outside the cube.

The human visual system is not thoroughly understood to the point where science has  much to say about iOS icon designs, but we do know why it has all these engineering issues. Yes, our visual system was not engineered, it evolved. And although we have good reasons to believe it gives us a pretty good approximation of the real world, evolution does not lead to perfection. These kinds of issues mean you find yourself having to nudge items in a design by a few pixels so they end up looking like they are aligned, even though they are not actually following the grid perfectly. The same goes with the size and shapes we use for icons. A range of shapes may all fit perfectly inside a grid, but factors such as the shape and the colours may mean one shape looks smaller or heavier than another. I am sure there is a research program here and we could spend time trying to develop a range of rules. But I suspect that every shape might need its own rules. And then the background colour and icon colour would also have an effect. And pretty soon a very complex formula, if one exists as all, would be needed. It is here we have to rely on the eye of a talented and experienced designer. This does not exempt designers from giving some rational, but it does afford some patience and forgiveness on those who expect a very precise answer. This is why it can often help to have more than one design solution to a problem so at least people can compare one icon with anther.

Something that I deeply feel is an important part of this craft and often goes unrecognised, is the importance of standards. A big part of design is taking the knowledge of what has been found to work over what has been known to fail. Examples of this are avoid 'yes and no' dialogs and instead offer clear actions. So in design we prefer 'Delete Photo? <Delete> <Cancel>' over 'Are you sure you want to destroy all your photos? Maybe you don't? <Yes> <No>'. The same goes for graphic design and this is where some of the new iOS 7 icons are real head scratchers. It is totally legitimate to change the style of what an iOS icon should be. But some of these changes appear to go against what has been accepted as good graphic design in general. 

 I stole this from&nbsp;http://mrgan.tumblr.com/post/53308781143/wrong

I stole this from http://mrgan.tumblr.com/post/53308781143/wrong

Session 221 - iOS User Interface Design from WWDC 2012 describes a great iOS icon as being 'Beautiful and Instantly Recognisable' as well as having a clear recognisable shape. Some of the new app icons, such as the App Store icon, have a shape that fills almost the entire space of the icon reducing how recognisable it is. This has been considered the mark of an amateur icon and to now suggest this standard is wrong is bound to provoke some serious discussion. Many people don't find apps based on the name under the icon. I bet we could arrange a test where by hacking a phone to make all the icons black rectangles, but still have their name underneath people would struggle like hell to find their apps. Similarly a shape with a decent amount of space around it is is both more visually appealing and faster to recognise. This is not an ambiguous statement, but an actual testable consequence.

I hope this was a small step to bring some clarity to why designers have issues with some of the icons in iOS 7 without resorting to 'its just my special opinion'.

 

Raising the discussion

It is a funny old business the technology industry. Despite owing its entire existence to the greatest search for truth and knowledge we have, science, we often seem pretty damn insistent on learning nothing. So before getting to the some real learnings from the past six years I wanted to just quickly cover some of the uninteresting discussions currently ongoing.

First!!
— Internet comment thread

There is a lot of fuss over who did anything first in this industry. Who invented the smartphone? It appears we should find this party and credit them with everything that came after. It is not clear why they should get all the credit, but for sure all the products that came after should get none, right? One problem is even if there is a clear inventor of the smartphone so much of that is built on the previous generations of mobile phones. So should we find who invented the mobile phone? But then we find so much of that work goes back to the original land line system, so do we then credit Alexander Graham Bell? But what about Samuel Morse of telegraph and morse code creation that came before? We can keep on going till we are crediting whoever came up with smoke signals or maybe who ever said the first human word?

The thing that hath been, it is that which shall be; and that which is done is that which shall be done: and there is no new thing under the sun.
— King James Version of Ecclesiastes 1:9

The next issue is originality. Take an iPhone and you can chop it up and see various parts were already visible in products that shipped years before.

Dedicated graphics chips (GPU's) were being promoted all the way back in 2004. And phones were already on the market by at least 2006.

Phones from 2005 already had accelerometers.

The Webkit browser engine used by both the desktop and mobile versions of Safari and Chrome was already in phones by 2005.

Countless devices already had touchscreens, including this Symbian model from 2000.

Capacitive technology was not only in use in phones it had already  used in the iPod click wheel. 

Todays phones are crazy complex. Just trowing technolgy together does not a great product make. Products do not just spontaneously evolve from these individual elements, just as life does not spontaneously evolve out of peanut butter. Back in 2007 many talented teams were trying to utilise all these great technologies in products and we all failed to make something like an iPhone. The lessons why still fail to be learned. So in the next post lets go back to 2007 and see what was so different and see what can be learned for work we do today.

On the question of constrained versus flexible products

Apparently before his death the film director Stanley Kubrik took all his outtakes into a field and burned them. Thus leaving us with just his finished films to evaluate. Its a position I have a lot of sympathy with. Similarly for reasons such competitive advantage most companies do not reveal the missteps in how they develop products. This can lead to some wild ideas about how great products are made and myths such as the design genius whose mind just gives birth to fully formed products. But this myth also devalues the mind boggling amount of hard work that goes into making anything decent. Part of the plans for Berry Forest was to lift the lid and share some of the messy reality that goes into designing and creating an app.

When I worked at Symbian a colleague of mine by the name of Scott Jenson had many catch phrases he used to highlight  design problems. One of which was 'flexibility is the root of all evil'. I never took this to mean products should avoid flexibility, but just that people constantly misjudge how hard it is to make what people later criticize as being highly constrained. It is a common complaint of Apple products. But it is also noticeable the same people making such claims also fail to deliver anything better. Why is that?

Background

mzl.inlntbrq.480x480-75.jpg

Toca Boca are the premier digital toy producer for iOS. They recently celebrated their 27 millionth download.  No doubt at the start of the project I was full of bravado and thinking oh we can better Toca Boca. Yet as the project progressed I really grew to appreciate how good their toys are. The closest toy Toca Boca has to Berry Forest is Toca Tailor. You select a character and then chose which clothes they wear. One of the key innovations is the ability to use the camera to take pictures which then become the textures for the clothes. We live in a crazy age where after spending $400 for a device you can pick up something that will delight your kids for hours for just $2.99. If you don’t have Toca Tailor already you should be snapping it up right now. 

 

mzl.cotwvtra.480x480-75.jpg

 When you drag on the clothes to your character in Toca Tailor they always snap to a predetermined position. For example it does not matter when you drag a pair of sunglasses. Once you let go they will position themselves to a specific point on the characters face. The size of glasses is also fixed and so is the rotation. Finally you cannot have more than one accessory.

The plan for the berry builder in Berry Forest was to have full flexibility. It was to be something like Mr. Potato head. Here you can create your own berry from an assortment of pieces. It is only as the builder developed that the consequence of this change started to emerge. One problem lead to another and soon 'lets be like Mr Potato head' had enormous consequences.

A touch of magic

Each berry piece is a rectangular image. In the first revision the areas you can touch to manipulate the piece just matched the original image size. Now if you have used a lot of apps and games you may have found some of them seemed to respond poorly to your touch. Often this is due to poorly placed or sized touch areas and this is easily done as these touchable areas are invisible. We addressed this with a debug mode that visualizes the areas that can be touched as semi transparent yellow shapes. By being semi transparent you can still roughly see them if they are several that overlap each other. 

squaretouch.jpg

In this image you can now see all the parts of the builder that can be touched. The huge rectangle that covers most of the berry is just from the yellow curly hair image. Now if the hair image is above the other pieces then that touch area is blocking all the others and you cannot pick them up. You can see it above that there would be no way to grab the eyes as they are completely covered by the hair touch area. A lot of the hair image is just transparent pixels though. To a normal person it looks like you can tap and drag the eyes with ease. But attempting to do that instead results in only the hair moving. You soon have frustrated and angry kids. Now as part of a visual rendering optimization we actually have a pretty good idea what the shape of the hair within that image actually is. So we reuse that data to mask the shape of the touch area and we now get this.

masked-touch.jpg

This results in a more usable berry builder. 

More on multitouch

My own kids are of course perfectly behaved. But I have observed that other children, not my own of course, tend to fight over the same app running on an iPad. Now part of this may just be natural disagreements, but some of it can just be a lack of proper multitouch. If both kids put their fingers on the screen at the same time the app stops working and soon my kids, sorry other kids are fighting and demanding no one else touch the iPad. For Berry Forest we have our own multi-touch system lovingly crafted by Kim. You could call it a cooperative multitouch system as it happily lets multiple people use the berry builder all at the same time. You can pick up as many items as the device can distinguish indivdual fingers. Which is five for the iPhone and ten for the iPad. We then allow a pinch gesture to be used to both scale the selected piece larger and smaller and 'drum roll please' pinch to rotate the item. This opens up a world of creative options such as moustaches that become eyebrows and hair that becomes skirts. As show in this video.

The problem is how do you know if the intent is to pick up and move two separate items or to just pinch manipulate a single one? The device cannot see the shape of you hand or if the fingers were attached to more than one child. All we know is more than one finger is touching the screen. We settled on a system where we presume two touches near each other mean you are attempting to pinch scale and rotate a piece. A side effect is if you cannot move two pieces that are close to each other at the same time. Most of the time this just works as intended.

outside.jpg

This then lead to a new problem. To calculate if an item should stick to the berry we were doing a simple calculation to see how far from the center of the berry the image was. But this still allowed items to appear far outside the berry as shown in this image.

Kim then went away and reused the same data that is used to both optimize the graphics and mask the touch areas to calculate if a piece is inside or outside the berry body. You can see a test video of this working here.

distancedebug.jpg

During development we only had a single strawberry body. When the other bodies such as blueberry and cloudberry were added problems specific to those bodies started to show up. they were not the same size and shape and so items that fitted one body did not fit another. So another tweak and another debug mode allowed specific ellipse that control what size the body is and how far an item can be placed outside it.

This is finally tweaked again as some items such as hair look great outside the boundaries of the body. While others such as eyes only look good within the body.

Too big to fail.

black.jpg

In the initial versions items could be scaled to any size. My main memory is of a strange bug report. A tester had been visited by their young cousins and after playing with the toy the background to the forest was black. This persisted even if Berry Forest was closed via the recent apps list and restarted. It just seemed to be impossible. I could understand bugs that make the whole app be blank, but just the forest? The tester sent me a screen shot. What I could see is that the screen was indeed very dark, but it was not uniformly black. It was just their cousin had pinch zoomed some black hair to be about 20 times the size of the entire screen!

Each piece now has a minimum and maximum size it can be pinch scaled to. At first this was the same size for everything, but it soon became clear that each type of piece needed its own constraints. So you can make the berry hair huge, but not go over the top with eye brows. 

Its not over yet!

We also had to address issues such as changing the eye lid colors to match the selected body color. If pieces overlap which one should be at the top? For this the last item you touch is left on top of everything else. What happens if parts are covering legs and arms? In some situations it would appear an arm was sticking out an eye. But in others the body parts obscured the leg and arm animations and made it look like the berry was sliding up the beanstalk and not climbing. We would knock down one issue only to have another three pop up.

When to give up?

There is no design law that states it will be possible to solve all the problems that arise for any app. Nor is their any magic process that can guarantee success. Each new solution will have unintended consequences. You just have to hope you can solve them all to a degree that the app reaches a point where the only issues left are minor enough that it still works as a complete product. Often all that keeps you going is a hunch you can address all the issues or sheer bloody mindedness. Here in this post it might look like as soon as one problem popped up we had a solution. But often I just sat at my desk feeling rather glum and suspecting we had hit an unsolvable issue. Then someone else would come up with a solution, or after a good nights sleep an answer would just pop into my mind. But for others we just had to admit defeat, if only because even if every problem is solvable, you don’t have infinite time. You need to ship and at some point you just have to ask yourself if it is worth carrying on with some feature or just leaving it out. It would have been neat to have resizable bodies. As this image shows it can have quite an effect on the personality of the berry. But shipping Berry Forest took priority.

I will follow up with some closing thoughts in a later post. 

size.jpg

Berry Forest can be found on the App Store for iPad and iPhone here.

Toe tapping design - Keeping parents in mind when creating a children's toy.

Sound design in apps is an important but all too often overlooked issue. "Turn the sounds off!" Is a common shout out in my house hold. So many of the games and toys available for iOS devices are beyond annoying to have to listen to. Music that is nothing more than a twenty second loop starts to drive anyone within earshot slowly insane. Berry Forest allowed an opportunity to try something different. Instead of worrying about the children playing the app, we instead focused on the poor parents.

We didn't have the resources or budget to hire a dedicated sound designer. So instead all the music and sounds are licensed. There are no 20 second loops. Instead a full three to four minute jazz track is used. To reduce even this getting monotonous each area of the game such as forest, underground, puzzles and bird game each have their own track. The main use of sound effects is in the bird game. Here again effort was made to reduce the problem of hearing the same sound over and over as you knock all the birds down. Each type of sound, such as a new version of a bird popping up, or a bird being frightened away has 3 possible sounds. Then some code randomly chooses one of the three sounds. This revealed a new issue which is you can still randomly choose exactly the same sound several times in a row. So there is an extra check to make sure the next sound is different to the last one that played. 

The feedback from the testers was pretty consistent. First they raised concerns that kids might not like jazz. But when children played with the toy none of them complained. As time passed the feedback on the sound design was overwhelmingly positive. Nearly 20% of the download size comes just from all the sounds and music. In exchange for a few more minutes download time hopefully many parents were saved from shouting  "turn those sounds off!"

If you want to checkout Berry Forest see our site at www.kanjoarts.com

Death to loading screens

Devices and apps have got a lot better thanks to a lot of hard work and tricks designed to hide away that devices are still computers and apps are just software. But despite all the advances there is still many problems that drive me mad and one of these is loading screens. You download a fun new game, launch it and then there might be a long wait with a progress dialog that fills up. Then you finally get in and you start a level only to be hit with another wait and a big Loading... screen. It does not have to be this way!

For Berry Forest we decided we were going to have a loading screen free application with no obvious trickery. First of all lets have a look at a video of Berry Forest running. I recorded this right off an iPad Mini.

 

Please note that although Berry Forest runs at 60 frames per second this video runs at  30.

This shows the app startup, a walking berry and the different parts of the app such as the underground camera studio, the puzzles atop a magic beanstalk and a bird game at the far end of the forest. Now most people probably saw nothing technically amazing. But if you are an app developer you might be scratching your head. There is no way round certain issues. No trick can get around the time it takes to load all the images the toy is made up from. Then they all need to be rendered. So what did we do?

1. We created a toy design where the parts of the world that are loading and unloading are always off screen so no one is ever the wiser to them popping in and out of existence.

2. A game render engine that is capable of a smooth 60 frames per second and does not stutter, freeze, or glitch while the new parts of the world are loaded and then rendered.

Berry Forest appears to be one seamless world. But it isn't'! Lets pull back the curtain and give you some visual clues to what is really going on by zooming out. 

When you tap the Berry Forest icon on your iPad or iPhone a static image is instantly shown to you. We then load in the cloud and logo images that match this image perfectly. This just takes a second and then we start the key turn animation. So no long waits to see something start to move. 

While the key turns we then load the main parts of the forest. Then the game pans down to the 3 logs and on walk the 3 berries. This area also works as the start screen where you can toggle the music and sound effects and see the credits screen. If while playing the game you want to turn off the music then you are forced to walk all the way back to the logs. It is not the most efficient solution  but it does preserve a single seamless world.

Then we select a berry and just by holding your finger down on the screen the berry will walk in that direction. Here you can find the special tree that acts as an entrance to the underground lair. When you tap the tree we immediately start loading the underground. While the Berry travels down in the lift the rest of the underground graphics are loaded and rendered. Look to the right and you can see the chairs and tables pop into existence.

Now as the Berry walks over to the camera studio there is something that is not as planned. Originally the plan was the wall would split apart and reveal the camera view finder instantly. To do this the camera was turned on while the Berry walked. But after much trial and error it seems to be impossible to start the camera on iOS and not have it cause the UI to stutter. Once the berry reaches the crack in the wall the camera is started and you just don't see any missing frames as so little in the scene is moving. It does sadly add a noticeable pause in an otherwise fluid animation.

The berry returns up to the forest and you will see the beanstalk pops into existence as its loading is triggered once you get near. The same goes for the bird game at the far end of the forest. 

Each area had its own technical problems to be solved to avoid any skipping in the toy. In some cases the design of the toy, such as with the underground camera had to be designed around issues that would cause frame skips.

Closing thoughts.

Looking back there were two other big issues that are constant recurring themes with this kind of work. The first is that a lot of teams just don't care about the performance of the UI. That is hitting and maintaining a silky smooth 60 frames per second experience. It can be really hard to debate this as sadly few people consider performance a feature. "Does it really matter if the UI skips a bit. You can still do everything in the toy". "Would you really want me to spend time solving this glitch instead of adding some other cool functionality?". But without the silky smooth UI the whole trick of hiding the loading would not work. And without that trick the fact this is software running on a computer would start to show its face. The toy would become secondary to the underlying technology. More on this in some later posts.

The second issue is that having this seamless world caused a lot of debate in the team. Angry debate. Was this worth doing? What counts as a fair trick, could we cross fade from one screen to another? After all films do this all the time. At one point instead of a beanstalk we had a house. How could the berry get into the house without something like a cross fade? In the 1948 film, Rope, by Alfred Hitchcock was released. It appears the whole film is a single take. But of course you could never film a 2 hour movie in a single take. So a number of tricks are employed. Some as simple as having cast member walk across and block the camera to disguise a cut. Now these are ticks, but fair tricks. Was there really a cut when someone walked past the camera, or was it actually part of a longer cut? If it still leaves the audience guessing then the trick is fair. So cross fades got the boot as the wrong kind of trick. Once this was agreed innovation such as the magic beanstalk were invented instead to solve the issue of where the puzzles should live. Some research showed up Disney's multi-plane camera technology used in many of their cartoons and used by us when the berry walks to the bird game at the end of the forest.

Again high standards lead to innovation. Achieving this seamless world was a lot of work, but also a lot of fun. I hope you enjoyed seeing behind the curtain.

 -------------------------------------------

Support this blog by purchasing Berry Forest from the App Store here. 

Innovation in Orientation

Berry Forest is a kids toy. You can create your own Berry while exploring a forest handcrafted out of modeling clay that is full of animals and mini games. You need to be aged between 2 and 6 to get the most from it. But I want to touch on an aspect that is more of interest to general application designers. Berry Forest has something that if not unique - I have not checked all 800,000 apps on the App Store - is certainly rare. That being a sophisticated app specific orientation change animation.

 

Orientation change what what?

The video above shows the standard iOS orientation animation. Orientation is the way a device is oriented, that is the way round you hold it. Portrait orientation is where the home button is at the top or bottom. While landscape is when the home button is at the left or right. As you turn the device round the current application animates to show itself in the new orientation. On iOS a number of Apple's own apps have interesting orientation change animations. Open the Stocks app on the iPhone an move your phone between landscape and portrait to see the stock graph animate about beautifully.

The video above shows the orientation change animation in Berry Forest. It kicks of with me showing the real app on an iPad Mini, just to prove no cheating is involved. Then the first stock rotation is shown. Nothing new here. Then after that is the special forest specific animation. 

The power of Zing™

Start Berry Forest and you should be struck by the ultra smooth 60 frames per second interface, even on older devices such as the iPhone 4. When we started out to make Berry Forest we knew we would be developing our own engine to build the app with. Building your own engine and tools is cool,  too many of the existing frameworks are difficult to work with and finally they don't offer the kinds of graphics and animation abilities we need. So Berry Forest was co-developed with this new engine that we call Zing. The more involved rotation animation is also something having our own engine made possible.

 

 iOS system UI

iOS system UI

Smoke and mirrors

Understanding how this animation was done also gives some insight into why so far this potential rich area for some fun animation has not yet been mined.  First of all we disabled the standard app rotation in iOS. In reality the app is always running in the default landscape orientation you see when the home button is on the left hand side of the screen. 

So inside Berry Forest we just rotate the scene and run that upside down if you flip over your device. Would there be any clues to how we do this? iOS has a system UI that shows a variety of menus and popovers. In the above screen shots you can see the volume indicator, the notification drop down and the recent apps bar. There are also items such as the low battery warning dialog. All of these would be upside down. Luckily iOS has an API where we can tell the device which way up any system UI should be shown. This just leaves one issue left. When showing the notification drop down or recent apps list we have no way to rotate the screen. iOS is in control here, not our app. So for now if these two parts of the system UI are showing you turning your device to a new orientation does nothing till you go back to the app or close it. It is a bit disappointing that this clue exists, but at least for now iOS offers no way that we are aware of to also us to do the detailed orientation animation we want and allow the standard system animation to be used when the system UI is showing.

Improvements

Unlike most iOS applications we don't stop any of the other parts of the app animating or disable touch. At the end of the video above you can see that the berry carries on walking to the logs and the camera keeps on panning, all while the separate layers of the forest spin round. There are also no dropped frames so the experience is smooth and fluid.

The in-forest animation works by rotating each of the six main layers that make up the forest one after the other. To add a bit of variety when you rotate the device one direction the layers rotate with the front layer starting first and then it works its way back. While in the other direction it starts with the back layer and works its way forward. 

All the layers still rotate in the same direction and with almost the same amount of easing as the stock iOS rotation animation, so it still has a subtle designed for iOS feel.  

Closing thoughts

First of all this animation serves no real purpose. But then again neither does Berry Forest because at its root it is a toy. If it has a purpose it is to entertain. It has creative aspects such as the ability to build a berry. The animation and responsive UI reward exploration. The puzzles may help the development on hand eye coordination and stretch your thinking. But no more or less than many other toys. Toys should be fun and encourage play. They also should sometimes have a bit of hidden fun. I wanted to have a cool orientation animation and the aim was to make it playful. At least on those two points it hits the mark. If there had been more time it would have been good to have created animations for other areas of the toy such as the underground Berry Builder. There was also a plan that after spinning the layers of the forest the berry would have been left all alone in the air. Then gravity would kick in and it would have fallen to the newly rotated floor. Again time got in the way of making this a reality. But there seems to be a rich array of possibilities for the future and now we solved the big issues to allow these kind of app specific orientation animations you can expect some even more fun ones from us in the future.

 

If you want to support this blog and the creation of more articles like this consider getting a copy of Berry Forest from the App Store. It is universal for iPad and iPhone and currently on sale for just 99 cents. 

By way of an introduction

Here goes nothing. In 2008 I authored a Nokia internal blog called Unintuitive. Chronicling issues with both products I worked and didn't work on, the technology used to create great user experiences and of course how fantastic Finnish chocolate is. It was a small part of a thriving community of internal dialog all centered around a central blog hub. Surprisingly it rapidly became an important part of the companies internal dialog and the community even made the news in Business Week. It gave people a chance to talk across the silos and bureaucracy that inevitably occur in any large company. As well as shine a light of sanity on the crazy. Ah, the time a new strategy was presented using a theme from The Matrix. But instead of inviting everyone to take the red pill, it was decided as it matched our corporate colors we should all take the blue pill.

One of the great blogs was The S&S Intelligence blog written by a certain Horace Dediu. A data centric blog that was a precursor to the phenomenal Asymco. By late 2010 these blogs had started to decline. Then a triple whammy of a  change in strategy, a disruptive new way to communicate and finally a technology migration (death march?) to SharePoint 2010 ended the conversation. Recently some of these blogs have been born again, but public for all to see. The Needle is now Thndl. and J-P's BoxScore became Real Box Score Some written by people who have moved on from Nokia and some who are still there. So here is the new public version of Unintuitive. It won't have anything about Nokia or directly related products. So despite my long history of working on Symbian products there will be nothing about that, or S40, the latest Asha phones or Window Phone. There will also be no Hulk and no comments. If you want to comment then feel free to mail me or just join the conversation on twitter where I live as @designker. So what will be here at Unintuitive? There is no clear plan right now. But with some friends in our spare time we have just shipped a children's toy for iPad and iPhone. You can see more over at Kanjo arts. After 15 years of working on the design of smartphones it has been a difficult and interesting change to suddenly create something so different. So lets start with some posts about what went into that. Expect a post about the perils of creating an unrestricted berry builder and the problems you get when you combine that with a multitouch UI. Coming soon...