Incoming Calls Shouldn’t Interrupt Video Recording on Smartphones

Summary: Smartphones should not interrupt users’ video recording sessions when calls come in, especially if users choose to ignore those calls, whether through inaction or via tapping "ignore" on a notification. Yet, this is what happens with the phones running the current leading mobile Oses, iOS (5, 6.1) and Android (including 4.1 JellyBean). In contrast, webOS v2.2.3 and Windows Phone 7.5 only stop recording when users choose to take the incoming call. The quick UX lesson is easy—that the latter behavior is how it should be; do not interrupt users’ activity unless absolutely necessary. The larger UX lessons include: prioritize users’ activities over system ones when possible and offer both more flexible multi-tasking and more thoughtful use of unobtrusive notifications to accommodate that prioritization.

Recording Life’s Little Moments

Imagine this scenario: it’s graduation season. You leave to attend a graduation ceremony, and you plan to record it for memory’s sake.

You’re recording away, and you get a phone call. The phone rings, and notifies you that you have a call coming in.

You don’t want to answer, and you just ignore the notification, waiting for the phone call to just go away.

Or, you don’t want to answer, and you respond to the notification choosing to ignore the call.

What should your smartphone do?

  1. Continue recording without a hitch.
  2. Stop recording.
  3. Stop recording, and do not save what has been recorded so far.

Users and those interaction designers who understand and prioritize human experience and their tasks know the answer is A.

How do today’s smartphones behave?

They behave in accordance with their interaction designers’ designs and their developers’ programming (with the caveat that bugs may be responsible for some disparities):

  • iPhone 4S with iOS v6.0.1 (Sprint): B
  • iPhone 4S with iOS v5.1.1 (Sprint):  B
  • Samsung Galaxy S3 with Android v4.0 (AT&T): B
  • Droid 4 with Android v2.3.6 (Verizon): B
  • Droid 3 with Android v2.3.4 (Verizon): C
  • HP Veer with webOS v2.2.3 (AT&T): A
  • HTC Arrive with Windows Phone 7.5 (Sprint): A

(Reported results are based on a very limited data set–one phone each from among this blogger’s circle of friends and family.)

I have an iPhone 4S, and this scenario happened to me during a graduation ceremony, twice.

UX: How do we learn from this?

As an interaction designer, what might one learn from this slice of user experience?

The simplistic reaction is to treat this smartphone behavior as a user experience bug and design a workaround–perhaps simply prevent the notification from stopping the recording until users actively choose to accept the call.

Because this problem could have OS-level architectural implications, this simplistic reaction is, at its best, a re-categorization of the incoming call notification as an existing class of notifications that do not interrupt users’ activity, or , at its worst, a band-aid or a one-off solution.

An arguably more insightful (and resource-intensive) reaction is to consider what wider implications can be inferred from this disparity between user behavior and smartphone behavior.

Or better yet, imagine a blue-sky scenario and consider a way beyond what might be expected today and the design implications of that.

Let’s Do This: Blue Sky Scenario

A "blue sky" scenario is an idealized one, unfettered by current limitations. They can be instructive, suggesting directions for guidelines, solutions, and further research.

So in the scene above, with a user recording an important video and a call coming in, what might be an ideal scenario be?

I note that a call is coming in. I take some attention from the activity of recording to consider the incoming call: who is it, and do I want to take the call?

Against the possibility that I would like to take the call, I’d like the phone to be able to continue recording and still allow me to take the call.


Notifications on phone (banner, iOS), BT-connected watch or headset, with options to answer, ignore, or reply via message.

Possible interaction flows: Incoming call during video recording. With the headset and/or the watch connected, the phone may even optionally not show its notification so as not to occlude the recording. Alternatively, we might leave the banner there to keep the locus of users’ attention on the phone while they interact with the notification.

Social environment aside, what if the phone could direct the call to a Bluetooth headset,  and then filter out audio input from that headset from the audio being recorded with the video?

Even more exotically, maybe that notification could be shunted onto a connected watch, so that the notification doesn’t scrunch or impinge on users’ view of the video that they’re recording.  The viability of that notion, of course, requires more exploration, involving as it does the question of users splitting their attention to a different display while  trying to record something.

That brainstorm aside, the audio filtering would be very resource-intensive, true, and possibly impossible given current smartphone computing power….

Lessons and Ramifications

Nevertheless, as a blue sky scenario, it is suggestive of guidelines and ramifications for interaction design, some of which we should already know:

  • Without users’ specifying ahead of time (for instance, via a setting for how disruptive an incoming notification can be), a notification should not disrupt or interfere with users’ activity.
    • In other words, the phone should wait for user direction before interrupting the recording. If the user does nothing, just waiting for the notification to go away, we have option A from the introduction.
    • iOS 5.1.1 does allow very granular control about how notifications  behave. Yet setting the "Phone" app’s notifications to None or Banners (the ostensibly non-modal choices) versus Alerts (the default, and modal) does not change my iPhone 4S’ behavior here; the incoming call notification remains a modal dialogue box that ends my recording upon appearance.
    • Notifications should never result in lost data. In other words, option C for the interaction, which the Motorola Droid 3 does, should not be acceptable.
  • Paying any attention at all to a notification and considering it while recording video should probably be considered an example of users multi-tasking.
  • The smartphone or mobile OS would need to support actual multi-tasking.
    • The smartphone may need to be able to multi-task to both record and present a notification and wait for users to decide whether to take the call.
    • The smartphone must support multi-tasking if it is to manage the video recording, a phone call, and the audio-processing to filter out the phone’s audio stream from the video’s audio stream.

Implications for the Future of Mobile OSs

Converged devices, considered in the context of users’ multi-tasking, argue for more flexible multi-tasking capabilities rather than more restrictive ones, and for notifications that do not preemptively disrupt the flow of users’ activities.

This is true even given any expectation of users having an increasing number of connected devices, because, ideally, those devices will be able to coordinate with each other in the background, while in the course of supporting users’ activities.

What does this mean for the current "crop" of mobile OSs?

iOS: Option B (Stop Recording)

iOS 5.1 comes from a line of work whose central assumption was a single-tasking environment, in which any notification was modal, which carries the assumption that that notification is more important than whatever users are doing at the time of the incoming notification.

iOS 4 and 5 introduced the ability to switch apps quickly, allowing limited multi-tasking for certain classes of background-able app activity and notifications that were not modal. That limited multi-tasking and those non-modal notifications could have been enough to enable the iPhone 4S to handle this "incoming call while recording" situation gracefully, but they were not used to handle this situation gracefully.

What happened with this interaction between video recording and incoming phone calls, then?

Whether this is an oversight, an intentional trade-off choice (perhaps for reasons of scope), or a bug, the iPhone 4S with iOS 5.1’s behavior here is probably just inherited from iOS’ single-tasking, modal notification legacy.

I’d hope that this issue is on their list of things to address as they evolve from their legacy. I suspect that pursuing anything like the blue-sky scenario above would require significantly re-working the multi-tasking framework that they introduced in iOS 4, and, given Apple’s history in dealing with users’ potential desire for flexibility, that doesn’t seem likely to be a high priority.

Android: Options B (Stop Recording ) and C (Stop Recording and Delete Recorded Video)

In contrast to iOS, Android is often touted as the mobile OS for power users and those who are willing to get their hands "dirty" to wrestle it into whatever form they please.

As such, multi-tasking has been among its claimed edges, and its notification drawer has clearly had an influence in iOS 5’s notification system.

And yet, here we have Motorola’s Droids 3 (Android v2.3.4) and 4 (v2.3.6), both stopping their recording process as a call comes in, even if users ignore the call by choosing not to act on the notification at all. The Droid 3 even loses the recording in progress. Thus, Android’s responses in this video recording scenario in versions 2.3.4 and 2.3.6 are options B and C, respectively.

Whether or not the loss of already-recorded video in the case of the Droid 3 is a bug, I would have expected Android to have been able to show behavior along the lines of option A. That it does not is somewhat puzzling.

Android should have the foundation to build itself into a system that can support continued recording in the scenario under scrutiny (thus, option A).

webOS: Option A (Continue Recording)

While never a commercial success, Palm’s (and then HP’s) webOS was often lauded as a promising mobile OS, built from the ground up to support multi-tasking and to provide an unobtrusive notification system.

With what was commonly recognized as the best multi-tasking and unobtrusive notifications as two of the OS’ central strengths, it is perhaps not surprising that this OS, along with Windows Phone 7, handles the situation at issue most elegantly.

As with all other webOS notifications, incoming call notifications do not disrupt the recording activity. Ignoring the notification through complete inaction by the user lets the notification go into the notification row and allows the recording to continue uninterrupted. Ignoring the notification by tapping the "Ignore" button dismisses the notification and also allows the recording to continue uninterrupted.

webOS has a very uncertain future, and has at least a great many challenges ahead of it. On its ability to handle this scenario, however, its central tenets of mult-tasking and unobtrusive notifications puts it at the head of the class, alongside Windows 7.5.

Windows Phone 7.5: Option A (Continue Recording)

Like webOS, Windows Phone 7.5, despite not having multi-tasking support, also handles this video recording issue elegantly–whether users choose to ignore the notification through inaction or through actively choosing an "Ignore" button, the video recording continues uninterrupted.

Its ability to continue recording in this scenario puts it at the head of the class, matching webOS–and this without multi-tasking support.

Back to the Issue at Hand

So, with all that said, in terms of current phones and OSes, how important is this issue?

In the particularly egregious case of the Droid 3’s behavior, losing users’ data should really be unacceptable, and it should be addressed with fairly high priority.

That data loss case aside, how important is addressing the issue of a notification of an incoming call stopping your recording?

In terms of justifying the resources necessary to address the issue, one might hear the question, "how many users would this affect?" Couldn’t users be encouraged to just use a workaround, like putting their phone into airplane mode before they start a video recording?

Those are a fair questions, but here’s another one: would you rather be the smartphone/OS that has to sweep these incidents under the rug, or the one that has to encourage users to remember to go into settings to switch to airplane mode before they record anything and have to remember to go back into settings to switch back out of airplane mode, or would you rather be the smartphone/OS that can brag about being able to let you just record your or your loved one’s life’s moments without fear of being interrupted by your "smart" phone or without having to switch modes first?

That latter question is rhetorical.


So what are the lessons here?

The easy answer is that incoming phone calls should not interrupt video recordings without users’ express choice. The question then becomes how to design and implement the interaction to provide that easy answer.

Beyond that, I would suggest that there are a few other implications about things to consider:

  • The nature  of notifications. When, if ever, should they take precedence over users’ activity such that the phone or OS is justified in halting users’ activity? The answer should be somewhere close to never, with exceptions made for danger to users or their data. This guideline should sound very familiar to UX, interaction (Ix), and user interface (UI) designers.
  • Multi-tasking: Under what conditions do people multi-task? And how is that relevant to a smartphone’s OS? Can an OS support that behavior? If so, should it? And if so, how can an OS support it? What is the importance of supporting it?
    • Ignoring a notification, reading a notification, considering a response to a notification, responding to a notification, these are all examples of users managing and likely splitting their attention between their current activity and considering another activity. This is multi-tasking by users that should be somehow supported by, or at least not betrayed by the system.
    • Why "True Multi-tasking"™ versus "fast app switching" or "limited multi-tasking"?
      • Can you predict all the situations in which users might need to consider or do more than one thing at once? No? Then maybe your system should be more flexible rather than less.
Posted in UX Gaffes | Tagged , , , , , , , , | 5 Comments

Amazon MP3s Now Buyable Via iOS: Grants +2 to Its “Your Music, Anytime, Anywhere, Any Device” Aura

The Amazon MP3 store is now optimized for iOS devices….[so now], you can purchase music from Amazon’s catalog…directly from an iPhone or iPod touch…. Additionally, purchases will be available for immediate streaming and download from within Amazon’s Cloud Player iOS app.

via Amazon now selling MP3s directly to iPhone, iPod touch users | The Verge.

This is Amazon fleshing out their coverage of user activities—with their ability to give you access to your already-purchased content from so many different hardware-tied ecosystems (iOS, Android, PC/Mac) already, they had a pretty thorough coverage of the user-focused goal of "my content, anywhere, from any device" (e.g., their Kindle, Cloud Player, and their respective apps). This move adds the coverage of the user activity "adding to my content" in a way that makes it immediately accessible—again, anywhere, from any device. That coverage of most of most users’ music and other content activities is a key part of iTunes’ success.

Along with their recent AutoRip feature, which puts any AutoRip-enabled CDs that you purchase or have purchased from them going back to 1998 automatically, instantly in your Cloud Player, Amazon’s music buying options may well be the fastest way to make your music purchases available "at any time, from any place, no transferring or synching–your music, everywhere" (from one of Amazon’s Cloud player videos).

More of what most users will want to do with Amazon’s content will now be even more platform-independent. This is Amazon building out their their content-delivery and storage ecosystem to out-flank all the hardware-tied ecosystems’ obstacles, offering those users a way to loosen those other ecosystems’ hold on them.

What could be next goals for Amazon in this vein? Maybe giving that same freedom to the other content that they sell: books, hopefully in various bundles with their ebook and Audible audiobook versions, and movies (offering bundles or just straight bundling streaming rights or digital copies with purchases of DVDs and Blu-ray discs).

Freeing users of arbitrary constraints? That’s a good thing.

Posted in Commentary | Tagged , , , , , , | Leave a comment

Nuance’s Wintermute Could Be a “PDI-Level Property/Service”

…Project Wintermute [is] a platform agnostic voice recognition service that lives in the cloud…. intelligently saving search queries and context across platforms.

via Nuance hopes to make cross-platform voice recognition a reality with Project Wintermute | The Verge.

As a platform-agnostic personal assistant, Wintermute would be perfectly positioned to free users from vendor-delineated ecosystems. Its ability to work with multiple—in an ideal world, all—vendors’ ecosystems, would free users to use whatever ecosystems’ components they wanted, with Wintermute serving then as a PDI-level (Personal Digital Infrastructure, i.e., the users’ ecosystem of hardware, software, content, services, and social connections) service, or a property of the PDI.

Imagine: Whatever you want to use or access, regardless of whether it is beholden to iOS, Android, Windows Phone 8, webOS, or Blackberry—just talk to Wintermute, and she’ll take care of it. Not quite Jarvis, but a step in the right direction.

1 Jarvis is the name of Tony Stark’s and Iron Man’s computer in the movie, "Marvel’s The Avengers" (2012).

Posted in Commentary, Editorials | Tagged , , , , , , | Leave a comment

User Experience of Ecosystems

Ecosystems: Past, Present, and…

The rise of Apple’s iPod inspired the use of the word “ecosystem” in the computer industry–at the time, primarily referring to the pairing of software and hardware that was iTunes and the iPod, respectively. The use of the word, bringing with it its ecologically-based notion of complex and critical interdependence–was intended to suggest a system that should be considered together as a whole, each part contributing to the success of the rest.

Today, “ecosystem” in the computer and consumer electronics world refers to much more–smartphones, tablets, computers, accessories, computer operating systems, mobile operating systems, application stores for each of those, even developers of the software for any of those, cloud services, and web sites. Most of what is written about these things focuses on a particular piece–reviews of smartphones, tablets, computers, operating systems; the time has come to consider the next level and to actively create a vision of what that future ecosystem should look like.

User-Focused Ecosystems: Personal Digital Infrastructures (PDIs)

I believe that ecosystems should be considered in terms of user experience and actively designed as a whole to be user-focused; sure, the user experience of any particular smartphone or tablet is important, but, in a world in which each individual is potentially the owner of an increasing number of increasingly capable, networked devices, the important questions should turn from those like "how well does this device work" or "how well do these devices work together" to those like "how well can this ecosystem serve its owner in an elegantly concerted way?"

Our separate devices, software, services, data, and content should not only work together, they should work together to support their user–a sort of personal digital infrastructure (PDI) that would be an intelligent substrate enabling effortless access to all of the things we need and want in our ecosystem and seamless interoperability within it.

The user experience challenge is how to ensure that a user’s ecosystem enables to spend their time on their activities and tasks rather than wrestling with the technologies and tools they use to get those done.

PDI Features

There’s much more to be considered here, but I’ll start a sketch of that PDI with a list of features–but for the ecosystem, not of any individual hardware or software context.

These are some of the features that such ecosystems should have to be more user-focused than todays examples:

  • Intelligently provide access to personal or work documents, media, information, etc. from any capable and authorized devices in the user’s stable. An intelligent personal cloud that can provide ready access to the things that you need wherever and whenever you need them.
    • Apple’s iCloud, Microsoft’s Live Mesh and Skydrive, and Dropbox all have elements of this, but they all have a ways to go.
  • Behaviorally, object singularity across devices and personal systems where appropriate: documents, media, notifications, cloud services, OS configurations and preferences, application configurations and preferences, saved game status, etc. This hinges on the ecosystem either central storage or automatic, effortless, and fast-enough-to-be-perceived-to-be-immediate syncing of all relevant objects.
    • On notifications, consider whether you would rather be notified just once about an incoming email via whatever device or software context you’re using and then dismiss it that once, or once on each of your phone, tablet, computer, or miscellaneous web browser.
  • Continuous client capabilities across devices as appropriate–i.e., the ability to maintain task state, history, and progress.
  • Ability to work with and integrate disparate vendors’ services and products, yet keep them separate.
    • Ability to synchronize data between sources or not according to intelligent defaults and user preference and the ability to work with existing systems and repositories rather than force migrations to a single-vendor ecosystem.

These features are glimpses into parts of what I hope will come as ecosystems develop further.

PDI Design Guidelines

But spot solutions and lists of functionality are often unwieldy. Being design-minded, I prefer larger pictures and guidelines by which to discover and situate them.

So, I propose a set of guidelines for ecosystems as we advance them:

  • Actions on any one device have system-visible, system-wide results.
  • The ecosystem should enable synergistic effects rather than just additive ones as more devices are added.
  • Support potentially simultaneous, interleaved, or synergistic usage of multiple devices.
  • Maximize task time; minimize, if not eliminate, tool time.
  • Use intelligent behavioral defaults, but ensure that they are override-able via appropriate settings options.

Looking to the Horizon

These features and guidelines are signposts, sketching out what I hope will be on the road ahead. In later posts, I’ll expand on these and others.

Posted in Editorials, Visions | Tagged , , , , , , , , , , , , , , , | Leave a comment