Apple CarPlay in depth

However, the news regarding CarPlay has been rather scarce in comparison. CarPlay didn't get much play during the opening ceremony of the WWDC, or Worldwide Developer's Conference, either. And in the developer sessions, they didn't really dig into the announced changes. Instead, they offered a few sessions dedicated to the inner workings of CarPlay, which is rare for Apple. But given the opportunity, we'll go over the current state of CarPlay with some details of its inner workings and then see what changes the bitten apple brand is preparing.


Current Status

Starting with the concept, it's worth asking "what is CarPlay?" It's a method devised by Apple to extend the operating system of its mobile devices, iOS, to the infotainment system of our cars. Its premise is very similar to that of Android Auto; not for nothing was CarPlay introduced 11 months earlier (June 2013). The true origins of CarPlay date back to 2010, when Apple and BMW announced a collaboration agreement whose fruit was "iPod Out", a feature that allowed certain cars of the Bavarian brand to display the interface of an iPhone or iPod in its infotainment system. Through it, BMW drivers could access their iOS device's music library and browse it from their car's screen using the vehicle's own controls. iPod Out would be released in 2011, and when it was extended to Mini vehicles, it would be called "PlugIn".

Today in CarPlay is available in more than 100 models from 40 different manufacturers, including Ferrari, due to the fact that Eddie Cue, Executive Vice President of Software and Internet Services for Apple, belongs to the board of directors of the brand of the Prancing Horse since 2012. In any case, the growing availability of CarPlay in new vehicles, like that of Android Auto, is supported by the fact that most manufacturers are opting to have both systems available in their cars, lest someone not buy a Honda NSX because they have a Samsung instead of an iPhone. Ferrari, due to its management ties, is an example of a manufacturer that has opted for only one system.


How does it work?

When we get into our car and sit down, if we connect our iPhone via a USB cable we will see the CarPlay interface take control of our vehicle's infotainment system. This doesn't mean that every time we connect our iPhone we necessarily have to use CarPlay; if we exit CarPlay with our phone connected and don't come back, the next time we connect our iPhone CarPlay will respect our preference to use our car's infotainment system. If, on the other hand, the last time our iPhone was connected CarPlay had control of our infotainment system's screen, it will return when we re-connect the iPhone.

This behavior also extends to the use of the speakers; if the last time our iPhone was connected we listened to music through CarPlay, when we connect it, the playback is resumed. If, on the other hand, even with the iPhone connected we were listening to music through the car's infotainment system, connecting the iPhone again will not affect playback. Therefore, CarPlay shares with the infotainment system of our car the control of the screen, speakers and microphone, respecting at all times our preferences for use. That is, we can listen to music from CarPlay but keep the infotainment system of our car on screen, and vice versa.

Software

In terms of applications, CarPlay meets the basic needs: calls, calendar access, messaging through Apple's iMessage service, turn-by-turn navigation with Apple Maps and music through Apple Music or third-party applications. Through its voice recognition system, we'll be able to type messages without taking our eyes off the road, and Apple's own voice assistant will take care of reading incoming messages to us. When it comes to basic functionality, CarPlay and Android Auto are on par.


The curious thing happens when we decide to pull back the curtain that is the user interface, and look at what's behind it. I've been a professional programmer of both Android and iPhone applications, so my vision of both systems goes beyond the simple differences at user level. And from this point of view, both mobile operating systems have a lot in common: they implement the same concepts, which means that practically everything you can do with one can be done on the other. When one gets ahead of the curve with a better implementation or a new feature, within a year or two at the latest, it appears in the other. CarPlay and Android Auto are examples of this: first Apple announced it, a year later Google did it.

And why am I telling you this? Because the internal differences I see between CarPlay and Android Auto are even smaller. As with Android Auto, in CarPlay all video and audio emanates from the iPhone, so the premise remains that to update CarPlay you don't need to update your car, you need to update your phone's iOS operating system. And regarding third-party apps, it turns out that Apple and Google have decided to restrict them in the same way: no third-party app can design its own user interface. And the only apps that are allowed for now, are audio players like the ones we would use for music, radio and podcasts.

What does this mean? That an audio player that wants to appear on the CarPlay screen simply sends you the list of possible content to play, and once the user chooses a song, CarPlay will ask that app to play that content, but no specific UI designs. The result? Increased security, as we discussed in the case of Android Auto, but for the user all apps end up looking the same.


The manufacturers' own apps, which are built into CarPlay, get a separate mention, but they're not third-party apps, so we'll discuss them below.

Fundamental requirements

Apple declares a series of minimum requirements that must be met by the infotainment system in cars that any manufacturer wants to make compatible with CarPlay. Those requirements fall into five categories: display, speakers and microphones, user interaction methods, sensors, and connection to an iPhone. The first two categories are very logical; in order to display the CarPlay interface, the car needs to have a touchscreen with a good resolution (the minimum supported is 800×480 and the maximum 1920×1080 or Full HD).

Regarding the speakers, it is also necessary that the vehicle has at least two audio channels, one main and one for notifications, so that for example the user can listen to music and at the same time receive instructions from the navigator to follow the desired route. The microphone is essential to be able to perform tasks using voice recognition. And since the physical connection to the iPhone is obvious, we will focus on the most interesting: the methods of user interaction and sensors.

CarPlay supports three methods of user interaction: the touchscreen, the car's own built-in knobs and controls, and voice recognition. Voice recognition, of course, is performed by Apple's Siri assistant. Siri, like any good digital assistant in 2016, is not only limited to reading our incoming messages and helping us respond by voice while driving, but can also perform actions such as changing the radio station, playing music by a particular artist, finding routes to our destination, calling a contact, showing us the day's agenda and so on.

Apple requires Siri to have a physical button on the steering wheel so that the user can access this assistant at any time, explaining that access to Siri is the equivalent in the car of the button that takes us to the home screen on our smartphones. In the event that the car's infotainment system has its own voice recognition system, Siri can be activated by a long press of the button (about 600 milliseconds), so the user can access the two systems when an iPhone is connected to the car.

Regarding the sensors, CarPlay needs to know the speed of the car at all times; why and what uses this information is put to is not specified. It's not hard to imagine that CarPlay could use it to send an SOS message if it detects a collision by abruptly slowing down, but we don't have any more information on this. If the car has satellite positioning systems such as GPS, it is a requirement that CarPlay has access to this information. Furthermore, if there are in-car positioning systems, CarPlay requires the use of both GPS (American system) and GLONASS (Russian system), and also recommends the use of the Galileo positioning system (European system).

Wireless mode

The first significant change that Apple will introduce in CarPlay will be the possibility of using this system wirelessly. That is, we will be able to leave our iPhone in our pocket or in the glove compartment of the car and see how CarPlay is enabled in our car after unlocking it and sit in it without doing absolutely nothing. Regardless of whether we connect the iPhone wirelessly or wired, CarPlay will respect our decision to control the infotainment system's display and sound source. However, it will probably take longer for the car to detect the presence of the iPhone wirelessly than when connected via a cable.

How do you make the connection? If you remember our extensive article on Android Auto, we explained why we opted for Google to implement Android Auto's wireless mode via Wi-Fi and not Bluetooth. In that case Google didn't detail how the feature would be implemented. Apple, on the other hand, has explained how it will do it: it's through a combination of Bluetooth and Wi-Fi. Bluetooth will be used to initiate communications between the car and the phone, and once these are established, it will switch to a Wi-Fi based connection to transmit the data itself, severing the Bluetooth link.

Recall that the iPhone must transmit to the car's infotainment system all video and audio, where video requires a large amount of data to display each and every pixel on screen. In addition, it is essential that the latency or delay between the user interaction, either through the screen or one of the controls, with the car is as low as possible, because otherwise the user will immediately notice this wait. And as we all know, if there's one thing that frustrates us all when using any kind of computer system, it's entering a command and seeing that it's not executed immediately.

In addition to Wi-Fi and Bluetooth, Apple points out that a prerequisite for wireless mode is that the car must have its own satellite tracking system. This is because, according to Apple, in wireless mode the chances of the iPhone being in a pocket or bag are very high, so the phone will have trouble detecting its position. In order for the user to always be able to count on the correct functioning of the maps, it is necessary that the car meets this requirement. In the case of a wired connection, CarPlay will use the iPhone's geolocation sensors if the car does not have them.

Given this requirement, it is possible that despite having CarPlay in our car, if we don't have the navigation kit we won't be able to enjoy the wireless mode. And even if we do have it, it remains to be seen if the requirement for satellite geolocation systems applies retroactively to all cars on the market with CarPlay; if an incompatibility for cars already on the road comes to light in a few months, I won't be surprised at all.

Turn-by-turn navigation in the instrument cluster

This is one point where Apple has barely wanted to give any details. Since CarPlay is part of the iOS operating system and this improvement has been introduced alongside the new iOS 10 version, it's possible that we'll have more details come its release around September. But for now, what we do know is that Apple is improving the navigation mode in its Apple Maps app for iOS and that these changes will be reflected in CarPlay.

In iOS 10, the navigation offers a better view of the route ahead, "zooming in" when we approach a turn and zooming out when we reach a straight or a road/highway that we don't have to leave soon. We can also move along the route to get an idea of this and at the same time see what are the areas of greater and lesser traffic density. Finally, CarPlay will offer us alternative routes (faster) if after choosing one the system finds a more optimal one.

In the car, and as seen in the photo above, it is added that CarPlay will be able to display the instructions of the route to follow on the auxiliary screen of the instrument panel. We understand that this new feature may not be possible on existing models after the update, unless the use of such a car resource was planned from the first version of CarPlay. Accessing a new in-car resource involves using communication protocols with the car to decide who gets to use that screen, similar to how the infotainment system and iPhone communicate to decide who gets control of the screen in the middle of the dashboard.

In any case, it seems that Apple, unlike Google, is not yet ready to take full control of the dashboard, or that it has decided not to go down that route, so CarPlay will be limited, according to what we've seen, to helping us follow our route without looking at the dashboard but at the instrument cluster.

Messaging via Siri and Manufacturer's Apps

In the "Software" section of the "Current Status" section we mentioned that CarPlay only allows third-party apps that are audio players, while Android Auto adds the ability to have third-party messaging apps, such as WhatsApp and Telegram. Apart from that, in our extensive article on Android Auto we also mentioned that Google is working on incorporating apps tailored for each vehicle.

The problem with allowing third-party messaging apps in CarPlay lies with Siri, which handles both voice recognition and transcribing the user's message for delivery without the need for the user to take their hands off the wheel. Siri is a problem because until the developer conference held recently, there was no API or programming interface so that other applications could teach Siri to perform actions with them. For example: if we tell Siri to "text Javier," it will always use Apple's messaging app, iMessage. Now, with the API just announced for the next version of iOS, an app like Telegram can teach Siri so that we can say "send a message to Javier via Telegram" and have the command recognized correctly.

This API now makes it possible for third-party messaging apps for CarPlay. Remember that both Apple and Google control the user interface at all times, both for preinstalled apps and third-party apps, so the glue that allows us to use other applications to send messages while maintaining the standard interface is voice recognition, a problem that Apple has just solved for the next version of CarPlay. Therefore, Apple has announced that third-party messaging apps will officially be supported in the next version.

As for the applications made by the vehicle manufacturer, these have been allowed by Apple since the launch of CarPlay, and are published by manufacturers as apps for the iPhone in the App Store and after being installed on our phone appear as one more on the main screen of CarPlay. It is likely, although we do not know for sure, that they have privileges within the operating system to access APIs not allowed for third-party applications. Why do I think this? Because for starters these apps will be able to draw their own user interfaces on the screen, something third-party apps can't do. So it's also feasible that these apps will have access to all the car data that CarPlay knows, such as vehicle speed and location information.

What I can tell you is that from now on manufacturer apps will have access to Siri, opening up the possibility of asking her to perform car-related tasks. For example, if such information is accessible to CarPlay (because the car's corresponding control unit sends it to it), we could ask Siri about the state of our tire pressure, or to tell us the average fuel consumption during the trip. These are small details that, once again, can keep our eyes off the road, even if only for a few seconds. And if the interface between the car and CarPlay allows it, maybe in the future we'll be able to ask Siri to lower all the windows in the car, or to raise them and turn on the climate control at a certain temperature, to give examples.

Of course, the more integration, the more careful manufacturers and developers (in this case Apple and Google) must be with security. And if we have learned anything so far, is that anyone can have an oversight (or two), and therefore, we will always have hackers to discover them.

Conclusions

For those of you reading this, CarPlay doesn't work with an iPad, so you can stop thinking about how you're going to connect your iPad to your car and leave it there so you don't have to use your iPhone. You also won't be able to dock your iPad Mini or iPhone 6/6s Plus to the dashboard and have CarPlay next to the instrument cluster because there is no such functionality; this is something I don't see Apple adding, but who knows?

Personally, the message I'm taking away from this Apple developer conference is that CarPlay hasn't been given much attention. There's been a lot of talk in recent years that Apple has a very limited software development team compared to other Silicon Valley giants, and that these teams are sometimes plucked from their day-to-day work to support another, more important team. I highly doubt that CarPlay has its own team; rather, I think it's an extra feature of iOS that was very important at launch, and that they're now focusing on other things.

Who knows if they're busy working more on their car's infotainment system. It's also true that Apple doesn't usually talk about their plans until they're finished, so maybe we'll be talking in a few months about changes that weren't announced at the developer conference but have made it to the final version. Until then, we'll be watching until these announced changes see the light of day with the release of iOS 10 and the new iPhone around September.



Add a comment from Apple CarPlay in depth
Comment sent successfully! We will review it in the next few hours.