Using High Accuracy GPS with Mobile Apps

There are times when the built-in GPS simply isn’t good enough for your Android or iOS app. Built-in GPS is good for finding a general location of a geographic feature such as a pond, field, or parking lot. If you need to locate a specific parking space, or a geographic feature within a field, or an inlet to a pond then you’ll need much greater precision of a commercial GPS unit.

Examples. Here is a more specific example to illustrate the concept. Let’s say the application users are mapping insect infected trees in a densely forested urban park. If you are using a smartphone GPS, with an average accuracy of 30+ meters when standing under the tree cover, the latitude/longitude coordinates may end up being unusable for future follow-up because there are dozens of trees within a 30+ meter radius. If you are using a commercial GPS with sub-meter accuracy you can have a much higher degree of certainty that someone will be able to locate that exact tree again.

I’ve seen entire days’ worth of data collection get thrown out the window because of significant location inaccuracies caused by using smartphones and dedicated hand-held consumer GPS units that are well known to outdoor recreational folks. I’ve accidentally sent a backcountry trail maintenance crew to a totally wrong location (they got really mad). All I had was a smartphone GPS and my location coordinate was over a thousand meters off even though the Android Location API returned an “accuracy” value of less than 50 meters. Oops — that’s the difference between precision and accuracy.

Good news. The good excellent news is there are small, handheld commercial GPS units that you can tether to your smartphone via Bluetooth such as the Trimble R1. Units like the R1 are able to override the onboard GPS and when configured correctly work just fine with the browser’s JavaScript-based Geolocation API or native iOS/Android geolocation.

The only downside can be cost. Some of these commercial units cost several thousand dollars and that may be outside the budget of many organizations.

Surprise, Surprise. I spend a ton of time talking to customers about this topic. Many people are very surprised because they regularly use GPS fitness tracker apps or Google Maps and the location information almost always seems very close to reality. The hidden trick is these apps use proprietary software algorithms that attempt to slightly improve accuracy, minimize location fluctuations or even intelligently snap your location to geographic features such as roadways, buildings or trails.

When you are building a custom mobile application, you will be forced to deal with raw, unfiltered geolocation data. And the coordinate data is coming from consumer-grade GPS units mass produced at the lowest possible cost. The antennas are tiny and usually on the back of the device rather than pointed towards the sky. The chipsets are designed to minimize battery usage, because GPS can be extremely battery intensive and generates a lot of heat. They have minimal ability to resolve communication difficulties between your device and GPS satellites, and GPS signals are already relatively weak.

Decisions, decisions. If your budget can’t handle several thousand dollars for a commercial GPS and you only have short term needs you can always look into renting a bluetooth unit. Just make sure to ask if it overrides the onboard GPS chip. You can find GPS rental companies online. There are also consumer-grade Bluetooth GPS units that only cost several hundred dollars. The consumer Bluetooth units aren’t nearly as accurate as commercial units; however, they offer a significant improvement over built-in GPS.

Advanced geolocation plugin for Cordova and PhoneGap for Android

The W3C’s browser-based, JavaScript Geolocation API is excellent as a one-size-fits-all interface, but that approach comes at a price and it can cause some serious limitations when it comes to implementing more stringent professional, commercial and government use-cases.

Challenges with the default Geolocation API. One of the primary limitations is the Geolocation API does not tell you how it got a location. All locations are lumped together in a black box. Let me explain. On a smartphone or tablet, location data comes from one of three places: the GPS chipset, the cellular provider’s Location Service, or the browser’s Location Service. The W3C Geolocation API simply lumps these data points together. The end result is typically seen by the end user as significant and disturbingly wild jumps back and forth in the reported location, sometimes over large distances. A key to minimizing these fluctuations is to gain back control and understand which location provider created the latitude and longitude point.

Cordova-plugin-advanced-geolocation. The good news is that Android, in particular, has a very detailed API called LocationManager for examining geolocation data that comes from the device.  And, even better news for JavaScript developers is that the API, along with its access to all on-device GPS and Network location providers, has been exposed thru a Cordova plugin that is available here:

What geolocation data is available? With this plugin you’ll be able to programmatically differentiate between the following geolocation data as well as get access to GPS satellites meta data:

  • Real-time GPS location – This is data from the on-board GPS or some devices will allow it to be the location data from an external GPS that is connected to the device via bluetooth.
  • Cached GPS location – Most devices cache the last-known GPS location and it’s persistent even when the device is restarted.
  • Real-time Network location triangulation – this is completely dependent on devices and cellular service providers. It may require WiFi to be turned on. It also may not be available in all countries or regions.
  • Cached Network location – Most devices cache the last-known network-based location and it’s persistent even when the device is restarted.

Use cases. With this plugin, you can now use your JavaScript skills to implement the following use cases and much more. For example, I’ve always wanted to play with the Satellite data to make a 3D map of the satellites using JavaScript. This plugin provides a huge advantage to developers building applications for capturing a single location such as field survey work as well as the following use cases and others that I haven’t thought of:

  • Determine a static outdoors location and only use GPS.
  • While indoors turn on only network location. Do not use GPS.
  • While in an urban area, use network location to get initial location before the GPS warms up and then turn off network location and only use GPS
  • Compare the differences between GPS and Network locations
  • …???

How does this plugin help minimize location fluctuations? This plugin comes with a configuration option for turning on a buffer. You can set the size of the buffer, each new geolocation from the device will be added to it, and then plugin will determine the geometric center based on all the locations in the buffer.

Are there any other advanced plugins? Yes, some Cordova plugins are focused on being activity based and will detect if you are walking, stopped, moving, etc. These plugins tend to work as apps that can be backgrounded. Feel free to browse the Cordova plug-in directory here.

The one thing that Android needs the most

Android has really missed the boat on one thing that iTunes and iCloud do really well. That is the Android eco-system doesn’t have a built-in, seamless solution for restoring a device from scratch.

There is no universal way to backup and restore Android’s home screen and your phone’s application organization, your application data and settings, photos, videos, messages, ringtones, miscellaneous phone settings, etc.

What this means is it’s a pain and potentially time consuming to rebuild your phone or tablet every time you buy a new Android, your current phone dies because you dropped it, or if you have to switch over to a replacement. The issue is further compounded by the fact that some apps prevent you from saving them to an SDCard. I’m not sure if this is intentional or simply an oversight by the developer when they configured the application for uploading to Google Play.

Third party apps have jumped in to try and fill the void. Many take a really good stab at addressing the issue, but the solutions and their features can be a hodge-podge. Some, such as Titanium Backup, require you root your phone which many people are wary of because it voids any warranties. Others, such as App Backup & Restore, aren’t able to back up the application data and that means all your settings are lost.

I would trade a well-done backup and restore functionality from Android for any new gimmicky feature or pseudo-incremental improvement. Universal back up and restore would be a huge bonus for the entire Android community.

Android Privacy Part 2: App Ops & Google Play Improvements

Let me start out by saying I like my Android phones. I like developing for Android even with all the inherent version support issues, etc etc. So, in my previous post I brought up privacy issues related to installing Android apps along with a suggested fix. That post was inspired by a major change in Android v4.4.2 that removed “App Ops,” even though I didn’t mention it by name. The removal of that functionality is now a very visibility topic thanks to a number of high–profile bloggers such as the Electronic Frontier Foundation (EFF) who also have taken exception to this change.

In a nutshell, App Ops or its equivalent, would allow you to manually toggle individual application permissions on and off.  You can search for articles on what App Ops is, or read my previous post on what it should be.

I’d take the issue of “control over what applications can access” even one step further and propose that it’s well past time that Google should begin reviewing Android app submissions similar to what Apple does for the App Store.  Seriously. In combination with App Ops, or similar functionality, this could only help reduce the amount of nefarious practices, content, viruses, Trojans and more. Case-in-point: for the first this September, Kaspersky Labs reported a particularly sophisticated Trojan virus along with distribution mechanisms specially gift wrapped just for Android users. And, then again in November they announced an Android-specific financial phishing Trojan aimed at stealing banking usernames and passwords.

To get an idea of what is allowed on Google Play all you need to do is compare Apple’s App Review Guidelines with Google Play Policies and Guidelines. For example, searching the related Android Developer Content Policy for the term “review”, it only shows up once, and that is in regards to serving up advertisements.

My hope is that Google takes heed and makes some necessary and timely changes so that we can all continue to enjoy our Android devices safely and securely.

Important fix needed for Android app permissions

I believe there is a significant flaw in how permissions are set when you install Android apps. You get two options – Accept all or nothing. For readers not familiar with how Android app permissions work, there is a configuration file for each app that sets permissions for that app only. Permissions are needed for any functionality that affects how the app accesses things like sensors (e.g. GPS), SD Cards and the internet. These permissions do not affect any other app on the phone.

I propose an important change should be implemented at the operating system level — You should be able to accept or deny each privilege at installation time. This would make it an opt-in approach rather than an opt-out. Sure, some of you will say there are apps that can help you do that afterwards, but for tens of millions of consumers that’s not good enough. The vast majority of consumers simply don’t do take advantage of that for a variety of reasons, so having the option to accept/deny up front is the best way to go.

Yes, there is a good chance that many (most?) users would still simply accept all. However, I think increasing numbers of users would become aware that they can opt out of certain things and take advantage of the convenience and the potential for added security that this approach provides.

Developers and companies that build Android apps will probably yell loudly that this will affect how their apps work. Note that there are no technical reasons as to why this wouldn’t work. If someone checks “don’t allow internet access”, we developers can gracefully disable parts of the application and provide notifications when users attempt to access the internet. If someone disallows geolocation, then we do the same thing. Users can always opt back in if they need to. If some vendors take the approach that if you opt-out of certain things then the entire app will be disabled, then so be it. I personally would be wary of installing an app that did that.

Take the example of the screenshot below. This is the installation screen from a very popular sports app. I wonder why does it need access to my phone calls, my Accounts, or even contents of my USB storage? It doesn’t even provide an option to move the app to USB and there are no capabilities in the app (that I’m aware of) related to making phone calls. I would love to be able to opt out of these.

Android permissions

Android 4.3 plus Snapdragon S4 has massive battery life improvements

Okay, I expected a battery life boost when I got my LG Nexus 4 (16GB), but what I got blew me away. When I finally got the phone last week as a replacement to my battery hogging Samsung S3 I didn’t really expect much in terms of battery life.

I immediately installed three apps that I use a lot and that draw some amount of standby power: corporate email, a second email account and Twitter.  Then I got a prompt to upgrade to Android v4.3 (thanks LG!) so I immediately installed the update. I fully expected to get 10 to 12 hours of use before the battery approached rock bottom. Well was I wrong…way wrong…because at the 32 hour mark (1 day 8 hours) I still had 42% battery left! See the screenshot at the bottom of the page. I’ve also seen other reports that 4.3, by itself, added an amazing boost to battery life.

Note, I did the exact same test on my S3 when it was brand new and it lasted almost 12 hours before it started throwing low power warnings. That’s pretty much been my general experience with the eight other various Androids that I’ve had since v1.5. The results from the Nexus 4 blew that out of the water. I understand this is a brand new phone with very few apps on it. And as I add other apps that eat power in stand-by I’ll fully expect to see a drop off in battery life, especially as the battery gets older and more discharge/recharge cycles on it.

I also wanted to explain my usage pattern for the phone. It is what I’ll refer to as light- to medium-duty. What I mean by that is I would check email, news and twitter every two to three hours. This would consist of browsing on the phone for roughly 7 – 10 minutes at a time in short bursts throughout the day. I very much use it as a business and Android dev phone.

My main suspicion is that the WXGA IPS screen combined with the Snapdragon processor and Android 4.3 is dramatically more power efficient than the Super AMOLED screen on the S3 running Android 4.1.2 with a Samsung Exynos 4 quadcore. I’ll also point out that the screens are roughly the same size, if you were thinking maybe the Nexus 4 had a smaller screen and that’s what saved on power then think again. In fact, the Nexus has a slightly higher pixel density:

Samsung Galaxy S3 – 720 x 1280, 4.8 inches (~306 ppi pixel)

Nexus 4 – 768 x 1280, 4.7 inches (~320 ppi)

My point is that when jumping between previous major Android versions in the past we never saw battery life improvements even close to this. And from a hardware perspective, my S3 and all of my other pre-4.2 Android’s had screens that were major gas hogs. The screens on those phones were always at the top of the list on the Android battery consumption monitor as the number one energy consumer.

Conclusion. Whoa! I’ve been harping on Android’s miserable battery life for a long time, and now the Android team along with their hardware manufacturing friends may have finally broken the trend. Time will tell as I continue to use the phone and load up on apps if 4.3’s battery life improvements continue to hold up under pressure.

Android battery at 32 hours