Using High Accuracy GPS with Mobile Apps

There are times when the built-in GPS simply isn’t good enough for your Android or iOS app. Built-in GPS is good for finding a general location of a geographic feature such as a pond, field, or parking lot. If you need to locate a specific parking space, or a geographic feature within a field, or an inlet to a pond then you’ll need much greater precision of a commercial GPS unit.

Examples. Here is a more specific example to illustrate the concept. Let’s say the application users are mapping insect infected trees in a densely forested urban park. If you are using a smartphone GPS, with an average accuracy of 30+ meters when standing under the tree cover, the latitude/longitude coordinates may end up being unusable for future follow-up because there are dozens of trees within a 30+ meter radius. If you are using a commercial GPS with sub-meter accuracy you can have a much higher degree of certainty that someone will be able to locate that exact tree again.

I’ve seen entire days’ worth of data collection get thrown out the window because of significant location inaccuracies caused by using smartphones and dedicated hand-held consumer GPS units that are well known to outdoor recreational folks. I’ve accidentally sent a backcountry trail maintenance crew to a totally wrong location (they got really mad). All I had was a smartphone GPS and my location coordinate was over a thousand meters off even though the Android Location API returned an “accuracy” value of less than 50 meters. Oops — that’s the difference between precision and accuracy.

Good news. The good excellent news is there are small, handheld commercial GPS units that you can tether to your smartphone via Bluetooth such as the Trimble R1. Units like the R1 are able to override the onboard GPS and when configured correctly work just fine with the browser’s JavaScript-based Geolocation API or native iOS/Android geolocation.

The only downside can be cost. Some of these commercial units cost several thousand dollars and that may be outside the budget of many organizations.

Surprise, Surprise. I spend a ton of time talking to customers about this topic. Many people are very surprised because they regularly use GPS fitness tracker apps or Google Maps and the location information almost always seems very close to reality. The hidden trick is these apps use proprietary software algorithms that attempt to slightly improve accuracy, minimize location fluctuations or even intelligently snap your location to geographic features such as roadways, buildings or trails.

When you are building a custom mobile application, you will be forced to deal with raw, unfiltered geolocation data. And the coordinate data is coming from consumer-grade GPS units mass produced at the lowest possible cost. The antennas are tiny and usually on the back of the device rather than pointed towards the sky. The chipsets are designed to minimize battery usage, because GPS can be extremely battery intensive and generates a lot of heat. They have minimal ability to resolve communication difficulties between your device and GPS satellites, and GPS signals are already relatively weak.

Decisions, decisions. If your budget can’t handle several thousand dollars for a commercial GPS and you only have short term needs you can always look into renting a bluetooth unit. Just make sure to ask if it overrides the onboard GPS chip. You can find GPS rental companies online. There are also consumer-grade Bluetooth GPS units that only cost several hundred dollars. The consumer Bluetooth units aren’t nearly as accurate as commercial units; however, they offer a significant improvement over built-in GPS.

Advanced geolocation plugin for Cordova and PhoneGap for Android

The W3C’s browser-based, JavaScript Geolocation API is excellent as a one-size-fits-all interface, but that approach comes at a price and it can cause some serious limitations when it comes to implementing more stringent professional, commercial and government use-cases.

Challenges with the default Geolocation API. One of the primary limitations is the Geolocation API does not tell you how it got a location. All locations are lumped together in a black box. Let me explain. On a smartphone or tablet, location data comes from one of three places: the GPS chipset, the cellular provider’s Location Service, or the browser’s Location Service. The W3C Geolocation API simply lumps these data points together. The end result is typically seen by the end user as significant and disturbingly wild jumps back and forth in the reported location, sometimes over large distances. A key to minimizing these fluctuations is to gain back control and understand which location provider created the latitude and longitude point.

Cordova-plugin-advanced-geolocation. The good news is that Android, in particular, has a very detailed API called LocationManager for examining geolocation data that comes from the device.  And, even better news for JavaScript developers is that the API, along with its access to all on-device GPS and Network location providers, has been exposed thru a Cordova plugin that is available here: https://github.com/Esri/cordova-plugin-advanced-geolocation.

What geolocation data is available? With this plugin you’ll be able to programmatically differentiate between the following geolocation data as well as get access to GPS satellites meta data:

  • Real-time GPS location – This is data from the on-board GPS or some devices will allow it to be the location data from an external GPS that is connected to the device via bluetooth.
  • Cached GPS location – Most devices cache the last-known GPS location and it’s persistent even when the device is restarted.
  • Real-time Network location triangulation – this is completely dependent on devices and cellular service providers. It may require WiFi to be turned on. It also may not be available in all countries or regions.
  • Cached Network location – Most devices cache the last-known network-based location and it’s persistent even when the device is restarted.

Use cases. With this plugin, you can now use your JavaScript skills to implement the following use cases and much more. For example, I’ve always wanted to play with the Satellite data to make a 3D map of the satellites using JavaScript. This plugin provides a huge advantage to developers building applications for capturing a single location such as field survey work as well as the following use cases and others that I haven’t thought of:

  • Determine a static outdoors location and only use GPS.
  • While indoors turn on only network location. Do not use GPS.
  • While in an urban area, use network location to get initial location before the GPS warms up and then turn off network location and only use GPS
  • Compare the differences between GPS and Network locations
  • …???

How does this plugin help minimize location fluctuations? This plugin comes with a configuration option for turning on a buffer. You can set the size of the buffer, each new geolocation from the device will be added to it, and then plugin will determine the geometric center based on all the locations in the buffer.

Are there any other advanced plugins? Yes, some Cordova plugins are focused on being activity based and will detect if you are walking, stopped, moving, etc. These plugins tend to work as apps that can be backgrounded. Feel free to browse the Cordova plug-in directory here.

Does using PhoneGap improve geolocation?

No, using PhoneGap or Cordova doesn’t change how you retrieve geolocation information from within a mobile application and it doesn’t improve accuracy. You still use the W3C HTML Geolocation API coding pattern.

The reason why is because PhoneGap runs the HTML/CSS/JS code in a native OS WebView. This is a chrome-less browser with mostly similar functionality as Chrome (Android) and Safari (iOS). There isn’t any additional functionality that would improve accuracy.

Some people have asked me if cordova-plugin-geolocation provides additional functionality. That plugin only works for devices that don’t already support geolocation and then it abides by the W3C Geolocation API spec.

The HTML Geolocation API in the browser also works offline. Yes, it’s true. You can use a web browser offline to get geolocation and even view a map without PhoneGap, here’s an example project that demonstrates the functionality in a number of ways. As long as the GPS on the device is enabled and working, and the user opt’s in at the geolocation-approval prompt then the application will receive GPS location data.

Are there any advantages to using PhoneGap for geolocation?

Yes, there are several potential advantages to using PhoneGap for geolocation. You can retrieve geolocation information while the device screen is locked and/or when the application is running in the background. You cannot do this with a typical web app. As soon as a browser is minimized a web app will stop running even if the GPS is still running.

Another advantage is you can write a PhoneGap plug-in to tap directly into the native OS location API. Taking this approach will give you a greater level of control over the information you are retrieving. And, you have coding tools and access to APIs to better manage the device’s battery life in a way that the HTML Geolocation API cannot deliver. In the case of Android you can tap into significantly greater information such as details on GPS satellites, granular information on device’s internet connection, as well as NMEA strings and more.

HTML5 Geolocation API: Getting the best single location

The Geolocation API is built into all modern mobile browsers and it lets you take either a quick, onetime snapshot, or you can get continuous location updates. Using the browser to get your approximate location is very, very cool, but it’s also fraught with many challenges. The vast majority of blog posts on this API talk about what it can do, this blog post focuses on how to best use it and understanding the data provided by the API.

To start things out, let’s take a quick look at a short list of some of the challenges when using the Geolocation API.

Challenge 1. You will not know where the location information is coming from. There’s no way to tell if it’s from the GPS, the cellular provider or the browser vendors location service. If you care about these things then the native Android SDK, for example, gives you a huge amount of control over what they call ‘location providers.’

Challenge 2. You cannot force the app to stay open. This means that the typical user has to keep tapping the app to keep it alive otherwise the screen will go to sleep and minimize your app.

Challenge 3. Speaking about minimizing apps, when the browser is minimized the geolocation capabilities stop working. If you have a requirement for the app to keep working in the background then you’ll need to go native.

Challenge 4. You’ll have very limited control over battery usage. Second only to the screen on your phone or tablet, the current generation of GPS chips are major energy hogs and can suck down your battery very quickly. Since the Geolocation API gives you very little control over how it works you cannot build much efficiency into your apps.

Challenge 5. Most smartphones and tablets use a consumer-grade GPS chip and antenna, and that limits the potential accuracy and precision. On average, the best possible accuracy is typically between 3 and 10 meters, or 10 – 33 feet. This is based on my own extensive experience building GPS-based mobile apps and working with many customers who are also using mobile apps. Under the most ideal scenario, the device will be kept stationary in one location until the desired accuracy number is achieved.

What’s it good for? Okay, you may be wondering what is browser-based geolocation good for? It’s perfect for very simple use cases that don’t require much accuracy. If you need to map manhole covers, or parking spaces, or any other physical things that are close together you’ll need a GPS device with professional-level capabilities.

Here are a few generic examples that I think are ideal for HTML5 Geolocation:

  • Simply getting your approximate location in latitude/longitude and converting it to a physical address.
  • Finding an approximate starting location for searching nearby places or things in a database or for getting one-time driving directions.
  • Determining which zip code, city or State you are in to enable specific features in the app.
  • Getting the approximate location of a decently sized geological feature such as a park, a building, a pond, a parking lot, a driveway, a group of trees, an intersection, etc.

What’s the best way to get a single location? The best way to get a single location is to not use getCurrentPosition() but to use watchPosition() and analyze the data for a minimum set of acceptable values.

Why? Because getCurrentPosition() simply forces the browser to barf up the best available raw, location snapshot right now. It literally forces a location out of the phone. Accuracy values can be wildly inaccurate, especially if the GPS hasn’t warmed up, or if you aren’t near a WiFi with your WiFi turned on, or if your cellular provider can’t get a good triangulation fix on your phone, or it returns a cached value from a different location altogether. There are many, many “what ifs?”

So, I recommend using watchPosition() and firing it off and letting it run until the return values meet some minimum criteria that you set. You need to know that while this is happening the location values returned may cover a fairly wide geographic area…remember our best accuracy values are 10 – 30 meters. Here’s a real-world example of Geolocation API location values that I captured over a 5 minute period while standing stationary in front of a building.

5 minute snapshot

What steps do you recommend? Here are five basic steps to help guide you towards one approach for getting the best location. This is a very simplistic approach and may not be appropriate for all use cases, but I think it’s adequate to demonstrate the basic concepts for working towards determining the best possible location.

Step 1. Immediately reject any values that have accuracy above a certain threshold that you determine. For example, let’s say we’ll reject any values with an accuracy reading greater than 50 meters.

Step 2. Create three arrays, one for accuracy, latitude and longitude. If the accuracy is below your threshold, or in this case < 50 meters, then push the values to the appropriate arrays. You will also need to set a maximize size for the array and create a simple algorithm for adding new values and removing old ones.

The array length could be 10, 20 or even 100 or more entries. Just keep in mind that the longer the array, the longer it will take to fill up and the longer the user will have to wait for the end result.

Step 3. Start calculating the average values for accuracy, latitude and longitude.

Step 4. Start calculating the standard deviation for accuracy, latitude and longitude.

Step 5. If your arrays fill up to the desired length and the average accuracy meets your best-possible criteria, and the standard deviation is acceptable then you can take the average latitude, longitude values as your approximate location.

For an example of this simple algorithm at work visit the following URL on your phone and step outside to get a clear view of the sky: http://esri.github.io/html5-geolocation-tool-js/field-location-template.html. [Updated link: Oct. 27, 2015]

Is a consumer smartphone GPS good enough?

When I presented at OSCON  (O’Reilly’s Open Source Conference) in Portland, Oregon this last week on native Android GPS and Geolocation, I was repeatedly asked the question “is the GPS in smartphones good enough?” In general the answer is “yes”. But, I should back this statement up by looking at several everyday types of scenarios to help illustrate my answer.

What does accuracy mean? First lets briefly look at what accuracy means. Accuracy, in a technical sense, means you get a latitude, longitude and accuracy number from the GPS. Then you can draw a circle using the accuracy number as radius around the latitude/longitude point. It’s highly likely that your actual location is somewhere within that circle. It’s unfortunate that consumer GPS devices don’t also come with a probability factor that would indicate how much to trust the accuracy number. As it is, we will have to take it on a certain amount of faith that our real location is, in fact, somewhere within the accuracy circle.

Under absolutely perfect conditions your typical smartphone GPS chip will deliver around 3 meters (~10 ft) in accuracy for several minutes at a time.  Standing on the highest mountaintop in the Rocky Mountains with no clouds in the sky might come close to being a perfect condition. Under what I’ll call “average” conditions, which reflect everyday in-city usage patterns, you can expect accuracy from 3 – 150 meters (10 ft – 500 ft) or greater and the accuracy number typically fluctuates quite a bit over a period of minutes or hours. GPS signals are affected by anything that interferes with your smartphone receiving the weak transmissions from GPS satellites circling above us. Nearby trees, cars, buildings, big weather storms can all reduce accuracy, and so can being inside a building or underneath trees.

Getting the weather. Now back to the everyday scenarios I mentioned. I bet that most people who own smartphones use them to check the weather at least once per day. Furthermore, weather is typically affecting a large geographic area so the vast majority of forecasts cover cities, counties, States, regions or even entire countries.

I’m going to argue that for this scenario a GPS accuracy of 1000 to 2000 meters, or 0.6 to 1.24 miles is good enough to get started with a finding places app. Values in this accuracy range can be easily and quickly retrieved by a typical smartphone.

Finding places around me. Almost everyone that owns a smartphone has used an app to search for food, gas, groceries etc that are nearby.  There’s no hard written rule, but I think most people would agree that users who are looking for places around them tend to be less concerned about high levels of accuracy. Some applications let you choose target levels of accuracy such as 1 mile, 10 miles or even up to 100 miles for the search radius.

I’m going to argue that for this scenario, as well, that a GPS accuracy of 1000 to 2000 meters will also work perfectly fine.

Real-time driving directions. There are only a handful of applications that do this on smartphones, and even fewer apps do it really well. There’s a lot of hidden math involved in making everything look smooth to the user. These apps wipe away all of the complexity: simply give it a starting point and an endpoint and then away you go.  We’ve all used these types of applications so we know they work well the majority of the time even with the occasional navigational glitch, hardware lockup or low battery.

Social Media location. Hundreds of millions of people use location-aware social media apps every day. Some of these apps provide you with discounts, give-aways and coupons for retail locations that are around you.  For a typical big box store, it’s easy for an app to place you in the parking lot of a Target, Walmart, or a large supermarket and promotions can be based on your location, time of day or day of the week. It’s more challenging for stores with smaller storefronts to use targeted advertising unless someone is simply “in the vicinity”.  For example, tiny stores, kiosks and shops trying to compete in high traffic tourist areas have to compete with many other vendors. In a crowded marketplace area, even with 3 meter accuracy you could be standing next to four or more different storefronts.

A final few words

Hopefully these short examples have successfully illustrated the point that for typical consumer-focused applications smartphone GPS is simply good enough. Certainly there are many, many more scenarios that could be examined so I tried to pick the most common ones. Because of the lower accuracy requirements you can get less accurate results faster from a GPS. Speedy results can mean everything for today’s consumers who have high levels of intolerance for application delays.

The opposite is also true, the greater the accuracy requirements the longer it can take to get a more precise GPS measurement. As I’ve mentioned in my others posts on this subject, it’s takes time for a GPS device to get a fix and then it will try its best to hold onto it as you move around. I suspect that most consumers are significantly less demanding about accuracy as compared to commercial and government users. If consumers were more demanding then there would be a greater uproar about GPS accuracy.

To better understand how to make the most of location data check out the other posts I’ve written in the reference section below.

References:

Six most common use cases for Android GPS

How accurate is Android GPS? Part 1 – Understanding location data.

How accurate is Android GPS? Part 2 – Consuming real-time locations

OSCON 2013 – Presentation on Android SDK Geolocation

If you are headed to OSCON, swing by my session on Mastering Android Geolocation. It’s a deep dive into the Android SDKs android.location package. If you’ve ever wanted to learn about the fundamental’s of the SDKs Geolocation capabilities then this is a must attend session. It will also give you a strong foundation to understand the underlying capabilities of the new Google Play Services SDK that includes Fused Location, Activity Recognition, and Geofencing APIs.

The presentation includes digging into the capabilities of my open source GPS Testing tool that lets you easily test different aspects of the Geolocation capabilities.

Here’s the details and I hope to see you there:

 Location, Location, Location: Mastering Android Geolocation
07/25/2013  5:00pm –  5:40pm PDT (40 minutes)
Room: Portland 251 (capacity: 200)
http://www.oscon.com/oscon2013/public/schedule/detail/28713