Using High Accuracy GPS with Mobile Apps

There are times when the built-in GPS simply isn’t good enough for your Android or iOS app. Built-in GPS is good for finding a general location of a geographic feature such as a pond, field, or parking lot. If you need to locate a specific parking space, or a geographic feature within a field, or an inlet to a pond then you’ll need much greater precision of a commercial GPS unit.

Examples. Here is a more specific example to illustrate the concept. Let’s say the application users are mapping insect infected trees in a densely forested urban park. If you are using a smartphone GPS, with an average accuracy of 30+ meters when standing under the tree cover, the latitude/longitude coordinates may end up being unusable for future follow-up because there are dozens of trees within a 30+ meter radius. If you are using a commercial GPS with sub-meter accuracy you can have a much higher degree of certainty that someone will be able to locate that exact tree again.

I’ve seen entire days’ worth of data collection get thrown out the window because of significant location inaccuracies caused by using smartphones and dedicated hand-held consumer GPS units that are well known to outdoor recreational folks. I’ve accidentally sent a backcountry trail maintenance crew to a totally wrong location (they got really mad). All I had was a smartphone GPS and my location coordinate was over a thousand meters off even though the Android Location API returned an “accuracy” value of less than 50 meters. Oops — that’s the difference between precision and accuracy.

Good news. The good excellent news is there are small, handheld commercial GPS units that you can tether to your smartphone via Bluetooth such as the Trimble R1. Units like the R1 are able to override the onboard GPS and when configured correctly work just fine with the browser’s JavaScript-based Geolocation API or native iOS/Android geolocation.

The only downside can be cost. Some of these commercial units cost several thousand dollars and that may be outside the budget of many organizations.

Surprise, Surprise. I spend a ton of time talking to customers about this topic. Many people are very surprised because they regularly use GPS fitness tracker apps or Google Maps and the location information almost always seems very close to reality. The hidden trick is these apps use proprietary software algorithms that attempt to slightly improve accuracy, minimize location fluctuations or even intelligently snap your location to geographic features such as roadways, buildings or trails.

When you are building a custom mobile application, you will be forced to deal with raw, unfiltered geolocation data. And the coordinate data is coming from consumer-grade GPS units mass produced at the lowest possible cost. The antennas are tiny and usually on the back of the device rather than pointed towards the sky. The chipsets are designed to minimize battery usage, because GPS can be extremely battery intensive and generates a lot of heat. They have minimal ability to resolve communication difficulties between your device and GPS satellites, and GPS signals are already relatively weak.

Decisions, decisions. If your budget can’t handle several thousand dollars for a commercial GPS and you only have short term needs you can always look into renting a bluetooth unit. Just make sure to ask if it overrides the onboard GPS chip. You can find GPS rental companies online. There are also consumer-grade Bluetooth GPS units that only cost several hundred dollars. The consumer Bluetooth units aren’t nearly as accurate as commercial units; however, they offer a significant improvement over built-in GPS.

Is Apple’s one-size-fits-all approach starting to fail?

My mother-in-law cracked me up when she asked “What’s the matter with Apple, how come they only make things in one or two sizes?” She’s as non-techie as they come and I think she nailed an important point. I’ve joked about this with friends and colleagues and even mocked about it in presentations. Yet, ever since the Apple September 10th announcement I’ve been wondering: can Apple innovate enough to stay a true technological leader, or are they starting to enter an era of simply copying others and wrapping ‘improvements’ under a luxury brand umbrella? I’m leaning towards to latter. After seeing Tim Cook’s presentation on September 10th, my opinion has started to solidify even more.

Factoid: Everyone will agree, for better or for worse, we are in an era of choice. Look at the variety of Androids. Think about the sheer number of cable and satellite TV channels, the variety of TV shows aimed at a dizzying variety of demographics, music has splintered from the days of major rock bands into hundreds of niche, self published indie groups.  Satellite radio. Most major car manufacturers now have dozens of car models and many of them you can significantly customize. And, even grocery stores now have a ridiculous number of choices for a lot of items that use to only have several manufacturers or producers.

So, what about Apple?

Apple has so far resisted playing to the status quo set by IBM, Microsoft and now Android of allowing endless variations of their products to fit a variety of needs and wants. It wasn’t that long ago, for example, that Dell Computer was a leader in selling desktop and laptop computers. That era has passed and I don’t believe Apple wants to follow in these giants footsteps. Perhaps there is some sort of evolutionary cycle that these leading public tech companies follow, like natural boom and bust cycles that we see in economics, neighborhoods, cities and world economies that Apple won’t be able to escape.

This leads me right back to the September 10th announcement.  Apple seems to have entered an era of incremental improvements in hardware and software: new colors (big whoop), finger print reader (Moto Atrix had that) and faster hardware.  But, the biggest pressure I think they’ll start facing is they are now behind the curve in allowing people to have choices. Real choices…not just new colors for custom cases made of soft silicon rubber. Choices are the way they world is headed right now. Case in point, how long have iPhones had 4-inch screens? How many tens of millions of larger Android screen phones have been sold? Can Apple simply ignore this and stick to their one-size-fits-all guns if the board of directors starts seeing missed opportunities and potentially lower sales?

Android, in comparison to iOS, has more varieties of sizes, manufacturers, shapes and colors than there are grapes for making wine. Buying an Android is like shopping for clothes at any mainstream department store. In addition to a bijillion patterns and colors, you have sizes like XS, SM, M, L, XL, XXL, etc. And then there is relaxed fit, straight fit and athletic fit along with different collar sizes.  This is brilliant from a consumer standpoint, and yes it’s a nightmare for application developers and IT shops that support them. But, developers and IT folks only represent a small fraction of an enormous world-wide porous marketplace full of more consumers, cultures and tastes than perhaps existed in history.

Buying a Mac is like walking into a top-notch art gallery. As you glide along into the next majestic room, shuffling your feet in hushed respect you can hear the mac genius say in an elegant foreign accent, “…and here on this masterfully carved solid white marble pedestal, fabulously embellished with an aluminum case and crystal clear retina view screen, and protected from theft by 12 visible and hidden security features is, ladies and gentlemen…the (audience gasps) Apple Macbook Pro.”

But, at some point even art galleries change up their exhibits as people’s tastes and interests change.  Even galleries have to innovate to stay ahead of the times. Can Apple change? Can Apple adapt to a new era of endless choices? Can Apple reinvent itself?

3 Steps for Determining if Your Website is Mobile Ready

Here are three step for helping determine the mobile ready strengths and weaknesses of your existing website. I’ve had a number of conversations from website teams recently asking the question: “Can we reuse our existing site for mobile users?” I was surprised to learn that the individuals asking me the question had, in fact, never visited their own site on a mobile device.

Note, this blog describes steps that need to be address before you decide whether to build for the web or native applications.

Step One – Create a small focus group of company outsiders, friends as well as employees.

  • Gather as many different types of mobile devices as possible including: iPad, iPhone, Android tablet, and several varieties of Android phones. Try to use a combination of older and newer devices. Don’t fool yourself by simply using all of the latest great versions, especially if your web visitors are the general public.
  • Get a mobile projector, such an Elmo or IPEVO.
  • Write down the common use cases, and the workflows associated with them. An example use case might be logging in to your site. And, a workflow would describe the steps a user takes to complete the login process  from beginning to the end.
  • Visit your website and run through the common use cases.
  • Turn off wireless, if possible, and let everyone experience typical internet speeds to simulate, for example, standing in line at the grocery store.
  • Trade off using different devices.
  • Hire a user interface (UX) designer if you don’t already have one. Bring them on board at the beginning, or as early as possible, in this evaluation process.

Step Two – Create a grading system to help assess the experience everyone had with each device.

  • Were you able to accomplish your task as easily and quickly as if you were at your desk with a full-size laptop or computer?
  • Did you have to do a lot of extra panning and zooming in and out to navigate through the use cases and workflows?
  • Was there any functionality that simply didn’t work, didn’t work correctly, or didn’t work as expected on the mobile device?
  • Were there any aspects of the site that looked different or wrong? For example, was all the text the right size? Was everything in the right place?
  • Were you satisfied with the amount of time it took for pages and images to load?
  • Were you able to comfortably use the site when rotating the phone between landscape and portrait views?
  • Were you okay with how quickly you were able to switch between different pages on the website?
  • Were you able to access secure resources without any problems?
  • And, perhaps most importantly, were there any obvious improvements you would like to see made to make mobile surfing experience better?

Step 3. Apply some commonly known mobile-specific conditions to your findings and see if helps to provide context to everyone’s experience.

  • One-handed plus gestures. It’s a fact that navigating a mobile web is significantly different from a desktop browser. There’s no mouse! Mobile browsing is usually done with one hand, while the other hand is used to hold the device. The screen is driven by what are called gestures. Examples of gestures are when you swipe your thumb upward on a page to scroll it downward, or when you use two fingers, usually the index finger and thumb, to pinch zoom the screen in or out.
  • Smaller Screens. And, of course the screens are much smaller than what you would find on a desktop or laptop. Different devices have different resolutions. And, navigating a full website can seem more cumbersome as you use gestures to navigate around, in comparison to the desktop experience of seeing the entire page, and using your a precision mouse to whip through the different links on a page.
  • Download Speeds. Download speeds on mobile devices vary considerably compared to your work machine hooked up to a reliable local area network (LAN). A site that seems zippy on your work machine, may load much differently on a typical smartphone. Also, for some older phones they may have much less processor power and that may lead to the perception of slower download speeds as the CPU chugs through displaying the page.

How do I interpret the results?

When you are done compile, discuss and analyze the findings with your internal teams and stakeholders.

Good. If most testers successfully navigated the majority of use cases and workflows then you are in good shape, and you may simply need to do some additional tweaking to your site.

Not so good. However, if most testers had unsatisfactory experiences then you’ll need to spend more time looking more closely as what worked and what didn’t work. You may find workflows that are great on a desktop machine that are clumsy and awkward on a mobile device.

Don’t be surprised. Portions of your site may have to be redesigned. You may not be able to include everything that’s in your full site into your mobile site. You may have to spend a lot of time optimizing the site to speed up page load times. Pay special attention to functionality that didn’t work on mobile. Mobile web browsers have well known limitations compared to full browsers. Looking at what didn’t work may help you decide if you need access to native device capabilities.

You’ve just taken a huge first step towards helping your team set the stage for stepping into the mobile world.

Check HTML5 Browser Height and Width using Canvas

I’ve been working on building mobile web apps the last few weeks and I needed a quick way to check the browsers screen size for both mobile and non-mobile. So, I built a small, rough app that quickly lets me do that. Here’s what the app does: 

–          Detect HTML5 Canvas support

–          For height give clientHeight, offsetHeight and scrollHeight.

–          For width give clientWidth, offsetWidth and scrollWidth.

–          Recalculate when browser is resized or rotated. Here’s the link to try it out.

Remember, it’s rough so if you need to tweak it for your needs feel free to grab the code via the link or QR Code and play: http://andygup.net/samples/windowresize/

At the heart of the app there are a few key pieces of code. There’s the Canvas element:

<canvas id="rectangle1" style="height: 100%; width: 100%; border:solid 1px #000000;"></canvas>

And, there’s the code that reads the height and width properties from the canvas element:

document.getElementById("main").innerText =
"clientHeight: " + rectangle1.clientHeight + ", clientWidth: " + rectangle1.clientWidth + "\r"
+ "offsetHeight: " +rectangle1.offsetHeight + ", offsetWidth: " + rectangle1.offsetWidth + "\r"
+ "scrollHeight: " + rectangle1.scrollHeight + ", scrollWidth: " + rectangle1.scrollWidth;

Here are a few guidelines on how to interpret the results:

It works great! This means you are using a modern browser that supports the HTML5 Canvas element.

Nothing shows up. If the mobile browser shows nothing, it probably doesn’t support HTML5. Example of this is Firefox for Android. Hey don’t laugh, we can’t change what browser people use.

Rotating phone doesn’t change numbers. If the numbers don’t change when you rotate the phone, then the browser has incomplete support for HTML5. Example of this is Opera for Android.

Just the height numbers don’t change or are inaccurate. The default height for canvas on most browsers is 150px. So if the browser can’t interpret the property height:100%, then you may simply see a height of “150” in the app.

Height number is too small. In some browsers, if a sub-window is open, for example Chrome developer tools, then the height number reflects the available browser window height not including the developer tools sub-window.

If you have suggestions for improvements the let me know!

[Minor update: May 26, 2012 – corrected a couple typos]

References:

Caniuse– Canvas

Mozilla – Determining the dimensions of an element.

HTML5 Geolocation API – how accurate is it, really?

If you are a developer building applications that require location information then you need to know what is really possible with the HTML5 Geolocation API and not a bunch of hype. The blog post attempts to give you some insight into how it works with desktop and mobile browsers as well as having a greater appreciation for what is and what isn’t possible. I’m going to show you that accuracy depends on many factors, some of which are beyond your control, and at best the location information returned by the API is just an approximation.

[Editors note: as of June 29th, 2013 Part 2 of this post is now available]

Most common use case. For the most part, HTML5 Geolocation works just fine in dense urban areas when you are stationary with your laptop or smartphone Wifi turned on. This is the use case most commonly cited when questions are asked about accuracy. This makes sense because urban areas have many public and private Wifi routers and cell phone towers are typically closer together. As you’ll see, HTML5 uses these and other methods to pinpoint your location. However, it’s not always that simple and below are some other use cases that you should take into consideration.  

How does the API work? Depending on which browser you are using, the HTML5 Geolocation API approximates location based on a number of factors including your public IP address, cell tower IDs, GPS information, a list of Wifi access points, signal strengths and MAC IDs (Wifi and/or Bluetooth). It then passes that information to a Location Service usually via an HTTPS request which attempts to correlate your location from a variety of databases that include Wifi access point locations both public and private, as well as Cell Tower and IP address locations. An approximate location is then returned to your code via a JavaScript callback.

As an example to show you what type of information is sent to a Location Service, I did some basic testing with Firefox 11. Firefox uses Google’s Location Service. On a related note, as far as I can tell with Firefox 11 it isn’t passing cookies any more where in Firefox 3.6 they use to pass a user ID token.

Firefox 11 browser sends queries to https://maps.googleapis.com/maps/api/browserlocation/json? The example results have been obfuscated, but by looking at it you should get the idea of what content is being sent:

GET /maps/api/browserlocation/json?browser=firefox&sensor=true&wifi=mac:01-24-7c-bc-51-46%7Cssid:3x2x%7Css:-37&wifi=mac:09-86-3b-31-97-b2%7Cssid:belkin.7b2%7Css:-47&wifi=mac:28-cf-da-ba-be-13%7Cssid:HERESIARCH%20NETWORK%7Css:-49&wifi=mac:2b-cf-da-ba-be-10%7Cssid: ARCH%20GUESTS%7Css:-52&wifi=mac:08-56-3b-2b-e1-a8%7Cssid:belkin.1a8%7Css:-59&wifi=mac:02-1e-64-fd-df-67%7Cssid:Brown%20Cow%7Css:-59&wifi=mac:2a-cf-df-ba-be-10%7Cssid: ARCH%20GUESTS%7Css:-59 HTTP/1.1

Which location service do browsers use?

Not all Geolocation services are the same, and they certainly don’t all use the same algorithms and exact same databases. Because of this the results typically vary across browsers that use different Geolocation services.

Here’s my best attempt to document which Geolocation service each of the major browsers are using. I haven’t done any definitive testing however I do know from experience that different browsers and even different laptops for smartphones will return different locations when tested from the exact same location. Some location services are better in some cities and others are better in other cities. I haven’t come across a definitive list, most likely because the information is constantly being updated. I’ve included a link to a demo application at the bottom of this blog where I encourage you to also test the API against different browsers.

  • Chrome uses Google Location Services.
  • Firefox on Windows uses Google Location Services.
  • Firefox on Linux uses GPSD – http://catb.org/gpsd/. I’m not sure if this includes Android. I haven’t had a chance to test it yet.
  • Internet Explorer 9+ uses the Microsoft Location Service.
  • Safari on iOS uses Apple Location Services for iPhone OS 3.2+.
  • I’m not sure what Safari on Windows uses. With all the public distrust between Apple and Google, I wouldn’t be surprised if Safari on Windows also uses Apple’s Location Service, but I haven’t found any documentation to verify this and I haven’t tested it.
  • Opera uses Google Location Services. On a related note, I’ve also noticed that mobile Opera on Android accesses the GPS. This is something to consider from a battery usage standpoint.

Not all browsers support HTML5. It’s important to note that not all browsers support the HTML5 Geolocation API, for example Internet Explorer 8. The HTML5 Geolocation API is built into the browser and is accessible using JavaScript methods that access the navigator object. In order to work it requires HTML5 support in the browser. You can research whether or not a particular browser supports Geolocation by going here: http://mobilehtml5.org/ or http://caniuse.com.

Additionally, if a user has disabled JavaScript for some reason, then your Geolocation app won’t work in their browser. JavaScript code is required to access the API.

HTML5 Geolocation requires an internet connection. If you lose your internet connection then you won’t be able to access the Location Service. With no internet connection most browsers will not return a location. Sometimes you can access a cached location that is stored in the browser by the API. But, that cached location is the last valid location that was calculated by the API.

Is Wifi turned on or off? If Wifi is turned off on your phone, desktop machine, laptop or tablet , the Geolocation API service will try to find your location by other methods which include your public IP address, Cell tower ID triangulation or GPS. Public IP addresses databases usually return a location for your internet providers Point of Presence or PoP. Furthermore, some internet provides offer rotating IP addresses. So you get to use one IP address for a particular time period such as 48 hours and then you get a different one. So a Public IP address is usually only good enough to locate you to a particular City, or a general area of the City, or a Country depending on where you are in the world.

As for Cell Tower IDs it depends on what type of information your particular phone and Telco Carrier provides to the API. Some smartphones only return information on the current tower that the phone is pinging, which obviously makes triangulation very difficult and decreases accuracy to within a radius around that tower.

I’ve noticed that the native Android browser is significantly less accurate without Wifi. Without it I typically see accuracy numbers in the 1000+ meters range. As soon as I turn Wifi back on and I’m in a neighborhood or downtown area, the accuracy drops to less than 75 meters almost instantly.

Are they in a rural or urban location? Granted the vast majority of users will be in urban locations. However if you have requirements for users traveling outside of urban areas then this section applies to you. Geolocation in rural areas is significantly less reliable. If Wifi is turned on but the user is not near any Wifi access points, then the Geolocation service will also attempt to fallback to the other methods mentioned above.  Triangulation can be much more difficult in rural areas where towers are spread further apart, and for browsers that don’t use GPS the accuracy will suffer significantly.

Are you moving or stationary? Being stationary in an urban area offers far better accuracy with the Geolocation API than when you are moving. On my native Android phones it’s rare to get an accurate reading while driving around town. Occasionally a sporadic result would be returned when you stop at a light. To date, I have never gotten a valid reading while driving on a highway at speeds over 50 mph.

Is a VPN turned on? If a VPN is turned on, then the location will resolve to the VPN’s public IP address. For example, a user in Denver is logged into the company VPN which host is hosted at their headquarters office in a suburb of Dallas, Texas. The HTML5 Geolocation API will resolve the location to the headquarters public IP address in Dallas and not the user’s actual location. Quite a few corporate users have VPNs for security reasons.

Custom Geolocation as a fallback? Depending on your requirements you may want to implement your own IP Geolocation using a company such as IP2Location. Or use a third-party Geolocation service, such as Skyhook, as a fallback. Remember IP Geolocation only returns locations to a City or an area within a City. So, if you need more accuracy than that for your application, then don’t bother with this approach.

The downside to custom IP Geolocation is that this requires writing a server-side service to grab the browsers IP address. All server-side languages such as PHP, C#.NET, Java and JSP support these capabilities. You also have to subscribe to another service that lets you query their database by IP address and get a return value of an approximate location. There is no current way to get this information from the browser, on the client-side, using JavaScript.

HTML5 Geolocation doesn’t meet my requirements, what do I do? If you have critical requirements for gathering more precise location information than the HTML5 Geolocation API is capable of delivering then I’d recommend building your application using a native API such as Android or iOS.

How can I test this? You can test HTML5 Geolocation in different browsers using a test application that I built. I recommend trying it on different browsers and comparing the results yourself:

http://andygup.net/samples/html5geo/

References

Mozilla FAQ

Mozilla Developer Network

Google Location Service

W3C Geolocation API in IE 9

Safari Developer Library

Opera Geolocation

IP Geolocation

W3C – Privacy of Geolocation Implementations

Apple Q&A on Location Data