Posts Tagged ‘html5’

In Part 1 we looked at the differences between partial and fully offline use cases. Part 2 provides an overview of the HTML5 Interfaces and JavaScript APIs that make it possible to go offline with web applications. Going offline involves working with multiple pieces and coding for specific patterns. I’ve tried my best to stick to technology that is widely available across the largest variety of browsers.

Offline dependencies

Offline web applications are dependent on three things.  It doesn’t matter if your application is partially or fully offline, you’ll still need to address these in your code.

  • Caching HTML, CSS and JavaScript
  • Data Storage
  • Offline/Online detection

Caching

Application Cache. The Application Cache, or AppCache, interface lets you specify and store HTML and CSS files as well as JavaScript libraries so that they are available from the browser’s native cache. Once an item is in the cache the browser will use it regardless of whether it’s online or offline. It’s almost like you never went offline!

The AppCache is an essential part of your application strategy for allowing offline browser reloads or restarts. Without this an application will simply fail to re-load while offline.

Data Storage

Browsers have a variety built-in JavaScript APIs for storing data. The data can be for maintaining the applications state such as for storing bookmarks and form data. Or, it can be used for storing information such as maps, address and phone lists, TO-DOs or points of interest for a vacation.

LocalStorage. The LocalStorage API is super-easy to use. It stores Strings in simple key/value pairs. It’s limited to about 5MBs on most browsers. The two main challenges you’ll run into with LocalStorage are hitting the storage limit and performance hits when serializing and deserializing data.

IndexedDB. IndexedDB is essentially an asynchronous noSQL database that lets you store a wide variety of datatypes so that you don’t have to deal with serialization/deserialization.  Datatypes include String, Object, Array, Blob, ArrayBuffer, Uint8Array and File. While many online sources will tell you that there isn’t a size limit, I’ll tell you that in general you should limit your storage on a mobile device around 50 – 100MBs to help prevent the browser from crashing.

WebSql. It’s widely recommended that you not build applications directly on WebSql. The World Wide Web Consortium (W3C) is letting this standard die off in favor of IndexedDB and LocalStorage. I’m really only including this here for reasons such as Safari 7 and and the Android native browsers before 4.4 only support WebSql. For more information on how to get around this read down to the section on IndexedDBShim.

3rd Party Browser Storage

If the built-in browser storage capabilities aren’t meeting your needs you still have other options.

IndexedDBShim. IndexedDBshim is a Polyfill for WebSQL-based browsers. Because IndexedDB isn’t natively supported on older versions Safari 7 and Opera you can use this 3rd party shim to transparently translate your IndexedDB code to work across Android and iOS.

PouchDB. PouchDB is an Open Source experimental library that is an attempt to smooth some of IndexedDB’s rough edges as well as provide additional functionality, such as the ability to sync with remote stores.

LocalForage (Mozilla).  LocalForage is also an attempt to bridge the gap between LocalStorage and IndexedDB. It gives you an interface that provides much wider browser coverage than IndexedDB by itself.  One of the downsides is the amount of storage you can use. If a user is on an older browser such as IE8 that’s limited to LocalStorage then that user will be limited to storing about 5MBs of data. If you requirements call for using more than that, such as downloading large address lists, then the app won’t work on that browser or you’ll have to build in some sort of paging mechanism that deletes the old data and brings in the new.

Offline/Online Detection

There are a number of ways to detect if the browser is online or offline as well as when the internet status changes.

NavigatorOnline.online.  Some browsers have a built-in detection mechanism. However, it is not always reliable and false positives are a distinct possibility. For that reason, you will have to build additional detection capabilities or lean towards a 3rd party library.

Offline.js. Offline.js is a small Open Source library (~3KB) that detects when you lose an internet connection and when it comes back up. While not perfect, it does handle a lot of cross-browser compatibility issues for you. And, if you find bugs you can always create a fix and submit pull requests.

References

Caniseuse – IndexedDB

Caniuse – LocalStorage

Caniuse – WebSQL

Let’s Take This Offline

Tags: , , , , , , ,
Posted in Browsers, JavaScript, Mobile | Comments Off

Going Offline with HTML5 and JavaScript, Part 1

There are two primary use cases for going offline with mobile HTML5 web applications and JavaScript: partial offline and full offline. Before diving into building offline apps, understanding the differences between these use cases will help you build the best applications for your requirements. The functionality in modern browsers has finally gotten to the point where it is feasible (and fun!) to build offline web applications.

Partial Offline. Partial offline means the vast majority of the time the application is online, however it can continue to work if the internet connection gets temporarily interrupted. A partially offline app understands that requests for remote resources, that is resources that don’t exist on the device, will automatically defer to local resource, or at least fail gracefully, during the period of time when an internet connection doesn’t exist. Partially offline apps typically cannot be reloaded or restarted offline. The coding required to handle this scenario is much lighter weight than the architecture required for going fully offline. An example of partial offline is a reader app that pre-caches certain HTML pages of your choice. If the internet connection gets disrupted you can continue reading and navigating between the cached pages.

The partially offline scenario exists because there is no such thing as a perfect internet connection for mobile. In reality, internet connections and download speeds are very choppy when measured over a period of minutes or hours. Sure, some internet providers market 4G connections as being extremely fast, or have the best coverage etc., etc., blah. The bottom line is cellular and even WiFi internet connections are not guaranteed. A good example of this is coffee shops. They don’t come with an uptime guarantee, so if a couple of yahoos sitting next to you are streaming HD Netflix then that will surely bring the internet connection to its knees.

In reality, if you don’t live danger close to a cell phone tower and are moving around doing your job or running errands chances are your internet connection will fluctuate up and down over time. Anyone that owns a smartphone has experienced this at one time or another. Dropped calls are perfect example. You may be shopping and going in and out of buildings, or hanging out in the back of a taxi, sitting in your car pulled off the side of the road, or perhaps even just watching your kids as they play in the neighborhood park. A web application architected for partially offline will let you keep surfing or working for a short period of time, and hopefully long enough until the internet connection comes back up.

Full Offline. A fully offline JavaScript application is one that starts out online to download all the necessary data and files, then it can be completely disconnected from the internet. These apps can survive browser restarts and reloads, they can stay offline indefinitely and/or they can be resynced online at some point in the future.

Fully offline apps need to be architected in a much more robust way than their partially offline cousins. Partial offline apps can be considered more fragile than fully offline apps because they can’t be restarted or reloaded while offline, and you have to be very careful to limit their capabilities while offline otherwise the user can easily break the app. Fully offline apps are built with the knowledge that they may be offline for extended periods of time and need to be self-sufficient because users will be depending on them. If a fully offline app breaks then the user will be completely hosed (and very unhappy) until some point in the future where the internet connection can be restored and they can resync the app.

Offline apps can break in any number of interesting ways such as throwing a 404 when the user hits the back button or simply crashing when the app unsuccessfully attempts to a load a new page. By their very nature, fully offline apps may have larger and more complex data storage and life cycle requirements. They cache entire HTML web pages and all their associated JavaScript libraries, images and supporting data.

Examples of full offline apps include mapping apps, web email, reader apps, and any other apps that require information from a database.  User data is typically downloaded locally, stored on disk and then accessed by offline web applications. Any data that’s stored in memory will be lost when if the device or browser is restarted while offline.

 

Web offline versus Native offline

When building out your requirements, it’s a best practice to do an honest comparison between offline web capabilities and the offline capabilities of native SDKs.  In general as of the writing of this post, it’s fair to say that native apps still offer much more robust offline capabilities than the latest versions of mobile browsers. There are a few exceptions where browsers may have similar capabilities but almost always the level of control is more limited.

Native apps have the advantage because they basically have direct access to the device operating system and many of the capabilities are simply integrated into the respective SDKs. Here is a partial list of capabilities that are commonly seen in native offline requirements:

Web apps, on the other hand, run within the browser and are subject to any limitations imposed by the browser. The browser, itself, is a native app and it restricts it’s own children (web apps) to certain security restrictions. A few examples of web app limitations include:

  • Access to a limited number of censors. Access is not consistent across different browser types.
  • Limited control over location services via HTML5 Geolocation API.
  • JavaScript cannot programmatically read and write non-cached files on the device without user intervention.
  • Internet connectivity detection typically dependent on 3rd party libraries such as offline.js. Support is inconsistent across some platform/browser combinations.
  • Indirect and limited control over battery life and optimization.
  • Browsers and any of their associated tabs stop running as soon as the browser is minimized. If you have a requirement for the app to “wake up” from a minimized state under certain conditions you will have to go native.

 

Summary

Partial offline applications are design to continue working gracefully during intermittent interruptions in connectivity. Because offline is considered a temporary or even momentary condition in this use case, partial offline apps can use lighter weight architecture and have smaller data storage needs than full offline apps.

Fully offline apps are designed to be taken offline for extensive periods of time. They have to meet more demanding requirements and need a comprehensive architecture that enables storing of HTML files, JavaScript libraries, and user data as well as being able to handle browser reloads and restarts while offline.

Lastly, when having conversations about building offline apps you should weight web versus native offline capabilities against your requirements. Native SDKs still offer much richer control over most of the aspects of offline functionality.

Stay tuned for additional posts on this subject. Part 2 will look at the features and APIs you can use to take applications offline.

 

References

10 ways to deal with intermittent internet connections

How accurate is Android GPS? Part 1: Understanding Location Data

Wikipedia – Browser Security

Tags: , , , , ,
Posted in JavaScript, Mobile | Comments Off

Deleting an HTML Application Cache

When you are testing web applications that use an Application Cache, also sometimes called the manifest file, you have to delete this file every time you make a change to the application. If you don’t then none of the changes you make to the application will show up. The very purpose of the Application Cache is to semi-permanently store your HTML, CSS, JavaScript and images. It’s becoming increasingly popular for speeding up web app performance, and a requirement for taking web apps offline. In fact, Google now uses an application cache for their home page.

Simply trying to delete your browser cache in the normal way won’t necessarily clear the Application Cache and its associated files. So here’s a quick rundown that will hopefully save you some time.

Chrome – browse to chrome://appcache-internals/.  There may be a number of different caches listed. Select ‘Remove’ for any cache that you want to go bye-bye.

Chrome (Mobile Android) – go to Settings > Privacy (under Advanced) > CLEAR BROWSING DATA, checkbox the ‘Clear the cache’ option and then select the ‘Clear’ button.

IE 10 – go to Tools > Internet Options > Settings > Caches and databases tab. Select the cache that you want to delete and the click the ‘Delete’ button.

Safari (Mobile) – For Safari iPhone and iPad go to Settings and select “Clear Cookies and Data.”

Safari (Desktop) – Simply attempting Develop > Empty Caches may not work. On a Mac you may have to: close your browser, manually delete the .db file by going to //library/Caches/com.Apple.Safari and move any item ending in .db to the trash, then restart browser. If this doesn’t work then try restarting your machine. Yep, it’s an awful workflow and it’s been a known bug in Safari dating back to at least version 6.

Firefox (Desktop) – go to Tools > Options > Advanced > Network > Offline data > Clear Now.

Want to learn more about Application Cache’s? Here’s a good technical overview from WHATWG describing what is an application cache. And, MDN has a good article on Using the application cache.

Tags: , , , ,
Posted in Browsers | Comments Off

The question that I get asked the most these days is “what development languages should I be learning [to stay competitive/excited/motivated/etc] ?” The high-tech industry, and software in particular, is changing at a ridiculously fast pace and that introduces a lot of uncertainty and confusion as well as great opportunity. My answer to this question is unequivocal: I think for the foreseeable future developers should be learning the basic concepts of JavaScript and Python. If you don’t already have these skills then you simply cannot go wrong with this approach. If you’ve been a long-time server-side developer, or you are just getting started with software development then knowing the patterns and practices for JavaScript and Python will serve you well.

Why?

There are three primary reasons and I’ll try to be short and to the point. First, there are at least 2.5 billion internet users world wide, and that number is growing. Their primary method of accessing the web is a browser and JavaScript is the lingua franca of the browser world. JavaScript is a scripting language and it is “the” fundamental building block that allows web pages to “do” things such as submitting your search request to a server, or helping to find your location from your phone. Almost all web pages being served up around the world have JavaScript in them.

Second, the majority of retail, commercial and governmental web applications have a requirement that calls for the use of “server-side” code. This is code, such as Python, that runs on a server and not in the browser application. The most common functionality of server-side code is passing data back and forth between a database and a web application. For example, if a web app asks for a username and password, that user name and password are almost always stored on a server somewhere and not, for security reasons, in the web page and on the client browser where it could be very easily stolen.

Third, you can absolutely apply these client-server patterns and practices to other languages used within the realm of web development. A User Interface designer who has been solely focused on layout and styling via Cascading Style Sheets (CSS) can now understand and appreciate how the underlying JavaScript code can affect the look, feel and behavior of a web page. Python skills can also be used a springboard for more quickly learning other powerful web development platforms such as ruby-on-rails.

The bottom line is if you understand both client development (JavaScript) and server development (Python), then you start to gain considerable value as someone who understands how to help the entire system work together in harmony.

A short note on jQuery.

Many (most?) new web developers learn jQuery first. However, even if you know jQuery you don’t necessarily understand JavaScript. The awesome jQuery libraries provide an interface that hides and simplifies a lot of native JavaScript hoopla, and in general can really make life significantly easier and save time when building modern cross-browser web apps. jQuery is built using JavaScript (and CSS3), but it is not JavaScript. Because of that, when something goes wrong or not as you expected (not if, but when!), and you have a general understanding of how JavaScript works, then you stand a much better chance of figuring out a timely work-around with significantly less stress, frustration and time wasted towards your projects deadline.

The absolute minimum recommended reading list:

JavaScript.

  • Douglas Crockford’s book “JavaScript: the good parts”.
  • Douglas Crockford’s website – He is considered a key brainchild behind the ongoing development and understanding of JavaScript.
  • W3schools – An excellent website for anyone using or learning JavaScript. It has tutorials and online Try It Yourself sample apps.

Python.

Tags: , , , , , , ,
Posted in JavaScript | Comments Off

Where Part 1 focused on non-GPS enabled devices, Part 2 is totally focused on mobile web geolocation. The great news is that the usage of HTML5 location services along-side the fact that there is a GPS chipset in most, if not all, modern smartphones and tablets dramatically improves the chances of getting an accurate location. And, besides that fact — mobile geolocation is simply a lot of fun to work with.

I also want to point out that there are an increasing number of really good blog posts covering the topic of “how to use” the API that look at the nitty-gritty of how the code works. This post is different in that I’ve tried to focus on “how to build successful applications” with the API, and how to get the most out of the API so that you can successfully implement your unique requirements.

What’s different about desktop vs. mobile HTML5 Geolocation? With mobile you can access the GPS if it’s available. It’s important to note that in order to access a device GPS you have to set the optional enableHighAccuracy property in your code. Contrary to what is shown in some samples on the internet, you can use this property with both the getCurrentPosition() and watchPosition() functions.

//One time snapshot
navigator.geolocation.getCurrentPosition(
     processGeolocation,
     // Optional settings below
     geolocationError,
     {
         timeout: 0,
         enableHighAccuracy: true,
         maximumAge: Infinity
     }
);

//Tracking users position
watchId = navigator.geolocation.watchPosition(
     processGeolocation,
     // Optional settings below
     geolocationError,
     {
         timeout: 0,
         enableHighAccuracy: true,
         maximumAge: Infinity
     }
);

How accurate is it??? This is the million dollar question, right? When using enableHighAccuracy() on a phone where all the appropriate permissions have been selected and granted, I’ve typically seen accuracy readings as low as 3 meters (~10 feet) that were obtained within 10 – 30 seconds of kicking off the geolocation functionality. I’d consider that excellent for most consumer and retail applications. You should be aware that like any location-based functionality you will get spurious (abnormal) results that fall way outside the norm, and sometimes these results are wildly wrong.

I’ve seen claims that using the enableHighAccuracy() property slows down the phones ability to deliver a location. I’m going to argue that those claims are misleading. It is true that the GPS, itself, can take a significant amount of time to warm up and start delivering high accuracy results. For an in-depth look at that topic see my post on the Six Most Common Use Cases for Android GPS. However, there are conditions where simply enabling the enableHighAccuracy() property doesn’t affect the speed in which you can get the initial result. More on these topics below.

What is the best way to try out various configuration scenarios? I’ve built an HTML5 Geolocation Testing tool that can be used in browser, or it can be repurposed to work in PhoneGap or Titanium. It is a jQuery-based mobile application that includes a map and settings view were you can adjust all the different properties and try out different configuration scenarios. It’s a work-in-progress so I welcome suggestions and pull requests.

 Why HTML5 Geolocation rather than native? Applications using HTML5 Geolocation typically have slightly different requirements than native GPS-based applications. Each platform has its advantages and disadvantages and it all comes down to your requirements, budget, timeframes and skill sets:

  • Ability to re-use existing JavaScript and HTML5 skills to build a high-accuracy mobile application.
  • Don’t have access to native platform developers or skillsets on Android, iPhone and/or Windows Phone.
  • Need a cross-platform stand-alone web app, or a web app that has been repurposed to work with PhoneGap or Titanium.
  • Quickly locate the user/consumer within a reasonable expectation of accuracy.
  • Typically it is a non-commercial, consumer grade application that does not have extremely high accuracy requirements (e.g. < 1 meter).

How fast can I get an initial location result? The answer is very fast, potentially within a few seconds, given the following scenarios:

  • If there was a cached GPS or Network location stored on the phone. The GPS location is, of course, from the GPS chipset. The Network location comes from your wireless carrier and is dependent on your phone and their capabilities.
  • How the timeout and maximumAge properties are set. If you set timeout = 0 and maximumAge = Infinity it will force the application to grab any cached location, if one is available. Other settings may result in delays.
  • If the phone or tablet has decent internet connectivity and Wifi enabled.
  • If the device is in an urban area with many wifi nodes broadcasting their SSIDs nearby.
  • The device has a clear and uninterrupted view of the sky. GPS’s listen for a very weak signal from multiple satellites. These signals can be partially or completely blocked by buildings, thick foliage, vehicle roofs, etc.

 How accurate is the initial location result? Hah, you might have guessed I’d say that it depends.  When you first kick off a geolocation request, accuracy does depend on a number of different factors that are mentioned above. And it’s safe to say that, in the vast majority of cases, the first location is not the most accurate and typically not the most dependable. If you want the fastest, most accurate location possible then you will most likely need to either do multiple snapshots, or use watchLocation until your desired level of accuracy is met. It’s important to note because I’ve been asked about this many times, you cannot expect the GPS, itself, to have enough time to lock onto a satellite and deliver a fast, accurate initial location. It may take dozens of seconds or even minutes. Yep, it’s true. Factors that affect initial location accuracy include:

  • Cached locations – how recently the user accessed location functionality. For example, applications like Facebook typically grab a location when you open the app. So frequent users of social media are more likely to have a fresh, cached location that non-social media users. If you are targeting business travelers, the cached location might the last city before they got on a plane. Or, it could be your home neighborhood and not where you work or go to games.
  • Wifi turned “on”. If the Wifi is turned on then the device can access the location service and there is a much greater chance that the initial result is fairly accurate. If you didn’t have a chance to read Part 1, when the Wifi is on your browser gathers local Wifi node information from your Wifi card, and it can use that information in a location service provider request over the internet to try and triangulate your position. Typically this means your initial location can be within a block or two of the actual position. Also, it is possible if Wifi is turned on that you can get a significantly more accurate initial location than if you were using GPS by itself with no Wifi or internet.
  • Internet connectivity strength. If you have a poor internet connection and no Wifi, then the browser’s requests to the location service can be delayed, blocked or even interrupted.
  • No VPN. Take note commercial application developers: as mentioned in Part 1, if VPN software is in use it can wildly affect accuracy and even place you in another State (or Country).

Can I use HTML5 Geolocation for mobile tracking? Yes, with caveats. Typically HTML5 tracking applications are built inside a native wrapper framework such as PhoneGap or Titanium. There are several immediate problems with stand-alone, browser-only HTML5 tracking applications. First, there is no built-in functionality to keep the screen from going to sleep. Second, when the screen goes to sleep the HTML5 Geolocation functionality also goes to sleep. Native-based tracking applications can work around these limitations and listen passively in the background when they are minimized. Third, you have little control over the GPS settings to help management battery consumption.

Can I use HTML5 Geolocation offline? Yes! If there is no cellular connection or Wifi available, then HTML5 Geolocation can still access cached locations and real-time GPS information. This is vastly different from what was discussed in Part 1 as related to applications targeted at laptops, desktops and tablets that may or may not have GPS. If a device does not have a built-in or externally available GPS then your offline application will not work.

Handling abnormal location results. Your application will occasionally encounter widely inaccurate results and you need to handle these gracefully for the best user experience possible. My recommendation is to check the timestamps and distance traveled between the current geolocation object and the previous one. If the distance or speed seems excessive then you’ll need to reject the result. In the reference section below is a link to more information on calculating the distance between two points containing latitude and longitude. As an example, see the attached screenshot with the spurious results indicated by red circles. Also note in the screenshot the accuracy level was 3 meters, so it’s important to understand that even at high accuracy levels you still need to very that each location meets your minimum requirements. This way your results will always look polished and professional to the end user.

Spurious results

What are some of the downsides of using HTML5 Geolocation versus native? The bottom line is that for simple location gathering and basic tracking HTML5 Geolocation is just fine. This should meet the requirements for most consumer applications. For anything more complex than that you should consider looking at going native.

  • It may not work on older phones and older browsers (depending on your definition of old). See below in the references section for a link to a fallback library to handle these situations.
  • HMTL5 Geolocation offers significantly less control over GPS settings. This can have an unacceptable impact on more complex applications.  Because of this, I also suggest that HTML5 Geolocation is not suitable for long-running tracking applications.
  • Battery life management. This is a direct result of bullet #2. It’s more challenging to manage battery life with HTML5 Geolocatoin if your requirements call for continuous use of the GPS.  Your control is very limited with respect to these two properties: timeout and maximumAge.
  • Cannot use it when the application is minimized. If your requirements calls for the ability to passively receive locations while in a minimized state then, as mentioned earlier, you will have to go native.
  • Very little control over how often you want location updates. You’ll need to do a bunch of custom coding to emulate what is already built into native application APIs. For example, the native Android API offers very detailed control over what type of geolocation data you can get access to, how you can access it and how often. Read more on that topic in my post on How Accurate is Android GPS Part 1 – Understanding Location Data and also take a look at Android’s LocationManager Class.

References

W3C Geolocation API Specification 

HTML5 Geolocation Test Tool

Mozilla – Using Geolocation

Calculating distance between two points.

Geolocation fallback library for older browsers

Tags: , , , , , , , ,
Posted in Browsers, HTML5, Mobile | 19 Comments »

Tonight I had the pleasure of presenting at the HTML5 Denver User Group. It looked like there was a really good turnout of over 130 people. I followed an excellent presentation on JSON by Tom Marrs. He really dived down into the guts of what JSON is, and I wished someone had done as good of a job explaining it when I was just getting started using JSON.

The concept of my presentation was that there are three main approaches to building web mobile applications: JavaScript mobile, Hybrid and Responsive. By a show of hands it looked like over 50% in the audience were brand new to web mobile and I had planned for that so hopefully the level of content was just right. If you want to discuss any of the content in more detail, feel free to drop me an email.

Here’s a link to a PDF version of the presentation

Tags: ,
Posted in HTML5, Mobile | 2 Comments »