Going Offline with HTML5 and JavaScript, Part 1

There are two primary use cases for going offline with mobile HTML5 web applications and JavaScript: partial offline and full offline. Before diving into building offline apps, understanding the differences between these use cases will help you build the best applications for your requirements. The functionality in modern browsers has finally gotten to the point where it is feasible (and fun!) to build offline web applications.

Partial Offline. Partial offline means the vast majority of the time the application is online, however it can continue to work if the internet connection gets temporarily interrupted. A partially offline app understands that requests for remote resources, that is resources that don’t exist on the device, will automatically defer to local resource, or at least fail gracefully, during the period of time when an internet connection doesn’t exist. Partially offline apps typically cannot be reloaded or restarted offline. The coding required to handle this scenario is much lighter weight than the architecture required for going fully offline. An example of partial offline is a reader app that pre-caches certain HTML pages of your choice. If the internet connection gets disrupted you can continue reading and navigating between the cached pages.

The partially offline scenario exists because there is no such thing as a perfect internet connection for mobile. In reality, internet connections and download speeds are very choppy when measured over a period of minutes or hours. Sure, some internet providers market 4G connections as being extremely fast, or have the best coverage etc., etc., blah. The bottom line is cellular and even WiFi internet connections are not guaranteed. A good example of this is coffee shops. They don’t come with an uptime guarantee, so if a couple of yahoos sitting next to you are streaming HD Netflix then that will surely bring the internet connection to its knees.

In reality, if you don’t live danger close to a cell phone tower and are moving around doing your job or running errands chances are your internet connection will fluctuate up and down over time. Anyone that owns a smartphone has experienced this at one time or another. Dropped calls are perfect example. You may be shopping and going in and out of buildings, or hanging out in the back of a taxi, sitting in your car pulled off the side of the road, or perhaps even just watching your kids as they play in the neighborhood park. A web application architected for partially offline will let you keep surfing or working for a short period of time, and hopefully long enough until the internet connection comes back up.

Full Offline. A fully offline JavaScript application is one that starts out online to download all the necessary data and files, then it can be completely disconnected from the internet. These apps can survive browser restarts and reloads, they can stay offline indefinitely and/or they can be resynced online at some point in the future.

Fully offline apps need to be architected in a much more robust way than their partially offline cousins. Partial offline apps can be considered more fragile than fully offline apps because they can’t be restarted or reloaded while offline, and you have to be very careful to limit their capabilities while offline otherwise the user can easily break the app. Fully offline apps are built with the knowledge that they may be offline for extended periods of time and need to be self-sufficient because users will be depending on them. If a fully offline app breaks then the user will be completely hosed (and very unhappy) until some point in the future where the internet connection can be restored and they can resync the app.

Offline apps can break in any number of interesting ways such as throwing a 404 when the user hits the back button or simply crashing when the app unsuccessfully attempts to a load a new page. By their very nature, fully offline apps may have larger and more complex data storage and life cycle requirements. They cache entire HTML web pages and all their associated JavaScript libraries, images and supporting data.

Examples of full offline apps include mapping apps, web email, reader apps, and any other apps that require information from a database.  User data is typically downloaded locally, stored on disk and then accessed by offline web applications. Any data that’s stored in memory will be lost when if the device or browser is restarted while offline.

 

Web offline versus Native offline

When building out your requirements, it’s a best practice to do an honest comparison between offline web capabilities and the offline capabilities of native SDKs.  In general as of the writing of this post, it’s fair to say that native apps still offer much more robust offline capabilities than the latest versions of mobile browsers. There are a few exceptions where browsers may have similar capabilities but almost always the level of control is more limited.

Native apps have the advantage because they basically have direct access to the device operating system and many of the capabilities are simply integrated into the respective SDKs. Here is a partial list of capabilities that are commonly seen in native offline requirements:

Web apps, on the other hand, run within the browser and are subject to any limitations imposed by the browser. The browser, itself, is a native app and it restricts it’s own children (web apps) to certain security restrictions. A few examples of web app limitations include:

  • Access to a limited number of censors. Access is not consistent across different browser types.
  • Limited control over location services via HTML5 Geolocation API.
  • JavaScript cannot programmatically read and write non-cached files on the device without user intervention.
  • Internet connectivity detection typically dependent on 3rd party libraries such as offline.js. Support is inconsistent across some platform/browser combinations.
  • Indirect and limited control over battery life and optimization.
  • Browsers and any of their associated tabs stop running as soon as the browser is minimized. If you have a requirement for the app to “wake up” from a minimized state under certain conditions you will have to go native.

 

Summary

Partial offline applications are design to continue working gracefully during intermittent interruptions in connectivity. Because offline is considered a temporary or even momentary condition in this use case, partial offline apps can use lighter weight architecture and have smaller data storage needs than full offline apps.

Fully offline apps are designed to be taken offline for extensive periods of time. They have to meet more demanding requirements and need a comprehensive architecture that enables storing of HTML files, JavaScript libraries, and user data as well as being able to handle browser reloads and restarts while offline.

Lastly, when having conversations about building offline apps you should weight web versus native offline capabilities against your requirements. Native SDKs still offer much richer control over most of the aspects of offline functionality.

Stay tuned for additional posts on this subject. Part 2 will look at the features and APIs you can use to take applications offline.

 

References

10 ways to deal with intermittent internet connections

How accurate is Android GPS? Part 1: Understanding Location Data

Wikipedia – Browser Security

Tags: , , , , ,
Posted in JavaScript, Mobile | No Comments »

Why I’m considering removing blog comments

Many others before me have disabled their blog comments. In fact, a blog post by Seth Goden from 2006 still sticks in my mind and all the outcry it generated from around the web. The longer my blog has been around the worst the spam has gotten. In a way that kinda makes sense as my domain name gets shared via spammer databases and my growing number of blog post links get stored in search engine results.

Spam comments are awful. As of today, Askimet says it has blocked over 250,000 spam comments since March 2014. During the time it took to write this post I’ve already received 16 spam messages. For every single legitimate comment, I get 1,000 spam comments. Seriously. If managed improperly, spam comments can crash your blog if your hosting provider limits the size of your blog database.

This happened to me earlier in the year. Silly me, I thought that it would be impossible for me to go over my previous hosting providers pre-set 2 gigabyte database size limit any time soon. Most of my blog posts are text based and take up very little room. The entire size of my blog database was around 15 MBs. I didn’t even know there was a problem until I couldn’t create any more blog posts. Upon further examination I had received over 75,000 spam comments over a 30 day period and they were waiting patiently for deletion. I’m not exactly clear on why Askimet didn’t correctly report these, or automatically delete them within 15 days, or what exactly happened. One thing was clear: my comments table in the database was full of tens of thousands of spam comments. I thought I had set them for automatic deletion but apparently Askimet only auto-deletes the worst of the worst spam and everything else goes to the spam queue. Caveat Emptor.

I still think I see the value of comments in that everyone including me can benefit from the “shared” feedback including myself. But, the increasing amount of time and headaches spent dealing with spam is making me seriously reconsider.

Tags: , ,
Posted in WordPress | No Comments »

5 ways that passwords get compromised

Over the last month as I’ve been doing normal updating of passwords I came across several major public company websites that gave me the following error:

The password you entered is invalid. Please enter between 8 and 20 characters or numbers only.

Anyone that knows anything about “Passwords 101” knows if you go with a minimum of 8 characters and/or numbers it could problematic if someone hacked into the password database. Security experts say that excellent password security includes alphabetical, numeric as well as non-alpha-numeric characters. A password of 8 characters and/or number could be hacked in micro-seconds.

This nicely dovetails with a number of conversations that I’ve had about this recently, and there is so much speculation about password security that I felt compelled to list the potential security holes for passwords. There is very little that you can do beyond minding your own passwords strength. The rest is up the companies and organizations that host your data. Here’s my list of the most common ways that passwords can get compromised.

Inadequate passwords – I suppose it makes sense to start off my list with this topic. But first I have a few important words about password strength. Simple passwords, such as those containing a limited amount of numbers and letters, for example “Test1234″, can be cracked in milliseconds on a typical laptop. When criminals get ahold of a username/password list the first they do is called a dictionary attack in which they try to compromise the easiest to break passwords first.

Unencrypted passwords stored on database – Not encrypting passwords in some way is the same thing as leaving the keys in the ignition of your car with the door unlocked. This is like ice cream to anyone that has access to the database, legally or illegally. They can simply download the ready-to-use user names and passwords.  It doesn’t matter how sophisticated of a password you have if it’s simply unencrypted. There is no way for the user to know if the passwords are encrypted or not, it’s completely up to the IT department that controls the database.

Phishing virus – This virus can provide usernames and passwords for a targeted organization. The intent of this virus is to trick someone into entering their username and password on a fake website that mimics another website, such as an email logon screen. Once someone enters their credentials, they are immediately available to the criminals. The best prevention against phishing viruses is to not open any suspicious attachments and to keep your virus software up to date. There is not a 100% cure against getting a phishing virus since any attachment, even from legitimate sources, could be compromised. But, a good start is not opening ones from someone you don’t know or one that has a suspicious sounding name. Trust your instincts.

Keystroke loggers – This is a spy program that can be installed on any computer. They can be very hard to detect and they do exactly as advertised: they log everything you do on a computer and then they typically relay that information to somewhere else on the internet where your data can examined. The best protection against keystroke loggers, and viruses as well, is a multi-faceted approach: anti-virus programs, spyware sweepers and software firewalls. In addition to that, occasionally viewing which programs are running on your computer and researching any program names you don’t recognize.

Network Sniffers – Sniffer software can monitor all internet traffic over a network.  Network sniffers can easily compromise public WiFi. The person who collects the sniffer data can sift thru the digital traffic information and then siphon out different types of login requests. Once they have the login information, if it’s encrypted they can run cracker programs against the encrypted information. As an internet surfer there is something you can to to help prevent getting your username and password siphoned off: purchase a consumer Virtual Private Network (VPN) product and always use it, especially if you are on an open WiFi at some place like Starbucks or an Airport.

Conclusion

Is there such thing as a truly secure password? Definitely not. This is especially true if the database server containing the password has less-then-adequate security measures in place to protect it from unauthorized intrusion.

Is there anything you can do to protect yourself? Absolutely. My minimum recommendation is to get a password manager program. They can create strong, unique passwords for every website that you need. Some password managers also let you securely share passwords between phone, tablet and laptop. Search for the words “password manager” to find out more. Here’s an article you can peruse as a starting point: pcmag.com. You should also keep your anti-virus software up-to-date and regularly run spyware scanners. Some folks go even one step further and install a software firewall that lets you control all communications to and from your computer. Last, but not least you can use VPN software when surfing the internet.

So what did I do about the websites mentioned above with poor security? In one case I wrote the CEO of the company and I also switched to using the maximum number of characters and numbers, which in the case mentioned above was 20. That website didn’t really store anything vital. In other case, I dropped the website like a lead balloon. If their password security is less-than-optimal I couldn’t help but question and wonder about the rest of their digital information security practices.

Tags: , ,
Posted in Browsers, Security | No Comments »

Mobile web developers: let users adjust font size

Many consumer web sites have done a fantastic job of deploying mobile web versions of their sites. However, over the last year I have heard an increasing number of complaints about websites specifically designed for mobile: you simply can’t adjust the font size. Depending on the device’s screen size, this can potentially cause painful choices: either deal with (usually) tiny font sizes and struggle to read content, or go to the full web site and be relegated to panning, zooming and rotating between portrait and landscape mode to try and fit as much content as possible onto a tiny screen.

To me, the solution is easy: allow users to set their own font-size. Here’s an example of settings I’m referring to, specifically setting the viewport’s user-scalable property to ‘no’ is what’s causing the problem:

<meta name="viewport" content="width=device-width, user-scalable=no">

There are certainly use cases, such as mapping applications, were you might want to prevent user scaling because it will affect other components inside the application.  However, if there aren’t any components affected by pinch zoom then you most likely can set the user-scalable property to yes, or simply omit the property.

My first suggestion is provide users with the ability to adjust the font size.  Here is an example that will let the user adjust font size for a specific DIV:

<meta name="viewport" content="width=device-width">
<script>
    var up = 1;
    var down = -1;
    var elementSize = document.getElementById(“someDiv”) .style.fontSize;

    function increaseFont(){
        up++;
        elementSize = up + “px”;
    }

    function decreaseFont(){
        down--;
        elementSize = down + “px”;
    }

    function resetFont(){
        elementSize = “14px”;
    }
</script>

Another option is to combine the above suggestion with media queries to adjust font-size automatically based on screen size, pixel ratio and/or min-resolution. Here is one example:


@media
(-webkit-min-device-pixel-ratio: 2),
(min-resolution: 192dpi) {
    body{
        font-size: 16px;
    }
}

Other examples of media queries can be found here: css-tricks.com.

Tags: , , ,
Posted in JavaScript, Mobile | 2 Comments »

Easily find image type in JavaScript

There are two easy ways to determine an image’s type using JavaScript: using an html Input tag with a type file, and using the DataView API. I’ve put together a github repository that contains all the code shown below. The sample app detects PNG, GIF, JPEG and BMP: https://github.com/andygup/DetectImageType.js

Here’s how to do it with an Input tag:

    <input type="file" id="fileInput" name="file"/>
    <script>
    var fileInput = document.getElementById("fileInput");
    fileInput.addEventListener("change",function(event){
        document.getElementById("name").innerHTML = "NAME: " + event.target.files[0].name;
        document.getElementById("type").innerHTML = "TYPE: " + event.target.files[0].type;
        document.getElementById("size").innerHTML = "SIZE: " + event.target.files[0].size;
    });
    </script>

And, here’s how to do it with the DataView API. The concept is to retrieve the image via an HTTP request with the response type set to “arraybuffer.” And then extract the hexadecimal signature or magic number of the image type. I’ve taken the liberty of only reading the first 2 bytes to get the magic numbers in my example. If you need more precision here’s a great site to use for more information on image signatures: http://www.filesignatures.net/index.php?page=search

  function getImageType(arrayBuffer){
        var type = "";
        var dv = new DataView(arrayBuffer,0,5);
        var nume1 = dv.getUint8(0,true);
        var nume2 = dv.getUint8(1,true);
        var hex = nume1.toString(16) + nume2.toString(16) ;

        switch(hex){
            case "8950":
                type = "image/png";
                break;
            case "4749":
                type = "image/gif";
                break;
            case "424d":
                type = "image/bmp";
                break;
            case "ffd8":
                type = "image/jpeg";
                break;
            default:
                type = null;
                break;
        }
        return type;
    }

The DataView API has really good browser support. The only issues you’ll have, not surprisingly, are with IE 8 and 9. For more info on support of the DataView API go here: http://caniuse.com/#search=dataview

Tags: , , ,
Posted in JavaScript | No Comments »

This is my 2014 wish list for where operating systems (OS) should be headed with laptops, tablets, smartphones and smart devices. Now before you lambast me or fill my ears with technical mumbo-jumbo about why some of these ideas aren’t possible, just take a slow, deep breath. I offer these concepts up as a challenge to take things to the next level, and not as fodder for a debate contest of what’s possible and what’s not. I hope these ideas are viewed as worthy goals rather than existing only in our imaginations thru science fiction.

I suggest it’s time we rethink operating system kernel theory and discard some of our historical notions of how operating systems are supposed to work. I’m continually amazed that even the newest operating systems, such as Android, have fundamental problems similar to what we’ve had since the earliest versions of Windows! So here’s my list…

No more OS lockups – It’s 2014 and computers still experience software related operating system crashes. In the last year, I’ve personally had brand new Windows machines, Mac’s and smartphones lock up in one way or another. No, it’s not just bad luck. I put the onus and ultimately responsibility back on the OS vendors. A 21st century OS should be hyper-intelligent about memory allocation and reclamation. The OS should be able to gracefully self-recover from everything short of a fatal hardware failure.

No more app crashes – I’m sure the OS developers will blame this on the application developers and vice-versa.  My take on this: app crashes should never happen. There are many well-known bad patterns that operating systems can monitor for and avoid. The OS should be able to detect bad application code and handle it without coming to a screeching halt. Examples that I’m thinking about include:

  • being aggressive about detecting and providing programmatic feedback on memory leak conditions,
  • automatically isolating run-away code blocks so they don’t lockup an entire application,
  • giving applications feedback on whether or not they are on a trajectory to run out of memory rather than simply killing them off,
  • provide not just guidelines but also build-time test tools for analyzing applications and provide pointed feedback on best practices.
  •  some may consider it draconian, but you could be more assertive on failing builds that don’t meet a minimum best-practice standard set by you, the OS manufacturer.

Dynamic updates – We should be able to update the OS and apps while they are running. I really don’t like having to reboot any device that gets updated, and in the case of Windows this can lead to multiple reboots and that is a major pain. This includes phones, computers, as well as TiVo’s, Hoppers and more. I’d like to see OSs model themselves after web pages that can replace specific content on-the-fly without having the refresh the entire page.

Instant boot – OS should allow smart, lazy loading of modules and applications as needed. Do we really need to wait for everything under the sun to load up front while we wait…and wait? My iPad takes some time to boot, my Android Nexus takes even longer, but my MacBook boots within seconds.

So that’s my short list. I hope some OS engineers have a chance to read this and give my suggestions thoughtful consideration.

Tags: , , ,
Posted in Innovation | No Comments »