Easy automation of JavaScript form testing

If you are writing unit tests that provide coverage for HTML forms then there is an easy, pure JavaScript way to automate testing of the underlying code that works on modern browsers. The nice thing about this approach is you don’t have to manually load a file every time you run the tests. You still need to test the HTML interface components but that’s a topic for a different blog post.

The concept is straightforward in that you need to emulate the underlying functionality of an HTML form. The good news is you don’t have to programmatically create an HTML Form or Input element. Here’s the pattern you need to follow:

  1. Create an xhr request to retrieve the file. Be sure to set the response type as blob.
  2. Take the xhr.response and create a new File Object using the File API.
  3. Inject the File Object into a fake Form Object or,
  4. You can also use FormData() to create an actual Form Object.
  5. The fake Form Object is now ready to pass into your unit tests. Cool!

Here’s how you create a JavaScript FormData Object. Depending on what data your code expects you can add additional key/value pairs using append():

var formData = new FormData();
formData.append("files",/* file array */files);

And, here’s what the basic pattern looks like to retrieve the file, process it and then make it available for your unit tests:


var formNode; // Unit tests can access form node via this global

function retrieveFile(){

    var xhr = new XMLHttpRequest();
    xhr.open("GET","images/blue-pin.png",true); //set path to any file
    xhr.responseType = "blob"; 

    xhr.onload = function()
    {
        if( xhr.status === 200)
        {
            var files = []; // This is our files array

            // Manually create the guts of the File
            var blob = new Blob([this.response],{type: this.response.type});
            var bits = [blob,"test", new ArrayBuffer(blob.size)];

            // Put the pieces together to create the File.
            // Typically the raw response Object won't contain the file name
            // so you may have to manually add that as a property.
            var file = new File(bits,"blue-pin.png",{
                lastModified: new Date(0),
                type: this.response.type
            });

            files.push(file);

            // In this pattern we are faking a form object
            // and adding the files array to it.
            formNode = {
                elements:[
                    {type:"file",
                        files:files}
                ]
            };

            // Last, now run your unit tests
            runYourUnitTestsNow();
        }
        else
        {
            console.log("Retrieve file failed");
        }
    };
    xhr.onerror = function(e)
    {
        console.log("Retrieved file failed: " + JSON.stringify(e));
    };

    xhr.send(null);
}

The problem with JavaScript Obfuscators and Minification – Tracking down errors

JavaScript obfuscators and minifiers do their job well. In fact, some obfuscators have anti-debugging features. However, if you are a legitimate developer building applications against one of these libraries, chances are you’ve gotten an indecipherable error such as “z=null line 14300” and it brings your development efforts to a halt. Error messages like this provide no useful information on what the problem really is, or give any hints on how you might be able solve it. You’ve probably even looked at the jumbled source code in a last ditch attempt to make some sense out of the error. And, whether it’s your own library or a mainstream ones as jQuery or Dojo, it doesn’t matter. The amount of productivity lost because of these errors in probably very large, not to mention the frustration it causes.

I hope the the developers of these obfuscators are reading this…because I have a proposed solution to the problem.

Now, I want to start out by mentioning that I fully understand why obfuscators exist for reasons such as source code protection and decreasing download size. What I propose takes this fully into account, yet makes your library developer friendly in a secure way:

During the obfuscation process create an index file that maps each variable, function and class to a real line number and store this file in a web folder.  Then create a small html file that lets you search the index and return the real line number. Provide an option for return the variable, function or class name, too.

The concept is that if there is an error, like the  “z=null line 14300” I mentioned above, developers can then at least have some hope of narrowing down the general area of the code where it might be occurring.

The bonus is, if you own an obfuscated commercial library, now your tech support people can also look up the general area where a customer might be having a problem. For security reasons you don’t have to share the index file, But, even then, there isn’t enough information in it to de-compile the library. Now, if I post my error to the forum:  What is “z=null line 14300”? Tech support will be able to tell me that I’m missing a custom property on a widget’s HTML DIV element. It’s a win-win situation.

What do you think?

7 required improvements for the Web, HTML and JavaScript

Here’s my 2012 web developer wish list for improvements that I’d like to see happen in the web developer world. If HTML and JavaScript want to be considered enterprise ready for commercial-grade deployments then here’s some things that are needed today.

For clarity, I consider a commercial software deployment to be one that contains over one thousand lines of code, at least two custom .js libraries and involves at least two developers and some sort of code versioning system.

  1. Refactoring. Not having this capability continues to be a huge productivity issue for large projects. Try refactoring across six JavaScript libraries and 1200 lines of code using Notepad++.
  2. Even stronger scope enforcement in JavaScript classes. One wrong misspelling and you can spend fun filled hours (or days) tracking down a private variable that turned itself into a global variable.
  3. Built-in support for code comments. Visual Studio does a fine job, for example. But, it’s still kind of a hack to make it work. I’d like the built-in ability to create comments for methods and classes directly and then be able to access those comments via intellisense throughout any file in the project. Again, this is all about productivity by having this information accessible at your fingertips.
  4. Better built-in JavaScript checking for IDEs. I’d like to see built-in JSLint-like capabilities that have been updated to the latest HTML, JavaScript and CSS3 versions, and not some third party plug-in that’s optional.
  5. Best practice whitepapers. These would be whitepapers written by the browser vendors that provide guidelines on the correct patterns to use when building apps against their browsers. Seriously, it’s been roughly 21 years since we started using browsers and there’s no guidance at all from the powers that be.  Honestly, I’m stunned that these don’t exist. That would be similar to Microsoft publishing .NET and then not providing any conceptual help documentation.
  6. Official tools for browser certification and testing. The folks that build the browsers don’t give us a way to verify if we are building our apps in the best way possible. If these items existed, then quality could get a lot better, and we’d all learn a lot too.
  7. Slower browser release cycles. A slower release cycle for browsers and more improved security and stability. I already blogged about this here.

The 1 Minute Primer for HTML 5

HTML 5 is getting a lot of press these days and I get a constant stream of questions from many non-techies, as well as developers, asking me to explain HTML5 in layman’s terms. So here it is.

HTML 5 is really a combination of three things: HTML, CSS and JavaScript. When all three of these technologies work together in a web browser then you have an HTML5 application. Period.

Why should we care about HTML 5? HMTL 5 brings many long awaited enhancements that make it easier for web developers to build more complex applications. More importantly, HTML 5 is being adopted by the major browser vendors: Google, Microsoft, Mozilla and Apple and this adoption is making it possible for developers to take advantage of the latest web technology that are built into web browsers.

How is HTML 5 “built into a web browser”? Web browsers have to interpret a web page first, and then display the content for you. Browsers contain logic that let’s them parse a pages’ code, and that code provides instructions for the browser to do certain things. Behind the scenes, in fact, the page you are looking at is built using code. It’s the browser that interprets the code and displays it in a way that makes sense to you. If you haven’t ever seen web page code then you can usually select View > Source on your browsers tool bar. Cool, right?!

HTML 5. HTML 5 is the latest version of the Hypertext Markup Language (HTML) specification which has been around in various forms since approximately 1991. HTML is a tag based language that defines the meaning and placement of elements of a web page. For example, a <button> tag defines a clickable button on a web page.

Cascading Style Sheets (CSS). Cascading Style Sheets, or more specifically CSS version 3 (a.k.a CSS3), provide the ability to apply styling to HTML elements. An example of styling would be to change the color of an HTML <button> from grey to green, as well as defining where on a web page it will be visible such as the top left corner.

JavaScript. JavaScript, which is really the meat behind HTML 5, is a type of programming language that lets developers implement actions within a web page. An example of an “action” would be when a web page visitor clicks a button that loads a picture. So, HTML defines the <button>, CSS styles the button, and JavaScript handles the action behind the scenes by retrieving the picture and then telling the browser how to display it for the end user.

This all sounds great, are there any downsides? Yes. First, HTML 5 is a standards-based specification that is still a work in progress. The specification and all its’ associated parts won’t be finalized for some time, possibly years. The good news is that browser vendors are keen to adopt this standard as much as possible. Second, implementation across different browsers isn’t 100% consistent. The good news is that there are tools and online resources to help developers work around many of these problems. Last, older versions of browsers (e.g. Internet Explorer 7 or 8, older versions of Safari, etc) don’t support HTML 5. There are strong campaigns under way to educate people to upgrade for security, performance and viewing experience.

So, there you have it. That’s a cursory pass at HTML 5 and I hope this post helps. I’ve added a few links at the bottom if you want to learn more about it.

Learn More:

 HTML5Rocks.com – includes information on features, tutorials and great slide decks.

w3Schools.com –  includes live “Try it” samples that let you explore the functionality.

W3C HTML 5 Specification –the World Wide Web Consortium is the group that writes the standards. If you are a techie, this is “the” specification that the browser vendors base their functionality on.

Tips for Clearing the Browser Cache: IE, Chrome and Firefox

When doing web development, especially JavaScript/HTML, it’s sometimes hard to tell if your changes loaded when you refreshed the web page, in fact sometimes your changes aren’t reflected. The best thing to do is delete the cache and then reload the page. So, this post will tell you how to that for the three most used browsers – Firefox, IE and Chrome.

Before I tell you how to do it, it’s good to know what the cache does and why. It’s basically a file directory where your browser stores temporary files such as web pages (e.g html files), images (e.g. png’s) and other web-related items including sound files ( e.g. mp3’s). The idea behind storing these files is the user’s experience: it’s faster to retrieve a local file than it is to retrieve it from some remote web site and the page appears to load faster to the user. Another reason is reduces server load for high usage sites because many of the files are loaded locally for repeat visitors. But, even though you may care about this your end users certainly don’t.

Chrome 16.x (Windows)

Go to the top right of the browser and click on the wrench symbol, then Tools > Clear Browsing data. Chrome will then load the Options page and let you choose by timeframe how far back to go when deleting data. When you are doing frequent web development the “past hour” option is awesome. That way you can delete your most recent work and all your other cookies and data will stay in the cache.

 

Firefox 10.x (Windows)

Go to the top left of the browser and click the pull down menu, then History > Clear Recent History. Firefox then loads a popup window that also lets you choose how far back in time to delete data. Firefox was the first browser to offer the “last hour” option. Again, it’s a really nice thing to have when you are doing frequent builds and constantly reloading the browser.

 

Internet Explorer 9 (Windows, of course)

For IE, go to Tools > Internet Options > Delete. Unlike Firefox and Chrome, IE deletes everything and I don’t know of a way to tailor the tool to not do that. This is something to be aware of it IE is your primary browser and you need to blow away the cache. All password cookies and anything else you have stored will be deleted. IE does, however, have a nice feature that I use a lot which is the Delete Browser History on Exit option. Again, if you are doing lots of builds (code then reload page to see changes) then consider checking this option and save yourself a bunch of time clicking through menus every time you reload a page.

 

 

Major public web sites miss the mark on using advanced web technology

As a developer I notice things about web sites that the average person wouldn’t think twice about. I also know that the level of technology that’s available to build user interfaces is well beyond what we had in the early 2000’s. And, the ability to build amazing, user-centric interfaces is as easy as ever. But, I’m always perplexed that most major web sites today incorporate very little of this technology in their full functionality web pages.

So, I’ve been doing an informal survey for the past month and my list includes major news sites, airline web sites and retailers. The vast majority of them aren’t much easier to use than they were five years ago with a great a selection of hyperlinks, tabs and full page refreshes. For the fun of it I decided to travel back in time using an internet time machine call the Wayback Machine and compare some of these sites to their predecessors. I challenge you to do the same.

I have a few suspicions as to why this is happening, or not happening as the case may be. First, plug-in based technology such as Adobe Flex and Microsoft Silverlight typically require some amount of time to load the initial payload into the browser. Sometimes you can create highly optimized or lazy-load packages, but it challenging. It’s rare to find one of these apps that load in the sub-second timeframe required in today’s hyper-competitive environment. The general impression is that the longer your page takes to load, the fewer the visitors you will have. So most major websites code is mostly made up of HTML, JavaScript, jQuery and CSS which most browsers have gotten really, really good at parsing extremely fast.

Second, it’s challenging to build Flex and Silverlight websites so that web crawlers can read text-based content. This seems fairly academic. If you can’t effectively index the content of your site, then potential visitors can’t search it via external search engines such as Bing and Google. Period.

These two items alone may explain why visually spectacular interfaces are limited to small portions of most public websites such as video plug-ins, or just specific sections of a much larger website. Where these more advanced interfaces typically reside are in back office applications where functionality trumps the need for millisecond application load times. There are some very cool exceptions for consumer apps such as the end-user experiences shown Mini Cooper’s build your own car online website. Yet, unfortunately for us as consumers, these are few and far between as consumer companies cater to the vast hunger for ever faster page load times.

The good news for advanced web technology in consumer apps is I’m seeing a large opening with mobile deployments. The plug-in technologies now have the capability to allow you to deliver visually enticing experiences across a wide array of devices. And this can be done, for the most part, without the tedium of worrying about all the vast nuances of different browser types and versions. Plus there is a bonus: the application is manually loaded and ready to go on your device minus the on-device load time when you turn on the app. I’m seeing some really innovative uses of the technology in what I call focused solutions, or applications built for a very specific purpose. Unfortunately most are in commercial beta and I can’t link to them. But, you’ll see them soon in an online marketplace right at your fingertips.

References:

Mobile Development with Adobe Flex 4.5

Silverlight for Windows Phone

Flex.org Showcase

Study: Consumers abandon slow loading websites (April 2010)

Let’s make the web faster (Google, May 2010)