Two languages that software developers should be familiar with

The question that I get asked the most these days is “what development languages should I be learning [to stay competitive/excited/motivated/etc] ?” The high-tech industry, and software in particular, is changing at a ridiculously fast pace and that introduces a lot of uncertainty and confusion as well as great opportunity. My answer to this question is unequivocal: I think for the foreseeable future developers should be learning the basic concepts of JavaScript and Python. If you don’t already have these skills then you simply cannot go wrong with this approach. If you’ve been a long-time server-side developer, or you are just getting started with software development then knowing the patterns and practices for JavaScript and Python will serve you well.

Why?

There are three primary reasons and I’ll try to be short and to the point. First, there are at least 2.5 billion internet users world wide, and that number is growing. Their primary method of accessing the web is a browser and JavaScript is the lingua franca of the browser world. JavaScript is a scripting language and it is “the” fundamental building block that allows web pages to “do” things such as submitting your search request to a server, or helping to find your location from your phone. Almost all web pages being served up around the world have JavaScript in them.

Second, the majority of retail, commercial and governmental web applications have a requirement that calls for the use of “server-side” code. This is code, such as Python, that runs on a server and not in the browser application. The most common functionality of server-side code is passing data back and forth between a database and a web application. For example, if a web app asks for a username and password, that user name and password are almost always stored on a server somewhere and not, for security reasons, in the web page and on the client browser where it could be very easily stolen.

Third, you can absolutely apply these client-server patterns and practices to other languages used within the realm of web development. A User Interface designer who has been solely focused on layout and styling via Cascading Style Sheets (CSS) can now understand and appreciate how the underlying JavaScript code can affect the look, feel and behavior of a web page. Python skills can also be used a springboard for more quickly learning other powerful web development platforms such as ruby-on-rails.

The bottom line is if you understand both client development (JavaScript) and server development (Python), then you start to gain considerable value as someone who understands how to help the entire system work together in harmony.

A short note on jQuery.

Many (most?) new web developers learn jQuery first. However, even if you know jQuery you don’t necessarily understand JavaScript. The awesome jQuery libraries provide an interface that hides and simplifies a lot of native JavaScript hoopla, and in general can really make life significantly easier and save time when building modern cross-browser web apps. jQuery is built using JavaScript (and CSS3), but it is not JavaScript. Because of that, when something goes wrong or not as you expected (not if, but when!), and you have a general understanding of how JavaScript works, then you stand a much better chance of figuring out a timely work-around with significantly less stress, frustration and time wasted towards your projects deadline.

The absolute minimum recommended reading list:

JavaScript.

  • Douglas Crockford’s book “JavaScript: the good parts”.
  • Douglas Crockford’s website – He is considered a key brainchild behind the ongoing development and understanding of JavaScript.
  • W3schools – An excellent website for anyone using or learning JavaScript. It has tutorials and online Try It Yourself sample apps.

Python.

State of the Internet Browser 2012 – consumer browser usage will decrease

Over the next two years I see consumer browser usage decreasing and people will increasingly spend more time using native mobile applications. This has a number of interesting implications.

The facts. As a web application developer I pay close attention to browser and browser-related technology usage statistics and trends. Like most people, I judge statistics based on my own experience and the experience of my co-workers, family and peers.  Here are some trends which I’ve been keeping an eye on:

  • Smartphones are rapidly replacing non-smart phones around the world.
  • The number of specialized smartphone applications is continuing to expand.*
  • The number of games for smartphones continues to grow rapidly.**
  • The amount of time people spend on their smartphone, whether it’s playing games or using specialized applications, is increasing.

Also based on my personal experience are the following additional observations that further tilt the balance in favor of native applications:

  • Performance. Native smartphone applications, when built correctly, almost always outperform web applications: I’m referring to actions such as page refresh, general drawing capabilities and to a lesser degree but still a factor is the look-and-feel. This is a general fact of application technology: compiled applications perform faster than interpreted applications. For the most part, once I’ve used a native application, such as Southwest Airlines check-in app, I loathe having to use their web page. It just seems so clunky and slow in comparison.
  • Games. Ah yes, we can’t forget game performance as well as their look-and-feel. Why would I want a mobile browser-based game? What’s the point of building a high-performance, beautiful user interface game in a browser? See my previous bullet’s comment about compiled application performance. Yes, yes, yes I know that HTML 5 is making big strides, but we are talking mobile applications and the technology as it exist today. You can’t tell your customers that they’ll have to wait another year for better game performance, because by then your favorite browser will have such and such HTML 5 functionality figured out. Your competitors would jump right in, tweak their native app and leave you in the dust!

A Corollary. If you generally agree with my bullets above, the perhaps you’ll agree that the corollary is this trend:

  • Consumers are spending less time on desktop and laptop machines “browsing the web” and more time using their smart phones.

In addition to the reasons I already listed, there are many reasons for this. I suspect the top reasons are because it’s so easy to use your smartphone, and it’s right by your side all the time even when you aren’t home. You most likely have seen people with their heads down playing with their smartphones during business meetings, while eating, while standing in line, while watching TV and even during sports events.

What about the Browser Vendors? These trends have interesting implications for browser vendors. They have to be aware of what’s going on. It’s possible that this is one of the many factors behind their massive push to add HTML 5 capabilities in an attempt to stave off what I’m going to call “user erosion”, as consumers spend less time using web browsers.

But, there are some facts to consider related to building applications that run in the browser:

  • Still functionality problems between different browsers. While the latest generation of browsers are the closest they have ever been to parity, in terms of JavaScript and HTML functionality, web developers are still hacking code to make certain things work equally across all browsers. These “hacks” cost extra time and money to code and maintain and the functionality differences between browsers cause customer frustration when things look different or don’t work as expected. This is especially true in large, retail-type consumer apps were you have little control over what browser your customers choose to use.
  • Faster but fast enough? Today’s browsers have the fastest parsers ever, but it’s a fact that they still aren’t as fast as native code, and they never will be. For the geeks reading this, browsers incur a CPU cost associated with parsing and then executing interpreted code. Smart engineers are going to continue to close the gap, but compiled code will always be faster and more powerful than code running in a browser. Period.
  • Memory usage. Browsers tend to be what we call “leaky”. The longer you use one without restarting it the more memory it will consume. I believe this is less of a problem in mobile browsers where windows get closed a lot more frequently than desktop/laptop browsers. However, it’s still an important consider this in mobile phones where more memory usage equals less battery life. Native apps can definitely leak memory, but they are also starting from a smaller initial footprint, and there are much better tools available for finding native app memory leaks. For browser apps, you also have the browser’s memory usage in addition to your application’s memory usage.
  • Security. Security is getting better for web browsers. But…it’s still easier to build a highly secure native app today than it is to build a secure web app. Also, for better or for worse, I suggest that many consumers perceive native apps to be more secure than web apps. Do you want to do your mobile banking over a web app or a native app? And whether a perception is right or wrong sometimes is irrelevant because it always strongly affects people’s behavior.

Concluding Remarks

Consumer-based companies are going to make important strategic choices based on information similar to what I’ve written above. My guess is that the most successful businesses will be the ones that adapt to what their customers want and if your customers are spending less time “on the web” then you should seriously consider adapting. Just to be clear, I’m definitely not saying that browsers are going away. No one has as crystal ball, and new technology is being created all the time. However, the momentum and sheer size of these trends, with hundreds of millions of people buying and using smart phones worldwide, makes it well worth studying its potential impact on your business.

References:

Mobile Apps Put the Web in Their Rear View Mirror
Mobile Apps vs. the Web – Which is Better For Business?
Gartner Report on Smart Phone Sales in 3rd Quarter 2011

* Companies are building specialized apps that essentially replace the need for customers to visit their web site. However, these apps offer much more control and typically provide a more consistent user experience that the web. Southwest Airlines, for example offers three types of mobile apps in addition to a mobile web site: http://www.southwest.com/html/air/products/mobile.html.

** Books and games, respectively have consistently been the top two categories for the most popular apps, for example: http://www.gottabemobile.com/2011/07/06/ipad-app-store-breakdown-top-apps-categories-chart/

A whole new world for app developers – it’s raining cats and browsers!

Holy smokes – how do we support all these different browser types and versions? I’ve been thinking about this for a while, but after attending the AdobeMAX conference this week I got to talk with other developers facing the same issues.  The web app world used to fairly simple: develop for the one or two major browsers running on desktop computers and you were done. We still grumbled a lot and had cross-browser problems that at the time were really annoying. And now we have to deal with not only desktop computers and laptops, but there are also dozens of different mobile browsers and hundreds of new tablet style devices. To make things even more interesting the pace at which new versions of browsers and mobile operating systems are being released is at an unprecedented (and perhaps unsustainable) rate.

What to do, what to do, what to do?  Well if we step back and look at this holistically, there are a few things to consider that can help lay the ground work for building apps in these crazy times. That will help you make better technical business decisions.

For web apps:

  1. What browsers and browser versions your customers are using? How often do you analyze this? If you don’t use a monitoring/analytics tool for this you can always download a free tool to analyze your web server logs. I’ve used Google Analytics in past, but there are plenty of other choices.
  2. What percentages of your visitors are using a particular browser version? Break down your stats again by the browser version. If you still have a large percentage of customers using IE 6 then you might want to consider continuing to support it.
  3. What percentage of your customers use mobile versus non-mobile?
  4. What percentage of customers are from your country? If most of your customers from outside your country then you’ll also have to include localization code.
  5. What type of device is each web site visitor using? Mobile devices can have vastly different screen resolutions and sizes.
  6. Which browsers represent visitors/customers that make you the most money? This is a very important number to know and getting this information can be a bit more involved than the other questions. If you mess up support for these group(s) you’ll really cause yourself business problems.

For native apps:

  1. Have you reviewed your marketplace stats page? One example is the Android Market offers application stats such as platform versions, devices, countries of origin, languages, etc.

What if you haven’t launched your app or website yet? If this is the case, then check out some of public tracking sites for hints. Some good places to start looking are: w3schools and Wikipedia which lets you drill down into different browser versions.  For mobile apps, most IDEs let you set up various layout scenarios that relate to rough categories of dpi (dots per inch). For example, here is an article from Adobe on Flex mobile development and handling different mobile device screen resolutions.

How do I interpret this information? In general, you want to have support for browsers and operating systems that the majority of your customers use. One example is in the retail/consumer industry you absolutely have to support what your customers use. If the vast majority of your customers use Internet Explorer then you will need to support IE or you will lose customers and money. If the vast majority of your visitors use Android, but your biggest spenders use iPhones then take that into your development considerations. If your website or application performs poorly for a small percentage of very high-spending customers then it could cost you your business. If you are lucky enough to have commercial customers consuming an internal-only application, or if you have an ultra-niche audience that only uses a single browser or smart phone operating system then you already have your answer. You build to suit their needs.

Any further advice? Test, test, test and then test some more on real devices and different versions of browsers. Emulators and simulators are not an adequate substitute for real-world checking. If you are a small shop, enlist family, friends and neighbors to test. Have them check out our app or website for a few minutes while they relax in the evening. You can also buy after market handsets on eBay and other places, just be careful that if a deal is too good to be true you probably shouldn’t purchase it. The reality is there are so many devices nowadays that you simply can’t test them all. So, pay special attention to feedback in your market place listing or on your forums.

Browser updates…too many too fast?

We’ve finally reached the point where the number of browser updates is out of control. There’s an all-out war between the various browser companies to see who can push out the most updates and improvements in the shortest period of time.

All these updates are causing a ripple effect on everything else that is dependent on the browser; for example, plug-in vendors, IT support staff, computer and smart phone vendors, application developers, any company that has a website, and your average consumer.  I’m guessing that on average the pace of updates is starting to outpacing business and consumer’s ability to keep up…and it seems to be accelerating.

Yes, I wholeheartedly agree we benefit from the advancements. No argument there. However that is balanced by reality. And, reality is architecting our products to support the latest and greatest. There is also the fact that most of us also have to maintain support for older versions of browsers and there is a cost associated with backward compatibility. And, there is a cost to upgrading. Not everyone is able to update all the time.

As a case in point, let’s take a look at Firefox since it’s fairly easy to find an archive of their older releases. We are only seven months into the year 2011 and Firefox has had two major releases: Firefox 4 and Firefox 5. However, if you include releases candidates, betas and updates to Firefox 3.x the total number climbs to around twenty-two releases so far this year. Yikes!

Sure, most consumers only saw the two major updates: Firefox 4 and Firefox 5. But, they also experience a plethora of plug-in updates. For example, Flash Player has had eight updates so far this year. Silverlight has had three general distribution releases this year and one beta release of Silverlight 5. And, I wasn’t counting but it seems like I’ve had a bunch of Adobe Reader updates in the last few months.

My concern is that the speed of browser innovation is starting to cause businesses and consumers to get fatigued. It begs the question: how long can the ecosystem of browser consumers maintain this pace? Or, at what point do people just start jumping off the bandwagon and simply starting skipping releases? Do most consumers really care if their browser is now 100ms faster in parsing JavaScript? How many new ways can we create tabs?

In summary, I think browser vendors should slow down and become better custodians of their systems. How about also focusing on innovation in security, memory leaks and best practices documentation and vendor-provided validation engines? Indeed, they have sparked tremendous innovation across the entire world wide web since Mosaic released in 1993. But, depending on how many websites you visit, you can still experience slow web pages, in-consistent cross-browser support and browser crashes. I know there are no easy answers because competition breeds creativity. Perhaps we’ll all go back to just looking at basic text, pictures and videos in the future. And, there will just be specific widgets with very focused functionality for other things.

References:

Flash Player Archive

Firefox Archive

Silverlight Release History

Improving Browser performance and stability – will web workers help?

The single-threaded nature of JavaScript is an old tradition that needs to go away. It was great in the wild-west, internet days of the 20th century. But, today we have more complex needs that are being driven by the advancements that are happening around good old JavaScript as we know it, such as…on-going advancements in HTML 5.  

The reason I bring this up is because I’ve been watching the discussion on Web Workers as it has evolved.  It’s a brave attempt to bring a standard for implementing some sanity on this ancient notion of single threading. Now, I do want to say that this post isn’t about debating the merits of web workers, per se. It’s about giving developers better tools on which to build web applications for end users. I’ll be the first to agree that many developers (but not all!), for a variety of reasons, build apps like factories, but without many quality checks.

One argument the pro-single threaded parties claim is that doing away with single-threading will make things even more complicated for the companies that develop browsers and the developers that build apps on them. And, in effect, you’d be giving them (web app developers) free license to create even more terribly built web pages that crash browsers.  For brevity sake, I’m only picking this one out of many possible arguments, as the one that comes up most often in discussions.

I also don’t ever recall seeing a browser vendor themselves saying something like this publicly, but it’s possible.  This is a very weak argument that won’t stand the test of time. Sure, as we build more complex apps then there will be more of both good and bad apps. That’s just the way things work. There’s no way we would ever have a single authority that reviews all web apps before they are published. Perhaps, similar to what Apple does with iPhone apps. Not only would it be impractical, but it certainly seems like it goes against the spirit of the internet and WWW.

I fall into the camp of evolving the tools to better to fit the ever-changing and growing needs of the end users. End users don’t understand the limitations of the browser technology.  They don’t need to and shouldn’t be expected to. All they know is that they want to see ever more visually stunning applications that run well and don’t crash all the time.

Developer tools and technology are much, much more advanced now than when the venerable Mosaic Web Browser hit the scene back in 1993. As an example, all eyes are on HTML 5 (more on that at a later date), and certainly we have the well-known browser plug-ins: Flash and Silverlight, and each has their own development kits. These technologies enable the building of some of the most eye-catching websites, and they really opened people’s eyes on what the web experience should be more like.

Now, I am eyes-wide-open about this. There are some well-documented, but not well understood existing limitations related to the web surfing/development experience as I blogged about here. But, merely saying things should not change because it will become too complicated isn’t a good enough reason to, well…not change.  There are lots of smart people out there that love solving these types of problems.

So, I have a few suggestions of my own for the browser vendors and others to debate and work on. I think web workers are huge step in the right direction. But I also think there’s some other more strategic things that browser vendors could be doing that I think would also help. To me these are just as important as evolving the web standards, perhaps even more so. This is about browser vendors officially providing guidelines for us on how to do our job better:

  • Best Practices Document. All the major vendors should publish web development best practices for HTML and JavaScript development. And, I’m not talking about the W3C standard. That is what’s expect, but not actually what’s implemented. For example, I did a quick search of “web development best practices” using Google and Bing and the very first result I found was a short, not-really-so-helpful article on the Apple web site that was written in 2008!
  • Online HTML/JavaScript Validation engine(s). Each browser vendor should publish their own online HTML/JavaScript validation engine. Or better yet would be if someone builds one site that checks all major browsers in one shot and provides actionable feedback. I’m aware of other types of validators such as this one by W3C for HTML and the like. But, in general right now it’s just a hodgepodge of 3-rd party tools and guesswork as to whether a web app is working right. And, if you are like me and running the web debugger all the time, you’d know how many broken web pages there really are.

References:

The browser as an operating system

Having a basic understanding of how our web applications affect browser performance is the key to determining whether the apps you build will be great, and which apps will be a miserable experience for your users. You can have the worlds’ best looking app with the nicest user-interface ever, but if it runs horribly on most visitors machines or phones then you’ve done your end users a massive disservice.

I contend that the browser as a web application programming environment should be treated as its own operating system with its own well defined dependencies. If you have a basic understanding of how these dependencies work, you’ll be able to build better, more stable, faster applications.

We are constrained in what we can build because browsers provide a finite environment in which to play in. To make things even more fun and challenging, in just the last five years we have gained access to some very powerful tools to build even more complex applications, such as Microsoft’s Silverlight API, and Adobe’s Flex/ActionScript API. Now we can build applications with very rich graphics in days or weeks that would have taken many months or even years before these tools became available. And, web applications until only recently gained the ability to semi-directly interact with the operating system to perform operations such as save or retrieve files from local hard drives. In the ‘dark ages’ we had to bounce files off a proxy server before being able to download them to the local machine. How we interact with the local machine is ultimately controlled by what the browser will allow.

The browser sandbox

Browsers provide us with a well-defined sandbox in which our apps can run, and from a developer’s perspective it includes the following:

  • A JavaScript engine
  • An HTML parser
  • User Interface rendering engine
  • Add-ins/Plug-ins (e.g. Flash Player, Silverlight, etc)
  • Cache space (includes cookies and local stores like in FlashPlayer)
  • Access to the internet
  • Access to local resources

It’s also important note, and if you’ve been building web apps for a while now you’ll know, that browser vendors don’t implement the various proposed standards exactly the same. For example, Internet Explorer may display a certain CSS tag different. Here’s an interesting comparison chart.

What about hardware?

I’m also not saying we don’t need to pay attention to the underlying operating system. In fact, we absolutely do need to pay attention to the following. However, we interact with them only indirectly, and because of that we tend to forget just how important they are:

  • CPU
  • Memory
  • Graphics card
  • Internet connection

Mobile devices are a great example. Mobile devices are getting more powerful all the time, but when they try and chug through a fully decked out web page, it takes them longer than a typical desktop or laptop. I’ve had developers tell me an app that’s running slow on everyone else’s machine was running just fine on theirs. What the developer forgot was he had the latest, greatest, hottest laptop out there with 8 GB’s of memory and an excellent internet connection! By way of example, I posted a screenshot at the bottom of this post of another website that even my quad-core laptop used between 50 – 80% of its resources to  load the page.

Simple Tests

Here are some simple tests to tell if you web app is a good one that will meet the needs of your end users:

  1. How much CPU does it consume? Test it on a moderately configured machine, typical of what your users might have.
  2. How much memory does it use? And, how much memory over time? Browsers can be notoriously leaky, but your program may be contributing to it.
  3. Is everything in the correct locations in the user interface at various common browser sizes? (e.g. 1024×768, 1280×800, etc.)
  4. Does it cause temporary slowdowns or lockups?
  5. Does it crash the browser or browser panel?
  6. Does key functionality and layout work consistently across the major browsers?
  7. Does your app work consistently across all the devices you wish to support, including mobile?

A Few References

Here are a few articles and websites for that are handy to have as references:

http://www.w3.org/  

https://wiki.mozilla.org/Main_Page

http://www.w3.org/TR/CSS2/cover.html#minitoc

http://taligarsiel.com/Projects/howbrowserswork1.htm

http://ejohn.org/blog/how-javascript-timers-work/

http://blog.chromium.org/2008/10/new-approach-to-browser-security-google.html

http://hacks.mozilla.org/2010/05/firefox-4-the-html5-parser-inline-svg-speed-and-more/

CPU Usage loading web site with quad-core laptop
Loading an unnamed website using a quad-core laptop