Wednesday, September 29, 2010

Why You (As A Developer) Should Question Technical Decisions

Many times throughout my career I've seen software developers take a lot of pain from their leaders. Throughout history, "leaders" have always been personified as an unstoppable wave in the ocean that everyone rides along with. One way--one direction--one man/woman's goal. A "leader" is the entity which has the final word. To some degree, I can't picture myself blindly following a leader without questioning his or her decisions--There has to be a reason for everything. Furthermore, you (as the developer on a team) should be asking "why" certain technical decisions are being made in specific situations. Positively, you may even end up saving a high degree of effort for your team just by having your leader realize how much of a bad mistake his or her decision was.

Decisions from a technical standpoint should always be strategic. One should always aiming for a goal to achieve and one should always have to look for key people, abilities, or events to transpire in order to make a strong, positive decision that best suits either the business or their team. Incidentally, being in the technical realm has its pains and pitfalls. In software development, a team captain has to take into account his or her experience, skill of his/her teammates, and the level of dedication they are willing to invest in building something heroic.

I live my life with this quote on my mind: "the only reason why you're unhappy is because you made yourself unhappy". There shouldn't be anyone to blame but yourself. This quote is generic enough to apply to anything: relationships, career, or school. Incidentally, from a career standpoint, I see many developers just tough it out without speaking up or challenging certain decisions from a superior officer. Ultimately: if you don't speak up and you have a feeling that a particular decision from your superior could inflict significant damage to your process, you'll end up shooting yourself in the foot.

It is always better to over-communicate than under-communicate. [Great] Technical leaders will listen to and acknowledge your opinion -- It's just a matter of voicing it out and backing up the facts. Be true to yourself. Be a developer. State your facts. "They" will listen. If they don't--then it is your time to move on.

Good luck and have fun,
Jaime Bueza

Jaime Bueza is a software developer in Vancouver, British Columbia, Canada. He has developed web applications for Nintendo, Electronic Arts, Ritchie Brothers, Kiwi Collections, and Cox Communications. When he's not developing useful software that constantly evolves with business requirements, he's creating tutorial videos for aspiring front-end developers.

Wednesday, September 22, 2010

The State of 4N/WebSockets and NodeJS Hosting

4N + WebSockets

For anyone that wants to develop on a 4N (Not Windows, nginx, NodeJS, NoSQL) stack, I've come up with a list of gotchas that you will eventually run into when developing your applications.

nginx does not support web sockets

If you're building an application that will require WebSockts for full duplex bi-directional communication between the client and the server, you're sore out of luck as nginx doesn't support the WebSockets protocol (ws:// for normal, wss:// for secure) yet as of September 2010. Based on the discussions on the NodeJS Google Group, some have gone out to say that NodeJS shouldn't be out in the front. For clarification, nginx is a battle-tested Russian event driven web server that powers high traffic sites like Wordpress, Hulu, Github, and SourceForge. By using a single master process, it is able to easily handle 10s of thousands requests by delegating to worker processes. Each worker handles multiple requests in an event-driven or asynchronous manner using special functionality from the Linux kernel (epoll/select/poll).

Furthermore, NodeJS is also event driven and is powered by the non-blocking beating drum of an extremely fast event loop. As many of you know, NodeJS is a platform on top of Google's V8 engine that allows developers to write server side Javascript and build powerful, scalable networking applications.

npm versions aren't excalibur swords--Learn how to hack together a solution

As I understand it, a lot of developers trying out NodeJS are Rails enthusiasts that are open-eyed, wanting to explore the vast world of server-side Javascript. I could be completely wrong here about the demographic but one thing I do know is that "npm install ..." is not as reliable as "gem install ..."

For example, while trying to build an MMO pokemon card game on the iPad for NodeJS, I couldn't figure out why Sockets.IO wasn't working until I realized that the repository on github had fixes that npm's repository didn't. The quick fix? Hack it up! I merged the code from 'master' on github into /usr/local/lib/node/.npm/ and it was smooth sailing from there. I notified the author of the library, he was gracious enough to inform me that it's already fixed in the upcoming code push to npm. Be warned--you will need to figure out some of these little things!

You may need to migrate to MongoDB, CouchDB, or MySQL (async driver)

If you're thinking about a rewrite of your application, you probably understand that your application will probably be the easiest part of the migration over to the NodeJS platform (it's just Javascript). The most difficult part in my opinion would be if you're migrating your data from an existing relational database. I would definitely not suggest that at all--What I do suggest is to migrate some of your frequently hit pieces of data (news, forum posts, etc) and have those migrated over...Or just not migrate your data if you're on MySQL/Postgres as there are an asynchronous drivers. If you're coming from Oracle, good luck.

NodeJS Hosting

- Gandi
- Joyent
- Heroku

Special Links of NodeJS Adoption

- THQ Selects Joyent To Provide Social Networking Infrastructure To Enable Rapid Prototyping And Deployment of Online Games
- Palm brings improved multi-tasking and NodeJS to webOS 2.0
- Plurk: Instant conversations using Comet
- Heroku adds NodeJS support

Tuesday, September 21, 2010

Why the web evolves so slowly in comparison to other industries

In technology, in one way or another, you will run into specific individuals that just don't care about their jobs. I've had to chance to see them infest my beloved discipline: software development. Time and time again, I see them dodge bullets from upper management by talking their way out of specific things that are too challenging or too lazy to take ownership of.

I work in web development: my role is to to develop cutting edge user interfaces, produce engaging online experiences, build intuitive software, and be a passionate technology leader. I've come to realize now after working in the agency business for so long that there are too many people that just don't care about the software they are actually building for their clients.

I've talked with several other developers from different companies and it appears to be the same. Consequently, I believe that the reason why we're forced to code like a drone is because a majority of other disciplines in the business don't understand what we really do as software developers. Furthermore, I believe that other disciplines actually think we build door knobs (components), that go into doors (applications), and the doors go to a house (technology platform). It's this imagery that completely dispels the drive and passion of building great software. In actuality, it is not that shocking. This happens time and time again, for example, in the gaming industry where it is very competitive--being on an old platform (SNES) could potentially cut off any chances of success for future games.

Back in 1996, Squaresoft ended their long, tumultuous relationship with Nintendo because Nintendo decided to go with cartridges, which didn't have sufficient memory to really give an engaging experience compared to FMVs (full motion videos)--so they ditched Nintendo for Sony PlayStation. Incidentally, this story was about how a game development company made the leap of faith because their old platform and archaic ways didn't provide them the features and advantage over their competitors they needed to deliver a solid product (Final Fantasy VII).

After reading this story--I had an epiphany. In game development, there is always this constant drive to create more engaging experiences. As technology evolves, so must your delivery whether it is a service or a line products. Many front-end developers tend to blame designers for being too heroic when time comes to deliver a working web application but I'm starting to see a different perspective. I don't necessarily think it is the designer's fault when they're told to design something cutting edge. They are just doing their job, and so are the developers. So, who do we blame for the lack of evolution on the web? Who do we blame when our competitors create better experiences at a lower cost and a faster delivery time? Who do we blame when the expectations of what you were building between the client and the delivery team are so divorced?

Why is it that the game development industry has had asynchronous non-blocking I/O 20 years before the web? How is it that games have been real-time for 20 years and just now there's a burst of web applications that are real time using web sockets?

I blame it on all the laziness and tunnel-vision technology leaders of our current day in web development. It's not the client, it is not the designers, it is not the lowly programmers that are anxious to build great things. It's the guys in charge of the technical decisions that are so lazy and closed-minded about learning new technologies that can help alleviate our development pains and speed up our delivery. Instead of adapting to how competitors are creating more engaging online experiences or evolving new techniques on top of more agile technology platforms, we leave it up to the guys in their garages to develop something brilliant and follow in their steps. The worst offender isn't the part where we aren't being a true leader--it is the thought of trying to copy a more technologically advanced competitor but on a slow, painful, and frustrating stack of technologies.

It is clear that everyone wants to evolve and build intuitive, useful web applications but how can you innovate when your technical leaders put a stranglehold on your creativity because of their negativity towards learning something new and building on top of more efficient platforms. Ultimately, web development is so full of arthritis. Douglas Crockford said that. In my opinion, techology should always be evolving to help the business reach its goals--Laziness should never be tolerated. If you want to just sit on your position, why don't you just become a doctor?

Douglas Crockford also said, "that's why software development is so slow, you basically have to wait for a generation to die off before you can get critical mass on the next good idea."

Good luck and have fun,
Jaime Bueza

Jaime Bueza is a software developer in Vancouver, British Columbia, Canada. He has developed web applications for Nintendo, Electronic Arts, Ritchie Brothers, Kiwi Collections, and Cox Communications. When he's not developing useful software that constantly evolves with business requirements, he's creating tutorial videos for aspiring front-end developers.

Thursday, September 16, 2010

NodeJS Hack Session: MMO Pokemon with NodeJS/WebSockets

The primary focus of this proof of concept is to determine how easy it is to build real-time web applications for all iPad, iPhone, droids, Safari, Chrome users on top of NodeJS (non-blocking event driven server side Javascript platform). The proof of concept was built within 6-8 hours including the following:

  • Uses Express framework for server side Javascript development (inspired by Rails / Sinatra / Django)
    • Uses EJS for templating language (much like Django templates and symfony) -- allows partials and passing parameters into partials like symfony PHP
  • Real-time chat using WebSockets
  • Arena Queueing System for real-time competitive matchplay using WebSockets
  • HTML5 Audio
  • CSS3 transitions for all hand cards, tappable cards, transparent panels, rounded corners, drop shadows
  • Fallbacks for Firefox, IE
    • Firefox/IE will fall back to Flash socket
    • IE will fall back to XHR long poll if the user doesn't have Flash installed
  • NoSQL CouchDB for fetching users and soon cards, achievements, friend associations

Technologies Used

  • NodeJS (Server side Javascript platform)
  • WebSockets for real-time
  • CouchDB for users, *achievements, *friend associations, *cards

NodeJS Packages Used

  • Socket.IO for networking (npm install
  • Express for MVC application development (npm install express)
  • CouchDB (npm install couchdb)

Lessons Learned and Thoughts

I didn't use nginx as a web server to front NodeJS because nginx can't proxy the WebSockets traffic just yet. Using Apache is pointless because that would defeat the purpose of using a non-blocking, non-threaded technology stack.

HTML5 Audio inconsistencies

Chrome can't play mp3s or wavs, only ogg audio files.
Safari can't play ogg files, but can play wav or mp3 files
Firefox can't play mp3 files, but can play ogg and wav files

Use an HTML 5 audio library for abstraction

+ Quick turnaround for implementing both backend (using couchdb/mongodb/async mysql) and websockets) and frontend functionality (using express mvc framework)
+ Easy and fast install, compile node, and it'll work
+ Simple and convenient package manager like gem for Ruby/Rails called "npm" (Node Package Manager)

- Must remember to use asynchronous programming (callbacks, callbacks, callbacks, do not block) when doing 

  • database access
  • networking
  • disk i/o

For more information on how fast NodeJS is coming along:

This is not a full game; as the intent was to accomplish the following goals

  1. Establish networking code for chat application within the game using WebSockets with fallbacks
  2. Establish networking code for Arena Queue System within game for competitive matchplay
  3. Use CSS3 to do all the rounded corners, animations, transparencies, and drop shadows
  4. Use HTML5 audio for hilarious interface sounds

Hopefully, this proof of concept will inspire other developers who want to shape the open Web into a more engaging experience (easy to build real-time web applications) on top of a broad subset of consumer technologies (iPad, iPhone, Droids, PC, Mac, etc).

If you're in the Vancouver area, shoot me an email at and I can give you a demo. Yes, it does look very sexy on the iPad with beautiful CSS3 animations.  :) Again, I'd like to reiterate this: my goal isn't to earn any money off this but to inspire others to join this initiative to create better web experiences for users across all technologies. Below are the source to the Pokemon card game on NodeJS with real-time chat, as well as, the real-time arena queue system for competitive match play.

github nodejs pokemon source (contact me if you can't set this up or if you have problems)

github nodejs arena queue system source

Good luck and have fun,
Jaime Bueza

Jaime Bueza is a software developer in Vancouver, British Columbia, Canada. He has developed web applications for Nintendo, Electronic Arts, Ritchie Brothers, Kiwi Collections, and Cox Communications. When he's not developing useful software that constantly evolves with business requirements, he's creating tutorial videos for aspiring front-end developers.

Saturday, September 4, 2010

SimpleYUI, thoughts on NodeJS development

I've been a huge fan of YUI for a few years, even back when they were named YUI-Ext. Recently, they rolled out SimpleYUI. When asking other web developers why they don't like YUI, they say, "their package system is too complicated" or "you can't just start accessing the DOM by including the library easily". Incidentally, back in the day, it was far worse--YUI didn't even have a selector engine until they ported someone else's into YAHOO.util.Selector (remember this guy?). This design flaw in the library is probably what cost them in terms of people adopting YUI. As a developer, you want a consistent way of doing the most common things (dom access).

Back in the day, you would have to call YAHOO.util.Dom.get(element or string for ID of element). This was terrible because they introduced YAHOO.util.Selector.query(cssSelector) right after and soon developers were choosing between fetching DOM elements by selector or by ID. MooTools has the same problem but separates it in $() versus $$(). How about jQuery? $(cssSelector or element). Everything is passed as a css selector or a DOM element and it stays consistent through $(). I am certainly glad that they have aligned towards having a consistent API for fetching DOM elements. At my last job, some projects were using the old YAHOO.util.Dom.get('') and some projects had YAHOO.util.Selector.query(). With SimpleYUI, you're able to get the simplicity of how you're using jQuery but you also get the power of swapping back to using YUI's powerful packages.

NodeJS: why you should jumping in to help bring NodeJS into the production-ready realm

When people ask me about whether we should adopt NodeJS into a production-level deploy, I have to state the clear and obvious answer: no, it isn't proven yet and we would burn too much budget trying to seek the answers that haven't been answered yet.

Every single time I say that exact sentence a part of me gets disheartened. Why? Because saying "no" to it is the most responsible thing to say if you're a technology leader. What happens when your project goes overbudget because you didn't have the technical skill set to handle this new technology? You get blamed for trying to push it into production.

I understand that it is new and unproven; so the question is: what can I do to ensure that NodeJS can be fully integrated into production deployments (big builds or small builds). I have been asking myself this for quite some time now and I find it way too easy for someone to disapprove of NodeJS because of how unproven it is; so the best thing to do for the NodeJS community would be the following:

1) More Node Knockout tournaments (I've showed this off to my coworkers and they are highly impressed with the apps developed)
2) More technical writers and video producers (viral video for how to setup your website in 5 mins on Node)
3) More web hosting solutions than just Joyent/Heroku/Linode, let people start creating sites on top of a solid backend!
4) More enterprise solutions, such as, application server replacements like JBoss

The most exciting part about NodeJS isn't the fact that we can move towards fast real-time web applications, or the fact that you can scale out to more users with less hardware--it's the fact that it has never been done before. Furthermore, having Javascript run on the front-end and the back-end is the most exciting part about NodeJS development; if you find yourself without a challenge, try NodeJS out!

Incidentally, after watching Douglas Crockford's Loopage talk, he said, "That's why software development is so slow, you basically have to wait for a generation to die off before you can get critical mass on the next great idea" and "We're about to take maybe the most important step we've ever taken in terms of the technology fo the web and Javascript is leading the way. Who would ever imagine that? Javascript is the technology leader, it's not just the thing we tolerate." Such an inspiring speech by Douglas Crockford. That said, my passion for advancing software development has been lit once again. Good stuff will be coming out in the near future! Believe it!

Good luck and have fun,
Jaime Bueza

Jaime Bueza is a software developer in Vancouver, British Columbia, Canada. He has developed web applications for Nintendo, Electronic Arts, Ritchie Brothers, Kiwi Collections, and Cox Communications. When he's not developing useful software that constantly evolves with business requirements, he's creating tutorial videos for aspiring front-end developers.