Important Phone Features

My AT&T contract expires in August and am considering what phone I should get. The progress of the iPhone is practically stagnating compared to how much Android is making progress. Not to mention the incredibly sucky service AT&T provides and how much cash you have to shell out for it. So I’m considering getting away from the iPhone. Anyways, for my next phone, I’m looking for:

  • Good reception
  • 4G
  • Act as a hotspot
  • Android or iPhone OS
  • The phone must have headphones that function as a remote to control songs (pause/play)
  • Physical Keyboard

That’s really it. Also, I’m looking for a platform that provides the following apps:

  • PDF reader
  • Chm reader
  • Weather
  • Google maps
  • Lirr schedule + map
  • Subway map
  • Rss reader
  • Twitter app
  • Fandango
  • Chat
  • YouTube
  • Radio podcasts
  • Wikipedia
  • Yelp
  • Imdb
  • Taxi
  • Soundhound
  • Facebook
  • Linkedin
  • Amazon

Which both the Android and iPhone OS provide. I would’ve gotten the EVO 4G except it didn’t have a physical keyboard. We’ll see what comes out this year.

Resetting Image in Word

Here’s a silly little tip I’ve just discovered. Often when you take a screenshot and paste it into Word, it’ll come up blurry. To reset it so that the image is the same size and also looks like the actual screenshot:

1. Right click on the image and choose Format Picture…

2. Click on “Picture” then “Reset Picture”…

That’s it!

Forcing Firefox to Show Image After Disabling Showing Images

I guess I didn’t know that if you disable Firefox to show images, images still show if you use a Data URI string, base-64 encoded. For example, let’s take a look at the following HTML:

<html>
<head>
<title>Test</title>
</head>
<body>
<img src="data:image/gif;base64,R0lGODlhuwAsAPcAAAAAAIAAAACAAICAAAAAgIAAgACAgMHAv8DcwKrK7wAAAAAA/wRAfQJHjQRR
owqRBgD/AAI2bRNnvw5j0BAQEBJWnBZoJhSGDhwdHBk6XhdLfiFUHhpcmB9xLxqmDidDXyVkHSMl
JidtqSty0it6JCyVFiyiFzEyMi6wGDk5OUFQazdowDttmjCAPzeXHkJCQUdITENyzUCIN0x9qEuE
tkiN3keTUk6Zwk2rNEq1J1BRUlVcUlJbblV1s1ehYFxcXV6HqFiAs2KRsljBLl1bWWVlZGefzl6a
4mtrinGX3WiycXC/D2jNNm9vbnaQyXaixYBhN3l8eXp6mYOOlXeq5oW9PX+41XjAgH7aEH3iSoQA
AIFxWISEAIaw54S8N4bMjI+EXYmIho+TqYulvIen2IytvZSx2pG764jsEZNoK5KQjZSYm5SUnJWU
1pDLPpvmOpbamppPM5mTfZqZkpycmZ2c15qylaHOY6abVqGfvKS7zaHkpKwIDa8OE66uqqqq566y
xq3Hg63G2a3ur7g1ObRKGbVWHLZ+MbR9UbS2uLTL37TafrkRFbopKbpmI726tLq9vbq5zcLB8L7R
m7r6vcIPGsUYIsk3IsY8QshKMcVNUcdqasZ7SsKAMryQXMiUPMa9zsrMzMDN1sulR86qpcy4pc7D
Ss3HwNDMxdIVJNdEJtZYPdhrStl0edSnfNHAttTQVtXVatPQyNbX2uMWC9saK9w0E+FMWNpZY91s
st1+LNx7VtyzcdqzydzXWeUBAeAnOOtiNudkxuhryOxzC+iBieaMzOmOHN+MWuamque0tuTl5fAP
CewuwOlFFe5IyeRWBPNY0Ox90PClheyqHvDGg+nQtO/v7/cAAPhSGPdSGP9sGviPavCVN+yjY/qz
IfWxWPK9bfXAofPELfLLSPTUMvPR0/neQfrcZPrmdPnqyfjsVvf3APv29v5iGv+vG/zphPvmpvv2
ev3znfv2tv742/7876KjpICAf/8AAAD/AP//AAAA//8A/wD///3+/SH5BAAAAAAALAAAAAC7ACwA
hwAAAIAAAACAAICAAAAAgIAAgACAgMHAv8DcwKrK7wAAAAAA/wRAfQJHjQRRowqRBgD/AAI2bRNn
vw5j0BAQEBJWnBZoJhSGDhwdHBk6XhdLfiFUHhpcmB9xLxqmDidDXyVkHSMlJidtqSty0it6JCyV
FiyiFzEyMi6wGDk5OUFQazdowDttmjCAPzeXHkJCQUdITENyzUCIN0x9qEuEtkiN3keTUk6Zwk2r
NEq1J1BRUlVcUlJbblV1s1ehYFxcXV6HqFiAs2KRsljBLl1bWWVlZGefzl6a4mtrinGX3WiycXC/
D2jNNm9vbnaQyXaixYBhN3l8eXp6mYOOlXeq5oW9PX+41XjAgH7aEH3iSoQAAIFxWISEAIaw54S8
N4bMjI+EXYmIho+TqYulvIen2IytvZSx2pG764jsEZNoK5KQjZSYm5SUnJWU1pDLPpvmOpbamppP
M5mTfZqZkpycmZ2c15qylaHOY6abVqGfvKS7zaHkpKwIDa8OE66uqqqq566yxq3Hg63G2a3ur7g1
ObRKGbVWHLZ+MbR9UbS2uLTL37TafrkRFbopKbpmI726tLq9vbq5zcLB8L7Rm7r6vcIPGsUYIsk3
IsY8QshKMcVNUcdqasZ7SsKAMryQXMiUPMa9zsrMzMDN1sulR86qpcy4pc7DSs3HwNDMxdIVJNdE
JtZYPdhrStl0edSnfNHAttTQVtXVatPQyNbX2uMWC9saK9w0E+FMWNpZY91sst1+LNx7Vtyzcdqz
ydzXWeUBAeAnOOtiNudkxuhryOxzC+iBieaMzOmOHN+MWuamque0tuTl5fAPCewuwOlFFe5IyeRW
BPNY0Ox90PClheyqHvDGg+nQtO/v7/cAAPhSGPdSGP9sGviPavCVN+yjY/qzIfWxWPK9bfXAofPE
LfLLSPTUMvPR0/neQfrcZPrmdPnqyfjsVvf3APv29v5iGv+vG/zphPvmpvv2ev3znfv2tv742/78
76KjpICAf/8AAAD/AP//AAAA//8A/wD///3+/Qj+APEJHEiwoMGDCBMqXMiwocOHECNKhCirosWL
GDNq3Mixo8ePIEOKHEmypMmTKFOqXMmypcuXMGPKnEmzps2bOHPq3Mmzp81TYV6cCBHiBIwwp1Bp
RIUqlFOlIQFIBYDv46l79x5B3WjDz6mKfg5sVUlpkNk9e+Co/cL2ipK3PlP6SRGiCB2B94qESOEH
Y6hZy5ZdGzwLUl+PVAdS7YgvxQsdO/CJ3Viiqx0ZD2xo1XgKV63PtVKJtkSatKavssoOQrv2y5XX
b+HGNekHgw5IoURNmSIqFCQdIQ5XnDV48LJ2/67JOtAx8cXFGMUggZECho4fRZqEeaTRTg4XMkz+
oPBwYbtGXMWQI//Hvv2/dq00VVyN9l/rK/9i+/Axu+SpEzo4JYoRQQRhhBF6hKLDCajJQlxxgzl1
zTJjYbSYVFVBZ9E9YoSSgg5NgIhPGHY1aJEPTAwxRA4ooHDBDsJdVMt6wLhnzi2p4HJNO5ZUhNYe
7Fng2mvsKbEff/2NhA8Gj+QmiAQVVCCBBAk+EkJVw0FonHGhcERVYl9iaZEU7Vj3TyhNFLGGGk1E
wR1GNmTBRIo5XLBBEfdUKEst/wDwD59S/dMKLuZY0so/jPioVpCvuVXkkSVJpdFUFU0lqUWWVgoA
pptulEIRApIhwQxjCKJIb7IUkcJFWmrZ5Ub+YcoS60VSRJICPqHQEaIf96QZY0U4ZCGnCxvAUAR3
XmHEJ3s1uvdPMpYkg6iicDDqlhLs7WeDDSR1mpG33lb6HKayghsuRiHM4ZQiRkhAgxBkCILqHCGw
qggZZjxY3KuT4nOhv2JWhIQUL2gXhRQB+tEEEb/KIsMQwoKgw5sD32Pisu1Zeg0mjSjzTx8VwcEW
o7Fluy23Ip0rq7gst9zypjCvrLJFGGS3hhgsNMAAAw1MQccaaWJgUShJdDGLIl2QASG/3wIsq9MX
3aOCDjAM/EMiByQSBsMahUFCDjhwpwISeUiBjx9bpeLs2q308XHIIq/d3sktjJQpy+COq7f+zHzP
XBEGdPgBiB4iOAAEEE/oAYlhcwhdUQxdJLDMLAkkYQaXXjo9q0WnqKFDETDAsEYoftDxQhF+1CGL
JBnZQUIUOujAgxhSAGJeRWq7FygmhGpCyD98VMTWF3KzR7fd5O6dt4WUruy837KEQEduijggwhil
9hbKKfRWBMgIVJyhiCBndBFDYMswbaHmUF90ABFR+OFUWGEcdUAds6iO0SxhpLCG7B3yQxQsVhFL
rMdZ5sBEHzBhjnaATBZDCpIFtGW8FlgQeS5zngZVdq5OeZAjn3KKIBqgM549wSmnUFVFnDCBGlBh
FqLoQg1GAJj0Ze5fGrqIGsJwDz/Qr2D+B8hDGyShP4voIBGQgMEPpnAAwyxMDajRRDGK5574VAQ2
QTLStip4wZQl74N4m5SFxrgRfISgSXqgAQ1YwIIgJCgUj8AAllg4giPAkAozrKGJ1ofDgHEuCmFo
wsJOdwBJtEEWf8iIIHNFB0gkIgowIMLZUHMKTViiEpVgBCOixR5M8KEPp7mio/4xwZOxx4It6EC3
LrVBVs6sg2TkDIAEJAqsOKVLC0INICYwghqIQhBHqMH5JrfH57AvhxZBxSPUUAQiFEEyefgDIAyJ
EVT4QQf4uEcoPAc/OjyimBchhSU24UBSYKRkpPSBKf9hwQ6oMkkjqY0ODuCU3TgliRj+iFEMeikK
PcyQDIGZBTgrJSZkVvMAPvTDI+ogTSJypgn4+N8OouDNgWKkFI0gRB9Kcc63+MACJ9uWBdrpTniS
xA9DsUsUKFoEDJzgVwdYwQjG8IQVBIGes5hFWDJHUJLMgpob8QM+sqOGb47kFITgwys6qq2QotKd
FrCASUkClBeEAAMYCMELkLKUMfQgCD18xAFOQVbEYIgkf/hDJDqiTIVatCOn2ERGjKROuqEylVGV
6lT3yte++vWvgA3sSAbwkZw+ZSal4OhHPPGJUZjisbCIhWQl64tYpMRSmM2sYD8ygH1A7x41NGwo
EOoSXsSjHvWoBnM4gghqjKMc6Ej+BzzmQY/a0mMe8liHZU8CgAD49rcAEIBwhwu9zcqiswPoU1BB
mwg/JOK5iaBDVFi5Px6gg7bxgMchOtKJ16ZDHai1hz3aUw/c+gIlvf2tb4M7XOEWV7DI/UdnZ+bD
ZdAlFEiExBpAAoAD+HdmTzjEOuBBj+x+Iwoc6e454jENdagiGxDOhjvEQQ95nNcivTCGNKRhjF7I
IhBVCHEgmJfZEhtXI/OV7z72MQCV+XAWEJ0F1g6wBj1Z6AD+OsDMbpCGc+iiGuk4xzt4kOBxwGMX
l1DHNCTsjm1sIxj1kAcsMByNKld5GF5Ygpa17IXnqHe97XXvTd5bks4eoE8rBkD+mjECXawtxylr
WK2XcFwulYlABONIBzN+MQ4oNMwihyBHNWyRjV3UIxjuSPQ23DGNeUxZFr2wspXxsOVK3wFTXw4A
e9tLZljdLXmgZomZ/3GAfSxgASsudbiQiOMw3DIUa3jTnKWS5nOtgQEVUMEcwpEG6criFcU8BDwy
AWFbWEMczkh0k92hjkdL4xnPEEYvepGLLd9hEnfYMqdKrFmTdDqDKRl1qU2N6lKr2iKQOMAJsJqC
v8A6EWZdcZpbjJF7ZCACDOCACGggh9OqIxxyloUndkFoCK+iHrtwRoSzwQ11VGQY0PawLEqBBSy4
wSJuqDimv7xp4vJ2jDGrswf+L7U8kAzgzONWc4v3cW6L0HgvL7BOTtfwZ03JW94AoHdGBJiBfF93
HvOAxzEu4opGLNwW4kBteMX7j4oII+IVobjFMa7xSmW642L2dqZIvkGQh9ojyU05i1ms5nOFAnZF
odrMay6rm+Nc5xthwBZ8AY94zAMd7/C1LBBB8Aivgh6scIbCIcwNazy8Gc2IxrRzUXEsLGISi2j8
trn96W7FMuTPax7mTS525JI9I6SDaAjuAYO1e8ntns15cTUABl3Q4hfoOMc3ZmARR4TjEsWuRjWS
rexsqGMUFekF4offDDw0/viXtjrHIcD85n/7W5fv+vI2z9k0jx25M4PELKL+gJsDFGFyNB+omt8+
gBavngXjQAc85IEOR5jB14boBjJs4QxWxOPBvZ8GOj5hEWMQH/FvgAYCKIBvsBXppV4A0HzO93HK
I32g9nwVMV/X93kaARizUASegz4050OyhinzVn4gWFz3oAEsAAa6BwVCoAhzUBGGQAzgkAmMUA3I
oHDKFgzx8A2dcBG98HTR5mGL8AZAuAgkRnnNEynRJ3Ka4koksXKeB3c7RwfHgT6BsQb34E2TknMg
WH7UtRE8wAAicANGYAVPAANQUQjEcAzegAzVwAi20IbOgHTjQA05yFv5UId2mIAKCAEQCBN7GIHd
1hEHEAZrEBgTMkA2pilgIfh8dKACGZABH3AsFhEH0EAM3RAOulcN8ZCJ/3YMx4AI6GWHd5iHejgb
fagSqHAAj2BUH1GELBEHhVAIhmAIjjCLtDiLnoheRFh5J7aLvNiLvviLwBiMwjiMxFiMHBEQADs=" />
<img src="TrayItems.gif" />
<img src="http://www.shinylight.com/wp-content/themes/Wiking/images/button_rss.png" />
</body>
</html>

Which outputs:

You can see for yourself here, which is the HTML page.

Now, let’s disable showing of images in Firefox – Uncheck “Load images automatically”:

Refresh the page and see that you get:

Strange that they haven’t blocked this as well. This happens in Firefox 3.6 for Windows. I tested it in Safari 4.0.4 on Windows and it none of them showed when images are disabled.

Reasons I’m Not a Fan of Google Chrome OS

I’m assuming you’ve seen what Chrome OS can do. Just Google “Chrome OS,” because it’s everywhere.

I would give Chrome OS an 8.0 / 10.0 for computer beginners, like the grandmas of the world. It’s gets a 2.0 / 10.0 for techies. Why? Three reasons.

  • Needs too many resources for what it delivers.
  • Cannot run executables, not even virtually/sandboxed.
  • Needs the cloud for storage.

Kind of annoying that I can’t just run it on any old laptop / netbook. System requirements are too demanding for what it promises and it seems it’s needed just to have that 2-4 second boot-up time, which honestly it doesn’t mean much to me. There’s never a time where I’m in such a hurry that I need my computer that fast, because I’m either doing something else, or I just use my phone.

I some people really care about boot-up time? I’d rather wait 10 more seconds and have the ability to run .EXE files and have it run on old PC’s. For a minimalist device, I have my table PC or iPhone already that can do all that stuff, and I don’t have to worry about malware on the iPhone. I don’t think techies have to worry to much about malware since they’re careful about getting them in the first place and have good Anti-viruses.

Lastly, an Internet connection is not ubiquitous yet. I’m on the subway a lot where there’s no WIFI and sometimes I travel to places without an Internet connection. I want to be able to at least read my latest documents without connecting to the cloud.

The OS is still in the works, so I suppose maybe they’ll address some of these features in the future.

High Performance Websites

This was a great read. Certainly learned a lot of useful lessons on front-end optimizations. Written by Steve Souders (previously lead Optimization efforts at Yahoo!), who also developed YSlow! – a Firefox add-on (actually an add-on to the Firefox add-on called Firebug) that gives your web pages a grade from A through F and tells you how to make it better.

If you have a build script, you may be able to automate a lot of these things, like combining JS and CSS files, and optimize PNG files (check out http://www.phpied.com/give-png-a-chance/ to see how to optimize from the command line). If you’re going to optimize JavaScript, I would recommend YUI Compressor (http://developer.yahoo.com/yui/compressor/) since it’s not as greedy as Google’s Closure Compiler for JavaScript. The Closure compiler (http://code.google.com/closure/compiler/) is relatively new and you may get even smaller files, but if your JavaScript is complex, it may have bugs because it’s a greedy compiler.

Anywhoot, here’s what I got from it:

  1. Reduce as many HTTP requests as possible.
  2. Minify JavaScript (don’t obfuscate, because it’s more trouble than it’s worth for the benefits you get)
  3. Minify CSS and optimize it (reduce duplication).
  4. Future-expire resources for caching (PNG, JPG, GIF, JavaScript and CSS).
  5. Minify HTML (get away from tables)
  6. Put CSS at the top, put JavaScript at the bottom.
  7. For design components (background images, button images, nav images), use CSS sprites.
  8. Use PNG for design components (they compress better than GIF, have partial transparency, and can have greater color palettes).
  9. Gzip non-binary data like HTML.
  10. Combine CSS and JavaScript into single files to reduce HTTP requests.

A summary of his optimization rules are found here, but of course, it’s not as detailed as the book: http://stevesouders.com/hpws/rules.php .

Stoyan Stefanov, another prominent developer who’s written tons on JavaScript and optimization, published 24 articles this month on optimization. I find these articles invaluable. It’s recent and he does actual optimization tests and tells you what tools he uses. Here’s the site: http://www.phpied.com/performance-advent-calendar-2009/

Asynchronous Upload to Amazon S3 (S3)

Requirements

Make sure you have an Amazon S3 Account (get it from tech). JavaScript is mandatory for this to work (to be able to POST to two different domains) upon user submission.

Summary of Problem

This is a summary outlining the solution used in a video uploader. It entailed a form that would leverage the user of our Amazon S3 (S3) account. In addition, because the video files could be large (and to avoid CF limitations), a critical requirement was to upload to that page S3 directly. At the same time, the form field information had to be saved to our database. This meant doing a double post – one to the XYZ domain, and the other to s3.amazon.com. This cross-domain POST could only be done via AJAX.

Here’s visualization:

As you can see, once the user clicks “Submit”, there’s an AJAX HTTP POST to XZY server to save the fields to the database, and then the JavaScript runs a form.submit() on the current form to submit the file via POST to S3.

Introducing Amazon S3

Amazon S3 (Simple Storage Service) is a cloud service whose sole purpose is to store files. To store files into it, one can use its REST API or Admin Web Console (AWS).

Screenshot of the AWS console, with Amazon S3 tab selected.

One gets to the AWS console via an account (can get it from tech) and going to http://aws.amazon.com/s3/ and clicking “Sign in.”

While S3 has many advantages, there are a set of drawbacks as well. To summarize, here’s a list:

Benefits:

  • The max upload size for a file is 5GB. Plenty.
  • For all the times I’ve tested, uploads of different file sizes, everything has gone super smoothly – like butter – so definitely reliable.
  • Amazon is super scalable (as you may already know), so parallel uploading from one or many users is really no problem.
  • Would not affect performance of our servers – there could be many uploads, and they would go fast, without slowing down any other web sites on servers.
  • The speed of the upload is limited to the user’s computer’s specs and internet provider – much faster than our servers.
  • Files can be made secure and unreadable, not just through obscurity – this is sometimes tricky to implement in ColdFusion.

Drawbacks:

To summarize, the reason for some of the drawbacks, is because it’s doing a POST request directly from one domain (us) to another (s3.amazonaws.com). It’s not being channeled through our CF servers.

There are two ways to interact with S3: the REST API, and doing a direct POST. With the REST API, the upload data has to be channeled through a server first before sending to Amazon – this was not what we were looking for, since our servers have issues with large files. So we looked into removing ourselves as the middleman and sending the data directly to S3 – via POST.

Here are the drawbacks, mainly three:

  • If S3 detected an error in the upload, e.g. if the file is too large, there’s no default error page, just a redirect to an XML document hosted on s3.amazonaws.com. There’s no way to set an error page – it’s on Amazon’s to-do list for future release. One can’t even customize the look and feel of the XML document you’re redirected to. Side note: if the upload was successful, it gets redirected to a page you specify (at least there’s some control here).
  • Progress bar reusable code is scare. There’s tons of code out there to do this, however, I could not find one that could cross-domain post. With traditional AJAX, you’re only allowed to do a POST/GET if the URL you’re using is the same domain as the caller page. One could get the code for a progress bar plugin (as there are tons out there) and rewrite it to do a POST and work with Amazon S3 – but that would take a considerate amount of work.
  • Lack of documentation. There’s not enough documentation for handling POST requests in the official Amazon Developer docs, which makes troubleshooting difficult. Doing POST submits is a relatively new feature of Amazon S3, compared to the rest of their APIs.

So the largest hurdle is to code functionality to get around the error page, since some JavaScript magic has to be put in place. That would be another day or so of work just for that, I believe. I already have some code in place that I put together while testing. If we left it as-is, when the user uploads, and if there was an error, the user would see something like this:

Which would, of course, be nonsensical. If the file was too large, they would see a message that the file was too large within the message tags. The user would then have to hit back to return to the form.

We can try another way, probably the easiest. When the user hits submits, it starts showing the animated spinner as it’s uploading. Also, we can tell the user that if he encounters an error page, just hit the back button. Also, keep in mind that there’ll be validation in place before the upload to check for file extension, at the very least. The only edge case to seeing that XML error message is if the file they submitted is over the limit *AND*  they have JavaScript turned off (that overrides the JavaScript file extension validation).

Creating a Basic HTML that POSTs a File to Amazon

Step 1 – Create a Bucket / Folder / Object:
The first thing we need to do a is create a Bucket on S3. To do this the easy way, go to the AWS: https://console.aws.amazon.com/s3/home and create a bucket:

Buckets are where you store your objects (i.e. your files of any format). You can create a folder for further organization:

As you can see here, there are 4 folders here. We can double-click on step1_new_submissions folder and see the objects that are contained within this:

You can right-click on one of those objects (files) and click “Properties”:

Notice that a Properties panel will expand below. To the right, you have three tabs: Details, Permissions, Metadata.

If you click on the Permissions Tab you’ll notice that by default the file that was selected has following permissions set by user USERNAME:

Go back to the details tab and click on the Link:

You’ll notice that you’ll be taken to an XML document in your browser that has the following:

It’s because you have not let it public access. To give it public access, you click on the Permissions tab again, and click “Add more permissions” , set the Grantee to “Everyone” and choose Open/Download and Save.

You can also set the make multiple objects public. Select an object, hold the SHIFT key, then select the other object to select the objects in between. Select “Make Public”:

You can also upload an object manually via the “Upload” button:

Then click “Add more files”

As the file starts uploading, you’ll see the bottom panel show details about the file transfer:

Step 2: Setting up the HTML Form

The S3 REST API is very flexible, as long as you execute the proper method from your application, while at the same time sending the file over from your server to S3 (via a REST method with the correct URI). Traditionally, it would look like this:

Notice how there’s a middle-man that serves as the captor of the data submitted by the form and the file. Then, it sends it long to S3. The middle-man here is crucial. Double the total bandwidth is spent here – the bandwidth to go from the user’s machine to the web server (in this case CF), and then the bandwidth spent transferring the file to S3.

The advantage to this layout is that because the web server acts as a middle-man server, it can modify the data, change its filename, and slice-and-dice anything within the file because the file submitted has to go through it first. Once the middle-man is done, then it sends it to the S3. Drawback is that there’s wasted resources from the middle-man, not to mention there may be limitations on the middle-man to handle large files > 1GB .

As a solution, S3 has a POST method solution where you can POST the file directly to S3:

Setting up the Form tag

Let’s see how we can cross-domain (a domain other than ours) to S3. Rather than doing the following (to post to the same domain):

<form action=”#CGI.SCRIPT_NAME#” method=”post” enctype=”multipart/form-data”>

We do the following:

<form action=”http://s3.amazonaws.com/PastaVideos” method=”post” enctype=”multipart/form-data”>

Where “PastaVideos” is the name of the bucket.

The format of the object URI is as follows:

Step 3: Setting up the Other Form Fields

This is where things get interesting. In order to set up an HTML form that can upload straight to S3, there’s a set of required input fields in the form. They are as follows:


Optional Form Fields

IMPORTANT: If you add any other additional form fields, it will throw an error. If there is in fact a need to add extra form fields, which will be pasted to another server, then you must append the prefix “x-ignore-“. Let’s say for example I have three input fields I want S3 to ignore, then do as follows:

<input type="text" name="x-ignore-lastname" tabindex="2" class="textfield">
<input type="text" name="x-ignore-address1" tabindex="3" class="textfield">
<input type="text" name="x-ignore-address2" tabindex="4" class="textfield">
<input type="text" name="x-ignore-city" tabindex="5" class="textfield">

This is completely legal and will not throw errors.

Grabbing x-ignore- fields in ColdFusion

If you want to grab these form variables via ColdFusion, do something like

Form.x-ignore-lastname

Will not suffice because of the dashes. You’ll have use the bracket/quotes format:

Form[“x-ignore-lastname”]

to grab them.

Also to check for existence or set a default value,

<cfparam name=”form.x-ignore-lastname” default=”parker” />

Or

<cfparam name=”form[“x-ignore-lastname” default=”parker” />

will not work.

You’ll have to use StructKeyexists( Form, “x-ignore-termsagree” ) to check for existence.

HTML Form Example

Putting all variables together from the previous table, we get something something like as follows:

<input type="hidden" name="key" value="step1_new_submissions/9AAAAAAA-D633-0944-9FBCCCCC6CFB161B_${filename}" />  
<input type="hidden" name="acl" value="private" />
<input type="hidden" name="AWSAccessKeyId" value="0N16468ABC47JDAQ2902" />
<input type="hidden" name="policy" value="eyJleHBpcmF0aW9uIjogIjIwMTgtMTAtMjFUMDA6MDA6MDBaIiwKICAiY29uZGl0aW9ucyI6IFsgCiAgICB7ImJ1Y2tldCI6lyZWN0IjogImh0dHA6Ly90ZXN0LXBhc3RhdmlkZW9zLm1pbGxlbm5pdW13ZWIuY29tL3RoYW5rcy5jZm0ifQogIF0KfQ==" />
<input type="hidden" name="signature" value="2AAAA/BhWMg4CCCCC32fzQ=" />
<input type="hidden" name="content-type" value="video/mov" />
<input type="hidden" name="success_action_redirect" value="http://XYZ.com/thanks.cfm" />      
<!--- Ignore All This Stuff... --->
<input type="text" id="x-ignore-firstname" name="x-ignore-firstname" value="peter" />
<input type="text" name="x-ignore-lastname" value="parker" />

Using Amazon’s HTML POST Form Generator

Because setting up the above HTML for the form could be tricky, Amazon has a tool that easily generates the HTML for the above code.

The following is a screenshot of the tool. The form can be found at: http://s3.amazonaws.com/doc/s3-example-code/post/post_sample.html


So the first thing you do is:

1. Fill out the IDs:

Where AWS ID = Access Key ID   and AWS Key = Secret Access Key

2. The next thing we’ll do is fill in the POST URL:

3. The third step is the trickiest step. This is a JSON document that must adhere to the JSON spec. By default, there’s already a default boilerplate JSON document there. Let’s analyze what it means:

Let’s use one a real one from the test-pastavideos.XYZweb.com page:

You’ll notice that it has content-length-range, which checks the max size, in this case being 1 GB, and it also redirects to the index.cfm page when successful.

After you copy and paste that JSON policy, press “Add Policy”. Notice how the fields in the section “Sample Form Based on the Above Fields” has been populated.

4. The fields may look something like this:

5. Now click on Generate HTML:

Note that with the HTML code above, whatever the user uploads will be renamed to “testfile.txt” on S3. To retain the user’s filename, you have to switch it to value=”${filename}”

6. You then copy that generated HTML and paste it into your page. Add any necessary optional fields with the x-ignore-prefixes.

That should give you a basic template for uploading to S3.

NOTE: You cannot change the values of any <input> fields except except the key. More about this in the next section.

Assigning the a unique ID to the object

Keep in mind of these items when assigning a unique filename:

  • You cannot change the user’s filename in JavaScript – once the user selects a file from his computer, you cannot append a GUID because JavaScript will not let you set the value of a file textbox, you can only read.
  • You cannot change append the GUID prefix after the form is submitted, because the filename will go to S3’s server, and there’s no way to run conditional logic once it’s on S3.

To get around these limitations, you generate the GUID or rather a ColdFusion UUID, then append it to your filename. Let’s take a look at an example of the pasta video form:

First let’s show the JSON policy document:

{"expiration": "2018-10-21T00:00:00Z",
  "conditions": 
  [ 
    {"bucket": "PastaVideos"}, 
    ["starts-with", "$key", "step1_new_submissions/"],
    {"acl": "private"},
    ["starts-with", "$Content-Type", "video/mov"],
    ["content-length-range", 0, 10737774],
    {"success_action_redirect": "http://pastavideos.XYZ.com/index.cfm"}
  ]
}

And now the HTML / ColdFusion code:

<cfset Data.videoUUID  = CreateUUID() />        
<form id="videoform" action="http://www.XYZ.com/index.cfm" method="get" enctype="multipart/form-data"> 
 
<!--- Start of Amazon S3 specific variables. --->
<input type="hidden" name="key" value="#Data.key##Data.videoUUIDXX#_${filename}" />   
<input type="hidden" name="acl" value="#Data.acl#" />

In the code above, when the HTML is rendered, it will already have filename S3 will use when it’s finished uploading. Remember, this is not the value of the <input type=”file” /> box.  So the user uploads, it will look like this on the AWS Console:

Now why is the action set to http://www.XYZ.com/index.cfm and method set to get? The HTML has this set, but when the page is loaded, JavaScript immediately runs and changes action to http://s3.amazonaws.com/PastaVideos and method to post:

<cfoutput>
  <!--- Only change the form variables if JavaScript is turned on.  --->  
  $( "##videoform" ).attr( "action", "#Variables.postURL#" ); 
  $( "##videoform" ).attr( "method", "post" );     
</cfoutput>

This is so that if JavaScript is turned off, it doesn’t POST to S3.

Other Sources of Information

Amazon S3 POST Example in HTML

AWS

Helpful Resources

Test Form

Documentation Home Page

Workflow using WS3 policies

Helpful Posts

Asynchronous Upload to Amazon S3 (S3)

Last Updated: 11/29/2010

Author: Dan Romero

Table of Contents

Summary of Problem.. 2

Introducing Amazon S3. 2

Benefits: 3

Drawbacks: 3

Creating a Basic HTML that POSTs a File to Amazon. 5

Step 1 – Create a Bucket / Folder / Object: 5

Step 2: Setting up the HTML Form.. 9

Setting up the Form tag. 10

Step 3: Setting up the Other Form Fields. 11

Optional Form Fields. 13

Grabbing x-ignore- fields in ColdFusion. 13

HTML Form Example. 14

Using Amazon’s HTML POST Form Generator. 14

Assigning the a unique ID to the object. 18

Other Sources of Information. 19

Requirements

Make sure you have an Amazon S3 Account (get it from tech).

JavaScript is mandatory for this to work (to be able to POST to two different domains) upon user
submission.

Summary of Problem

This is a summary outlining the solution used in a video uploader. It entailed a form that would leverage the user of our Amazon S3 (S3) account. In addition, because the video files could be large (and to avoid CF limitations), a critical requirement was to upload to that page S3 directly. At the same time, the form field information had to be saved to our database. This meant doing a double post – one to the XYZ domain, and the other to s3.amazon.com. This cross-domain POST could only be done via AJAX.

Here’s visualization:

As you can see, once the user clicks “Submit”, there’s an AJAX HTTP POST to
XZY server to save the fields to the database, and then the JavaScript runs a
form.submit() on the current form to submit the file via POST to S3.

Introducing Amazon S3

Amazon S3 (Simple Storage Service) is a cloud service whose sole purpose is to store files. To store files into it, one can use its REST API or Admin Web Console (AWS).

Screenshot of the AWS console, with Amazon S3 tab selected.

One gets to the AWS console via an account (can get it from tech) and going to http://aws.amazon.com/s3/ and clicking “Sign in.”

While S3 has many advantages, there are a set of drawbacks as well. To summarize, here’s a list:

Benefits:

The max upload size for a file is 5GB. Plenty.

For all the times I’ve tested, uploads of different file sizes, everything has gone super smoothly – like butter – so definitely reliable.

Amazon is super scalable (as you may already know), so parallel uploading from one or many users is really no problem.

Would not affect performance of our servers – there could be many uploads, and they would go fast, without slowing down any other web sites on servers.

The speed of the upload is limited to the user’s computer’s specs and internet provider – much faster than our servers.

Files can be made secure and unreadable, not just through obscurity – this is sometimes tricky to implement in ColdFusion.

Drawbacks:

To summarize, the reason for some of the drawbacks, is because it’s doing a POST request directly from one domain (us) to another (s3.amazonaws.com). It’s not being channeled through our CF servers.

There are two ways to interact with S3: the REST API, and doing a direct POST. With the REST API, the upload data has to be channeled through a server first before sending to Amazon – this was not what we were looking for, since our servers have issues with large files. So we looked into removing ourselves as the middleman and sending the data directly to S3 – via POST.

Here are the drawbacks, mainly three:

If S3 detected an error in the upload, e.g. if the file is too large, there’s no default error page, just a redirect to an XML document hosted on s3.amazonaws.com. There’s no way to set an error page – it’s on Amazon’s to-do list for future release. One can’t even customize the look and feel of the XML document you’re redirected to. Side note: if the upload was successful, it gets redirected to a page you specify (at least there’s some control here).

Progress bar reusable code is scare. There’s tons of code out there to do this, however, I could not find one that could cross-domain post. With traditional AJAX, you’re only allowed to do a POST/GET if the URL you’re using is the same domain as the caller page. One could get the code for a progress bar plugin (as there are tons out there) and rewrite it to do a POST and work with Amazon S3 – but that would take a considerate amount of work.

Lack of documentation. There’s not enough documentation for handling POST requests in the official Amazon Developer docs, which makes troubleshooting difficult. Doing POST submits is a relatively new feature of Amazon S3, compared to the rest of their APIs.

So the largest hurdle is to code functionality to get around the error page, since some JavaScript magic has to be put in place. That would be another day or so of work just for that, I believe. I already have some code in place that I put together while testing. If we left it as-is, when the user uploads, and if there was an error, the user would see something like this:

Which would, of course, be nonsensical. If the file was too large, they would see a message that the file was too large within the message tags. The user would then have to hit back to return to the form.

We can try another way, probably the easiest. When the user hits submits, it starts showing the animated spinner as it’s uploading. Also, we can tell the user that if he encounters an error page, just hit the back button. Also, keep in mind that there’ll be validation in place before the upload to check for file extension, at the very least. The only edge case to seeing that XML error message is if the file they submitted is over the limit *AND*  they have JavaScript turned off (that overrides the JavaScript file extension validation).

Creating a Basic HTML that POSTs a File to Amazon

Step 1 – Create a Bucket / Folder / Object:
The first thing we need to do a is create a Bucket on S3. To do this the easy way, go to the AWS: https://console.aws.amazon.com/s3/home  and create a bucket:

Buckets are where you store your objects (i.e. your files of any format). You can create a folder for further organization:

As you can see here, there are 4 folders here. We can double-click on step1_new_submissions folder and see the objects that are contained within this:

You can right-click on one of those objects (files) and click “Properties”:

Notice that a Properties panel will expand below. To the right, you have three tabs: Details, Permissions, Metadata.

If you click on the Permissions Tab you’ll notice that by default the file that was selected has following permissions set by user USERNAME:

Go back to the details tab and click on the Link:

You’ll notice that you’ll be taken to an XML document in your browser that has the following:

It’s because you have not let it public access. To give it public access, you click on the Permissions tab again, and click “Add more permissions” , set the Grantee to “Everyone” and choose Open/Download and Save.

You can also set the make multiple objects public. Select an object, hold the SHIFT key, then select the other object to select the objects in between. Select “Make Public”:

You can also upload an object manually via the “Upload” button:

Then click “Add more files”

As the file starts uploading, you’ll see the bottom panel show details about the file transfer:

Step 2: Setting up the HTML Form

The S3 REST API is very flexible, as long as you execute the proper method from your application, while at the same time sending the file over from your server to S3 (via a REST method with the correct URI). Traditionally, it would look like this:

Notice how there’s a middle-man that serves as the captor of the data submitted by the form and the file. Then, it sends it long to S3. The middle-man here is crucial. Double the total bandwidth is spent here – the bandwidth to go from the user’s machine to the web server (in this case CF), and then the bandwidth spent transferring the file to S3.

The advantage to this layout is that because the web server acts as a middle-man server, it can modify the data, change its filename, and slice-and-dice anything within the file because the file submitted has to go through it first. Once the middle-man is done, then it sends it to the S3. Drawback is that there’s wasted resources from the middle-man, not to mention there may be limitations on the middle-man to handle large files > 1GB .

As a solution, S3 has a POST method solution where you can POST the file directly to S3:

Setting up the Form tag

Let’s see how we can cross-domain (a domain other than ours) to S3. Rather than doing the following (to post to the same domain):

<form action=”#CGI.SCRIPT_NAME#” method=”post” enctype=”multipart/form-data”>

We do the following:

<form action=”http://s3.amazonaws.com/PastaVideos” method=”post” enctype=”multipart/form-data”>

Where “PastaVideos” is the name of the bucket.

The format of the object URI is as follows:

Step 3: Setting up the Other Form Fields

This is where things get interesting. In order to set up an HTML form that can upload straight to S3, there’s a set of required input fields in the form. They are as follows:

Tag
Type
Name
Value

input
hidden
key
The location of the object you’ll be uploading. You can consider it as the concatenation of the folder(s) and the filename (don’t use the bucket name):

step1_new_submissions/4059C3_${filename}

The ${filename} is the original filename of the file the user is uploading. The “4059C3_” is a made up ID that is concatenated to the ${filename} that will add uniqueness to the objects in the bucket, if multiple people are uploading to it.

input
hidden
acl
The access control list. Can be set to:

private – Lets the public user upload a file, but not be able to access it once he uploads. To make it accessible, one has to go into the AWS console and change the rights.

public-read – Lets the public user see it after he has uploaded it or anyone else see it.

input
hidden
AWSAccessKeyId
To get this key id, you need to access it by going to:

Then security credentials:

Get the Access Key ID. This id is also called the AWSAccessKeyId.

You should also grab the “Secret Access Key” by clicking on the “Show” on the adjacent column:

Keep the Secret Access Key private! Only the Access Key ID can be made public.

input
hidden
policy
Policy is a Base64 encoded JSON document that outlines the privileges and details of the files being uploaded. More details about this in the next section.

input
hidden
signature
The signature is the policy, HMAC-encrypted using the Secret Access Key.

input
hidden
content-type
The content type is the what kind of mime content the file that will pass through the form will be.

input
hidden
success_action_redirect
This is the URL of where to go when the upload succeeds. It could be any URL.  Also, when redirected, it will add the three additional URL variables:

bucket=PastaVideos

key=step1_new_submissions%2A192DCAE1-D625-0944-9FBCCD5C6CCB161B_ajax5-loader.mov

etag=%22356060aa56ce8955d38ed8c58661497a%22

Optional Form Fields

IMPORTANT: If you add any other additional form fields, it will throw an error. If there is in fact a need to add extra form fields, which will be pasted to another server, then you must append the prefix “x-ignore-“. Let’s say for example I have three input fields I want S3 to ignore, then do as follows:

<input type=”text” name=”x-ignore-lastname” tabindex=”2″ class=”textfield”>

<input type=”text” name=”x-ignore-address1″ tabindex=”3″ class=”textfield”>

<input type=”text” name=”x-ignore-address2″ tabindex=”4″ class=”textfield”>

<input type=”text” name=”x-ignore-city” tabindex=”5″ class=”textfield”>

This is completely legal and will not throw errors.

Grabbing x-ignore- fields in ColdFusion

If you want to grab these form variables via ColdFusion, do something like

Form.x-ignore-lastname

Will not suffice because of the dashes. You’ll have use the bracket/quotes format:

Form[“x-ignore-lastname”]

to grab them.

Also to check for existence or set a default value,

<cfparam name=”form.x-ignore-lastname” default=”parker” />

Or

<cfparam name=”form[“x-ignore-lastname” default=”parker” />

will not work.

You’ll have to use StructKeyexists( Form, “x-ignore-termsagree” ) to check for existence.

HTML Form Example

Putting all variables together from the previous table, we get something something like as follows:

<input type=”hidden” name=”key” value=”step1_new_submissions/9AAAAAAA-D633-0944-9FBCCCCC6CFB161B_${filename}” />

<input type=”hidden” name=”acl” value=”private” />

<input type=”hidden” name=”AWSAccessKeyId” value=”0N16468ABC47JDAQ2902″ />

<input type=”hidden” name=”policy” value=”eyJleHBpcmF0aW9uIjogIjIwMTgtMTAtMjFUMDA6MDA6MDBaIiwKICAiY29uZGl0aW9ucyI6IFsgCiAgICB7ImJ1Y2tldCI6lyZWN0IjogImh0dHA6Ly90ZXN0LXBhc3RhdmlkZW9zLm1pbGxlbm5pdW13ZWIuY29tL3RoYW5rcy5jZm0ifQogIF0KfQ==” />

<input type=”hidden” name=”signature” value=”2AAAA/BhWMg4CCCCC32fzQ=” />

<input type=”hidden” name=”content-type” value=”video/mov” />

<input type=”hidden” name=”success_action_redirect” value=”http://XYZ.com/thanks.cfm” />

<!— Ignore All This Stuff… —>
<input type=”text” id=”x-ignore-firstname” name=”x-ignore-firstname” value=”peter” />

<input type=”text” name=”x-ignore-lastname” value=”parker” />

Using Amazon’s HTML POST Form Generator

Because setting up the above HTML for the form could be tricky, Amazon has a tool that easily generates the HTML for the above code.

The following is a screenshot of the tool. The form can be found at: http://s3.amazonaws.com/doc/s3-example-code/post/post_sample.html

So the first thing you do is:

1.       Fill out the IDs:

Where AWS ID = Access Key ID   and AWS Key = Secret Access Key

2.       The next thing we’ll do is fill in the POST URL:

3.       The third step is the trickiest step. This is a JSON document that must adhere to the JSON spec. By default, there’s already a default boilerplate JSON document there. Let’s analyze what it means:

Let’s use one a real one from the test-pastavideos.XYZweb.com page:

You’ll notice that it has content-length-range, which checks the max size, in this case being 1 GB, and it also redirects to the index.cfm page when successful.

After you copy and paste that JSON policy, press “Add Policy”. Notice how the fields in the section “Sample Form Based on the Above Fields” has been populated.

4.       The fields may look something like this:

5.       Now click on Generate HTML:

Note that with the HTML code above, whatever the user uploads will be renamed to “testfile.txt” on S3. To retain the user’s filename, you have to switch it to value=”${filename}”

6.       You then copy that generated HTML and paste it into your page. Add any necessary optional fields with the x-ignore-prefixes.

That should give you a basic template for uploading to S3.

NOTE: You cannot change the values of any <input> fields except except the key. More about this in the next section.

Assigning the a unique ID to the object

Keep in mind of these items when assigning a unique filename:

You cannot change the user’s filename in JavaScript – once the user selects a file from his computer, you cannot append a GUID because JavaScript will not let you set the value of a file textbox, you can only read.

You cannot change append the GUID prefix after the form is submitted, because the filename will go to S3’s server, and there’s no way to run conditional logic once it’s on S3.

To get around these limitations, you generate the GUID or rather a ColdFusion UUID, then append it to your filename. Let’s take a look at an example of the pasta video form:

First let’s show the JSON policy document:

{“expiration”: “2018-10-21T00:00:00Z”,

“conditions”: [

{“bucket”: “PastaVideos”},

[“starts-with”, “$key”, “step1_new_submissions/”],

{“acl”: “private”},

[“starts-with”, “$Content-Type”, “video/mov”],

[“content-length-range”, 0, 10737774],

{“success_action_redirect”: “http://pastavideos.XYZ.com/index.cfm”}

]

}

And now the HTML / ColdFusion code:

<cfset Data.videoUUID  = CreateUUID() />

<form id=”videoform” action=”http://www.XYZ.com/index.cfm” method=”get” enctype=”multipart/form-data”>

<!— Start of Amazon S3 specific variables. —>

<input type=”hidden” name=”key”

value=”#Data.key## Data.videoUUIDXX#_${filename}” />

<input type=”hidden” name=”acl” value=”#Data.acl#” />

In the code above, when the HTML is rendered, it will already have filename S3 will use when it’s finished uploading. Remember, this is not the value of the <input type=”file” /> box.  So the user uploads, it will look like this on the AWS Console:

Now why is the action set to http://www.XYZ.com/index.cfm and method set to get? The HTML has this set, but when the page is loaded, JavaScript immediately runs and changes action to http://s3.amazonaws.com/PastaVideos and method to post:

<cfoutput>

<!— Only change the form variables if JavaScript is turned on.  —>

$( “##videoform” ).attr( “action”, “#Variables.postURL#” );

$( “##videoform” ).attr( “method”, “post” );

</cfoutput>

This is so that if JavaScript is turned off, it doesn’t POST to S3.

Other Sources of Information

Amazon S3 POST Example in HTML

http://aws.amazon.com/code/Amazon%20S3/1093?_encoding=UTF8&jiveRedirect=1

AWS

http://aws.amazon.com/developertools/

Helpful Resources

http://wiki.smartfrog.org/wiki/display/sf/Amazon+S3

http://docs.amazonwebservices.com/AmazonS3/latest/dev/index.html?AccessPolicyLanguage_UseCases_s3_a.html

http://docs.amazonwebservices.com/AmazonS3/latest/gsg/

http://docs.amazonwebservices.com/AmazonS3/2006-03-01/dev/index.html?UsingHTTPPOST.html

http://docs.amazonwebservices.com/AmazonS3/2006-03-01/dev/index.html?HTTPPOSTExamples.html

http://docs.amazonwebservices.com/AmazonS3/2006-03-01/dev/index.html?Introduction.html#S3_ACLs

Test Form

http://s3.amazonaws.com/doc/s3-example-code/post/post_sample.html

Documentation Home Page

http://aws.amazon.com/documentation/s3/

Workflow using WS3 policies

http://docs.amazonwebservices.com/AmazonS3/latest/dev/

Helpful Posts

http://aws.amazon.com/articles/1434?_encoding=UTF8&jiveRedirect=1

https://forums.aws.amazon.com/message.jspa?messageID=89017

https://forums.aws.amazon.com/message.jspa?messageID=88188

Running Google Chrome Frame

A few weeks ago, Google released Chrome Frame, which is an IE plugin that can make your IE render a webpage like Google Chrome. After you install it, however, by default it’s only triggered when you have the following meta tag on your webpage:

<meta http-equiv="X-UA-Compatible" content="chrome=1">

To always have it running so that every page (regardless if the webpage has the meta tag), you will have to make a quick Registry edit.

How to be a Web Designer (Part 1)

Web Designer vs. Web Developer

First, let’s start by defining what a web designer is. “Web designer” has the word “designer” in it, so it implies having artistic ability and emphasizes artistic tasks (e.g. coming up with creative layout, knowing about color, typography, capturing your audience, and being familiar with industry-standard software). Attach the word “web” before it, and it implies what I’ve just defined with an additional ability to know about the World Wide Web and doing things on the “web.” At the heart of a web designer, there is this definition.

There is another term that sometimes many non-tech people use, however. It is web developer. Like the term web designer, it’s got the word “web” so you have to know about the web (how it works, etc.). It also has the word “developer” in it, which comes from “software development.” Software development involves programming (a.k.a. “developing”) software, like making a Windows program that keeps track of your contact information, or developing a small program that plays audio / video. These programs, before the web became popular, used to be strictly made for operating systems, like Windows and Macs. As the web grew, they since have been developed for the Web (e.g. a forum, Google Spreadsheets, Google Maps; they are all example of web programs a.k.a “web applications”). Web developers usually tend to handle these and write the code behind the scenes to process data. They work together with web designers, who handle the overall look and feel of the pages – what the users see.

One thing to keep in mind, however, is that sometimes the roles of the two may overlap, especially for consultants. If you work for yourself, you’ll sometimes want to handle an entire project by yourself. For example, you may have a client that asks you to build a web site. So you first design the pages. Then, the client asks you to implement a forum, a chatroom, a custom web application that keeps track of the sales of his company and an events calendar that has to be in sync with his PDA. These additional tasks cannot be designed but must be programmed. So at this point, you have the option of a) doing these tasks yourself; b) hiring a programmer (web developer); c) purchasing pre-made packages that you integrate or install yourself. The same goes for a web developer. The web developer might know how to implement these, but if the client asks to make the look-and-feel look vibrant, professional, and easy to use, and the web developer has no artistic talent, the web developer would have to hire a web designer.

Additionally, the titles of web designer and web developer can vary. I’ve seen and heard web designers / developers called different things. While I don’t particularly agree with some of these titles, these are titles that I’ve heard from experience and job hunting:

Web Designers Web Developers
Designer Developer
Graphic Designer Software Engineer
HTML Developer Programmer
Web Producer Internet Engineer
Interactive Designer Web Application Developer


Core Requirements

For a web developer, you have to have a software development background in order to build web applications, and have networking knowledge to understand the Internet. Companies tend to ask for a computer science or engineering degree from this individual. Also, a demonstration of previously created applications may be asked.

For a web designer, you have to have an art background in order to design pages that contain style, good aesthetics, be capturing, and create a look-and-feel that establishes or resonates the company’s brand. Companies tend to ask for an art-related degree from this individual. Also, a demonstration of a portfolio (from print and more importantly, online) may be asked.

Competitive Requirements

It’s chaos in the web community. Competition is brutal. Just having an art background, knowing some HTML, Dreamweaver and Photoshop, is not enough. After establishing the core requirements (like earning a degree), one goes to the next level to polish up these requirements that make you a true professional in the field. Depending on how many of these you master, it could directly impact how much money you make and which doors open for you in your path. These are the skills/knowledge/traits:

  1. Computer Basics
  2. Digital Media
  3. Project Management
  4. Information Architecture
  5. Usability
  6. Internet & Networking
  7. Programming
  8. Business
  9. News Active
  10. Determined Personality

You could probably make a career out of most of these, to show you how involved each could be. Let’s go into more detail for each one.

1. Computer Basics – The more you understand about your computer, the more productive you can be. It may take too much time trying to wait technical support to fix your computer if it’s a simple issue. Also, the more you learn about computers, especially if you play with a lot of software, the more you find out about features they have that could apply to web sites you design. You’re in an IT-related field, and therefore you should at least know the very basics of how a computer works. While you won’t be a network administrator or hardware specialist anytime soon, it is much faster when you help yourself with very basic tasks rather than always relying on outside support. Also, software that is common in most work places should be a given: Microsoft Word, Excel, and PowerPoint in the case of presentations.

2. Digital Media – This should definitely be one of your strong points. By media, I mean both visual and audio. You should become very familiar with Photoshop. Depending if your role is also doing print work, you may need to know Illustrator or QuarkXpress, both industry-standard. Know also digital formats, screen colors, and typography. Animation or fancy functionality may sometimes be necessary, so Flash may be required on a project. Sometimes Flash animation may require sound, or you may even have to edit sound clips for a video being streamed via Flash, so you’ll have to learn an industry sound editing program like SoundForge. Editing a video might even be necessary, so a program like Premiere might be of use. And Of course, last but not least, web authoring applications, such as Dreamweaver and web imaging program and prototyping tool like Fireworks. To make great pages, you will need to understand the code that is needed behind them, and they are: HTML, CSS, and JavaScript. (JavaScript is a programming language on its own. Web designers are usually not responsible to have this mastered – just knowing the basics and learn to use pre-made scripts.)

3. Project Management – Your boss or a client asks you, “How long is it going to take?” What do you say? How much money are you going to invest yourself? What’s your budget? Will you have to hire additional support to make the site? How will you handle quality assurance? How many progress meetings are necessary and when will they be? What are the deliverables when the project is complete?

There are whole books written on this topic itself, and as you can see, it is essential. Not establishing a project plan (even a simple one) is one of the reasons many projects miss deadlines, are delivered with poor quality, go over budget, or even fail.

4. Information Architecture – Your boss or a client asks you, “I want to make a site about my stamp collection. I have 10,000 stamps of various types and a description for each. How do I make this site?” It is your responsibility to think about all this information and organize it in a way that is easily presentable. How could you categorize these? By year? By genre? By value? There’s a great depth of information variables to consider and it’s your job to structure them. Once this structure is complete, only then can you build a great site.

5. Usability – Once the information architecture has been established for the stamp collection web site mentioned in #4 above, you have to consider how the web site can be made so that it is easy to use. Should you use a navigation menu with sublinks? If so, will users bother clicking on the links? Will they find it confusing? The site has to be easy and intuitive to use. The content users seek on the site should be easily accessible and easy on the eyes to read.

6. Internet & Networking – Being in the IT-field, it would be an embarrassment not to know how at least the basics of the Internet. To understand the Internet, it’s very essential to understand the basics of networking, e.g.: data transmission, protocols, servers (web, database), firewalls, online security, hacking, etc. – at least what they are. These all affect web sites.

7. Programming – Understand at least how programming can affect the pages you design. Know what languages are out there. You don’t necessary have to know how to program, as this is not your job. However, the more you understand it, the better you can work with web developers. Programming adds functionality to a site. For example, when you click on “purchase order” in a shopping cart site, someone programmed that page so that your credit card gets charged and it sends an email to the warehouse so your item can start the delivery process. There are hundreds of programming languages, but the most popular languages at this time for the web are as follows: PHP, ASP.NET, Java, ColdFusion, Perl, SQL.

8. Business – The more you know about business, the more your possibilities will grow. Understanding how the business functions, may give you a clearer idea on how to put together better pages in regards to functionality. Understanding the company’s finances may help you budget your project plan if you’re the project manager. Understanding company policies and dealing with lawyers will help put together better contracts between you and vendors you plan on hiring if your project is of larger scale.

9. News Active – If you’re always involved in the web community, then you’ll understand all the technological trends on the web. There’s always talk of web design methodologies and stories that come from other web designers that are always worth learning from. Staying active by receiving daily newsletters from various web design sites really keeps you sharp and on your toes. Subscribing to printed magazine like Practical Web Design and Computer Arts show you the latest trends, and the various examples of screenshots on its pages may lead to ideas in your own pages. RSS feeds, podcasts, Twitter – there are several ways to stay up to date.

10. Determined Personality – Companies always want to see a very determined individual. One of the differences between tech people and non-tech is in the confidence of learning something new on the computer. Because hardcore techs always use the Internet (are very familiar with Googling anything, and reading ebooks, blogs, forums, etc.) learning any task becomes way easier. Some people who do not have this mindset of always searching for information might be somewhat intimidated if they are asked to learn something new. The mindset of “I don’t care what it is, I can learn this, I will learn this, and I will master it no matter what” should apply for every project.

Outlook Emails/Contacts in “To” Dropdown

Needed to backup my contacts in Outlook, but it would not save the emails in the email “To” dropdown field, when autocomplete triggers. This is because it’s stored in Outlook’s NK2 cache file. This file is not in readable form, but you if you have the patience, you can view the emails if you don’t mind looking through funky ASCII characters of binary data. In the process of looking for and trying various tools, I realized that not all tools were able to extract all contacts. Keep this in mind.

There’s a few tools out there that you can use to extract a list of emails saved in the NK2 file. Here they are.

Ingressor – is the best program, I was able to extract the most amount of emails/contacts, but you have to pay for it. I think it extracts everything, but you can’t save anything or copy anything. Use OCR software?

Debunk2– It’s free, but it missed 1 (could possibly miss more on different NK2 files) contact.

NK2.info– It’s free, but it missed 2 contacts, and it doesn’t show the names accurately, like Debung2 and Ingressor.