Handling Images from the Command Line

Recently, I needed to do some work from the Windows Command Line, and I needed to deal with a few images. Along the way, I found some great tools. All these tools are free. Can come in useful when automating.

Manipulation

ImageMagick – This is a collection of command line tools. You can do image conversion, view properties, transform, transparency, join, overlay, add special special effects, and tons more. Also has APIs for C, C++, Java, .NET, Perl, PHP, Python, Ruby, and others. Highly recommend it.

Screen Capture


CmdCapture – Takes a screenshot of your desktop from the command line.

IECapt – Capture Internet Explorer’s rendering of a web page into a BMP, JPEG or PNG image file.

Cutycapt – Capture WebKit’s rendering of a web page into a variety of vector and bitmap formats, including SVG, PDF, PS, PNG, JPEG, TIFF, GIF, and BMP.

wkhtmltopdf – Convert HTML to PDF using the Webkit rendering engine, and Qt.

Automate FTP Uploads

I thought this might be useful for those who are constantly uploading the same files. Whether you need to deploy a project live or constantly upload the same group of files, you can do this by creating a batch file in Windows using ftp.exe.

First of all, if you’re not familiar with FTP.exe, it’s just a command console way to upload/download files from an FTP server. You can run this program and start typing away…. OR, you can load a text file filled with FTP commands in FTP.exe. For example, the following are commands in a file called upload.bs:

1
2
3
4
5
6
7
8
9
open western.dreamhost.com
username
password
binary
hash
send index.htm
send frame.htm
send car.jpg
bye

The above should be straightforward. The “open” command is used to connect to the FTP server. The “username” should be your usename and “password” should be your actual password. “binary” is how your data should be handled on transfer – if it’s not text, it’s binary data, and should be “binary.” “hash” makes the console display pound signs (#) to serve as a progress bar as it’s uploading to the server. The “send” commands are used to upload the files. “bye” is exits the FTP session.

That was just a FTP script session file. You now need to call that file from FTP.exe on WindowsXP. You do that as follows:

1
@%windir%\system32\ftp.exe -s:upload.bs

You can include that call in a file called “upload.bat” – To learn more about FTP scripting and how to merge the two files into just one batch file, you can check out this site. Also, there’s more information about this found here: tinyapps.org.

ANT Script to Deploy

A friend of mine asked me to post a simple script to show the power of ANT. Here it is with comments that show some of the things you can do with it. To read how to set up ANT and run scripts, please read my earlier post on doing so.

This script gets the latest from an SVN repo, deploys using FTP, then sends an email using a Gmail account. Here’s the ant script (deploy.ant), followed by a properties file (i.e. the config file, deploy.properties), which the ANT script pulls from:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
<?xml version="1.0"?>
<!-- ======================================================== -->
<!-- Sample build file used to retrieve svnant's sources -->
<!-- ======================================================== -->
 
<!-- TODO: Mail if the script failed --> 
 
<project name="STAGING-Deploy" basedir="." default="empty">
 
  <property file="STAGING-Deploy.properties" />            
  <!-- Import SVNAnt tasks from jar files.  -->
  <typedef resource="org/tigris/subversion/svnant/svnantlib.xml" />
 
  <target name="Deploy"> 
    <!-- Step 1: Export the latest files from the repo to local machine. -->
    <antcall target="export" />
 
    <!-- Step 2: FTP the files from the local machine to STAGING. -->
    <antcall target="ftp_files" />
 
    <!-- Step 3: Cleanup. -->
    <antcall target="cleanup" />  
  </target>  
 
 
  <!-- SVN Checkout (Latest) -->
  <target name="checkout_latest">
    <svn username="${svnant.repository.user}" password="${svnant.repository.passwd}">
      <checkout url="${svnant.projectrunk.url}" revision="${svnant.checkout.revision}" destPath="${svnant.checkout.destpath}" />
    </svn>    
  </target>   
 
  <!-- SVN Update -->
  <target name="update_latest">
    <svn username="${svnant.repository.user}" password="${svnant.repository.passwd}">
      <update dir="${svnant.update.directory}" recurse="${svnant.update.recurse}" revision="${svnant.update.revision}" />
    </svn>    
  </target>  
 
  <!-- SVN Export  -->
  <target name="export">
    <svn username="${svnant.repository.user}" password="${svnant.repository.passwd}">
      <export srcUrl="${svnant.export.srcurl}" destPath="${svnant.export.destpath}" force="${svnant.export.force}" />
    </svn>    
  </target>
 
  <!-- FTP the export of the latest revision to STAGING. -->  
  <target name="ftp_files">    
    <!-- Import FTP tasks from jar files.  -->
    <taskdef classname="org.apache.tools.ant.taskdefs.optional.net.FTP" name="ftp" />
 
    <ftp server="${ftp.server}" port="${ftp.port}" remotedir="${ftp.destination}" userid="${ftp.userid}" password="${ftp.password}" 
         passive="${ftp.passive}" depends="${ftp.depends}" binary="${ftp.binary}" retriesAllowed="${ftp.retriesallowed}" verbose="${ftp.verbose}">
 
      <fileset dir="${svnant.export.destpath}" />
 
    </ftp>    
  </target>  
 
  <!-- Cleans Up: Remove temp info, etc. -->
  <target name="cleanup">  
    <delete includeEmptyDirs="true">
      <fileset dir="${svnant.export.destpath}"/>      
    </delete>
 
    <!-- Email Notification -->
    <mail mailhost="${mail.host}" mailport="${mail.port}" subject="${mail.subjectalert}" ssl="${mail.ssl}" user="${mail.username}" password="${mail.password}">
      <from address="${mail.from}"/>      
      <to address="${mail.to}"/>
      <message>The nightly build has completed.</message>
    </mail>
 
    <echo>Cleanup complete. </echo>
  </target>
 
  <target name="empty">    
    <echo>Pass in a default target to call </echo>    
  </target>
 
</project>

Now let’s check out the properties file (deploy.properties):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
# -----------------------------------------------------------------------------
# STAGING-DEPLOY
# Variables are initialized here.
# -----------------------------------------------------------------------------
#
# NOTE: Backslashes have to be escaped with another backslash (on Windows)
#
 
# === INIT =====================================================================
svnant.projectrunk.url=svn://192.168.3.142/TESTING/trunk
svnant.repository.user=YOURUSERNAME
svnant.repository.passwd=YOURPASSWORD
 
# === CHECKOUT (INITIAL) =======================================================
# Check out a working copy of the latest to a local directory.
svnant.checkout.destpath=C:\\testing123
svnant.checkout.revision=HEAD
 
# === UPDATE TO LATEST =========================================================
# Update the working copy of your local directory to latest revision
svnant.update.directory=C:\\Experiments\\Firehawk\\src_latest
svnant.update.revision=HEAD
svnant.update.recurse=true
 
 
# === EXPORT ===================================================================
# Copies From Source Repo:
svnant.export.srcurl=${svnant.projectrunk.url}
 
# Into a temporary directory in your ant home (that will be FTP'd):
svnant.export.destpath=${ant.home}\\export-temp
 
# Overwrite (needs to in case the file changed)? 
svnant.export.force=true
 
PBNStagingFTP
# == FTP INFO ==================================================================
# FTP what was exported (latest version) to this location on staging:
ftp.destination=/testing/
ftp.server=TESTING
ftp.port=21
ftp.userid=USERNAME
ftp.password=PASSWORD
ftp.passive=yes
ftp.depends=yes
ftp.binary=yes
ftp.retriesallowed=5
ftp.verbose=yes
 
# == MAIL INFO ==================================================================
# Currently using GMAIL for now.
mail.host=smtp.gmail.com
mail.port=465
mail.subjectalert=[STAGING ANT Build] Test build
mail.ssl=yes
mail.username=USERNAME
mail.password=PASSWORD
mail.from=someone@somewhere.com
mail.to=someone@somewhere.com

Executable Extension

Want to run a script in the command console that is not of type EXE, COM, BAT by just typing in the name of the script? Have a script in Perl or Python that you have that you want to run without typing the name of the interpreter with switches? It's pretty easy to do in Windows.

  1. Open up System Properties and click on Environment Variables:



  2. Select the PATHEXT System variable and click on the Edit button.



  3. Add the extension you want. In this case, let's add the extension for .PL so we can run Perl scripts without typing in the name of the Perl interpreter:



  4. Hit OK on the Windows.

  5. In an Explorer Window, Go to Tools -> Folders



  6. In the Folder Options Window, click on the File Types tab, and then click New:



  7. Type .PL for the File Extension



  8. Click OK then Click on the Advanced button to open up the Edit File Type and click on New...



  9. Enter the name of the interpreter:



  10. Hit OK for all Windows.

  11. You're set and good to go.

Now let's test it. Make a .pl file under c:\ called "add.pl". Make add.pl be:

1
2
$sum = $ARGV[0] + $ARGV[1];
print "\nSum of the numbers is: $sum\n\n";

 

Now let’s call it via the console by typing:

1
add 5 5

9

JavaScript for the Windows Command Console

JSDB is a great JavaScript interpreter based on Mozilla’s Spidermonkey JavaScript engine. If you like working off the command console, I suggest you give it a go. Of course, not everything is supported, as there is no HTML/DOM context in the console. To compensate for it, the author of this tool has added some great features, like being able to connect to databases, use includes ( easily via the load(“file.js”) function ), and other network facilities, like fetching HTML content from other websites.

LogParser to Query IIS logs using SQL

LogParser is a great way to query IIS logs (any text log, actually, that is delimited).

Once you have it installed (default install is to C:\Program Files\Log Parser 2.2), let’s try to query log file ex090915.log from directory C:\WINDOWS\system32\LogFiles\W3SVC1942853941 . The way you would do this, is this:

1
LogParser "select date, s-ip, cs-method from C:\WINDOWS\system32\LogFiles\W3SVC1942853941\ex090915.log" -rtp:-1

As you can probably imagine, “date”, “s-ip”, and “cs-method” are the column headers from the log file. The select statement goes in quotes. Also, rather naming a table, you give the path to the log file. What’s the argument -rtp:-1 ? If you don’t include this argument, every 10 results, it will prompt you to “press a key…,” then will show you the next batch of results. In any case, the select state we just ran will spit out the following in the console:

date       s-ip            cs-method 
---------- --------------- ---------
2009-09-15 192.168.157.128 GET
2009-09-15 192.168.157.128 GET
2009-09-15 192.168.157.128 GET
2009-09-15 192.168.157.128 GET
2009-09-15 192.168.157.128 GET
2009-09-15 192.168.157.128 GET
2009-09-15 192.168.157.128 POST
2009-09-15 192.168.157.128 POST
 
 
Statistics:
-----------
Elements processed: 27
Elements output:    27
Execution time:     0.02 seconds

LogParser will even generate graphs (.gif format) of your results.

If you want to use a GUI for your queries, I suggest you try Log Parser Lizard.

Setting Up Ant

ant_logo_large

Ant (Another Neat Tool) is a tool to automate tasks needed to run successful builds, deployments, and tests. You can run Ant as a stand-alone tool, or run it from Eclipse.


Needed JAR Files

In Ant, a "task" is a single line of instruction that gets run in the build file, as opposed to a "target" which is a group of tasks. There are a core set of tasks in Ant: copy, delete, echo, and many more (http://ant.apache.org/manual/). Other important tasks (like FTP/SVN/Mail functionality), however, are not included. For that reason, we need to include these "add-ons" into Ant in the form of JAR files into Ant's classpath.

The following is a list of additional tasks we need, that are not part of Ant's core task list (download these before reading on):


Setup to Run Stand-Alone
  1. Install the JRE, if you don't already have it.

  2. Download latest version of ANT: http://ant.apache.org/bindownload.cgi and unzip to c:\Ant

  3. Download the above latest JAR Files in Zip files. Zip files you download may have more than just JAR files, but you need to isolate these JAR files and put the JAR files into Ant's classpath, C:\Ant\lib


  4. Put all the JAR files under C:\Ant\lib

  5. Add the following variables to your SYSTEM Variables
    ANT_HOME=C:\Ant  

     
    Add this to your existing PATH variable:

    C:\Ant\bin;  

     
    Also, make sure there's a CLASSPATH and JAVA_HOME system variables setup for the JRE.
    e.g. 

    CLASSPATH=.;C:\Program Files\Java\jre1.6.0_07\lib\ext\QTJava.zip  
    JAVA_HOME=C:\Program Files\Java\jdk1.6.0_07  

  1. That should be it. Try to run, go to the command console and type "ant". To load a build file, type:
    ant \-buildfile build.ant  


Setup to Run From Eclipse

The typical Eclipse setup already comes with the core Ant tasks, so you just have to get the optional ones. Download the attachment and preferably put them here: C:\Ant\lib

  1. Run Eclipse

  2. Go to Run > External Tools > External Tools Configurations

  3. Double click on the Ant Build Icon
    This will open up the "New_Configuration Wizard"

  4. Name the build DeployStaging (there should already be a name in the field, but it's called "New_configuration"

  5. Under the Main tab, put in the Buildfile (in this case, "Browse File System...") - browse to where your buildfile is. (No need to include the .properties file, as the build file will call it.)

  6. Under the Classpath tab, click on User Entries then click on "Add External JARs..." and point it to the directory which has all the JARs you downloaded (C:\Ant\lib).

  7. Select all the JARs.

  8. Click Apply.

  9. Click Close.

  10. Save if prompted.

  11. Select the build file and then Go to Run > Run

  12. To Modify the script you've just setup, go to Run > External Tools > External Tools Configurations

  13. To select another default target (for example, for to get the latest changes), go to the Targets tab, uncheck the default, and select the target you want (in this case, "update_latest").

  14. Click Apply and close.

  15. To see the build progress in detail, open up the console view by Window > Show View >Other > General > Console.

Computer Tools for the Power User

Here’s a list several tools I use both at home and work. I usually carry these on my USB keychain for easy access. They’re mostly lightweight, fast, and some don’t even require a installation. These are free and NOT Shareware! They are for Windows/Windows XP.

Windows

WinMerge
Used for file comparisons. You can compare text files and see, line by line where these files are different.

Notepad++
Great editor for HTML. This integrates nicely into Internet Explorer to “view source.” It can collapse HTML tags.

Notepad2
Microsoft Notepad on steroids. It uses color coding for files of various extensions. Uses transparency so that you can see through the editor window what’s behind it.

Agent Ransack
Search for text in files. The results show you the file and what line your keyword search is in.

Password Manager
It saves your passwords. You can create categories for what kind of passwords they are. Once you save it, you hit CTRL+U to save the user name, and CTRL+C to save the password to the clipboard. It uses a secure 448 bit Blowfish algorithm to ensure the data are not crackable. The password used for securing access to stored data is not saved anywhere.

JR Screen Ruler
A visual ruler you can use to measure pixels on the screen.

WinMacro
Record whatever you do at your Windows Desktop, and can replay them exactly. This is great to record tasks and save them for later.

STRIFE
Get a checksum for a file using hashes: crc16, crc32, xsum, md5.

PuTTY
Telnet/SSH Client

Process Explorer
See information about processes: what files or directories they have open. Kill processes, etc.

StartupRun
Know exactly what’s running when you start windows and turn those annoying programs off.

Simple File Schredder
Simple File Shredder is a utility that securely deletes your files so they can’t be restored.

Restore Deleted Files
This restores files that have been deleted. Use this as a last line of defense. The great thing about this program is that it does not require to install – unlike other applications, where you need to install first.

Angry IP Scanner
Angry IP scanner is a very fast IP scanner and port scanner. It can scan IP addresses in any range as well as any their ports. Its binary file size is very small compared to other IP or port scanners.

SetFileDate
Change the date stamp for any file you want.

PopupManager
Great popup blocker.

ZoneAlarm
Good firewall software.

System Spec
System Information Utility. Get a report for practically everything you wanted to know about your computer.

SyncBack
Synchronize files from different folders. I use this to synchronize files from my computer and USB keychain and USB drive. Lots of features.

Microsoft PowerToys for Windows XP
PowerToys add fun and functionality to the Windows experience. What are they? PowerToys are additional programs that developers work on after a product has been released. These tools include: Synchronize file tool, Image resizer to resize a batch of files, power calculator, multiple desktops, tweak UI settings, and more productivity tools.

Windows Internet Browser Plugins

Web Developer Plugin for Firefox
Provides a variety of tools for quickly creating, understanding, and troubleshooting Web pages. This feature is added to the menu when you right click on a page.

Internet Explorer Developer Toolbar
Provides a variety of tools for quickly creating, understanding, and troubleshooting Web pages. This feature is added to the menu when you right click on a page.

IE Context Menu Toolset
This is a set of tools which extend the right-click context menu of Internet Explorer. By simply installing these tools, your browser will have added functionality which can greatly help in debugging pages, designing sites, diagnosing form problems, tweaking design, etc.

Fiddler
Fiddler is a HTTP Debugging Proxy which logs all HTTP traffic between your computer and the Internet. Fiddler allows you to inspect all HTTP Traffic, set breakpoints, and “fiddle” with incoming or outgoing data. Fiddler is designed to be much simpler than using NetMon or Achilles, and includes a simple but powerful JScript.NET event-based scripting subsystem.

Command Console Tools

MaxDIR
Like the dir command but displays files/directories with color and in columns.
Way much better! The drawback is if your OS is not using FAT12 or FAT16 type disks, the Bytes Consumed will be wrong.

HexEdit
Great Hexeditor. The only Hexeditor I use.

FastSum
Get md5 hashes of files or text you type in. Useful if you need to find the md5 hash of a word.

Other Great Sites for Small Tools

These two sites have great little apps for networks, programming, graphics, etc. These are also free.

Systinternals Freeware
TinyApps

Reprogram Your Keyboard Keys

I recently got a new Microsoft keyboard without the Right-Windows-Key. I use this all the time ’cause I’m a shortcut fanatic, and I couldn’t get used to using the one on the left. Not only that, but I use a CTRL/SHIFT + INSERT a lot to copy/paste and the keys were way different from my past keyboard…

along comes AutoHotKey….

This program rocks. With it, you can create a script that runs and stays in memory. Remap keys and buttons on your keyboard, joystick, and mouse. In any case, with this tool, I was able to remap the menu key (the one usually between the Right-Windows-Key and CTRL key) that I never use to a windows key. That was close enough to me. Once installed, I was able to create a script with this command:

AppsKey::RWin

And that’s it! That mapped the menu key to the windows key! I’ll be experimenting with this program a little more and trying to get more shortcuts. You can also compile your scripts into a program that runs without an installation – in case you want your shortcuts / remapping run on a different machine.

The URL is: http://www.autohotkey.com/