Posted on

Testing Database Import Scripts with Snapshots

Can’t stress enough how handy database snapshots (compatible only in Enterprise and Developer editions of SQL Server 2005+) come in when testing bulk imports, data scrubbing, or any sort of data / schema modification script. In nearly no time, I can create a snapshot (via shortcut snippet), run my script – don’t like the results? My script crapped out the data? I can run an instant undo by recovering from the snapshot, which works pretty fast. If I want to create another snapshot, I usually tend to overwrite the snapshot I created, and then create it again.

It’s so handy that I have a snippet for it:

Anywhoot, here’s how you create a snapshot:

1
2
3
4
5
6
7

-- CREATING A SNAPSHOT
CREATE DATABASE YourDatabase_snap ON         -- Name of new snapshot
(
  NAME = yourdb_data,                        -- Logical filename of source db you want to snapshot
  FILENAME = 'c:\YourDatabase_data_1800.ss'  -- Sparse file to create
)
AS SNAPSHOT OF YourDatabase;                 -- Source db name
1
2
3
4

-- RECOVERING FROM A SNAPSHOT
USE master;
RESTORE DATABASE YourDatabase                    -- Source db name where the data resided
FROM DATABASE_SNAPSHOT = 'YourDatabase_snap';    -- Snapshot db name

Deleting a snapshot is just like dropping a database.

1

DROP DATABASE YourDatabase_Snap
Posted on

Fast Delete

I needed to delete 30,000 files that took about 60 GB. I know traditionally it takes forever to delete it in Windows. Even using the “del” command in the console, it’s always been slow for me. Enter Robocopy. I never gave it a chance until today. The way it can delete is by syncing directory against an empty one which was just what I needed. It was able to do it in about 50 seconds. Awesome.

You’ll have to download Robocopy as part of the Windows Server 2003 Resource Kit Tools.

What I used to delete this big directory:

robocopy /MIR c:\empty-directory c:\my-files-to-delete
Posted on

Reasons I’m Not a Fan of Google Chrome OS

I’m assuming you’ve seen what Chrome OS can do. Just Google “Chrome OS,” because it’s everywhere.

I would give Chrome OS an 8.0 / 10.0 for computer beginners, like the grandmas of the world. It’s gets a 2.0 / 10.0 for techies. Why? Three reasons.

  • Needs too many resources for what it delivers.
  • Cannot run executables, not even virtually/sandboxed.
  • Needs the cloud for storage.

Kind of annoying that I can’t just run it on any old laptop / netbook. System requirements are too demanding for what it promises and it seems it’s needed just to have that 2-4 second boot-up time, which honestly it doesn’t mean much to me. There’s never a time where I’m in such a hurry that I need my computer that fast, because I’m either doing something else, or I just use my phone.

I some people really care about boot-up time? I’d rather wait 10 more seconds and have the ability to run .EXE files and have it run on old PC’s. For a minimalist device, I have my table PC or iPhone already that can do all that stuff, and I don’t have to worry about malware on the iPhone. I don’t think techies have to worry to much about malware since they’re careful about getting them in the first place and have good Anti-viruses.

Lastly, an Internet connection is not ubiquitous yet. I’m on the subway a lot where there’s no WIFI and sometimes I travel to places without an Internet connection. I want to be able to at least read my latest documents without connecting to the cloud.

The OS is still in the works, so I suppose maybe they’ll address some of these features in the future.

Posted on

Console2 Alternative to CMD

You know what, I tried PowerShell and I guess I never got used to it. I’ve been using DOS for the longest time that I prefer its simplicity when navigating through files. As far as scripting with it – forget it – it’s very cryptic and limiting, not to mention awkward. That’s why for scripting now I’m using Python. Nonetheless though, I sometimes need a playing field to run the scripts for, or running console utilities. That’s where Console2 comes in.

Console2 is a great tool that provides tabbing (you can tab through multiple consoles) and several ways to customize the UI. Here’s an example:



I prefer this configuration:


How I Have It Setup



Download Console2:

I have the Console2 open up when I hit CTR+\ via AutoHotkey. Here’s the snippet for this to happen in AutoHotkey. I have this run when Window starts (it must stay in memory).

^\::Run C:\dan-local\dos-win\Console.exe

Change the CMD prompt by creating a PROMPT system variable:




Download the Console2 Config File

Other Tips


Autocomplete
Good tip about command console = CTRL+I
can select files in the current directory, kinda like autofill-in
you can also put in parts of the file and it autoputs it for you

Rename tab
CTRL+R

New Tab with default transparency
CTRL+F1

New Tab with lighter transparency
CTRL+F2

Switch to NEXT adjacent tab
CTRL+TAB

Switch to Previous adjacent tab
CTRL+SHIFT+TAB

CTRL+[NUMBER]
Goes to the tab instance. To make it easier, it’s best if you rename your tabs,

[INSTANCE NAME].[NAME OF TAB]

So for example,

1.Batch File
2.wget list
3.ftp the files

Posted on

Base Tag Not Supported In Hotmail

It’s interesting how the <base> works in various clients. I thought all major web clients would support it, but apparently it is not the case.

Let’s look at how some email clients process this tag:

Yahoo.com! Web Mail

Base tags are modified by their mail server and relative URLs are changed to absolute. So it works.

Gmail Web Mail

Base tags are modified by their mail server and relative URLs are changed to absolute. So it works.

Gmail Account via Thunderbird

Base tags are shown and its functionality is intact. URLs remain relative. So it works.

iMS Account via Outlook 2007

Base tags are shown and its functionality is intact. URLs remain relative. So it works. More info on iMS.

ShinyLight Account via iPhone

Can’t see the HTML, but all URLs worked.

Hotmail

Unfortunately, this is the ugly duckling. Hotmail servers remove the base tag and leave your relative links as relative. Big FAIL.

Posted on

Compress and Move Log Files

Sometimes log files bog a system down. For one of our servers, I made this little Python script that compresses (via WinRAR) the log files in a directory, and then moves them to a backup location. The only little catch is that I wanted to leave the latest log files for in that directory. Log files are created daily, so the the latest log files have a datestamp of today. Here’s how I did it.

First Create the Python Script:

import os
import datetime
 
dateStamp  = datetime.datetime.now().strftime("%Y-%m-%d") 
imsLogPath = 'd:\\LogFiles\\'                     
# Don't use a mapped drive but use UNC for network drives. Task Schedule seems to choke when it calls Python.
newRARPath = '"\\\\192.168.1.2\\Root\\backups\\' + dateStamp + '.rar"'
rarPath    = '"C:\\Program Files\\WinRAR\\rar.exe" a -m5 ' + newRARPath 
 
# Get Latest Files
smtpLatest   = os.popen(r"dir /od /a-d /b " + imsLogPath + "SMTP*.log").read().splitlines()[-1]
postLatest   = os.popen(r"dir /od /a-d /b " + imsLogPath + "POST*.log").read().splitlines()[-1]
ischedLatest = os.popen(r"dir /od /a-d /b " + imsLogPath + "iSched*.log").read().splitlines()[-1]
relayLatest  = os.popen(r"dir /od /a-d /b " + imsLogPath + "Relay*.log").read().splitlines()[-1]
qengLatest   = os.popen(r"dir /od /a-d /b " + imsLogPath + "Qeng*.log").read().splitlines()[-1]
 
# Get List of All Files
allFiles     = os.popen(r"dir /od /a-d /b " + imsLogPath + "*.log").read().splitlines()
 
# Remove Latest Files from All Files List
allFiles.remove( smtpLatest )
allFiles.remove( postLatest )
allFiles.remove( ischedLatest )
allFiles.remove( relayLatest )
allFiles.remove( qengLatest )
 
# allFiles Array Has the list of files
 
# Flatten Array allFiles to be used as a parameter in system command
flatLogPathList = ""
for filenameWithPath in allFiles:
  flatLogPathList = flatLogPathList + imsLogPath + filenameWithPath + " "
 
 
# Execute WinRar
path = rarPath + " " + flatLogPathList.rstrip()
os.system( '"' + path + '"' )
 
# Delete all log files
os.system( '"del ' + flatLogPathList.rstrip() + '"' )

Then I set up the Scheduled Task:

With these Settings:

Posted on

Remove Comments from IIS Logs

If you think that Log Parser is a bit on the slow side (i.e. if you’re dealing with big IIS logs) and you want to bulk import your logs into SQL Server, then you’ll have to remove # comments from the log files. Microsoft has the PrepWebLog Utility to do this, but it seems to choke for files that are > 100 MB. Also, you’ll have to write this as a batch file so it goes through a whole directory of files.

I wrote a Perl script that’s relatively fast (faster than PrepWebLog) and it can crawl folders/subfolders recursively. Here it is:

# parse.pl
# example: 
#   parse c:\temp\logs\logs*\*.log
#
# Requirement: no spaces in the directory names and file names.
# This gets called via run.bat. 
 
 
sub getFileList 
{    
    # This function returns an array of file list based on filter
    # This is the filter they can put in.       
    # Returns a file with full path. 
    # Example of filters: getFileList ( "*.log" );
    @files = <@_[0]>;
    return @files;    
}
 
 
sub remove_comments
{
  # Remove # pound sign comments from files. 
  # @_[0] = filename
 
  open (my $in, "<", @_[0] ) 
      or die "in: @_[0]";
 
  open (my $out, ">", "@_[0].txt") 
      or die "out: @_[0]";
 
  while( my $line = <$in>)
  {
      print $out $line
          unless $line =~ /^#/;
  }
 
  close $in;
  close $out;
}
 
 
########## MAIN #############
$arg = @ARGV[0];
 
# Location of root directory of logs files
#$arg = 'c:\temp\logs\logs*\*.log';
 
# Replace slashes
$arg =~ s/\\/\\\\/g;
 
# Loop through all the log files. 
for $file (getFileList ($arg))
{  
  print ( "Processing file $file ... \n" );    
  remove_comments( $file );  
}

The Perl script gets called via run.bat:

REM No spaces in directory and file names.
perl Parse.pl D:\statesites\W3SVC*\*.log
pause
Posted on

Outlook Emails/Contacts in “To” Dropdown

Needed to backup my contacts in Outlook, but it would not save the emails in the email “To” dropdown field, when autocomplete triggers. This is because it’s stored in Outlook’s NK2 cache file. This file is not in readable form, but you if you have the patience, you can view the emails if you don’t mind looking through funky ASCII characters of binary data. In the process of looking for and trying various tools, I realized that not all tools were able to extract all contacts. Keep this in mind.

There’s a few tools out there that you can use to extract a list of emails saved in the NK2 file. Here they are.

Ingressor – is the best program, I was able to extract the most amount of emails/contacts, but you have to pay for it. I think it extracts everything, but you can’t save anything or copy anything. Use OCR software?

Debunk2– It’s free, but it missed 1 (could possibly miss more on different NK2 files) contact.

NK2.info– It’s free, but it missed 2 contacts, and it doesn’t show the names accurately, like Debung2 and Ingressor.

Posted on

Automate FTP Uploads

I thought this might be useful for those who are constantly uploading the same files. Whether you need to deploy a project live or constantly upload the same group of files, you can do this by creating a batch file in Windows using ftp.exe.

First of all, if you’re not familiar with FTP.exe, it’s just a command console way to upload/download files from an FTP server. You can run this program and start typing away…. OR, you can load a text file filled with FTP commands in FTP.exe. For example, the following are commands in a file called upload.bs:

1
2
3
4
5
6
7
8
9

open western.dreamhost.com
username
password
binary
hash
send index.htm
send frame.htm
send car.jpg
bye

The above should be straightforward. The “open” command is used to connect to the FTP server. The “username” should be your usename and “password” should be your actual password. “binary” is how your data should be handled on transfer – if it’s not text, it’s binary data, and should be “binary.” “hash” makes the console display pound signs (#) to serve as a progress bar as it’s uploading to the server. The “send” commands are used to upload the files. “bye” is exits the FTP session.

That was just a FTP script session file. You now need to call that file from FTP.exe on WindowsXP. You do that as follows:

1

@%windir%\system32\ftp.exe -s:upload.bs

You can include that call in a file called “upload.bat” – To learn more about FTP scripting and how to merge the two files into just one batch file, you can check out this site. Also, there’s more information about this found here: tinyapps.org.