Importing Excel Data into SQL Server

Sometimes it can be a pain. Why? Because SSMS (or should I say SSIS) uses the Windows Excel driver, and it infers data types of the columns incorrectly, and will choke when you import. Sometimes it will work, sometimes not. So you have to import it as text. However, first you have to re-save the Excel file as a CSV (which will by default use commas as delimiters, and quotes for qualifiers).

So…

Step 1:
In Excel, save the file as a CSV, in this case, file.csv.

Step 2:
In SSMS, under the Import wizard, choose “Flat File Source:

Step 3:
Leave everything default, except use a one quote in the qualifier textbox:

STEP 4:
In the Advanced section, select all the columns and make sure the OutputColumnWidth is something big enough, larger than the size of the largest column. In this case, we’ll make it 150. Make sure the DataType is string [DT_STR]

Step 5:
Leave the rest of the settings except the target database/table where you’re importing, and Finish the import.

Step 6:
Done.

Better SSMS Objects Management with SQLTreeo

This is so cool and handy. You can hide and organize database objects (procs, tables, views, databases) in folders. This organization is saved locally to your machine, on a the server level, via extended properties. If you decide to save the info on the server, someone else who has the extension will view it organized accordingly.

Check it out.

Search For Column Name in All Databases

Whipped up this little script today to search for a column in all my databases.

--------------------------------------------------------------------------------------------------
-- Search for the column name
--------------------------------------------------------------------------------------------------
DECLARE @searchFor AS VARCHAR(50)     = 'CustomerID' -- keyword to search for
DECLARE @databaseList AS VARCHAR(MAX) = 'Database1,Database2,Database3,Database4' -- comma delimited
 
DECLARE @orderBy AS VARCHAR(50)       = 'DateCreated ASC' -- You can also append a space and the ASC/DESC
-- Order By One of These
-- ObjectID, ColumnName, Table, Database, Date Created, Modified Date
--------------------------------------------------------------------------------------------------
 
DECLARE @sql NVARCHAR(MAX) = ''
-- loop
SET @databaseList = @databaseList + ','
DECLARE @pos AS INT
DECLARE @val AS VARCHAR(255)
 
WHILE PATINDEX('%,%', @databaseList) <> 0
BEGIN       
  SELECT  @pos = PATINDEX('%,%', @databaseList)
  SELECT  @val = LEFT(@databaseList, @pos - 1)       
  SELECT  @databaseList = STUFF(@databaseList, 1, @pos, NULL)
  SET @sql = @sql + 
             'SELECT  *
              FROM    ( SELECT  portalColumns.OBJECT_ID AS ObjectID ,
                                portalColumns.[NAME] ColumnName ,
                                ''[' + @val + '].dbo.['' + portalTables.[NAME] + '']'' AS [Table] ,
                                ''[' + @val + ']'' AS ''DatabaseName'',                  
                                create_date AS ''DateCreated'' ,
                                modify_date AS ''ModifiedDate''
                        FROM    [' + @val + '].sys.columns portalColumns
                                INNER JOIN [' + @val + '].sys.tables portalTables ON portalColumns.object_id = portalTables.object_id
                        WHERE   portalColumns.name LIKE ''%' + @searchFor + '%''
                      ) AS Results
              UNION '                       
END
 
SET @sql = LEFT(@sql, LEN(@sql) - 5) + ' ORDER BY ' + @orderBy
 
--PRINT @sql    
 
EXECUTE sp_executesql  @sql

Dapper – A C# Micro ORM Library Makes Data Access a Snap – Part 1

I’ve recently gotten a chance to try out Dapper – a micro ORM library by Sam Saffron, which originally spawned from StackOverflow’s codeset. If you’re looking for an extremely light-weight library to access your SQL Server, I highly recommend it. It focuses more on convention than configuration. It also does not have the overhead that EntityFramework or NHibernate have, and it emphasizes speed and ease-of-use.

Dapper is a collection of Extension Methods that enhance the Connection object (or rather extends the IDbConnection interface). It also embraces the use of inline SQL in your code, which is useful when retrieving sets that are less trivial. With its buddy, Dapper-Extensions by Thad Smith, it makes DB interaction even easier.

Both of these libraries can be downloaded from NuGet, and you’ll need them to run the following code.

There’s really just one thing that I found irritating about the Dapper project: the documentation. While there is documentation found at its Google Code home page, I find it very lacking.

Anywhoot, to get started first we need to create our table. Once we have that and add some data, we’ll use the Dapper to extract the data and map to objects.

----------------------------------------
-- Customer Schema
----------------------------------------
CREATE TABLE [dbo].[Customer](
[CustomerID] [uniqueidentifier] NOT NULL,
[FirstName] [nvarchar](50) NULL,
[LastName] [nvarchar](50) NULL,
[Age] [int] NULL,
[IsAllowed] [bit] NULL,
[DateRegistered] [datetime] NULL,
[AdditionalDetails] [nvarchar](max) NULL,
CONSTRAINT [PK_Customer_1] PRIMARY KEY CLUSTERED
(
[CustomerID] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
 
ALTER TABLE [dbo].[Customer] ADD  CONSTRAINT [DF_Customer_CustomerID]  DEFAULT (newid()) FOR [CustomerID]
GO
 
----------------------------------------
-- Data
----------------------------------------
INSERT INTO [dbo].[Customer]([CustomerID], [FirstName], [LastName], [Age], [IsAllowed], [DateRegistered], [AdditionalDetails])
  SELECT N'5458dff8-cea2-4bdb-9431-1dea56f109f8', N'Bruce', N'Wayne', 31, 1, '20080604 08:13:44.000', N'I am Batman' UNION ALL
  SELECT N'1767bc57-af03-4b17-891f-2aa9af244180', N'Peter', N'Parker', 25, 1, '20120103 12:22:28.000', N'I am Spider-Man' UNION ALL
  SELECT N'dc2bf42d-045b-4189-bd6b-8bf1bf120291', N'Eddie', N'Brock', 34, 0, '20040123 02:45:41.000', N'We are Venom'

OK, so this is what we have so far:

Because Dapper focuses more on convention over configuration, it’s important to note that when you create your class:

  • The name of your class must match the name of the table, otherwise an exception is thrown.
  • All column names must match all class property names, otherwise an exception is thrown.
  • If a column is not used (in NOT written out in the SELECT statement) and the class property exists, it will set to NULL or assign a default value – more on this later.

Now that we’re aware of these points, let’s see the code to get started:

using System;
using Dapper;
using System.Data.SqlClient;
 
class Customer
{
  public Guid CustomerID { get; set; }
  public string FirstName { get; set; }
  public string LastName { get; set; }
  public int Age { get; set; }
  public bool IsAllowed { get; set; }
  public DateTime DateRegistered { get; set; }
  public string AdditionalDetails { get; set; }
}
 
class Program
{
  private static void Main()
  {
 
    using (SqlConnection conn = new SqlConnection("Data Source=NARUTO;Initial Catalog=TESTDATABASE;Integrated Security=True"))
    {
      conn.Open();
 
      // Put dapper code here.
 
      conn.Close();
    }
 
    Console.ReadLine();
  }
}

OK, so there’s nothing out of the ordinary here. We’re creating the class we want to map our data to, creating a Connection object, opening, closing, and then disposing (via using). Now let’s suck up some data within our Main() method:

private static void Main()
{
  using (SqlConnection conn = new SqlConnection("Data Source=NARUTO;Initial Catalog=TESTDATABASE;Integrated Security=True"))
  {
    conn.Open();
 
    IEnumerable customers = conn.Query("SELECT * FROM Customer");
 
    foreach (Customer c in customers)
    {
      Console.WriteLine("CustomerID: " + c.CustomerID);
      Console.WriteLine("FirstName: " + c.FirstName);
      Console.WriteLine("LastName: " + c.LastName);
      Console.WriteLine("Age: " + c.Age);
      Console.WriteLine("IsAllowed: " + c.IsAllowed);
      Console.WriteLine("DateRegistered: " + c.DateRegistered);
      Console.WriteLine("AdditionalDetails: " + c.AdditionalDetails);
      Console.WriteLine("-----------------------------------------------");
    }
 
    conn.Close();
  }
 
  Console.ReadLine();
}

As we see, the Query extension method pulls in the data based on the select statements. The Query method returns a strongly-typed list of Customer. You should see the following:

Now what if we want to get the number of records in the table? Well, because Query returns an IEnumerable, we cannot use the Count property. To do so, we convert to a List:

List<Customers> customers = (List<Customers>)conn.Query("SELECT * FROM Customer");
Console.WriteLine(customers.Count);

That should give us the count. To see how the data types were mapped, we can take a look at the Dapper source code, SqlMapper.cs inside the static method SqlMapper(). For reference, you can take a peek at these two sources to understand SQL Server/CLR/.NET framework data types:

http://stackoverflow.com/questions/425389/c-sharp-equivalent-of-sql-server-2005-datatypes

http://msdn.microsoft.com/en-us/library/ms131092.aspx

For our example though, we saw that the data types were mapped:

Mismatches and Nulls

In the query that we ran, we were making the assumption that all columns in the db match all the properties in the class. What if we have a mismatch? What if we have this?

IEnumerable customers = conn.Query("SELECT LastName FROM Customer");

Notice that we’re only retrieving the LastName, while we have other properties in our Customer class. Well, this is what you would see:

So from the above, we notice that Dapper will handle Nulls by:

  • Setting strings as Nulls
  • Setting a Guid to 00000000-0000-0000-0000-000000000000
  • Setting an int to 0
  • Setting bool to false
  • Setting a DateTime to 1/1/0001 12:00:00 AM

Which will all happen when Dapper cannot find a class property to map to from the column value. Also, if you column name that DOES NOT match a property, it will throw a SqlException of “Invalid Column Name”.

Parameterization

What if we want to pass a parameter into our SQL? Let’s pass a parameter:

IEnumerable customers = conn.Query("SELECT * FROM Customer WHERE AGE &gt; @Age", new { Age = 30 } );

Notice that I’m passing in an anonymous object. I could also have done:

Customer someCustomer = new Customer {
Age = 33,
FirstName = "Clark Kent"
};
 
IEnumerable customers = conn.Query("SELECT * FROM Customer WHERE AGE &gt; @Age", someCustomer );

Which yields:

The property in someCustomer (FirstName = “Clark Kent”) is silently ignored because it’s not used in the SELECT statement.

When the above is run, SQL Server Profiler shows that the following is run:

exec sp_executesql N'SELECT * FROM Customer WHERE AGE &gt; @Age',N'@Age int',@Age=33

So we know it’s parameterized.

To call a stored procedure and map it to objects, we do, assuming we have the following sproc:

CREATE PROCEDURE GetCustomers
(
@IsAllowed BIT
)
AS
BEGIN
 
SELECT * FROM Customer
WHERE IsAllowed = @IsAllowed
 
END

We do:

// The second @ token does not have to be named "@IsAllowed" and can be named anything as
// long as the C# property in the anonymous object is named the same thing.
IEnumerable customers = conn.Query("GetCustomers @IsAllowed = @IsAllowed", new { IsAllowed = 33 } );

Also accepted:

IEnumerable customers = conn.Query("GetCustomers @IsAllowed = @Allow", new { Allow = 33 } );

Or you can be more explicit by specifying the command type (StoredProcedure):

IEnumerable customers = conn.Query("GetCustomers", new { IsAllowed = 33 }, commandType: CommandType.StoredProcedure );

Or writing it a more secure way:

DynamicParameters parameters = new DynamicParameters();
parameters.Add("@IsAllowed", 'f', dbType:DbType.Int32);
 
IEnumerable customers = conn.Query("GetCustomers", parameters, commandType: CommandType.StoredProcedure );

To insert data, we do as follows:

Customer customer = new Customer{
  FirstName = "Mazinger",
  LastName = "Z",
  Age = 40,
  IsAllowed = true,
  DateRegistered = DateTime.Now,
  AdditionalDetails = "Metallic defender"
};
 
conn.Execute(@"INSERT INTO Customer
                                  ( FirstName ,
                                  LastName ,
                                  Age ,
                                  IsAllowed ,
                                  DateRegistered ,
                                  AdditionalDetails
 
                   )
                   VALUES  ( @FirstName ,
                                 @LastName ,
                                 @Age ,
                                 @IsAllowed ,
                                 @DateRegistered ,
                                 @AdditionalDetails
                               )"
                 , customer);

This ends part 1 of the tutorial. I will post the 2nd part soon.

msdb.dbo.sp_sqlagent_get_perf_counters high CPU

That stored proc generates server stats, and depending on the stat results, it will fire off the alerts in SQL Server. If that instance of that stored proc hangs or has abnormally high CPU, just kill it. The agent will fire it again when needed.

Also, by default, SQL Server comes with a set of alerts that the server can use (called, “Demo”). I would keep the alerts active if possible and see what alerts are being triggered and why – check logs.

There was also a bug on SQL Server 2008 that was fixed that stemmed from this proc causing high CPU, even if all Alerts are disabled. I’m not sure if this fix applied to SQL Server 2000 with latest SP.

More details: http://connect.microsoft.com/SQLServer/feedback/details/536354/execute-msdb-dbo-sp-sqlagent-get-perf-counters-hogs-cpu-although-no-alerts-enabled#details

Getting Started with the MongoDB C# Driver

Here’s my attempt on writing a super-barebones tutorial for using the MongoDB C# Driver. It’s a very hands-on tutorial-by-example, showing you the basics of the driver by creating console apps.

I’m assuming you’ve installed the official MongoDB C# driver and read the intro. Heads up on how you set up the driver. The .msi sets it up in the GAC. The zip file has the assemblies that can just be referenced in your project.

After you’ve done those two things, you can proceed. 🙂

Dependencies

In projects, you have the option of utilizing the following namespaces:

using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Bson.IO;
using MongoDB.Bson.Serialization;
using MongoDB.Bson.Serialization.Attributes;
using MongoDB.Bson.Serialization.Conventions;
using MongoDB.Bson.Serialization.IdGenerators;
using MongoDB.Bson.Serialization.Options;
using MongoDB.Bson.Serialization.Serializers;
using MongoDB.Driver.Builders;
using MongoDB.Driver.GridFS;
using MongoDB.Driver.Wrappers;

The most commonly used are Bson, Driver, Serialization, and Serialization.Attributes. This tutorial will use these four in all code samples, so make sure you include them in your app, even if I don’t have it in the code samples. You can learn more about the API/namespaces by checking out the docs.

Connecting to a MongoDB Database

Connecting is pretty simple. Once you include the C# driver assemblies, you connect as follows:

namespace ConsoleApplication1
{
  class Program
  {
    static void Main(string[] args)
    {
      MongoServer  server = MongoServer.Create("mongodb://work2");
      MongoDatabase Cars = server.GetDatabase("Cars");
    }
  }
}

That’s pretty much all you need to connect. Of course, I recommend putting the connection string in the Web.config, or some initializer.

Inserting a document on MongoDB

There are tons of ways to do this, but one of the simplest ways is to create a class, create an instances of that class, and pass it into a MongoDB object that serializes that object into BSON and submits it to your database.

So for this example, before inserting a document into the database from C#, one has to do two things (aside from the establishing a connection and choosing which db to write to):

  1. Create a Generic Collection object. In this case, it’s a MongoDB collection.
  2. Create a class that’s to be used by the Generic Collection.

Those two go together.

Also, note that while working with MongoDB in C#, the word “collection” may be thrown around loosely. In MongoDB you have a “Collection” that’s comparable to a Table in a traditional ER database. A MongoDB “Collection” holds “documents,” which is comparable to a “record” in a traditional ER database. On the opposite side, in C#, a collection is a data structure that unlike an Array, can be resized dynamically and is more flexible as far as functionality.

The MongoDB Generic Collection is a C# collection that can hold any type of object, but it has to know what type of object it’s holding. You let it know what kind of type by creating a class, and then when you create a collection, you specify the name of the class. So for example, this is the syntax for creating a Generic Collection that has Car objects (where person is defined in a class):

After that line of code, you populate the vehicles object (remember it’s a MongoDB Generic collection that holds Cars objects), by calling the method “GetCollection” from the Cars object we creating in the last section “Connecting to a MongoDB database”:

In the above statement, if MongoDB does not find a collection on the database called “vehicles,” it will create it for you.

What does that Cars class look like? Pretty simple:

  public class Cars
  {
    public string _id { get; set; }
    public string brand { get; set; }
    public string lname { get; set; }
    public Int32 miles { get; set; }
    public int age { get; set; }
  }

Then of course, before sending it to the MongoDB database, you create an object and set the values of the object:

      Cars newCar = new Cars();
      newCar._id = System.Guid.NewGuid().ToString();
      newCar.brand = "Honda";
      newCar.miles = 186000;
      newCar.age = 1;

And then insert it in MongoDB:

vehicles.Insert(newCar);

To put it in perspective, here’s the entire program:

namespace ConsoleApplication1
{
  public class Cars
  {
    public string _id { get; set; }
    public string brand { get; set; }
    public string lname { get; set; }
    public Int32 miles { get; set; }
    public int age { get; set; }
  }
 
  class Program
  {
    static void Main(string[] args)
    {       
 
      MongoServer  server = MongoServer.Create("mongodb://work2");
      MongoDatabase Cars = server.GetDatabase("Cars");
 
      MongoCollection vehicles;
 
      vehicles = Cars.GetCollection("vehicles");
 
      Cars newCar = new Cars();
      newCar._id = System.Guid.NewGuid().ToString();
      newCar.brand = "Honda";
      newCar.miles = 186000;
      newCar.age = 1;      
 
      vehicles.Insert(newCar);
    }
  }
}

The insert method should serialize the Cars object (newCar) and create the MongoDB document, mapping the properties of the Cars class to MongoDB document key-pair values.

Let’s see how this got created in our database:

Run:

db.vehicles.find()

(e.g. ignore the lname property. This was removed in the source code)

Inserting Complex Objects in MongoDB

Let’s try inserting an object with a more complex data structure. Let’s assume we have the following class:

  public class Cars
  {
    public string _id { get; set; }
    public string brand { get; set; }
    public string lname { get; set; }
    public Int32 miles { get; set; }
    public int age { get; set; }
 
    public Hashtable history { get; set; }
    public Hashtable complex { get; set; }
    public int[] magicNumbers { get; set; }
  }

Now let’s declare the class and its properties:

      newCar._id = System.Guid.NewGuid().ToString();
      newCar.brand = "Honda";
      newCar.miles = 186000;
      newCar.age = 1;
 
      newCar.history = new Hashtable();   
      newCar.history.Add("2010-08-14", "Puss 'N Boots");
      newCar.history.Add("2011-12-01", "Puff the Magic Dragon");
      newCar.history.Add("2009-03-07", "Fraggle Rock");      
 
      newCar.magicNumbers = new int[3];
      newCar.magicNumbers[0] = 60;
      newCar.magicNumbers[1] = 186000;
      newCar.magicNumbers[2] = 1440;
 
      newCar.complex = new Hashtable();
      newCar.complex.Add( newCar.magicNumbers[1], newCar.history );
      newCar.complex.Add( System.Guid.NewGuid().ToString(), newCar.history);

It outputs the following (formatted for better reading):

{
  "_id": "a07513c8-9cb5-46cd-a9cb-425cabc0dd76",
  "brand": "Honda",
  "lname": null,
  "miles": 186000,
  "age": 1,
  "history": 
  {
    "2010-08-14": "Puss 'N Boots",
    "2009-03-07": "Fraggle Rock",
    "2011-12-01": "Puff the Magic Dragon"
  },
  "complex": 
  [
    ["eb7b3d4d-87b1-43c1-9fe7-5a5e4b5b79de",
    {
        "2010-08-14": "Puss 'N Boots",
        "2009-03-07": "Fraggle Rock",
        "2011-12-01": "Puff the Magic Dragon"
    }],
    [
    186000,
    {
        "2010-08-14": "Puss 'N Boots",
        "2009-03-07": "Fraggle Rock",
        "2011-12-01": "Puff the Magic Dragon"
    }]
  ],
  "magicNumbers": [60, 186000, 1440]
}

Now let’s look at the entire code for all this:

namespace ConsoleApplication1
{
  public class Cars
  {     
    public string _id { get; set; }   
    public string brand { get; set; }  
    public string lname { get; set; }  
    public Int32 miles { get; set; }  
    public int age { get; set; }
 
    public Hashtable history { get; set; }
    public Hashtable complex { get; set; }
    public int[] magicNumbers { get; set; }
  }
 
  class Program
  {   
    static void Main(string[] args)
    {
 
      MongoServer  server = MongoServer.Create("mongodb://work2");          
      MongoDatabase Cars = server.GetDatabase("Cars");
 
      MongoCollection<Cars> vehicles;
 
      vehicles = Cars.GetCollection<Cars>("vehicles");
 
      Cars newCar = new Cars();
 
      newCar._id = System.Guid.NewGuid().ToString();
      newCar.brand = "Honda";
      newCar.miles = 186000;
      newCar.age = 1;
 
      newCar.history = new Hashtable();   
      newCar.history.Add("2010-08-14", "Puss 'N Boots");
      newCar.history.Add("2011-12-01", "Puff the Magic Dragon");
      newCar.history.Add("2009-03-07", "Fraggle Rock");      
 
      newCar.magicNumbers = new int[3];
      newCar.magicNumbers[0] = 60;
      newCar.magicNumbers[1] = 186000;
      newCar.magicNumbers[2] = 1440;
 
      newCar.complex = new Hashtable();
      newCar.complex.Add( newCar.magicNumbers[1], newCar.history );
      newCar.complex.Add( System.Guid.NewGuid().ToString(), newCar.history); 
 
      vehicles.Insert<Cars>(newCar);         
 
      Console.ReadLine();                
    }     
  }
}

Reading the 1st MongoDB document from the database

Before you query the database, you must make sure the class and member types match the values/objects that the MongoDB connection will return.

Let’s try to fetch the 1st document. To see what we’re expecting, let’s fetch it via the console first:

db.vehicles.find().limit(1)

which returns:

{
  "_id" : ObjectId("4d95f30058430000007a5943"),
  "ObjectId" : ObjectId("000000000000000000000000"),
  "brand" : "Ford",
  "lname" : null,
  "miles" : 2500,
  "age" : 1
}

Now that we know what it returns, let’s make sure we have the class in place that will handle the kind of key-value pairs in the MongoDB collection:

  public class Cars
  {
    public MongoDB.Bson.ObjectId _id { get; set; }
    public MongoDB.Bson.ObjectId ObjectId { get; set; }
    public string brand { get; set; }
    public string lname { get; set; }
    public Int32 miles { get; set; }
    public int age { get; set; }
 
    // Complex Properties
    public Hashtable history { get; set; }
    public Hashtable complex { get; set; }
    public int[] magicNumbers { get; set; }
  }

Notice that the properties history, complex, and magicNumbers are not part of the MongoDB document we queried. That’s OK. The MongoDB C# drivers are smart enough to map to to a null value when deserializing from MongoDB to a C# object.

Now how do you map values from a MongoDB connection to a C# object? Like so:

    Cars car = vehicles.FindOne();

The FindOne() method will return the first document. Now let’s add some context to this line of code, by adding MongoDB connection code (that we established before), writing it to the screen, and checking for nulls in the complex properties:

namespace ConsoleApplication1
{
  public class Cars
  {
    public MongoDB.Bson.ObjectId _id { get; set; }
    public MongoDB.Bson.ObjectId ObjectId { get; set; }
    public string brand { get; set; }
    public string lname { get; set; }
    public Int32 miles { get; set; }
    public int age { get; set; }
 
    // Complex Properties
    public Hashtable history { get; set; }
    public Hashtable complex { get; set; }
    public int[] magicNumbers { get; set; }
  }
 
  class Program
  {
    static void Main(string[] args)
    {
      MongoServer  server = MongoServer.Create("mongodb://work2");
      MongoDatabase Cars = server.GetDatabase("Cars");
 
      MongoCollection vehicles;
      vehicles = Cars.GetCollection("vehicles");
 
      Cars car = vehicles.FindOne();
 
      Console.WriteLine(car._id);
      Console.WriteLine(car.ObjectId);
      Console.WriteLine(car.brand);
      Console.WriteLine(car.lname);
      Console.WriteLine(car.miles);
      Console.WriteLine(car.age);
 
      if (car.history == null)
        Console.WriteLine("history was null");
      else
        Console.WriteLine(car.history);
 
      if(car.complex == null)
        Console.WriteLine("complex was null");
      else
        Console.WriteLine(car.complex);
 
      if (car.magicNumbers == null)
        Console.WriteLine("magicNumbers was null");
      else
        Console.WriteLine(car.magicNumbers);                  
 
      Console.WriteLine("ha made it this far!");
      Console.ReadLine();
    }
  }
}

This will print the following on to the screen:

Read All Documents from a MongoDB Collection

Let’s try to output all the documents from a collection. We’re going to create a new MongoDB collection under the Cars database. Through the MongoDB client, we’ve created four documents. Let’s read them from the console first:

db.bankaccount.find()

The four documents look like:

{ "_id" : ObjectId("4d9650301b6e000000004116"), "name" : "Batman" }
{ "_id" : ObjectId("4d9650391b6e000000004117"), "name" : "Spider-Man" }
{ "_id" : ObjectId("4d96503f1b6e000000004118"), "name" : "Superman" }
{ "_id" : ObjectId("4d9650431b6e000000004119"), "name" : "Spawn" }

So the first thing is to create a class that represents each MongoDB document:

public class BankAccount
  {
    public MongoDB.Bson.ObjectId _id;
    public string name;
  }

The class is used to create an object that will contain the data that is deserialized (from MongoDB to a C# object).

Here’s the code to query all documents, iterating through them and showing them on the screen:

      MongoCursor cursor = BankAccountCollection.FindAll();
 
      foreach (BankAccount ba in cursor)
      {
        Console.WriteLine(ba._id);
        Console.WriteLine(ba.name);
        Console.WriteLine("=============================");
      }

Looks straightforward enough. What is a cursor? We can think of a MongoDB Cursor as a DataSet in .NET. Later, we’ll see MongoDB Queries, which resemble a SqlCommand object in .NET. Queries are configurable objects to search for document data. The Query object is then passed into the a find method (there are several) of the MongoCollection. The find method then typically returns a Cursor, which is then iterated through. (More about this later.) Also note that a Cursor is not “populated” with documents until the there’s an attempt to retrieve the first result from the server.

Let’s see the entire program in action.

namespace ConsoleApplication1
{
  public class BankAccount
  {
    public MongoDB.Bson.ObjectId _id;
    public string name;
  }
 
  class Program
  {
    static void Main(string[] args)
    {
      MongoServer  server = MongoServer.Create("mongodb://work2");
      MongoDatabase BankAccountDatabase = server.GetDatabase("Cars");
 
      MongoCollection BankAccountCollection;
      BankAccountCollection = BankAccountDatabase.GetCollection("bankaccount");
 
      MongoCursor cursor = BankAccountCollection.FindAll();
 
      foreach (BankAccount ba in cursor)
      {
        Console.WriteLine(ba._id);
        Console.WriteLine(ba.name);
        Console.WriteLine("=============================");
      }      
 
      Console.WriteLine("\n\nha made it this far!");
      Console.ReadLine();
    }
  }
}

The above program outputs the following:

If we wanted to retrieve a specific query, rather than doing this (from the above code):

 MongoCursor cursor = BankAccountCollection.FindAll();

We would do:

      var query = Query.EQ("name", "Batman");
      var cursor = BankAccountCollection.Find(query);

One last thing to mention about the Find() method, as stated from docs:
The Find method (and its variations) don’t immediately return the actual results of a query. Instead they return a cursor that can be enumerated to retrieve the results of the query. The query isn’t actually sent to the server until we attempt to retrieve the first result (technically, when MoveNext is called for the first time on the enumerator returned by GetEnumerator). This means that we can control the results of the query in interesting ways by modifying the cursor before fetching the results.

Get only key-value pairs from a collection

In this example, we’re going to try to do something like this in SQL:

SELECT Column1, Column2
FROM  TABLE

We’re also going to be using the collection “bbu”. Let’s try to translate this query:

db.bbu.find({},{whichAPI:1}).limit(10)

Which would return the following:

{ "_id" : ObjectId("4d90fbd85843000000004359"), "whichAPI" : "SL" }
{ "_id" : ObjectId("4d90fbd8584300000000435a"), "whichAPI" : "SL" }
{ "_id" : ObjectId("4d90fbd8584300000000435b"), "whichAPI" : "SIT" }
{ "_id" : ObjectId("4d90fbd8584300000000435c"), "whichAPI" : "SL" }
{ "_id" : ObjectId("4d90fbd8584300000000435d"), "whichAPI" : "SL" }

First, we’re going to define the class. We’re going to establish all the properties that are needed for a class, but then only query and populate two colums from the class.

Define the class and all of its properties.

public class BBU
  {
    public MongoDB.Bson.ObjectId _id;
    public int counter;
    public string whichAPI;
  }

Now the actual program:

  class Program
  {
    static void Main(string[] args)
    {    
 
      MongoServer  server = MongoServer.Create("mongodb://work2");
      MongoDatabase BBUDatabase = server.GetDatabase("bbu");
 
      MongoCollection BBUCollection;
      BBUCollection = BBUDatabase.GetCollection("bbu");
 
      MongoCursor cursor = BBUCollection.FindAll().SetFields("id","whichAPI").SetLimit(5);
 
      foreach (BBU ba in cursor)
      {
        Console.WriteLine(ba._id + "----" + ba.whichAPI);
        Console.WriteLine("=============================");
      }      
 
      Console.WriteLine("\n\nha made it this far!");
      Console.ReadLine();                        
 
    }
  }

In the example above, SetLimit(5) will only return the top 5 documents.

Conclusion

I hope you got a good a sense of the basics of using the MongoDB C# driver. I’ve only scratched the surface of the API. I’ll be discussing more advanced topics in future posts.

For the meantime, I point you to the MongoDB Google Group: http://groups.google.com/group/mongodb-user?pli=1 where you’ll get informative responses, usually very quickly.

I also recommend you following the following folks on Twitter and these blogs/sites:

Lastly, I recommend you check out the MongoDB video presentations using the C# Driver and working in a Microsoft environment.

SSMS Template Explorer (Snippets)

I sometimes see people miss this. A very resourceful feature is the Template Explorer in SQL Server Management Studio. Here’s how to take advantage of it:

1. First you show the panel in SQL Server:

2. Double click on the snippet/template. We’ll double click on the Create Multi-statement table-value Function:

3. After you do so, it’ll open up a new query window with the source of the snippet.

4. Do CTRL+SHIFT+M to fill in the parameters:

Checklist to Consider Before Migrating from SQL Server to NoSQL

Here’s a checklist of items to consider before migrating to a NoSQL db from SQL Server. If you come from SQL Server land, you may be used to these features and facilities. So before shifting over to a different world, consider if the db you’re jumping into (MongoDB, CouchDB, etc.) meets your needs by analyzing these items. Keep in mind that some features may not exist in NoSQL, not because of immaturity, but because it may not apply to the nature of a NoSQL database.

Handle Load

Compare against SQL Server:

Reading

Writing

Resource Monitoring

CPU Usage

Memory Use

Disk use

Transactions

Are there software design patterns to rollback

Locking

Reading

Programming

Syntax

SELECT / UPDATE / DELETE / INSERT / JOINS

Reusable Code Modules

Compiled Stored procs

Functions

Custom Data Types

Dynamic Management Views

Indexing

Clustered

Non-Clustered

Toolset / Resources

IDE

Administration

Profiler

Excution Planner

Community

Books

Administration

Complexity

Backing up

Full

Differentials

Transaction Log

Recovery

Replication / Clustering

Snapshots

Security

C# Integration

Facilities

Full Text Search

Snapshots

Kooboo CMS First Impressions

Kooboo is a CMS based on ASP.NET MVC. Recently, I got a chance to take it for a spin, and here are some of my thoughts. Keep in mind that the drawbacks here may come just from my ignorance of the tool use. 🙂 I’ll update this as the more I learn about the inner workings.

Benefits

  • Admin Panel’s UI is intuitive for designers/programmers
    • Easy to add pages
    • Easy to add your own themes/styles
    • Easy to create your own type of content
    • Easy to add content
  • Lots of Features, more than Orchard
  • Mature, has been around for a while (2008)
  • Views are coded in Razor
  • Can connect to MongoDB and other datasource types
  • Versioning of any piece of content and view differences
  • Manage website resources easily – images/documents/etc.

Drawbacks

May not be a drawback once I figure out the “how” and get better understanding.

  • Once a site is created, when I migrated from XML to MongoDB, I lost all the website data from the XML files.
  • Admin Panel’s UI may not be intuitive to non-designers/programmers.
  • Site directory structure Kooboo generates is not the same as the traditional ASP.NET MVC.

Let’s take a look. For a site I created using Kooboo, named “batman”:

  • When a content type is created, it does not create a C# class file. (I didn’t see one at least, in the directory structure.) It does, however, create a MongoDB collection for the content, there’s just no C# class mapped to it.
  • There’s no clear way to bind a View to a model class as in traditional ASP.NET MVC since, Kooboo doesn’t create a C# class file. It doesn’t follow the traditional file/folder naming convention: for each View, you map that to a model.
  • Community not as large as other CMS communities (Orchard, Umbraco, DNN).

I’ll keep exploring, but this is what I’ve found so far.

Kooboo Setup Notes

I had to apply these settings to make Kooboo work on my machine connecting to MongoDB on my VM:

In Web.config:

For the MongoDB.config (when you follow the instructions to set it up http://www.kooboo.com/Documents/Detail/CMS/v3/MSSQL-SQLCE-RavenDB-and-MongoDB ), I copied it to:

Kooboo_CMS/

And you may have to copy it to C:\kooboo\Kooboo_CMS\Kooboo_CMS\bin as well. (my installation was a little flaky)

You’ll have to build the project (in VS2010) if you already ran the site (the default data store is XML files).

When all is set and done, make sure that the db was created on MongoDB. In this case, it created the db samplesite.