As you may have noticed, it is not so easy to upload documents in bulk into SharePoint 2010. In fact it was quite a bit better in SharePoint 2007. Let’s take a look at our options

1.   SharePoint 2007 Multiple Upload: The Upload Multiple Files ActiveX control worked quite well and had good performance. But it did not traverse subdirectories.

2.   SharePoint 2010 Multiple Upload: In SharePoint 2010 this has been replaced with a Silverlight control, which while has interface advantages with drag and drop does limit you to only 100 documents of folders at a time. So now what happens if you want to upload an entire directory of documents with many subfolders?

3.   Windows Explorer View: This option allows you to upload series of documents and nested subfolders into SharePoint. As nice as this option may be, the performance is really hit or miss.

4.   SharePoint Workspace: This is actually one of my favorite options. You’ve got your drag and drop functionality same as with Silverlight. The performance is excellent and highly optimized to your internet connection. The limitations are in the amount of documents though. They don’t officially support more than 500 documents.

So here I am thinking, SharePoint Workspace is the winner. However, I tried to drag a folder with almost 2GB of content with 4000 documents and 600 folders. Let’s just say the result was disastrous. Workspace locked up, I gave it the benefit of the doubt however. After 20 minutes of a completely frozen application, I decided to see if anything was going on in SharePoint land. Indeed the documents were being upload, but slooooowly. So it was at this point that I opted for a custom solution.

Lucky for me, SharePoint 2010 now includes a very rich and powerful .NET Client Object Model. So all I needed to do was write a windows forms application that used this API to traverse the local files and folders and upload them into SharePoint. There actual code was pretty minimal but I did want to make sure I had an application that was both performant and didn’t freeze the application. This involved basically calling all the SharePoint code from a separate thread and updating the user interface with BeginInvoke(). Here is how I did it:

Building the User Interface

The first step was dragging out all the controls I would need. A button for Import, and text boxes for the parameters plus a label for status.

Starting the thread

Once the user hits the Import button, I didn’t want the application to freeze up. So I started a new thread called dobutton1work()

Thread t = new Thread(new ThreadStart(dobutton1work));

t.IsBackground = true;

t.Start(); 

Opening the site

The next step is actually opening up the site. This is a combination of the base server URL and the site relative URL which is what the ClientContext object expects. Then I initialize the web and list objects, load the folders of the library and run ExecuteQuery(). This is necessary because the way Client Object model differs mainly from traditional object model is commands and objects need to be explicitly requested, queued up, and then executed. The initial execution of the ClientContext will be the slowest part of the application. But once this is done, all the other methods run pretty quickly.

clientContext = new ClientContext(serverurl + siterelativeurl);

web = clientContext.Web;

oList = web.Lists.GetByTitle(libraryname);

clientContext.Load(web);

clientContext.Load(oList);

clientContext.ExecuteQuery();

folders = web.Folders;

clientContext.Load(folders);

clientContext.ExecuteQuery();

Update the UI

In Windows Forms, the user interface actually runs on its own thread and doesn’t allow other threads to make changes. To get around this, you must call the BeginInvoke method on a specific UI control and then call Thread.Sleep(). I refactored this into its own method:

private void updateLabel(string message)

{

    label1.BeginInvoke(new MethodInvoker(() => label1.Text = message));

    Thread.Sleep(1);

}

Upload the Documents

Uploading the documents itself was done using a recursive function call uploadDocuments which tooks a local disk path as a parameter. I build out the current path and translate it to a path which can be used by SharePoint. The actual upload is done using a function called SaveBinaryDirect() and the function’s performance is excellent. After doing the file uploads, I iterate through the list of folders and create them in SharePoint to match the structure on my local disk.

private void uploadDocuments(string dir)

{

    //figure out the current path

    if (dir.Length > initialdirectory.Length)

    {

currentpath = dir.Substring(initialdirectory.Length + 1);

currentpath = currentpath.Replace("\\", "/");

currentpath = "/" + currentpath;

    }

    else

currentpath = "";

    //iterate through files

    string[] files = Directory.GetFiles(dir);

    foreach (string file in files)

    {

counter++;

updateLabel(counter + " / " + filecount);

string filename = file.Substring(file.LastIndexOf("\\") + 1);

//upload document to sharepoint

using (FileStream fileStream = new FileStream(file, FileMode.Open))

        Microsoft.SharePoint.Client.File.SaveBinaryDirect(clientContext,

siterelativeurl + "/" + libraryname + currentpath + "/" + filename, fileStream, true);

    }

    //iterate through folders

    string[] dirs = Directory.GetDirectories(dir);

    foreach (string directory in dirs)

    {

string dirtocreate = directory.Substring(initialdirectory.Length + 1);

createFolder(dirtocreate);

uploadDocuments(directory);

    }

}

Creating the Folders

Creating the folders is done with the following code. It is simply a matter of building the correct SharePoint Path and having a pointer to the FolderCollection object on the document library.

private void createFolder(string path)

{

Folder folder = folders.Add(serverurl + siterelativeurl + "/" + libraryname + "/" + path);

    oList.Update();

    clientContext.ExecuteQuery();

}

Running the application worked beautifully. It uploaded all the documents in a matter of minutes. The only thing to keep in mind from an authentication standpoint is that it will be expecting passthrough windows authentication to your SharePoint environment. If you are not attached to the domain, I have found the other thing that works is browsing to the SharePoint site in Internet Explorer, and adding the site to your “Intranet Sites” security zone and saving your password. That will also correctly authenticate to the site. Here is a link to the application and complete source code:

http://www.thesharepointblog.net/Documents/SharePointBulkUploader.zip

Enjoy!