SharePoint Events

  9/03 - Webcast: SharePoint 2013 vs. SharePoint Online: Workflow
  9/04 - Webcast: Power Business Intelligence for SharePoint Online and Office 365
  9/05 - Webcast: SharePoint 2013 vs. SharePoint Online: Search
  9/30 - Webcast: Project Management Workflow in SharePoint
  10/01 - Webcast: SharePoint 2013 vs. SharePoint Online: Enterprise Content Management
  10/02 - Webcast: SharePoint 2013 vs. SharePoint Online: Business Intelligence

 SharePoint Videos

  Why SharePoint 2013
  Keys to Successful SharePoint Initiatives and User Adoption
  Out of the Box Business Intelligence and Reporting in SharePoint 2013
  Driving the Business Case and User Adoption for SharePoint
  Automate Business Processes with SharePoint 2013 and Business Connectivity Services
  SharePoint and Office 365 Migration Made Easy
  Full Lifecycle Content Management with SharePoint 2013
  Compliant Document Generation and Assembly in SharePoint
  Application of Content Security and Corporate Risk Protection in SharePoint
  How to Redline, Markup, Collaborate and Review Content in SharePoint
  Advanced Content Lifecycle and Records Management in SharePoint
  Advanced Workflow Made Easy with SharePoint 2013 and Nintex
  Full Lifecycle Content Management with SharePoint 2013
  SharePoint 2013 Solutions for the Financial Services Industry
  ProjectReady 2013: SharePoint Solutions built for the AEC
  SharePoint 2013 Solutions for the Legal Industry
  Driving the Business Case and User Adoption for SharePoint
  Investment Considerations for SharePoint 2010 and SharePoint 2013
  SharePoint 2013 and Enterprise Content Management
  SharePoint 2013 Launch
(More Links...)

 Archives

Opening SharePoint Links in a new windowUse SHIFT+ENTER to open the menu (new window).
Mail Enabled Lists vs. The Missing Windows 2008 POP3/IMAP Server Use SHIFT+ENTER to open the menu (new window).
7 Tools for SharePoint DevelopersUse SHIFT+ENTER to open the menu (new window).
Public Facing Masterpage TechniquesUse SHIFT+ENTER to open the menu (new window).
How to Quickly Deploy and Activate a Timer Service to Your Site CollectionUse SHIFT+ENTER to open the menu (new window).
Custom SharePoint Master Page Feature with WSP BuilderUse SHIFT+ENTER to open the menu (new window).
Date Math with InfoPathUse SHIFT+ENTER to open the menu (new window).
Enterprise Search Tricks and Tips Part 1Use SHIFT+ENTER to open the menu (new window).
Populating Word Documents With SharePoint Data. Try The DIP!Use SHIFT+ENTER to open the menu (new window).
Programmatic Deep Dive into Blank SharePoint Lookup ColumnsUse SHIFT+ENTER to open the menu (new window).
1 - 10 Next
Adventures in Uploading Multiple Documents in SharePoint 2010
By: Neil Barkhina
 

As you may have noticed, it is not so easy to upload documents in bulk into SharePoint 2010. In fact it was quite a bit better in SharePoint 2007. Let’s take a look at our options

1.   SharePoint 2007 Multiple Upload: The Upload Multiple Files ActiveX control worked quite well and had good performance. But it did not traverse subdirectories.

 

2.   SharePoint 2010 Multiple Upload: In SharePoint 2010 this has been replaced with a Silverlight control, which while has interface advantages with drag and drop does limit you to only 100 documents of folders at a time. So now what happens if you want to upload an entire directory of documents with many subfolders?

 

3.   Windows Explorer View: This option allows you to upload series of documents and nested subfolders into SharePoint. As nice as this option may be, the performance is really hit or miss.

 

4.   SharePoint Workspace: This is actually one of my favorite options. You’ve got your drag and drop functionality same as with Silverlight. The performance is excellent and highly optimized to your internet connection. The limitations are in the amount of documents though. They don’t officially support more than 500 documents.

So here I am thinking, SharePoint Workspace is the winner. However, I tried to drag a folder with almost 2GB of content with 4000 documents and 600 folders. Let’s just say the result was disastrous. Workspace locked up, I gave it the benefit of the doubt however. After 20 minutes of a completely frozen application, I decided to see if anything was going on in SharePoint land. Indeed the documents were being upload, but slooooowly. So it was at this point that I opted for a custom solution.

Lucky for me, SharePoint 2010 now includes a very rich and powerful .NET Client Object Model. So all I needed to do was write a windows forms application that used this API to traverse the local files and folders and upload them into SharePoint. There actual code was pretty minimal but I did want to make sure I had an application that was both performant and didn’t freeze the application. This involved basically calling all the SharePoint code from a separate thread and updating the user interface with BeginInvoke(). Here is how I did it:

Building the User Interface

The first step was dragging out all the controls I would need. A button for Import, and text boxes for the parameters plus a label for status.

Adventures_in_Uploading_Multiple_Documents_in_SharePoint_2010

Starting the thread

Once the user hits the Import button, I didn’t want the application to freeze up. So I started a new thread called dobutton1work()

Thread t = new Thread(new ThreadStart(dobutton1work));

t.IsBackground = true;

t.Start(); 

Opening the site

The next step is actually opening up the site. This is a combination of the base server URL and the site relative URL which is what the ClientContext object expects. Then I initialize the web and list objects, load the folders of the library and run ExecuteQuery(). This is necessary because the way Client Object model differs mainly from traditional object model is commands and objects need to be explicitly requested, queued up, and then executed. The initial execution of the ClientContext will be the slowest part of the application. But once this is done, all the other methods run pretty quickly.

clientContext = new ClientContext(serverurl + siterelativeurl);

web = clientContext.Web;

oList = web.Lists.GetByTitle(libraryname);

clientContext.Load(web);

clientContext.Load(oList);

clientContext.ExecuteQuery();

folders = web.Folders;

clientContext.Load(folders);

clientContext.ExecuteQuery();

 

Update the UI

In Windows Forms, the user interface actually runs on its own thread and doesn’t allow other threads to make changes. To get around this, you must call the BeginInvoke method on a specific UI control and then call Thread.Sleep(). I refactored this into its own method:

private void updateLabel(string message)

{

    label1.BeginInvoke(new MethodInvoker(() => label1.Text = message));

    Thread.Sleep(1);

}

 

Upload the Documents

Uploading the documents itself was done using a recursive function call uploadDocuments which tooks a local disk path as a parameter. I build out the current path and translate it to a path which can be used by SharePoint. The actual upload is done using a function called SaveBinaryDirect() and the function’s performance is excellent. After doing the file uploads, I iterate through the list of folders and create them in SharePoint to match the structure on my local disk.

private void uploadDocuments(string dir)

{

    //figure out the current path

    if (dir.Length > initialdirectory.Length)

    {

currentpath = dir.Substring(initialdirectory.Length + 1);

currentpath = currentpath.Replace("\\", "/");

currentpath = "/" + currentpath;

    }

    else

currentpath = "";

 

    //iterate through files

    string[] files = Directory.GetFiles(dir);

    foreach (string file in files)

    {

counter++;

updateLabel(counter + " / " + filecount);

string filename = file.Substring(file.LastIndexOf("\\") + 1);

 

//upload document to sharepoint

using (FileStream fileStream = new FileStream(file, FileMode.Open))

        Microsoft.SharePoint.Client.File.SaveBinaryDirect(clientContext,

siterelativeurl + "/" + libraryname + currentpath + "/" + filename, fileStream, true);

    }

 

    //iterate through folders

    string[] dirs = Directory.GetDirectories(dir);

    foreach (string directory in dirs)

    {

string dirtocreate = directory.Substring(initialdirectory.Length + 1);

createFolder(dirtocreate);

uploadDocuments(directory);

    }

}

Creating the Folders

Creating the folders is done with the following code. It is simply a matter of building the correct SharePoint Path and having a pointer to the FolderCollection object on the document library.

private void createFolder(string path)

{

Folder folder = folders.Add(serverurl + siterelativeurl + "/" + libraryname + "/" + path);

    oList.Update();

    clientContext.ExecuteQuery();

}

 

Running the application worked beautifully. It uploaded all the documents in a matter of minutes. The only thing to keep in mind from an authentication standpoint is that it will be expecting passthrough windows authentication to your SharePoint environment. If you are not attached to the domain, I have found the other thing that works is browsing to the SharePoint site in Internet Explorer, and adding the site to your “Intranet Sites” security zone and saving your password. That will also correctly authenticate to the site. Here is a link to the application and complete source code:

http://www.thesharepointblog.net/Documents/SharePointBulkUploader.zip

Enjoy!

By: Neil Barkhina
        

Comments

Mark

2.   SharePoint 2010 Multiple Upload

Where is this?  I have been searching all over trying to figure out how to get this control you mention.  I am trying to get single file uploading working in my silverlight application and I have been running into tremendous issues getting this to work (rather tiny file size limits and no matter how much config tweaking I do it doesn't go away).  So I'm wondering if this control could help.
at 11/29/2010 9:45 AM

Ramesh Natarajan

I would like to load files into sharepoint workspace and then want the sharepoint workspace to take care of sync into sharepoint.

The idea is : The legacy application will call my .NET code and the code will put files into workspace. (The legacy application can be offline/online to sharepoint).

Is there any way of achieving it? I have written code to directly upload to sharepoint (Kind of your Upload 2010 Option mentioned), however I would like to achieve it through workspace. Is there any Calls/Methods to achieve this?
at 1/25/2011 8:20 AM

Faisal Iqbal

This application is not working. I have a sitecollection http://local:89/sites/test1 and library 'Shared Documents'. I also have a sub folder inside Shared Documents. This client object model seems to have some limitations.
at 1/27/2011 6:56 AM

Neil

can you please send me all the settings that you are populating into the application?
at 1/27/2011 1:31 PM

Kiren Patel

Loading the document library from a network or local drive gives consistent error as:

9/8/2011 3:14:52 PM - Filecount: 203
9/8/2011 3:14:52 PM - Connecting to site
9/8/2011 3:14:55 PM - The underlying connection was closed: A connection that was expected to be kept alive was closed by the server.    at System.Net.HttpWebRequest.GetResponse()
   at Microsoft.SharePoint.Client.SPWebRequestExecutor.Execute()
   at Microsoft.SharePoint.Client.File.SaveBinary(ClientContext context, String serverRelativeUrl, Stream stream, String etag, Boolean overwriteIfExists, SaveBinaryCheckMode checkMode)
   at Microsoft.SharePoint.Client.File.SaveBinaryDirect(ClientContext context, String serverRelativeUrl, Stream stream, Boolean overwriteIfExists)
   at SharePointBulkUploader.Form1.scanDir(String dir) in C:\Kiren Patel\Projects\SharePointBulkUploader\Form1.cs:line 152
   at SharePointBulkUploader.Form1.scanDir(String dir) in C:\Kiren Patel\Projects\SharePointBulkUploader\Form1.cs:line 162
   at SharePointBulkUploader.Form1.dobutton1work() in C:\Kiren Patel\Projects\SharePointBulkUploader\Form1.cs:line 82
at 9/8/2011 3:15 PM

SUDHEER

I follow the below process to reproduce the issue:
1) I navigate to the Document Library in a sharepoint site
2) click on "Multiple Documents"
3) select the files and click ok.
4) After uploading files, I click on 'Done'

Immediate next to the above step, the current upload page reverts to the home page of the site instead of pointing to the document library page.

Thanks & Regards,
Sudheer Baisani.
at 10/12/2011 7:52 AM

ADOLFO BEJARANO

I'm getting this error after I enter all the parameters into the respective boxes and hit the 'Import" buttom:
SharePointBulkUploaders has stopped working
Please advise, what should I do to make the application work??
Thanks,

Adolfo
at 11/3/2011 5:25 PM

Bosco Montenegro

This is a great tool, thank you so much for sharing it. Excellent work.
at 5/7/2012 11:59 AM

Aishu

I am getting "The underlying connection was closed: A connection that was expected to be kept alive was closed by the server" error. Can u pls help
at 5/15/2012 11:13 PM

Add Comment

Items on this list require content approval. Your submission will not appear in public views until approved by someone with proper rights. More information on content approval.

Your Name *


e-mail address *


Website (optional)


Comment *


Attachments

 Subscribe

  GigWerks RSS  Gig Werks Mailing List 

 Contact Us

 Connect

 Resources

  On Demand SharePoint Webcast Recordings
  Upcoming Webinars
  SharePoint Resources
  Business Intelligence Resources
  Gig Werks Website



©2009 Gig Werks. All rights reserved. Privacy Policy