SharePoint 2013 Search not provisioning correctly

Imagine one fine sunny day you built a new SharePoint 2013 Search Application, for a new server or an old one, and the rain came in the form of an error accessing the application after Central Admin stated clearly that it was created successfully. Here’s what you would be experiencing:

After creating the Search Service Application, the following issues arise:

  1. The search topology displays the following message: “Unable to retrieve topology component health states. This may be because the admin component is not up and running.”
  2. The default content source “Local SharePoint Sites” is inconsistent. It doesn’t always appear after creation of Search, sometimes with start addresses of existing web apps listed already, other times not.
  3. Starting a full crawl results in stuck in ‘starting’.

So you can’t configure your search. In PowerShell, all search components are listed as available.

Event Viewer shows:

Application Server Administration job failed for service instance Microsoft.Office.Server.Search.Administration.SearchServiceInstance (cf15c8c7-1980-4391-bd97-17de75c4dd5d).

Reason: Failed to connect to system manager. SystemManagerLocations: http://sp2013/AdminComponent1/Management

Technical Support Details:

System.InvalidOperationException: Failed to connect to system manager. SystemManagerLocations: http://sp2013/AdminComponent1/Management—> System.ServiceModel.EndpointNotFoundException: There was no endpoint listening at http://sp2013/AdminComponent1/Management/NodeController that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details.

Server stack trace:

   at System.ServiceModel.Channels.ConnectionUpgradeHelper.DecodeFramingFault(ClientFramingDecoder decoder, IConnection connection, Uri via, String contentType, TimeoutHelper& timeoutHelper)

   at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.SendPreamble(IConnection connection, ArraySegment`1 preamble, TimeoutHelper& timeoutHelper)

   at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.DuplexConnectionPoolHelper.AcceptPooledConnection(IConnection connection, TimeoutHelper& timeoutHelper)

   at System.ServiceModel.Channels.ConnectionPoolHelper.EstablishConnection(TimeSpan timeout)

   at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.OnOpen(TimeSpan timeout)

   at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)

   at System.ServiceModel.Channels.ServiceChannel.OnOpen(TimeSpan timeout)

   at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)

   at System.ServiceModel.Channels.ServiceChannel.CallOnceManager.CallOnce(TimeSpan timeout, CallOnceManager cascade)

   at System.ServiceModel.Channels.ServiceChannel.EnsureOpened(TimeSpan timeout)

   at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)

   at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)

   at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:

   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)

   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)

   at Microsoft.Ceres.CoreServices.Services.Node.INodeControllerManagementAgent.get_NodeName()

   at Microsoft.Ceres.CoreServices.Tools.Management.Client.AbstractSystemClient.TryConnect(Uri nodeManagementUri, ICollection`1 locationsToTry, ICollection`1 triedLocations, ICollection`1 nodeSystemManagerLocations)

   — End of inner exception stack trace —

   at Microsoft.Office.Server.Search.Administration.SearchServiceInstance.Synchronize()

   at Microsoft.Office.Server.Administration.ApplicationServerJob.ProvisionLocalSharedServiceInstances(Boolean isAdministrationServiceJob)

After much digging and playing around, it seems this error goes away if I changed the app pool the search was running under to the SharePoint Web Services Default. But this is not what we want, Search should run under its own app pool. So after much more digging, I discovered that this is another bug I have found in SharePoint 2013, fixable by applying the following hotfixes:

Give the server a good old reboot and things should work fine. That’ll save you a few days of trouble.

Peace love and happiness!

The Art of Installing SharePoint 2013 in a 3 Tier Topology – Part One

Ok, so you’re a support person/ newbie/ trainee SharePoint guru, and you have been asked to install SharePoint 2013 with the following topology or similar:

Now SharePoint 2013 is similar to 2010 but they are very different. Although this topology is quite rare, I had to re-create it for my app development, architecture and POC creation, and I’m going to take you through the steps of creating it from scratch. I’m going to assume this is your first SharePoint installation, so all the gurus out there, be patient! It could be a long post so I might put some links to the same information I used for reference to help me complete some of my tasks. There’s a lot of info out there on installing SharePoint 2013 (SP2013 from here on in) so by all means, have a gander and design your own custom installation solution using different aspects of others as I have done.

First off please read and understand the hardware and software requirements for SP2013, they are very important, the nature of the beast is that SharePoint is getting more splendid and all these beautiful features require more and more memory. – this is a good link to read to get you thinking about browser and IP support as well as software boundaries, limits and capacity management. Before installing SharePoint 2013, you need to really understand what it is you are doing, it will save many days of frustration and prevent many grey hairs! Ok, so here we go…

Hardware RoadMap

I created all of this on a machine with 16GB RAM, my OS was Server 2008 r2 with Hyper V role installed. All servers are on 64 bit processors.

My plan for memory allocation was as such:

AD Server – 2GB Ram and 80GB Hard drive On 64 bit processor

Database Server – 3GB and 80GB Hard drive (8GB Ram usual for SP2013 but I have used 3 and not suffered any repercussions yet, you can always increase this if the need arises especially if you are using Hyper environments) On 64 bit processor

SharePoint Server – 8GB and 80GB hard drive (Microsoft recommends at least 12 GB for production deployments but really, I think they are being overly cautious because of performance issues (Page Loads etc.) encountered with SharePoint 2010. I work with my 8GB just fine, no problems yet, but this can always be increased if required). On 64 bit processors

I created all these hyper V servers using my Hyper V manager on my guest OS. See these 2 links to create a virtual machine from a guest OS (Windows Server 2008 R2 in my case):

The next links helped me understand the different network options available in Hyper V and how basic networking in Hyper V environments work:

One other very important step is providing internet access to your machines once you have created them. I was connecting to the internet on my guest OS using a wireless connection and used the following link to facilitate that:

Software RoadMap

My software list is as follows:

AD Server – Windows 2008 R2 with AD Services role installed. I used the following link to help with setting up my domain and promoting my server to be a domain controller:

To install the OS once I had created my hyper V machine was to attach the installation media, i.e. point Hyper V to where your Windows Server 2008 r2 installation media is on your guest OS. Don’t forget to install DHCP, WINS and Enterprise Certificate Services.

*Note Before I installed the AD role on the server I update the machine with the latest service packs and updates as the Domain controller is not really meant to have any internet access for security reasons.

SQL Server – Windows Server 2012 and SQL Server 2012 standard (No longer enterprise version of Windows now, only Standard and DataCenter).

I used this link to help me set up Server 2012 and add it to my domain (Created on the domain controller):

The process of installing SQL is quite laborious so I will help you through that one in the next blog post. Don’t forget to install updates after you have installed all your required software.

SharePoint Server – Windows Server 2012, SharePoint Server 2013, Visual Studio 2013, Office 2013, SharePoint Designer 2013, and any other Dev tools you usually use. I included ULS viewer and CSK Dev Visual Studio tools from Codeplex, SP2013 Client Component Tools SDK (, and whatever else you might use for your development machine.

So I knew I had a big task ahead of me to set this all up, so I made a list of all my required tasks and it looked something like this:

  1. Create, install and configure AD Server and Domain and create Service and Farm accounts (See SharePoint installation blog post for a list of these)
  2. Create, install and configure Database Server
    1. Install Server 2012 and configure
    2. Install SQL Server 2012
    3. Add SharePoint Admin and Farm account to Database server and apply correct perms (Spadmin needs DBCreator and Security Admin permission as well as an admin on the SQL and SharePoint servers). See here ( and here (
    4. Set Up Max degree of parallelism ( and some info here (
    5. Configure SQL Server Security (
  3. Create, install and configure SharePoint 2013 Server
    1. Set up server with Web server and Application server roles
    2. Install and configure SharePoint 2013
    3. Install and configure Visual Studio 2013, Office 2013, SharePoint Designer 2013, ULS Viewer and other software
    4. Configure Managed Accounts and set up logging, email etc
    5. Set Up AD synchronisation
    6. Download and install SP2013 Client Components SDK
    7. Update all servers
    8. Backup all servers and create future backup schedule
  4. Pour a much needed Long island Ice tea and assume the recovery position! Lol that’s optional!

Ok so for part 2, I’m gonna assume you have your own plan ready to go, and you have an AD server with a working domain. Ok Lets install and configure SQL 2012