Search: from 2003 to today

Every generation of SharePoint search, and how to manage today's search topology.

Search is the SharePoint component that has changed the most. Each generation brought a different engine, a different topology and a different way to administer it.

2003: two separate engines

Windows SharePoint Services 2.0 sites used SQL Server full-text search on the content database. It worked within a site and nothing more.
SharePoint Portal Server 2003 had its own engine built on the Microsoft Search (MSSearch) service, with content sources, crawls, search scopes and a portal-wide index.

2007: the Shared Services Provider era

WSS 3.0 had a basic built-in search limited to site content.
MOSS 2007 search ran in the Shared Services Provider. One index server crawled content and built the index, then propagated it to the query servers.
Admins got content sources, crawl rules, scopes, managed properties and best bets.
The single index server was the weak spot: it was a single point of failure and a hard scaling limit.
Microsoft also released Search Server 2008 (and a free Express edition) for search without the rest of SharePoint.

2010: scale-out search and FAST as an add-on

SharePoint Server 2010 search split crawling from querying. You could run several crawl components (each with a crawl database) and several query components, with the index split into partitions and mirrors.
Refiners, people search with phonetic matching, and better relevance arrived.
FAST Search Server 2010 for SharePoint was a separate product and farm for organisations that needed more: entity extraction, deep refiners, document thumbnails and custom content processing.

2013: one engine, rebuilt

SharePoint 2013 merged FAST into SharePoint itself. The search system is made of six component types:

Component	What it does
Crawl	Fetches content from content sources
Content processing	Parses documents, extracts metadata, maps crawled properties to managed properties
Analytics processing	Usage and search analytics, popularity, recommendations
Index	Stores the index; scales out in partitions, made resilient with replicas
Query processing	Handles queries, applies query rules
Search administration	Runs the search system itself

Other changes that are still with us:

Result sources replaced scopes and federated locations.
Query rules, result types and display templates control what users see.
Continuous crawl for SharePoint content keeps the index fresh within minutes.
The topology is changed only with PowerShell, never in Central Administration.

2016 to Subscription Edition: hybrid and modern

Cloud hybrid search (2016, also back-ported to 2013) indexes on-premises content into the Microsoft 365 index, so users see on-premises and cloud results together in SharePoint Online.
The Search MinRole role keeps search on its own servers.
2019 and Subscription Edition add the modern search box and results in modern sites. The engine underneath is the 2013 design.

SharePoint Online: Microsoft Search

In Microsoft 365 you don't manage a topology. Microsoft Search gives one search across SharePoint, OneDrive, Teams and Outlook. Admins manage bookmarks, acronyms, verticals and result types in the Microsoft 365 admin center, and Microsoft Graph connectors bring in content from other systems. The community PnP Modern Search web parts are widely used to build custom search pages.

Working with search in PowerShell (2013 and later)

Health at a glance:

$ssa = Get-SPEnterpriseSearchServiceApplication
Get-SPEnterpriseSearchStatus -SearchApplication $ssa -Text
Get-SPEnterpriseSearchCrawlContentSource -SearchApplication $ssa |
    Select-Object Name, CrawlState, CrawlCompleted

See the active topology:

$active = Get-SPEnterpriseSearchTopology -SearchApplication $ssa -Active
Get-SPEnterpriseSearchComponent -SearchTopology $active |
    Select-Object Name, ServerName | Sort-Object ServerName

Change the topology (clone, change the clone, activate it). This example adds a query processing component on another server:

$ssa    = Get-SPEnterpriseSearchServiceApplication
$active = Get-SPEnterpriseSearchTopology -SearchApplication $ssa -Active
$clone  = New-SPEnterpriseSearchTopology -SearchApplication $ssa -Clone -SearchTopology $active

$si = Get-SPEnterpriseSearchServiceInstance -Identity "SP-SRCH02"
Start-SPEnterpriseSearchServiceInstance -Identity $si
New-SPEnterpriseSearchQueryProcessingComponent -SearchTopology $clone -SearchServiceInstance $si

Set-SPEnterpriseSearchTopology -Identity $clone

Pause search during patching or maintenance, and resume it afterwards:

Suspend-SPEnterpriseSearchServiceApplication -Identity $ssa
Resume-SPEnterpriseSearchServiceApplication  -Identity $ssa

Tips from the field

Crawl a dedicated target. Point the crawl server at a web server that isn't in the user load balancer (for example with a hosts file entry on the crawl server), so crawls don't slow down users.
Give the crawl account read access only, through the web application's "Full Read" user policy. Never use a farm admin account.
Keep an eye on the crawl log. A spike in errors usually means a permissions change, an expired certificate or a site that was moved.
Resetting the index is a last resort. It empties search for every user until the full crawl completes. Try a full crawl of the problem content source first.
Plan index size. Each index partition holds a limited number of items. Add partitions before you reach the limit, not after.