Search Benchmarks

PHP vs ASP vs ASP.NET vs JavaScript vs CGI

Zoom provides five platform options for searching: PHP, ASP, ASP.NET, JavaScript, and CGI. These searching front ends processes the user search queries, searches through the index data, and displays the results on your website.

Below are some benchmarks we have made comparing the 5 different versions of the Zoom search scripts provided, so that you can see how Zoom performs on sites of certain sizes with the available scripting platforms.

In the tests below, we benchmarked several different scenarios:

  • Searching a small/typical website
    with 400 pages, and 296,601 total words (11,995 unique words).
  • Searching a medium website
    with 60,000 pages, and 5,638,632 total words (256,874 unique words).
  • Searching an large website (CGI and ASP.NET only)
    with 125,000 pages, and 10,532,537 total words (360,023 unique words).
  • Searching an huge website (FastCGI, CGI and ASP.NET only)
    with 1,000,000 pages, and 369,293,182 total words (2,817,237 unique words).

Note: The CGI (C++) version can search sites larger than 60,000 pages and 300,000 unique words, but neither the PHP or ASP version is capable of this, so the largest site compared here is under this 60,000/300,000 limit.

We measured the performance of each of these sets of data with a range of search query test cases.

Test server specifications:

  • Intel Core i7-4790 @ 3.60 GHz
  • Windows 8.1 Pro (64-bit)
  • Plextor M6e PCI-E SSD 256 GB
  • 16 GB of RAM
  • PHP 5.5.9 on Apache 2.4.7
  • ASP and ASP.NET tested on IIS 8.5.9600.16384
  • CGI (C++) application tested on Apache 2.4.7
  • FastCGI (C++) application tested on:
    • IIS 8.5.9600.16384
    • Apache 2.4.7 with mod_fcgid 2.3.9
  • JavaScript tested on Google Chrome 35.0.1916.153
  • Zoom Search Engine V7 (Windows build 1004), Enterprise Edition
  • Zoom Search Engine V6 (build 1029), Enterprise Edition

Searching a small/typical site with Zoom Search V7

400 pages and 296,601 total words (11,995 unique words)

Time is in seconds to generate and display results (the smaller the values the better)

Single word queries 2 words 3 words Wildcard Exact phrase
Platform Test 1 Test 2 Test 3 Test 4 Test 5 Test 6 Test 7
PHP 0.0397 0.0320 0.0317 0.0323 0.0327 0.0587 0.0557
ASP .0767 0.0600 0.0637 0.0633 0.0647 0.0967 0.1304
JavaScript* 0.2543 0.2513 0.2520 0.2623 0.2733 0.2967 N/A*
ASP.NET 0.0063 0.0070 0.0070 0.0083 0.0087 0.0100 0.0103
CGI (C++) 0.0040 0.0040 0.0040 0.0040 0.0040 0.0050 0.0050
*Each JavaScript time includes an assumed download duration, to represent the downloading of the index files by the browser when it is hosted on a remote web server. This assumes certain bandwidth conditions. Note that JavaScript would be much faster when run directly from a local CD/DVD or hard disk. PHP, ASP, ASP.NET and CGI do not need the index files to be downloaded.

Benchmarking PHP vs. ASP vs ASP.NET vs Javascript vs Binary CGI (C++) on a typical site search
Test 1 to 5 are random queries involving single to multiple search words.
Test 6 involves the use of wildcards, and Test 7 is an exact phrase query.
Note that wildcard and exact phrase searches have a significant impact on speed.
JavaScript does not support exact phrase searches so it does not have a result for Test 7.

 

Searching a medium site on Zoom Search V7

60,000 pages and 5,638,632 total words (256,874 unique words).

Time is in seconds to generate and display results (the smaller the values the better)

Single word queries 2 word 3 words Wildcard Exact phrase
Platform Test 8 Test 9 Test 10 Test 11 Test 12 Test 13 Test 14
PHP 0.8643 0.8413 0.8247 0.9120 0.9437 2.5020 1.0930
ASP 2.3673 2.2343 2.4000 2.4603 2.5027 2.8097 2.4907
JavaScript* 3.6637 3.2943 3.4163 3.8317 5.0010 4.9400 N/A*
ASP.NET 0.1673 0.1147 0.1107 0.1113 0.1137 0.1207 0.1153
CGI (C++) 0.0733 0.0747 0.0750 0.0747 0.0767 0.0803 0.0743

Comparing performance of PHP, ASP, ASP.NET and CGI (C++) on a medium sized site
Test 8 to 12 are random queries involving single to multiple search words.
Test 13 involves the use of wildcards, and Test 14 is an exact phrase query.
Note that the CGI (C++) and ASP.NET versions scales significantly better with larger sites.

Searching a large site on Zoom Search V7

125,000 pages, and 10,532,537 total words (360,023 unique words).

Time is in seconds to generate and display results (the smaller the values the better)

Single word queries 2 word 3 words Wildcard Exact phrase
Platform Test 15 Test 16 Test 17 Test 18 Test 19 Test 20 Test 21
ASP.NET 0.2070 0.1540 0.1570 0.1607 0.1640 0.1730 0.1647
CGI (C++) 0.1037 0.1023 0.1037 0.1073 0.1077 0.1123 0.1080

ASP.NET vs binary (C++ compiled) CGI search benchmarking on a large site with over 10.5 million words
Test 15 to 19 are random queries involving single to multiple search words.
Test 20 involves the use of wildcards, and Test 21 is an exact phrase query.
Note that this test was designed for the CGI(C++) and ASP.NET platforms only.
The PHP, ASP, and JS versions can not be used to search this amount of data.

Searching a huge site on Zoom Search V7

1,000,000 pages, and 369,293,182 total words (2,817,237 unique words).

Time is in seconds to generate and display results (the smaller the values the better)

Single word queries 2 word 3 words Wildcard Exact phrase
Platform Test 22 Test 23 Test 24 Test 25 Test 26 Test 27 Test 28
ASP.NET 1.1930 1.0503 1.0313 1.0433 1.0430 1.1760 1.3020
CGI (C++) 0.8100 0.7360 0.7347 0.7380 0.7347 0.8190 0.8713
FastCGI (IIS, C++) 0.0677 0.0423 0.0373 0.0400 0.0373 0.1253 0.2197
FastCGI (Apache, C++) 0.0480 0.0347 0.0300 0.0313 0.0297 0.0787 0.0580

ASP.NET vs binary (C++ compiled) CGI search benchmarking on a large site with over 10.5 million words
Test 22 to 26 are random queries involving single to multiple search words.
Test 27 involves the use of wildcards, and Test 21 is an exact phrase query.
Note that this test was designed for the FastCGI(C++), CGI(C++) and ASP.NET platforms only.
The PHP, ASP, and JS versions can not be used to search this amount of data.

Benchmark Notes

  • Search script settings: 10 results per page, "any" search word, context descriptions (except for JavaScript), exact phrase matching and stemming enabled (except for JavaScript), highlighting is enabled, logging is disabled, and timing is enabled.
  • Operations are performed and measured on a local computer, and assume no network latency, or shared CPU processing load.
  • Assumes no graphics, or extraneous layout in the template file, nor any extra header or footer scripting
  • All test results are a calculated average of 3 runs. The first test for each site is performed after a fresh reboot and thus do not have the files cached.

Benchmark Conclusions

  • On average, the PHP version is faster than the ASP version, while the CGI (C++) version is more than 10 times faster than both PHP and ASP.
  • The PHP version is only slightly faster than the ASP version for smaller sites, but as the size of a site grows, the difference increases. This indicates that the PHP version scales better than the ASP version.
  • The CGI (C++) version performs the best and scales significantly better than the other versions as the size of a site increases (up to 70 times faster in some cases). The native ASP.NET Server Control also performs well. They are the only platform capable of searching through an enterprise size site of over 65,000 pages (as demonstrated in the "large site" test).
  • The JavaScript version produces the slowest results. This emphasises the fact that it should only be used exclusively for offline distributions (in which case it performs well because there is no download time), or as a last resort for small online website. Due to the technical restrictions of the JavaScript platform, it also has the least capable support of features (it does not allow context descriptions, exact phrase matching nor stemming), and can not be used on larger sites due to memory restrictions enforced by browsers.
  • The use of wildcards result in slower searches.
  • The use of exact phrase searches can result in slow searches, depending on the commonality of the words in the phrase.
  • There is a slight increase in the search delay when the index files are not already loaded in the system's cache (as expected).
  • When searching over a fairly large site, one should consider using FastCGI, as FastCGI is about 10 times faster than CGI and about 15 times faster than ASP.NET.

Return to the Zoom Search Engine product page to find out more.