The Search for Search Continues

Posted on Monday, July 20, 2009

A few days ago I posted a link to wpSearchMu and called it “finally a sane way to have sitewide search in WPMU.” See, we use WordPress MU for contexts.org and I mostly like it, but there are a few things that drive me crazy. Like there’s no built-in way to search across the entire site. If you wanted to create a multi-blog WordPress system, you’d think being able to search all the blogs in that system at once would be pretty high on the todo list. It’s not been, apparently. ((Not to sound too snarky as WPMU is pretty much run by a single guy who does a pretty phenomenal job. He’s also responsible for the must-have WP Super Cache plugin. Also, search overall doesn’t seem to be a very big priority for WordPress: regular WordPress search is pretty lousy as well, even when it’s working properly on single blogs.))

So here’s why I say the options for sitewide search are insane. a) There’s a plugin that apparently enables this, though I’ve never been able to get it to work and it seems like every time there’s a new version of WPMU, it breaks. b) There’s another plugin, Sitewide Tags, that gives you sitewide search, but it does this by creating a special blog on your site that simply duplicates every single post on your blog into one special blog and then lets you search that. This approach just bothers me. Or c) you do what I’ve always done, which is just embed a “site:contexts.org” google search into your search page. It won’t update instantly when posts are created, it looks kind of generic…but it works.

wpSearchMu, which is based off the wpSearch plugin, takes another approach: it installs a copy of the open source Lucene search engine and then creates a local index of your posts that updates every time someone posts. It works beautifully, installs with no problems whatever, except…

[17-Jul-2009 16:54:52] Peak memory usage: 13969748
[17-Jul-2009 16:55:42] Peak memory usage: 13968528
[17-Jul-2009 16:56:15] Peak memory usage: 16787488
[17-Jul-2009 16:56:17] Peak memory usage: 14029656
[17-Jul-2009 16:56:21] Peak memory usage: 14007952
[17-Jul-2009 16:59:37] Peak memory usage: 11151812
[17-Jul-2009 16:59:39] Peak memory usage: 11156788
[17-Jul-2009 16:59:42] Peak memory usage: 11133696
[17-Jul-2009 16:59:44] Peak memory usage: 11149024

This is the output in my php error log from adding the following snippet at the very end of my footer.php ((By the way. put this in your footer, disable all your plugins and then enable them one by one and watch WP’s memory consumption grow…)):

error_log('Peak memory usage: '.memory_get_peak_usage());

Guess where I uninstalled wpSearchMu? Because it includes the entire Lucene search engine on every page load it adds significantly to WordPress' memory footprint. That’s a pretty high price to pay on every page load for something that a relatively small percentage of visits are actually going to use. Out of curiosity I checked the regular, non-MU version of the plugin and found the same thing:

[17-Jul-2009 16:59:44] Peak memory usage: 11149024
[17-Jul-2009 17:09:37] Peak memory usage: 13967872
[17-Jul-2009 17:09:41] Peak memory usage: 13974660
[17-Jul-2009 17:09:49] Peak memory usage: 13952464
[17-Jul-2009 17:09:52] Peak memory usage: 13968344

Again, you can clearly tell where I enabled wpSearch.

Like I said, the plugin works really well otherwise & people who know more than me may be able to figure out how to fix this (maybe by finding a way to only conditionally include the Lucene stuff?). I’m not even trying to single this plugin out — I suspect there are much, much worse offenders out there. And this may not be a big deal for many use cases, but in our case, I’m trying to be conservative about memory usage. ((I’m in the process of redesigning the theme for contexts.org, so that’s why I’ve been paying attention to this stuff. We’re on shared hosting. Relatively good shared hosting generously provided by the University of Minnesota, but still shared hosting that can bog down at times.))

WPMU is going to be merged into WP at some indeterminate point in the future, so hopefully this will be built in by time that happens, but we’ll see. But for now, the search continues…