Yahoo! Search BOSS Integration with CakePHP: CakePHP, Yahoo, API

I implemented full text search with the Yahoo! Search BOSS API on top of CakePHP in one night, ending at 5AM - lol.  It has auto-suggest for spelling errors, and I added paging as well. The BOSS API impressed me, it was very clean and I was able to implement seamless full text search into my website.

The full source is at the bottom of the article, but let me walk you through the basic parts.

Start by gathering the results. I'm sure there is a better way to do this in CakePHP, but for now I'm using straight PHP5.  I got most of this off Webmaster-Source, but I added a few things including: setting the site search context site:domain.com as well as getting the total results found totalhits.  You can see all the xml data fields that you get back on the search response fields page.

Here is the code for gathering the search results, make sure you get $search_term from the params, then switch out the BOSS API key your_API_key and domain.com with yours. You get the API key by registering on the Yahoo! BOSS website.

PHP:

//Gather data and prepare query
$thequery = urlencode($search_term);
$yhost = 'http://boss.yahooapis.com';
$apikey = 'your_API_key';
$site = 'site:yourdomain.com';
$url = "$yhost/ysearch/web/v1/$thequery+$site?appid=$apikey&format=xml";
// if (!empty($start)) $url .= "&start=$start"; Add this to provide paging
 
//Get the search results
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, 0);
$data = curl_exec($ch);
curl_close($ch);
$results = new SimpleXmlElement($data, LIBXML_NOCDATA);
$total = $results->resultset_web->attributes();
$total = $total['totalhits'];
$results = $results->resultset_web->result;
 
$yhost = 'http://boss.yahooapis.com';
$url = "$yhost/ysearch/spelling/v1/$thequery?appid=$apikey&format=xml";

We can loop through our results in our view. $results will contain title, dispurl and abstract. I use the shortcut, <? which is shorthand for <?php echo.

PHP:

<? $i=1; foreach($results as $result): if(!empty($result->title)): ?>
    <li>
        <span class="count">#<?= $i++; ?>:</span>
        <? $title = str_replace('"', '"', $result->title); ?>
        <a class="search_result" title="<?= $title; ?>" href="<?= $result->clickurl; ?>">
            <?= $title; ?>
        </a><br />
        <span class="abstract">
            <?= str_replace(
                array('<b>yourdomain.com</b>', '<b>YourDomain.com</b>', '<b>...</b>'),
                array('yourdomain.com', 'YourDomain.com', '...'), 
                $result->abstract
            ); ?>
        </span><br />
        <div class="url"><?= strip_tags($result->dispurl); ?></div>
    </li>
<? endif; endforeach; ?>

Next, we want to get any suggestions in case they spelled something wrong.  This is obviously optional, but I think it is a key to search. People are going to misspell things (including me). Using the spelling API, look up the suggested search term.

PHP:

//Get the suggested term
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, 0);
$data = curl_exec($ch);
curl_close($ch);
$suggest = new SimpleXmlElement($data, LIBXML_NOCDATA);
$suggest_term = $suggest->resultset_spell->result->suggestion;

Then it is easy to display the suggestion in our view:

PHP:

<? if(!empty($suggestTerm)): ?>
    <strong>Did you mean: <?= $html->link($suggestTerm, '/searches?q=' . $suggestTerm); ?></strong>
<? endif; ?>

I also implemented paging in the view, this is probably pretty crappy code... written very close to 5AM so you may want to do this your own way. It basically uses the $start and $total variables in the URL to calculate the pages, and what page you are on.

PHP:

<?php
    if ($start > 0):
 
        if ($start == 10)
            $startParam = '';
        else
            $startParam = "&start=" . ($start - 10);
       
        echo $html->link('Prev', $self . "?q=$searchTerm" . $startParam) . "&nbsp;&nbsp; ";
    endif;
 
    $page = 0;
    for ($i=0; $i<$total; $i=$i+10) {
        $page++;
   
        if ($i > ($start-50) && $i < ($start+50)) {
            if ($i == $start) {
                echo " $page ";
            } else if ($i === 0) {
                echo $html->link($page, $self . "?q=$searchTerm");
            } else {
                echo $html->link($page, $self . "?q=$searchTerm&start=$i");
            }
        } else if ($i == ($start-60)) {
            echo '... ';
        } else if ($i == ($start+60)) {
            echo '... '; break;
        }
    }
    echo " &nbsp;&nbsp;";
 
    if (count( $results ) === 10):
        echo $html->link('Next', $self . "?q=$searchTerm&start=$next");
    endif;
 
    $countPages = ceil($total / 10);
    echo "&nbsp;&nbsp; ($total results on $countPages pages)";
endif;
?>

Ok, putting it all together here is the full source.

Full source of controllers/searches_controller.php

PHP:

<?php
class SearchesController extends AppController {
    var $name = 'Searches';
    var $uses = array();
 
        public function index() {
        
        params['url']['q']) ? $this->params['url']['q'] : null;
        $start = isset($this->params['url']['start']) ? $this->params['url']['start'] : 0;
        
        if (!empty($search_term)) {
        
            //Gather data and prepare query
            $thequery = urlencode($search_term);
            $yhost = 'http://boss.yahooapis.com';
            $apikey = 'your_API_key';
            $site = 'site:yourdomain.com';
            $url = "$yhost/ysearch/web/v1/$thequery+$site?appid=$apikey&format=xml";
            if (!empty($start)) $url .= "&start=$start";
 
            //Get the search results
            $ch = curl_init($url);
            curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
            curl_setopt($ch, CURLOPT_HEADER, 0);
            $data = curl_exec($ch);
            curl_close($ch);
            $results = new SimpleXmlElement($data, LIBXML_NOCDATA);
            $total = $results->resultset_web->attributes();
            $total = $total['totalhits'];
            $results = $results->resultset_web->result;
            
            $yhost = 'http://boss.yahooapis.com';
            $url = "$yhost/ysearch/spelling/v1/$thequery?appid=$apikey&format=xml";
            
            //Get the suggested term
            $ch = curl_init($url);
            curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
            curl_setopt($ch, CURLOPT_HEADER, 0);
            $data = curl_exec($ch);
            curl_close($ch);
            $suggest = new SimpleXmlElement($data, LIBXML_NOCDATA);
            $suggest_term = $suggest->resultset_spell->result->suggestion;
            
            $next = $start + 10;
            $this->set(compact('search_term', 'results', 'suggest_term', 'start', 'next', 'total'));
            
        }
    }
    
}
 
?>

Full source of views/searches/index.ctp

PHP:

<?php if (!empty($searchTerm)): ?>
    
<?php
    $this->pageTitle = "Search results for, \"$searchTerm\"";
?>
 
    <h2>Search results for, "<?= $searchTerm; ?>"</h2>
    <? if(!empty($suggestTerm)): ?>
        <strong>Did you mean: <?= $html->link($suggestTerm, '/searches?q=' . $suggestTerm); ?></strong>
    <? endif; ?>
    <div class="container search_results">
        <? if(empty($results)): ?>
            No results found for "<?= $searchTerm; ?>". 
        <? else: ?>
            <h4>Results <?= $start+1 . '-' . (count($results)+$start) . ' of ' . $total; ?></h4>
            <ul>
                <? $i=$start+1; foreach($results as $result): if(!empty($result->title)): ?>
                    <li>
                        <span class="count">#<?= $i++; ?>:</span>
                        <? $title = str_replace('"', '&quot;', $result->title); ?>
                        <a class="search_result" title="<?= $title; ?>" href="<?= $result->clickurl; ?>">
                            <?= $title; ?>
                        </a><br />
                        <span class="abstract">
                            <?= str_replace(
                                array('<b>yourdomain.com</b>', '<b>YourDomain.com</b>', '<b>...</b>'),
                                array('yourdomain.com', 'YourDomain.com', '...'), 
                                $result->abstract
                            ); ?>
                        </span><br />
                        <div class="url"><?= strip_tags($result->dispurl); ?></div>
                    </li>
                <? endif; endforeach; ?>
            </ul>
            
            <? 
            
            if ($total > 10):
                if ($start > 0):
                
                    if ($start == 10)
                        $startParam = '';
                    else
                        $startParam = "&start=" . ($start - 10);
                        
                    echo $html->link('Prev', $self . "?q=$searchTerm" . $startParam) . "&nbsp;&nbsp; "; 
                endif;
                
                $page = 0;
                for ($i=0; $i<$total; $i=$i+10) {
                    $page++;
                    
                    if ($i > ($start-50) && $i < ($start+50)) {
                        if ($i == $start) {
                            echo " $page ";
                        } else if ($i === 0) {
                            echo $html->link($page, $self . "?q=$searchTerm");
                        } else {
                            echo $html->link($page, $self . "?q=$searchTerm&start=$i");
                        }
                    } else if ($i == ($start-60)) {
                        echo '... ';
                    } else if ($i == ($start+60)) {
                        echo '... '; break;
                    }
                }
                echo " &nbsp;&nbsp;";
                
                if (count( $results ) === 10):
                    echo $html->link('Next', $self . "?q=$searchTerm&start=$next");
                endif;
                
                $countPages = ceil($total / 10);
                echo "&nbsp;&nbsp; ($total results on $countPages pages)";
            endif;
            ?>
        <? endif; ?>
    </div>
    
    
 
<?php endif; ?>