{"id":15,"date":"2011-02-01T03:39:43","date_gmt":"2011-02-01T03:39:43","guid":{"rendered":"\/blog\/?p=15"},"modified":"2011-02-27T04:09:35","modified_gmt":"2011-02-27T09:09:35","slug":"indexing-my-web-site-for-search","status":"publish","type":"post","link":"https:\/\/www.rfdm.com\/blog\/?p=15","title":{"rendered":"Indexing my web site for search"},"content":{"rendered":"<p>I&#8217;ve been looking for site indexing code to support searching my web site.\u00a0 Yes, you can certainly use Google for some of my content.\u00a0 But Google won&#8217;t typically see content that&#8217;s not linked in.\u00a0 Then there&#8217;s the data hidden away in databases instead of HTML.<\/p>\n<p>Looking around at this time, nothing really excites me.\u00a0 I want something fairly light weight, fast, and capable of indexing content from my HTML, PHP, blogs and photos.\u00a0 I also want something that easily integrates into the look and feel of my site, today and 2 years from now.<\/p>\n<p>Conceptually, I like xapian.\u00a0 There&#8217;s also the &#8216;omega&#8217; package built on xapian.\u00a0 However, omega&#8217;s &#8216;omindex&#8217; doesn&#8217;t work for me; I&#8217;ve got considerable textual content in PHP files and &#8216;omindex&#8217; skips PHP.\u00a0 It also doesn&#8217;t handle WordPress blogs, gallery3 databases, etc.\u00a0 I&#8217;m also not terribly fond of the output of omega (though that&#8217;s not difficult to change).<\/p>\n<p>I&#8217;ve decided I&#8217;ll write my own indexer and search using xapian as the back end.\u00a0 In fact I&#8217;ve already written the code to index HTML and PHP files, and have a design for the code to index my blog posts.\u00a0 I have a test search program that emits simple HTML.\u00a0 All looks good so far; after indexing a handful of pages, searches yield appropriate document weights and rankings.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I&#8217;ve been looking for site indexing code to support searching my web site.\u00a0 Yes, you can certainly use Google for some of my content.\u00a0 But Google won&#8217;t typically see content that&#8217;s not linked in.\u00a0 Then there&#8217;s the data hidden away in databases instead of HTML. Looking around at this time, nothing really excites me.\u00a0 I &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.rfdm.com\/blog\/?p=15\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Indexing my web site for search&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12,13],"tags":[23,37,20,34,16,14],"class_list":["post-15","post","type-post","status-publish","format-standard","hentry","category-software-development","category-web-development","tag-c","tag-dwmsiteindex","tag-indexing","tag-php","tag-search","tag-xapian"],"_links":{"self":[{"href":"https:\/\/www.rfdm.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/15","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.rfdm.com\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.rfdm.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.rfdm.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.rfdm.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=15"}],"version-history":[{"count":5,"href":"https:\/\/www.rfdm.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/15\/revisions"}],"predecessor-version":[{"id":65,"href":"https:\/\/www.rfdm.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/15\/revisions\/65"}],"wp:attachment":[{"href":"https:\/\/www.rfdm.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=15"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.rfdm.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=15"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.rfdm.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=15"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}