無論是通用搜索還是垂直搜索，其關鍵的核心技術之一就是網絡爬蟲的設計。本文結合
HTMLParser 信息提取方法，對生活類垂直搜索引擎中網絡爬蟲進行了詳細研究。通過深入分析生活類網站網址的樹形結構的構架，開發了收集種子頁面URL 的模擬搜索器，并基于HTMLParser 的信息提取方法，從種子頁面中提取出與生活類主題相關的目標URL。經實驗測試證明該爬蟲的爬準率達93.552% ，爬全率達96.720% ，表明該網絡爬蟲是有效的，達到中等規模的垂直搜索企業級應用的要求。
關鍵詞：網絡爬蟲；垂直搜索； HTMLParser

Abstract：Whether general search engine or vertical search engine, the design of web crawler is the core technology. In this article, a novel system of life-theme web crawler based on HTMLParser information extraction is thoroughly studied. In this system, a simulation searcher is designed for collecting the seed URL by analyzing tree structure of life-theme website, then, based on the discussion of HTMLParser information extraction, the target URL that relate to life-theme is extracted from the seed pages. Empirical studies show that the Pr ecision = 93.552% and the Re call = 96.720%, proving its effectiveness and achieving requirements for general enterprise-level application of vertical search engine.
Key words：web crawler; vertical search engine;HTMLParser