基于LUCENE的站内搜索引擎的设计与实现

论文字数:16707,页数:41 有开题报告，任务书，PPT

摘   要
 LUCENE[1]是apache软件基金会jakarta项目组的一个子项目，是一个开放源代码的全文检索引擎工具包，即它不是一个完整的全文检索引擎，而是一个全文检索引擎的架构，提供了完整的查询引擎和索引引擎，LUCENE的目的是为软件开发人员提供一个简单易用的工具包，以方便的在目标系统中实现全文检索的功能，或者是以此为基础建立起完整的全文检索引擎。
 作为一个开放源代码项目，LUCENE从问世之后，引发了开放源代码社群的巨大反响，程序员们不仅使用它构建具体的全文检索应用，而且将之集成到各种系统软件中去，以及构建Web应用，甚至某些商业软件也采用了LUCENE作为其内部全文检索子系统的核心。apache软件基金会的网站使用了LUCENE作为全文检索的引擎，IBM的开源软件eclipse的2.1版本中也采用了LUCENE作为帮助子系统的全文索引引擎，相应的IBM的商业软件Web Sphere中也采用了LUCENE。LUCENE以其开放源代码的特性、优异的索引结构、良好的系统架构获得了越来越多的应用。
 这个系统的实际需求来源于本人在企业里实习时开发的2007世界特殊奥林匹克运动会官方网站，这个官方网站也使用了站内搜索引擎的功能，这个搜索引擎是我用LUCENE在.NET平台上实现的，现在网站运行稳定，站内搜索使整个网站的功能更加强大，为用户提供了更为便利的搜索功能。
 本文对搜索引擎的原理、组成、数据结构、工作流程等方面做了深入而细致地研究与分析。并且通过LUCENE来设计和实现一个全文检索站内搜索引擎系统，最后通过增量索引和优化索引两个方面来说明如何提高LUCENE的高效性。
 
关键词： 全文检索，搜索引擎，LUCENE ，jakarta
 Design and Realization of Search Engine in Site Base On LUCENE

Abstract
                
 LUCENE is a sub-project of jakarta project team in apache software foundation, and is a tool kit of full-text search engine of open source, that is not a complete full-text search engine, but is a framework of full-text search engine to provide integral inquire engine and index engine. LUCENE is designed to provide a simple, easy-to-use tool kit for software developers，and it’s convenient to realize the full-text retrieval function  in the target system , or as a  basis to establish the integral  full-text search engine.
 As an open source project, LUCENE brings tremendous response in the open source community after its appearance. The programmers not only use it to build concrete full-text retrieval applications, but also make it integrate into various of systems software, and build web applications, even some commercial software are using LUCENE as its core of full-text retrieval subsystems. Apache Software Foundation website uses LUCENE as a full-text search engine, LUCENE is used to help full-text index engine of subsystem in the 2.1 version of IBM''s software revenue eclipse. As the IBM’s corresponding commercial software Web Sphere also uses LUCENE. LUCENE gets more and more applications with open source characteristics, excellent index structure, good system architecture.
 The actual requirements of this system origins from my development on “2007 Special Olympics World official website” when I practiced in the enterprise. The official website also uses the function of station search engine, I use the LUCENE to realize the search engine in .NET platform. Now for stable operation, station search make the function of the whole site become more powerful, and to provide users with a more convenient search function.
 I have carefully studied and analysis search engines principle, composition, data structure, and work flow, and have designed and realized a full-text retrieval stations search engine by means of LUCENE. Finally, both to I illustrate how to improve the efficiency of LUCENE through two aspects, the increment index and the optimization index.
 
Key Words: Full Text Retrieval，Search Engine，jakarta

目       录
1.绪论 1
1.1课题背景 1
1.2课题目前研究情况及存在问题 2
1.3论文组织结构 2
2.全文检索与LUCENE 3
2.1全文检索与全文检索简介 3
2.2全文检索系统与数据库比较 4
2.3 LUCENE简介 8
2.4 LUCENE的应用、特点及优势 9
2.5互联网搜索引擎的研究 10
2.6中文分词的简单介绍 11
3.LUCENE系统结构 12
3.1 LUCENE系统结构组织 12
3.2 数据流分析 12
3.3 LUCENE索引文件格式分析 14
3.3.1 LUCENE源码实现分析的说明 14
3.3.2 LUCENE索引文件格式 14
3.4 LUCENE的倒排序原理 17
3.5 LUCENE搜索结果排序 20
4.系统设计与实现 21
4.1系统需求 21
4.2开发环境与工具 22
4.3系统组织结构 24
4.4流程实现 25
4.4.1根据网站中不同模块建立生成动态索引 25
4.4.2搜索界面 26
4.4.3 搜索结果界面 27
5.关键技术 28
5.1 LUCENE增量索引 28
5.2优化索引 28
5.3 LUCENE文件格式的通用性 29
5.4对于私密文件的处理 31
总  结 32
致  谢 33
参考文献 34
附录A 主要源程序 35