Apache Tika 1.9 发布,内容抽取工具集合
Apache Tika 1.9 发布,此版本包括一些改进和 bug 修复,详细改进请看 更新日志:
* The ability to use the cTAKES clinical text knowledge extraction system for biomedical data is now included as a Tika parser (TIKA-1645, TIKA-1642). * Tika-server allows a user to specify the Tika config from the command line (TIKA-1652, TIKA-1426). * Matlab file detection has been improved (TIKA-1634). * The EXIFTool was added as an External parser (TIKA-1639). * If FFMPEG is installed and on the PATH, it is a usable Parser in Tika now (TIKA-1510). * Fixes have been applied to the ExternalParser to make it functional (TIKA-1638). * Tika service loading can now be more verbose with the org.apache.tika.service.error.warn system property (TIKA-1636). * Tika Server now allows for metadata extraction from remote URLs and in addition it outputs the detected language as a metadata field (TIKA-1625). * OUTPUT_FILE_TOKEN not being replaced in ExternalParser contributed by Pascal Essiembre (TIKA-1620). * Tika REST server now supports language identification (TIKA-1622). * All of the example code from the Tika in Action book has been donated to Tika and added to tika-examples (TIKA-1562). * Tika server now logs errors determining ContentDisposition (TIKA-1621). * An algorithm for using Byte Histogram frequencies to construct a Neural Network and to perform MIME detection was added (TIKA-1582). * A Bayesian algorithm for MIME detection by probabilistic means was added (TIKA-1517). * Tika now incorporates the Apache Spatial Information System capability of parsing Geographic ISO 19139 files (TIKA-443). It can also detect those files as well. * Update the MimeTypes code to support inheritance (TIKA-1535). * Provide ability to parse and identify Global Change Master Directory Interchange Format (GCMD DIF) scientific data files (TIKA-1532). * Improvements to detect CBOR files by extension (TIKA-1610). * Change xerial.org's sqlite-jdbc jar to "provided" (TIKA-1511). Users will now need to add sqlite-jdbc to their classpath for the Sqlite3Parser to work. * ExternalParser.check now catches (suppresses) SecurityException and returns false, so it's OK to run Tika with a security policy that does not allow execution of external processes (TIKA-1628).
下载:
http://www.apache.org/dyn/closer.cgi/tika/apache-tika-1.9-src.zip
Maven 2: http://repo1.maven.org/maven2/org/apache/tika/
更多内容请看 发行说明。
Tika是一个内容抽取的工具集合(a toolkit for text extracting)。它集成了 POI, Pdfbox 并且为文本抽取工作提供了一个统一的界面。其次,Tika也提供了便利的扩展API,用来丰富其对第三方文件格式的支持。
在当前的0.2-SNAPSHOT版本中, Tika提供了对如下文件格式的支持:
-
PDF - 通过Pdfbox
-
MS-* - 通过POI
-
HTML - 使用 nekohtml将不规范的html整理成为xhtml
-
OpenOffice 格式 - Tika提供
-
Archive - zip, tar, gzip, bzip等
-
RTF - Tika提供
-
Java class - Class解析由 ASM完成
-
Image - 只支持图像的元数据抽取
-
XML
Tika的API十分便捷,核心是Parser interface,其中定义了一个parse方法:
public void parse(InputStream stream, ContentHandler handler, Metadata metadata)
用stream参数传递需要解析的文件流, 文本内容会被传入handler,而元数据会更新至metadata。
可以使用Tika的ParserUtils工具来根据文件的 mime-type
来得到一个适当的Parser来进行解析工作。或者Tika还提供了一个AutoDetectParser根据不同的二进制文件的特殊格式 (比如说Magic Code),来寻找适合的Parser。