內容抽取Java工具集合,Apache Tika 1.9 發布
Tika是一個內容抽取的工具集合(a toolkit for text extracting)。它集成了POI, Pdfbox 并且為文本抽取工作提供了一個統一的界面。其次,Tika也提供了便利的擴展API,用來豐富其對第三方文件格式的支持。
在當前的0.2-SNAPSHOT版本中, Tika提供了對如下文件格式的支持:
-
PDF - 通過Pdfbox
-
MS-* - 通過POI
-
HTML - 使用nekohtml將不規范的html整理成為xhtml
-
OpenOffice 格式 - Tika提供
-
Archive - zip, tar, gzip, bzip等
-
RTF - Tika提供
-
Java class - Class解析由ASM完成
-
Image - 只支持圖像的元數據抽取
-
XML
Tika的API十分便捷,核心是Parser interface,其中定義了一個parse方法:
public void parse(InputStream stream, ContentHandler handler, Metadata metadata)
用stream參數傳遞需要解析的文件流, 文本內容會被傳入handler,而元數據會更新至metadata。
可以使用Tika的ParserUtils工具來根據文件的mime-type
來得到一個適當的Parser來進行解析工作。或者Tika還提供了一個AutoDetectParser根據不同的二進制文件的特殊格式 (比如說Magic Code),來尋找適合的Parser。
* The ability to use the cTAKES clinical text knowledge extraction system for biomedical data is now included as a Tika parser (TIKA-1645, TIKA-1642). * Tika-server allows a user to specify the Tika config from the command line (TIKA-1652, TIKA-1426). * Matlab file detection has been improved (TIKA-1634). * The EXIFTool was added as an External parser (TIKA-1639). * If FFMPEG is installed and on the PATH, it is a usable Parser in Tika now (TIKA-1510). * Fixes have been applied to the ExternalParser to make it functional (TIKA-1638). * Tika service loading can now be more verbose with the org.apache.tika.service.error.warn system property (TIKA-1636). * Tika Server now allows for metadata extraction from remote URLs and in addition it outputs the detected language as a metadata field (TIKA-1625). * OUTPUT_FILE_TOKEN not being replaced in ExternalParser contributed by Pascal Essiembre (TIKA-1620). * Tika REST server now supports language identification (TIKA-1622). * All of the example code from the Tika in Action book has been donated to Tika and added to tika-examples (TIKA-1562). * Tika server now logs errors determining ContentDisposition (TIKA-1621). * An algorithm for using Byte Histogram frequencies to construct a Neural Network and to perform MIME detection was added (TIKA-1582). * A Bayesian algorithm for MIME detection by probabilistic means was added (TIKA-1517). * Tika now incorporates the Apache Spatial Information System capability of parsing Geographic ISO 19139 files (TIKA-443). It can also detect those files as well. * Update the MimeTypes code to support inheritance (TIKA-1535). * Provide ability to parse and identify Global Change Master Directory Interchange Format (GCMD DIF) scientific data files (TIKA-1532). * Improvements to detect CBOR files by extension (TIKA-1610). * Change xerial.org's sqlite-jdbc jar to "provided" (TIKA-1511). Users will now need to add sqlite-jdbc to their classpath for the Sqlite3Parser to work. * ExternalParser.check now catches (suppresses) SecurityException and returns false, so it's OK to run Tika with a security policy that does not allow execution of external processes (TIKA-1628).
下載:
http://www.apache.org/dyn/closer.cgi/tika/apache-tika-1.9-src.zip
Maven 2:http://repo1.maven.org/maven2/org/apache/tika/
更多內容請看發行說明。