WebSPHINX - 可定制的Java Web爬蟲
WebSPHINX是一個Java類包和Web爬蟲的交互式開發環境。Web爬蟲(也叫作機器人或蜘蛛)是可以自動瀏覽與處理Web頁面的程序。WebSPHINX由兩部分組成：爬蟲工作平臺和WebSPHINX類包。
- Multithreaded Web page retrieval in a simple application framework
- An object model that explicitly represents pages and links
- Support for reusable page content classifiers
- Tolerant HTML parsing
- Support for the robot exclusion standard
- Pattern matching, including regular expressions, Unix shell wildcards, and HTML tag expressions. Regular expressions are provided by the Apache jakarta-regexp regular expression library.
- Common HTML transformations , such as concatenating pages , saving pages to disk, and renaming links
本文由用戶 jopen 自行上傳分享，僅供網友學習交流。所有權歸原作者，若您的權利被侵害，請聯系管理員。
轉載本站原創文章，請注明出處，并保留原始鏈接、圖片水印。
本站是一個以用戶分享為主的開源技術平臺，歡迎各類分享！