JStorm 2.1.1 發布,分布式計算系統
Storm 是一個類似Hadoop MapReduce的系統, 用戶按照指定的接口實現一個任務,然后將這個任務遞交給JStorm系統,Jstorm將這個任務跑起來,并且按7 * 24小時運行起來,一旦中間一個worker 發生意外故障, 調度器立即分配一個新的worker替換這個失效的worker。因此,從應用的角度,JStorm 應用是一種遵守某種編程規范的分布式應用。從系統角度, JStorm一套類似MapReduce的調度系統。 從數據的角度, 是一套基于流水線的消息處理機制。實時計算現在是大數據領域中最火爆的一個方向,因為人們對數據的要求越來越高,實時性要求也越來越快,傳統的Hadoop Map Reduce,逐漸滿足不了需求,因此在這個領域需求不斷。
新特性:
-
1.5~6x performance boost from worst to best scenarios compared to JStorm-2.1.0
-
Add application-level auto-batch
-
Add independent control channel to separate control msgs from biz msgs to guarantee high priority for control msgs
-
Dramatic performance boost in metrics, see "Improvements" section
-
Support jdk1.8
-
Add Nimbus hook and topology hook
-
Metrics system:
-
Support disable/enable metrics on the fly
-
Add jstorm metrics design docs, see JSTORM-METRICS.md
-
JStorm web UI:
-
Add zookeeper viewer in web UI, thanks to @dingjun84
-
Add log search and deep log search, support both backward search and forward search
-
Support log file download
-
Support changing log level on the fly
-
Change error structure in zk, add errorLevel, errorCode and duration.
-
Add supervisor health check
-
Add -Dexclude.jars option to enable filtering jars manually
Bug 修復:
-
Fix deadlock when starting workers
-
Fix the bug that when localstate file is empty, supervisor can't start
-
Fix kryo serialization for HeapByteBuffer in metrics
-
Fix total memory usage calculation
-
Fix the bug that empty worker is assigned when configured worker number is bigger than the actual number for user defined scheduler
-
Fix UI log home directory
-
Fix XSS security bug in web UI
-
Don't start TopologyMetricsRunnable thread in local mode, thanks to @L-Donne
-
Fix JSTORM-141, JSTORM-188 that TopologyMetricsRunnable consumes too much CPU
-
Remove MaxTenuringThreshold JVM option support jdk1.8, thanks to @249550148
-
Fix possible NPE in MkLocalShuffer
下載地址:
來自: http://www.oschina.net//news/71306/jstorm-2-1-1