ElasticSearch簡單使用

d66g 9年前發布 | 42K 次閱讀 ElasticSearch 搜索引擎

What is ElasticSearch ？

Elasticsearch是一個基于Apache Lucene(TM)的開源搜索引擎。無論在開源還是專有領域，Lucene可以被認為是迄今為止最先進、性能最好的、功能最全的搜索引擎庫。

但是，Lucene只是一個庫。想要使用它，你必須使用Java來作為開發語言并將其直接集成到你的應用中，更糟糕的是，Lucene非常復雜，你需要深入了解檢索的相關知識來理解它是如何工作的。

Elasticsearch也使用Java開發并使用Lucene作為其核心來實現所有索引和搜索的功能，但是它的目的是通過簡單的RESTful API來隱藏Lucene的復雜性，從而讓全文搜索變得簡單。

不過，Elasticsearch不僅僅是Lucene和全文搜索，我們還能這樣去描述它：

分布式的實時文件存儲，每個字段都被索引并可被搜索

分布式的實時分析搜索引擎

可以擴展到上百臺服務器，處理PB級結構化或非結構化數據

而且，所有的這些功能被集成到一個服務里面，你的應用可以通過簡單的RESTful API、各種語言的客戶端甚至命令行與之交互。

上手Elasticsearch非常容易。它提供了許多合理的缺省值，并對初學者隱藏了復雜的搜索引擎理論。它開箱即用（安裝即可使用），只需很少的學習既可在生產環境中使用。

Elasticsearch在Apache 2 license下許可使用，可以免費下載、使用和修改。

隨著你對Elasticsearch的理解加深，你可以根據不同的問題領域定制Elasticsearch的高級特性，這一切都是可配置的，并且配置非常靈活。

Config elasticSearch.properties

#配置elasticsearch index
csg_bill_index_name=hsc_test_index
配置elasticsearch type
csg_bill_type_name=billdetails
配置elasticsearch es address
csg_bill_es_address=l-test.h.beta.cn0:9300
配置elasticsearch cluster
csg_bill_es_cluster=elasticsearch_f</pre> 
Create elasticSearch TransportSessionFactory 

/**

Created by xueping.you on 15-7-30.
*/
@Service
public class ESTransportSessionFactory  {
private final static Logger LOGGER = LoggerFactory.getLogger(ESTransportSessionFactory.class);
@Value("${csg_bill_es_address}")
 private String elasticAddress;
@Value("${csg_bill_es_cluster}")
 private String elasticCluster;



private TransportClient transportClient;


@PostConstruct
public void init(){
    checkArgument(!Strings.isEmpty(elasticCluster) ,
     "ElasticSearch Cluster Name Null or EmptyString");
    TransportAddress[] transportAddresses = null;
    List<String> addressList = 
        Lists.newArrayList(
            Splitter.on(",").trimResults().omitEmptyStrings().split(elasticAddress)
    );
    checkArgument(!CollectionUtils.isEmpty(addressList) ,
     "ElasticSearch Cluster Address Can't be Empty");
    transportAddresses = new TransportAddress[addressList.size()];
    Splitter splitter = Splitter.on(":").omitEmptyStrings().trimResults();
    for(int i=0; i<addressList.size() ; i++){
        List<String> singleAddressPair = 
            Lists.newArrayList(splitter.split(addressList.get(i)));
        checkArgument(singleAddressPair.size()==2 ,
                "ElasticSearch Address format address:port error" + addressList.get(i)
        );
        transportAddresses[i] = new InetSocketTransportAddress(
                singleAddressPair.get(0),
                Integer.parseInt(singleAddressPair.get(1))
        );
    }
    Settings settings = ImmutableSettings.builder()
        .put("cluster.name" , elasticCluster)
        .build();
    transportClient = new TransportClient(settings);
    transportClient.addTransportAddresses(transportAddresses);

    LOGGER.info("ElasticSearch Init Done ElasticAddress={} , ElasticClusterName={}" 
        , elasticAddress , elasticCluster);
}

@PreDestroy
public void destroy() {
    if (transportClient != null) {
        transportClient.close();
    }
}

public TransportClient getTransportClient() {
    return transportClient;
}


}</pre>

Service Method Interface 

/**

Created by xueping.you on 15-7-30.
*/
@Service
public class ESBillDetailServiceBase {
 private final static Logger LOGGER = LoggerFactory.getLogger(ESBillDetailServiceBase.class);
@Resource
 private ESTransportSessionFactory esTransportSessionFactory;
@Value("${csg_bill_index_name}")
 private String ES_INDEX;
@Value("${csg_bill_type_name}")
 private String ES_TYPE;
private static ObjectMapper objectMapper = new ObjectMapper();
static {

 objectMapper.setDateFormat(new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"));

}
/**

Method1 將訂單號 , 對賬單號 , 產品Id , 賬單號 ,結算供應商ID,結算對象Id,create_ts進行索引
@param esBillDetails
*/
public void insertAllOnDuplicateIdCover(List<ESBillDetail> esBillDetails){
 Stopwatch stopwatch = Stopwatch.createStarted();
 if(CollectionUtils.isEmpty(esBillDetails)){

 return;

}
TransportClient client = esTransportSessionFactory.getTransportClient();
 BulkRequestBuilder bulkRequestBuilder = client.prepareBulk();
for(ESBillDetail esBillDetail : esBillDetails){

 String source;
 try {
     source = objectMapper.writeValueAsString(esBillDetail);
 } catch (Exception e1) {
     LOGGER.error("create billDetail index error source : " + esBillDetail, e1);
     QMonitor.recordOne(QMonitorConstants.ES_BILL_DETAIL_INDEXING_ERROR);
     continue;
 }
 IndexRequestBuilder indexRequestBuilder = 
     client.prepareIndex(ES_INDEX, ES_TYPE, 
     esBillDetail.getId()).setSource(source);
 bulkRequestBuilder.add(indexRequestBuilder);

}
BulkResponse bulkResponse = bulkRequestBuilder.execute().actionGet();
 if(bulkResponse.hasFailures()){

 Iterator<BulkItemResponse> itemResponseIterator = bulkResponse.iterator();
 while(itemResponseIterator.hasNext()){
     BulkItemResponse itemResponse = itemResponseIterator.next();
     if(itemResponse!=null && itemResponse.isFailed()){
         LOGGER.info("indexing billDetail error indexMessage={},
             errorMassage={}",Joiner.on(',')                     .join(itemResponse.getIndex(),itemResponse.getType(),itemResponse.getId()),
                 JsonUtils.toJSONString(itemResponse));
     }
 }
 QMonitor.recordOne(QMonitorConstants.ES_BILL_DETAIL_INDEXING_ERROR);

}
QMonitor.recordOne(QMonitorConstants.ES_BILL_DETAIL_INDEXING_DONE , stopwatch.elapsed(TimeUnit.MILLISECONDS));
}
/**

Method2 通過訂單號 ，對賬單號, 產品Id , 賬單號 的任意條件進行查詢，支持分頁
*/
public List<ESBillDetail> query(ESBillDetailQuery esBillDetailQuery , Pagination pagination){
 try {

 Stopwatch stopwatch = Stopwatch.createStarted();
 List<ESBillDetail> esBillDetails = Lists.newArrayList();
 BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
 for(Map.Entry entry : esBillDetailQuery.entrySet()){
     ESBillDetailQuery.Param param = (ESBillDetailQuery.Param)entry.getKey();
     param.decorateBoolQueryBuilder(entry.getValue(),boolQueryBuilder);
 }
 SearchRequestBuilder requestBuilder = esTransportSessionFactory.getTransportClient()
         .prepareSearch(ES_INDEX)
         .setTypes(ES_TYPE)
         .setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
         .setQuery(boolQueryBuilder);
 if(pagination!=null){
         requestBuilder.setFrom(pagination.getStartIndex())
                 .setSize(pagination.getPageSize());
 }
 SearchResponse searchResponse = requestBuilder.execute().actionGet();
 SearchHits searchHits = searchResponse.getHits();
 for(SearchHit searchHit : searchHits){
     Map<String , Object> map = searchHit.getSource();
     ESBillDetail esBillDetail = objectMapper.readValue(
             JsonUtils.toJSONString(map) ,
             new TypeReference<ESBillDetail>() {});
     esBillDetails.add(esBillDetail);
 }
 LOGGER.info("Query ESDetail done query={} , time={}" , esBillDetailQuery , stopwatch.elapsed(TimeUnit.MILLISECONDS));
 QMonitor.recordOne(QMonitorConstants.ES_BILL_QUERY , stopwatch.elapsed(TimeUnit.MILLISECONDS));
 return esBillDetails;

}catch (Exception e){

 LOGGER.error("elastic search query Error query={} , page={}" , esBillDetailQuery , pagination , e);
 QMonitor.recordOne(QMonitorConstants.ES_BILL_DETAIL_QUERY_ERROR);
 throw new RuntimeException("ES 查詢異常");

}
}
/**

此方法慎用！！！！！！！！！！！！！！！！！！！
Method3 通過匹配條件刪除ES 中的記錄
*/
public boolean delete(ESBillDetailQuery esBillDetailQuery){
 try {
 Stopwatch stopwatch = Stopwatch.createStarted();
 BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
 for(Map.Entry entry : esBillDetailQuery.entrySet()){
     ESBillDetailQuery.Param param = (ESBillDetailQuery.Param)entry.getKey();
     param.decorateBoolQueryBuilder(entry.getValue(),boolQueryBuilder);
 }
 DeleteByQueryResponse deleteByQueryResponse = esTransportSessionFactory.getTransportClient()
         .prepareDeleteByQuery(ES_INDEX )
         .setTypes(ES_TYPE)
         .setQuery(boolQueryBuilder).execute().actionGet();
 LOGGER.info("Delete ESDetail done query={} , time={}" , esBillDetailQuery , stopwatch.elapsed(TimeUnit.MILLISECONDS));
 QMonitor.recordOne(QMonitorConstants.ES_BILL_DELETE , stopwatch.elapsed(TimeUnit.MILLISECONDS));
 return deleteByQueryResponse.status().equals(RestStatus.OK) ? true : false;

 }catch (Exception e){
 LOGGER.error("elastic search delete Error query={} , page={}" , esBillDetailQuery , e);
 QMonitor.recordOne(QMonitorConstants.ES_BILL_DETAIL_QUERY_ERROR);
 throw new RuntimeException("ES 刪除記錄異常");

 }
}</pre>

ESBillDetailQuery 
/**


Created by xueping.you on 15-7-30.
*/
public class ESBillDetailQuery extends GenericQuery<ESBillDetailQuery.Param> {
//全為Str類型
 public enum Param{

 ids {
     @Override
     public <T, Q extends QueryBuilder> void decorateBoolQueryBuilder(T param, Q queryBuilder) {
         List<String> strsParam = (List<String>)param;
         BoolQueryBuilder boolQueryBuilder = (BoolQueryBuilder)queryBuilder;
         if(CollectionUtils.isEmpty(strsParam)){
             return;
         }
         BoolQueryBuilder subCodeQuery = QueryBuilders.boolQuery();
         for(String strParam : strsParam){
             subCodeQuery.should(QueryBuilders.matchQuery("billNo" , strParam).operator(MatchQueryBuilder.Operator.AND));
         }
         boolQueryBuilder.must(subCodeQuery);
     }
 },
 billNos {
     @Override
     public <T, Q extends QueryBuilder> void decorateBoolQueryBuilder(T param, Q queryBuilder) {
         List<String> strsParam = (List<String>)param;
         BoolQueryBuilder boolQueryBuilder = (BoolQueryBuilder)queryBuilder;
         if(CollectionUtils.isEmpty(strsParam)){
             return;
         }
         BoolQueryBuilder subCodeQuery = QueryBuilders.boolQuery();
         for(String strParam : strsParam){
             subCodeQuery.should(QueryBuilders.matchQuery("billNo" , strParam).operator(MatchQueryBuilder.Operator.AND));
         }
         boolQueryBuilder.must(subCodeQuery);
     }
 },
 orderNos {
     @Override
     public <T, Q extends QueryBuilder> void decorateBoolQueryBuilder(T param, Q queryBuilder) {
         List<String> strsParam = (List<String>)param;
         BoolQueryBuilder boolQueryBuilder = (BoolQueryBuilder)queryBuilder;
         if(CollectionUtils.isEmpty(strsParam)){
             return;
         }
         BoolQueryBuilder subCodeQuery = QueryBuilders.boolQuery();
         for(String strParam : strsParam){
             subCodeQuery.should(QueryBuilders.matchQuery("orderNo" , strParam).operator(MatchQueryBuilder.Operator.AND));
         }
         boolQueryBuilder.must(subCodeQuery);
     }
 },
 productIds {
     @Override
     public <T, Q extends QueryBuilder> void decorateBoolQueryBuilder(T param, Q queryBuilder) {
         List<String> strsParam = (List<String>)param;
         BoolQueryBuilder boolQueryBuilder = (BoolQueryBuilder)queryBuilder;
         if(CollectionUtils.isEmpty(strsParam)){
             return;
         }
         BoolQueryBuilder subCodeQuery = QueryBuilders.boolQuery();
         for(String strParam : strsParam){
             subCodeQuery.should(QueryBuilders.matchQuery("productId" , strParam).operator(MatchQueryBuilder.Operator.AND));
         }
         boolQueryBuilder.must(subCodeQuery);
     }
 },
 reconcileIds {
     @Override
     public <T, Q extends QueryBuilder> void decorateBoolQueryBuilder(T param, Q queryBuilder) {
         List<String> strsParam = (List<String>)param;
         BoolQueryBuilder boolQueryBuilder = (BoolQueryBuilder)queryBuilder;
         if(CollectionUtils.isEmpty(strsParam)){
             return;
         }
         BoolQueryBuilder subCodeQuery = QueryBuilders.boolQuery();
         for(String strParam : strsParam){
             subCodeQuery.should(QueryBuilders.matchQuery("reconcileId" , strParam).operator(MatchQueryBuilder.Operator.AND));
         }
         boolQueryBuilder.must(subCodeQuery);
     }
 };

 public abstract <T, Q extends QueryBuilder> void decorateBoolQueryBuilder(T param ,  Q queryBuilder );

}



public ESBillDetailQuery() {
}

public ESBillDetailQuery(Integer startIndex, Integer maxCount) {
    super(startIndex, maxCount);
}

public ESBillDetailQuery(Class<Param> paramType) {
    super(paramType);
}

}</pre>

來自：http://blog.csdn.net/yxp20092010/article/details/47260413

本文由用戶 d66g 自行上傳分享，僅供網友學習交流。所有權歸原作者，若您的權利被侵害，請聯系管理員。

轉載本站原創文章，請注明出處，并保留原始鏈接、圖片水印。

本站是一個以用戶分享為主的開源技術平臺，歡迎各類分享！

本文地址：http://www.baiduhome.net/lib/view/open1440506794310.html

ElasticSearch 搜索引擎

ElasticSearch簡單使用

What is ElasticSearch ？

Config elasticSearch.properties

相關經驗

相關資訊

相關文檔

目錄