本文內容主要集中在應用層,通過下面幾個部分介紹當前最流行的搜索工具:Elasticsearch,了解這些內容后,可以快速開始使用它。
Elasticsearch 是一個分布式、RESTful 風格的搜索和數據分析引擎。
它基于Lunece實現,使用java語言編寫。Lunece是一個優秀的搜索引擎庫,但它使用起來非常復雜。
Elasticsearch通過對 Lunece的封裝,隱藏了復雜性,提供了使用簡單的RESTful Api。
同時也實現了分布式集群特性,具有存儲數據大,查詢性能好,擴展方便等特點。
在業務開發中,基于ES的特性,通常有下面這些場景需要使用它:
要使用ES,需要了解幾個最基本的概念,節點(node),索引(index),類型映射(mapping)和文檔(doc)。
節點是組成ES集群的基本單位,每個節點是一個運行的ES實例。每個物理機器上可以有多個節點,使用不同的端口和節點名稱。
節點按主要功能可以分為三種:主節點(Master Node),協調節點(Coordianting Node)和數據節點(Data Node)。下面簡單介紹下:
索引是ES中的邏輯概念,是文檔的容器。對ES的操作,基本都是對索引操作,一個ES集群中,可以創建多個索引。
索引定義了一組文檔的數據模型和處理方法。每個索引可以有多個主分片和副本分片,分別保存在不同的節點。
mapping定義了一個索引中,文檔保存的每個字段的數據類型。根據數據類型的不同,在添加文檔時對每個字段的處理也不同。
例如,對text類型的字段,會先使用分詞器分詞,生成倒排索引,用于之后的搜索。對keyword類型的字段,不會分詞,搜索時只能精確查找。
一個簡單的mapping示例如下:
{
"javalogs": { //索引名稱
"mappings": {
"properties": {
"log_content": { //text類型,分詞,用于之后的分詞索引
"type": "text"
},
"date": {//時間類型
"type": "date"
},
"log_level": { //keyword類型,不分詞
"type": "keyword"
},
"ip": {
"type": "keyword"
}
}
}
}
}
在6.x版本中,每個索引中還可以有多個type,區分不同的mapping。在7.x中,type被取消,每個索引只有一個type:_doc
下面通過一張圖來描述,節點(node),索引(index)和文檔(doc)之間的關系。
一切知識都要通過實踐掌握,所以在了解基本的概念和邏輯后,下面就進入實踐環節。
這里推薦使用docker來搭建本地開發環境,docker對應windows和mac系統都有桌面版本,使用非常方便。因為網絡限制,直接使用docker官方倉庫拉取鏡像會很慢,所以在安裝完成后,需要在設置中將倉庫的地址替換為國內源,這里推薦https://docker.mirrors.ustc.edu.cn,速度很快,設置如下:
{
"registry-mirrors": [
"https://docker.mirrors.ustc.edu.cn"
]
}
下面我們使用docker安裝Elasticsearch和kibana鏡像,kibana是es官方配套的可視化分析工具,使用它的頁面dev tools可以很方便的通過api操作es。
因為要同時部署兩個docker鏡像,這里推薦使用docker-composer,桌面版安裝完成后就帶有該命令,需要的配置如下:
services:
kibana:
image: kibana:7.2.0
container_name: kibana-simple
environment:
- TIMELION_ENABLED=true
ports:
- "5601:5601"
networks:
- mynetwork
elasticsearch:
image: elasticsearch:7.2.0
container_name: es-simple
environment:
- cluster.name=mytestes #這里就是ES集群的名稱
- node.name=es-simple #節點名稱
- bootstrap.memory_lock=true
- network.publish_host=elasticsearch #節點發布的網絡名稱
- discovery.seed_hosts=es-simple #設置集群中的主機地址
- cluster.initial_master_nodes=es-simple #手動設置可以成為master的節點集合
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- esdata1:/usr/local/elasticsearch/simpledata
ports:
- 9200:9200
networks:
- mynetwork
volumes:
esdata1:
driver: local
networks:
mynetwork:
driver: bridge
創建一個名稱為docker-compose.yaml文件,復制下面的配置到文件中,然后再文件所在目錄執行docker-compose up,之后會啟動兩個docker實例,分別是elasticsearch和kibana。
在本地瀏覽器中,訪問http://127.0.0.1:5601/,可以看到kibana的界面如下:
創建好的kibana已經默認添加了Elasticsearch的配置,通過管理工具可以很方便的查看ES集群的狀態,索引情況,刪除索引等。
kibana-monitor
下面通過dev tools創建索引,dev tools提供的命令提示很方便,并且可以把已寫好的請求保存在瀏覽器緩存中,非常適合用來學習Elasticsearch。
create-index
這里通過ES提供的RESTful Api創建了第一個索引, 并且設置了該索引中的mapping,ES的地址已經設置過,這里可以不寫完整的域名,對應的curl完整請求如下:
curl --location --request PUT 'http://127.0.0.1:9200/javalogs' \
--header 'Content-Type: application/json' \
--data-raw '{
"mappings": {
"properties": {
"log_content": {
"type": "text"
},
"date": {
"type": "date"
},
"log_level": {
"type": "keyword"
},
"ip": {
"type": "keyword"
}
}
}
}'
下面介紹下Elasticsearch中常用的api,這些例子都是直接在kibana的dev tools中運行的,如果想用curl訪問,可參考前一節中的轉換例子。
//自動生成_id
POST javalogs/_doc
{
"log_content" : "get user_id 123456",
"date" : "2020-04-15T11:09:08",
"log_level": "info",
"ip": "10.223.32.67"
}
//指定_id
POST javalogs/_doc/111
{
"log_content" : "api response in 55ms",
"date" : "2020-04-15T11:09:07",
"log_level": "info",
"ip": "10.223.32.67"
}
ES在文檔查詢時,對于不分詞的查詢,直接按值查詢即可,例如下面這樣:
//不分詞類型查詢
POST javalogs/_search
{
"query": {
"match": {
"ip": "10.223.32.67"
}
}
}
這里主要說下分詞類型的查詢,對于分析類型的field在查詢時,也會默認把查詢的語句分詞。假設有兩個文檔如下:
//文檔1
{
"log_content" : "call aaa service error",
"date" : "2020-04-15T11:09:07",
"log_level": "error",
"ip": "10.223.32.67"
}
//文檔2
{
"log_content" : "call bbb service error",
"date" : "2020-04-15T11:09:08",
"log_level": "error",
"ip": "10.223.32.67"
}
當搜索條件為call aaa service時,實際上會把兩個文檔都搜索出來。 這是因為在搜索時,條件call aaa service會被分詞為call,aaa和service,所有包含這三個詞的文檔都會被搜索出來,例如下面:
//普通搜索,兩個文檔都會返回
POST javalogs/_search
{
"query": {
"match": {
"log_content": "call aaa service"
}
}
}
那如果想要只搜索包含call aaa service的文檔,應該如何做呢?
按照上面的分析,需要同時包含這三個詞,并且按照給定的順序,才返回對應的文檔,那么這個可以使用match_phrase實現,示例如下:
//文檔必須同時包含三個詞,并且順序與搜索條件一致才會返回。這里只會返回-文檔1
POST javalogs/_search
{
"profile": "true",
"query": {
"match_phrase": {
"log_content": "call aaa service"
}
}
}
那如果條件是包含call,aaa和service,但是不一定是連著的,該如何搜索呢?可以使用operator操作符實現。
例如有第三個文檔如下:
//文檔3
{
"log_content" : "call inner aaa service error",
"date" : "2020-04-15T11:09:08",
"log_level": "error",
"ip": "10.223.32.67"
}
要想把文檔1和文檔2都搜索出來,查詢的示例如下:
//文檔中同時包含call,aaa和service就會返回,不按順序。會返回-文檔1和文檔2
POST javalogs/_search
{
"query": {
"match": {
"log_content":
{
"query": "call aaa service",
"operator": "and"
}
}
}
}
上面就是對Elasticsearch的簡單介紹和實戰操作示例,希望能幫助大家快速入門使用ES。
以上內容屬個人學習總結,如有不當之處,歡迎在評論中指正
在前面一篇文章(坐標語法)里講到,英語詞類可以分為不變詞(介詞,冠詞,連詞與感嘆詞)與變態詞(名詞,代詞,形容詞,副詞,動詞,數詞)。對于不變詞,我們主要是掌握其用法,對于變態詞主要是掌握其變法。
一.介詞的用法:
介詞是一種沒有形態變化,不能單獨使用,介于動詞與名詞,名詞與名詞之間或名詞之前的一種虛詞。介詞不能單獨使用,必須帶上賓語構成介詞短語,或依附在不及物動詞后面使用。英語不及物動詞不能直接帶賓語,必須加上適當的介詞才能帶賓語。介詞短語可以在句子中充當狀語,定語,表語和賓語補語。介詞的具體用法如下:
介詞位置 | 語法功能 | 例詞、例句 |
位于動詞之后 | 使不及物動詞能帶賓語 | listen to , look at , wait for , knock at , worry about , talk with / about , work on , dream about , depend on, 等。 |
位于動詞之后 | 介詞依附在及物或不及物動詞后,構成固定短語 | look after照看 look for 尋找look up 查找look through 瀏覽hear from 收到來信think of 想起ask for 索要try on 試穿try out 試用send for 派人去請lead to 導致laugh at 嘲笑give in 屈服turn off/on 關\開(電器)stand for 代表break into 闖入hand in 上交work out算出 take off 脫掉,起飛等。 |
位于形容詞之后 | 系表結構中,后面的形容詞加上一定的介詞構成句型 | be good at 擅長于 be good /bad for 對…有好/壞處be famous for 因…面聞名be proud of 對…感到驕傲be difficult for對…有困難 be strict with 對…嚴格 be wrong with出毛病 be late for遲到 be interested in對…感興趣 |
位于名詞之前 | 介詞位于表時間的名詞前,構成短語做時間狀語 | at ,in ,on, for ,during ,since ,before ,after ,by, from, until ,within 等。 |
位于名詞之前 | 介詞位于表方位的名詞前,構成短語做地點狀語 | in ,at ,on ,under ,over ,above ,below, into, through, cross ,between, along, beside, behind from等。 |
位于名詞之前 | 介詞位于名詞前,構成短語作狀語,表示比較、方式關系 | in,by, with ,than, as ,like,at等。 |
位于名詞之前 | 介詞位于名詞前,構成短語作狀語,表示原因、條件關系 | for, becaus of ,with ,without, besides, except等。 |
位于名詞之前 | 介詞位于名詞前,構成固定短語 | by the way 順便on time 準時in time 及時in fact 事實上door to door挨家挨戶 |
位于名詞之間 | 介詞位于名詞之間,構成短語作定語 | The man with glasses over there is my bother.那邊帶眼鏡那個男的是我的兄弟。 |
二:介詞短語與短語介詞
介詞短語與短語介詞是兩個不同的概念,介詞短語的詞性為介詞,在句子中不能單獨使用,而短語介詞是由介詞加上賓語構成,其詞性相當于形容詞或副詞。中學介詞短語與短語介詞歸納如下:
1.短語介詞歸納:
1. according to 根據 2. because of 由于
3. in spite of 盡管 4.next to 緊鄰; 在…近旁; 僅次于
5. in front of 在…前面 6. ahead of在…之前
7. on behalf of 代表 8. with reference to 關于,參照
9. but for 要不是 10. apart from 除了,(除…以外尚有)
11. except for (不同類中的)除此以外 12. as for至于;就…方面說來
13. in regard to 關于;對于 14. in terms of根據; 以…為單位
15. instead of 而不是;反而 16. nothing but 僅僅;只不過
17. more than 超過; 不只是 18. other than 不同于
19. all over 遍及 20. out of . 出于;出自
21. such as 比如 22. thanks to幸虧; 由于
23. due to 由于;因…造成 24. up to 達到;多達
25. in need of 需要;缺少 26. in favor of贊同; 支持
27. in charge of 主管;負責 28. in honor of為慶祝;為紀念
29. by means of 依靠; 借助于 30. for fear of以免;以防
31. in search of 搜索; 為了尋找 32. in touch with與…保持聯絡
33. regardless of 不顧 34.as far as 遠至,到...程度
35. as for 至于;關于 35. far from 遠非;遠離
36. fond of喜歡 37. in accordance with 與...一致,按照
38. in addition to 除...之外(還) 39. in case of 假如,防備
40. in line with 與...一致 41. in place of 代替,取代,交換
42. in return for 作為對...報答 43. in step with 與...一致/協調
44. in the course of 在...期間/過程中 45. in the event of 如果...發生,萬一
46. on the point of 即將...的時刻 47. opposite to在對面
48. short of缺少 49. as far as 遠至,到...程度
50. as well as 和……一樣 51. as good as 和...幾乎一樣
52. for the sake of 為了,為了...的利益
2.介詞短語歸納:
1. as a matter of fact 實際上 2. as a result(of) 因此,由于
3. as a rule 通常,照例 4. as far as 就...而言
5. as follows 如下 6. as usual 像平常一樣,照例
7. as well 同樣,也,還 8. at a loss 茫然,不知所措
9. at a time 一次,每次 10. at all costs 不惜一切代價
11. at all events 不管怎樣,無論如何 12. at all times 隨時,總是
13. at all 絲毫(不),一點也不 14. at any rate 無論如何,至少
15. at best 充其量,至多 16. at first sight 乍一看,初看起來
17. at first 最初,起先 18. at hand 在手邊,在附近
19. at home 在家,在國內 20. at intervals 不時,每隔...
21. at large 大多數,未被捕獲的 22. at last 終于
23. at least 至少 24 at length 最終,終于
25. at most 至多,不超過 26. at no time 從不,決不
27 at one time 曾經,一度;同時 28. at present 目前,現在
29. at someone's disposal 任...處理 30. at the cost of 以...為代價
31. at the mercy of 任憑...擺布 32. at the moment 此刻,目前
33. at this rate 照此速度 34. at times 有時,間或
35. beside point 離題的,不相干的 36. beyond one's ability超越某人的能力
37. beyond question 毫無疑問 38. by accident 偶然
39. by all means 盡一切辦法,務必 40. by chance 偶然,碰巧
41. by far 最,...得多 42. by hand 用手,用體力
43. by oneself 自動地,獨自地 44. by means of 用,依靠
45. by mistake 錯誤地,無意地 46. by no means 決不,并沒有
47. by the way 順便說說 48. for good 永久地
49. for the better 好轉 50. for the moment 暫時,目前
51. for the time being 暫時,眼下 52. in a hurry 匆忙,急于
53. in a moment 立刻,一會兒 54. in a sense 從某種意義上說
55. in a way 在某種程度上 56. in a word 簡言之,總之
57. in addition 另外,加之 58. in advance 預先,事先
59. in all 總共,合計 60. in any case 無論如何
61. in brief 簡單地說 62.in common 共用的,共有的
63. in debt 欠債,欠情 64. in detail 詳細地
65. in general 一般來說,大體上 66. in hand 在進行中,待辦理
67. in no case 決不 68. in no time 立即,馬上
69. in no way 決不 70. in order 按順序,按次序
71. in other words 換句話說 72. in particular 特別,尤其
73. in person 親自,本人 74.in place 在合適的位置,就位
75. in public 公開地,當眾 76. in return 作為報答/回報/交換
77. in short 簡言之,總之 78. in sight 被見到;在望
79. in tears 流著淚,在哭著 80.in the distance 在遠處
81. in the end 最后,終于 82.in the least 絲毫,一點
83. in the long run 長期 84. in the way 擋道
85. in time 及時 86. in turn 依次,輪流;轉而
87. in vain 徒勞,白費力 88.off duty 下班
89. on a large/small scale 大/小規模地 90. on board 在船(車/飛機)上
91. on business 因公 92. on duty 上班,值班
93. on earth 究竟,到底 94. on fire 起火著火
95. on foot 步行 96. on one's own 獨立,獨自
97. on purpose 故意地 98. on sale 出售,廉價出售
99. on second thoughts 經重新考慮 100. on the contrary 正相反
101. on the spot 在場;馬上 102. on the whole 總的來說,大體上
103. on time 準時 104. on(an/the) average 平均,通常
105.out of breath 喘不過氣來 106. out of control 失去控制
107. out of date 過時的 108. out of order 出故障的
109. out of place 不適當的 110. out of sight 看不見,在視野外
111. out of the question 毫無可能的
三.介詞考考你(用適當的介詞填空)
1. There were a lot of people standing at the door and the small girl couldn’t get ________ .
2. Careless driving often brings ________ an accident.
3. ________ time going by, the boy has grown into a strong man.
4. The girl danced ________ the music of Paul Mallrat’s band.
5. This song is familiar ________ all the students.
6. He rent a house ________ the year.
7. We know nothing about him ________ that his son joined in the army.
8. One ________ five will have the chance to join in the game.
9. It was a real race ________ time to get the project done. Luckily, we made it.
10. A serious study of physics is impossible ________ some knowledge of mathematics.
11.The Smiths are praised _______ the way they bring up their children.
12.An artist who was recently traveling on a ferry to the southern island discovered ______ chance a long lost antique Greek vase.
13.Before you pay a visit to a place of interest, look in your local library ______ a book.
14.I have an appointment_______ Dr. Smith, but I need to change it.
15. The monkey the tree was eating the bananas the tree.
16. You can go there my car.
17. Tom! You are wanted the phone.
18. As you can see, the bridge is made stone.
19. He suddenly returned a rainy night.
20. I’m sorry; I took your umbrella mistake.
21. You look pale. What’s wrong you?
22. This village is well-known its beautiful sights.
23. If you find my book, please call me ______ 31397671.
24. Betty only watches football ___________ TV.
25. Ann is my good friend. She often helps me ________ math.
26. Miss Gao is very strict ________ her students ________ their studies.
27. The girl ______ the purple coat is his sister.
28. You’d better take a raincoat ____you.
29. He has lived here ___ 1965.
30. When we see stars twinkling in the sky at night we may wonder that ___ so many starts in the universe there is life out there in space.
31. Her parents went away ___ business, so she had to stay at home alone.
32. Don’t waste water. Water is very important and no one can live __________it.
33. Bill has made great progress ______ the help of his teachers.
34. They were astonished _____ the news of his escape from prison.
35. He was absorbed ____ a book and didn’t hear your repeated call.
36. The dying man is _____ the help of the doctor.
37. I called _____ you last night, but you were out.
38. Can you say _____ certain when he will come here?
39. The boy is curious _____ everything. I am sure he is very clever.
40. Our national scenery Zhangjiajie is so charming that it is _____ description.
41. The struggle against sand storm will not end _____ victory unless we pay much attention to the balance of nature.
42. In order to finish the work in time,they always worked far______ the night.
43. When we heard the news that our team has won more than 30 gold medals Athena Olympic Games, we sang and danced _____ joy.
44. It is reported that the thirty-ninth president of America Jimmy Carter has won the Nobel Prize _____ peace in the year of 2002.
45. When he was in America, he made a journey of seven hundred miles _____ purpose to get a glimpse of the Niagara Falls.