跨域资源共享以及Spring MVC的支持

本来想写文章详细整理一下「跨域资源共享,以及在Spring MVC中如何实现」,但是发现网站上已经有很多文章总结了,
相信我在怎么写也和绝大多数文章的差别不大,所以我选中了5篇比较不错的文章(地址在文章参考资料中)作为引用。
所以读者直接看引用的这几篇文章就好了。

##参考资料

使用七牛做博客图床

经过对比了国内的一些图床以后,最后还是选择了七牛做图床。免费版的对于我这个小博客完全就够用了。
操作方式如下:

  • 首先注册七牛账号,然后并认证。认证方式我选择的是支付宝认证。
  • 然后创建一个空间

  • 在「对象管理」->「存储资源列表」->「添加」

  • 这块会生成一个测试域名,这个域名绝大多数情况下就够用了

  • 然后推荐使用极简图床,配置好自己的空间信息就可以使用了

  • 配置的时候域名写自己之前空间的测试域名就好了。至于空间名称就写自己之前创建的空间的名称

  • AK和SK在「个人面板」->「密匙管理」中可以找到

本地搭建dubbo的运行环境

本篇文章主要讲述如何在本地搭建dubbo的运行环境

安装zk

dubbo推荐使用zk来作为自己的注册中心,当然使用其余的实现来作为注册中心也是可以的。比如我之前就使用redis实现了一个注册中心。

Dubbo 未对 Zookeeper 服务器端做任何侵入修改,只需安装原生的 Zookeeper 服务器即可,
所有注册中心逻辑适配都在调用 Zookeeper 客户端时完成。

安装zk:

1
2
3
4
wget http://archive.apache.org/dist/zookeeper/zookeeper-3.3.3/zookeeper-3.3.3.tar.gz
tar zxvf zookeeper-3.3.3.tar.gz
cd zookeeper-3.3.3
cp conf/zoo_sample.cfg conf/zoo.cfg

配置:

1
vi conf/zoo.cfg

如果不需要集群,zoo.cfg 的内容如下 :

1
2
3
4
5
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/home/dubbo/zookeeper-3.3.3/data
clientPort=2181

如果需要集群,zoo.cfg 的内容如下(其中 data 目录和 server 地址需改成你真实部署机器的信息)

1
2
3
4
5
6
7
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/home/dubbo/zookeeper-3.3.3/data
clientPort=2181
server.1=10.20.153.10:2555:3555
server.2=10.20.153.11:2555:3555

并在 data 目录下放置 myid 文件:

1
2
mkdir data
vi myid

myid 指明自己的 id,对应上面 zoo.cfgserver. 后的数字,第一台的内容为 1,第二台的内容为 2,内容如下:

1
1

启动

1
./bin/zkServer.sh start

如果是windows环境的话,直接双击zkServer.cmd文件。不过建议修改zkServer.cmd文件,在文件的最末尾处增加一行,内容为pause,修改完以后zkServer.cmd文件
的内容为

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
@echo off
REM Licensed to the Apache Software Foundation (ASF) under one or more
REM contributor license agreements. See the NOTICE file distributed with
REM this work for additional information regarding copyright ownership.
REM The ASF licenses this file to You under the Apache License, Version 2.0
REM (the "License"); you may not use this file except in compliance with
REM the License. You may obtain a copy of the License at
REM
REM http://www.apache.org/licenses/LICENSE-2.0
REM
REM Unless required by applicable law or agreed to in writing, software
REM distributed under the License is distributed on an "AS IS" BASIS,
REM WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
REM See the License for the specific language governing permissions and
REM limitations under the License.

setlocal
call "%~dp0zkEnv.cmd"

set ZOOMAIN=org.apache.zookeeper.server.quorum.QuorumPeerMain
echo on
call %JAVA% "-Dzookeeper.log.dir=%ZOO_LOG_DIR%" "-Dzookeeper.root.logger=%ZOO_LOG4J_PROP%" -cp "%CLASSPATH%" %ZOOMAIN% "%ZOOCFG%" %*

endlocal

pause

这样当系统环境有问题的时候就可以在终端看见具体是啥问题了。

停止:

1
./bin/zkServer.sh stop
1
2
telnet 127.0.0.1 2181
dump

或者:

1
echo dump | nc 127.0.0.1 2181

用法:

1
dubbo.registry.address=zookeeper://10.20.153.10:2181?backup=10.20.153.11:2181

或者:

1
<dubbo:registry protocol="zookeeper" address="10.20.153.10:2181,10.20.153.11:2181" />

比如以dubbo-demo-provider中的dubbo-demo-provider.xml文件为例子,使用本地的zk修改以后内容为:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
<beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:dubbo="http://dubbo.apache.org/schema/dubbo"
xmlns="http://www.springframework.org/schema/beans"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.3.xsd
http://dubbo.apache.org/schema/dubbo http://dubbo.apache.org/schema/dubbo/dubbo.xsd">

<!-- provider's application name, used for tracing dependency relationship -->
<dubbo:application name="demo-provider"/>

<!-- use multicast registry center to export service -->
<!--<dubbo:registry address="multicast://224.5.6.7:1234"/>-->
<dubbo:registry protocol="zookeeper" address="localhost:2181"/>

<!-- use dubbo protocol to export service on port 20880 -->
<dubbo:protocol name="dubbo" port="20880"/>

<!-- service implementation, as same as regular local bean -->
<bean id="demoService" class="org.apache.dubbo.demo.provider.DemoServiceImpl"/>

<!-- declare the service interface to be exported -->
<dubbo:service interface="org.apache.dubbo.demo.DemoService" ref="demoService"/>

</beans>

安装dubbo

其实就是下载dubbo的源码以及对应的maven依赖:

1
2
3
4
git clone https://github.com/apache/incubator-dubbo.git
cd incubator-dubbo
运行 dubbo-demo-provider中的com.alibaba.dubbo.demo.provider.Provider
如果使用Intellij Idea 请加上-Djava.net.preferIPv4Stack=true

配置:

1
2
resource/META-INFO.spring/dubbo-demo-provider.xml
修改其中的dubbo:registery,替换成真实的注册中心地址,推荐使用zookeeper

运行demo

启动provider

本地启动zk以后,运行dubbo-demo-provider中的Provider类,日志输出为

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
Connected to the target VM, address: '127.0.0.1:7562', transport: 'socket'
[20/08/18 11:43:20:020 CST] main INFO support.ClassPathXmlApplicationContext: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@551aa95a: startup date [Mon Aug 20 23:43:20 CST 2018]; root of context hierarchy
[20/08/18 11:43:20:020 CST] main INFO xml.XmlBeanDefinitionReader: Loading XML bean definitions from class path resource [META-INF/spring/dubbo-demo-provider.xml]
[20/08/18 11:43:20:020 CST] main INFO logger.LoggerFactory: using logger: org.apache.dubbo.common.logger.log4j.Log4jLoggerAdapter
[20/08/18 11:43:20:020 CST] main WARN extension.SpringExtensionFactory: [DUBBO] No spring extension(bean) named:defaultCompiler, try to find an extension(bean) of type java.lang.String, dubbo version: , current host: 192.168.116.1
[20/08/18 11:43:20:020 CST] main WARN extension.SpringExtensionFactory: [DUBBO] No spring extension(bean) named:defaultCompiler, type:java.lang.String found, stop get bean., dubbo version: , current host: 192.168.116.1
[20/08/18 11:43:20:020 CST] main INFO config.AbstractConfig: [DUBBO] The service ready on spring started. service: org.apache.dubbo.demo.DemoService, dubbo version: , current host: 192.168.116.1
[20/08/18 11:43:21:021 CST] main INFO config.AbstractConfig: [DUBBO] Export dubbo service org.apache.dubbo.demo.DemoService to local registry, dubbo version: , current host: 192.168.116.1
[20/08/18 11:43:21:021 CST] main INFO config.AbstractConfig: [DUBBO] Export dubbo service org.apache.dubbo.demo.DemoService to url dubbo://192.168.116.1:20880/org.apache.dubbo.demo.DemoService?anyhost=true&application=demo-provider&bind.ip=192.168.116.1&bind.port=20880&dubbo=2.0.2&generic=false&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=6872&qos.port=22222&side=provider&timestamp=1534779801013, dubbo version: , current host: 192.168.116.1
[20/08/18 11:43:21:021 CST] main INFO config.AbstractConfig: [DUBBO] Register dubbo service org.apache.dubbo.demo.DemoService url dubbo://192.168.116.1:20880/org.apache.dubbo.demo.DemoService?anyhost=true&application=demo-provider&bind.ip=192.168.116.1&bind.port=20880&dubbo=2.0.2&generic=false&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=6872&qos.port=22222&side=provider&timestamp=1534779801013 to registry registry://localhost:2181/org.apache.dubbo.registry.RegistryService?application=demo-provider&dubbo=2.0.2&pid=6872&qos.port=22222&registry=zookeeper&timestamp=1534779801006, dubbo version: , current host: 192.168.116.1
[20/08/18 11:43:22:022 CST] main INFO transport.AbstractServer: [DUBBO] Start NettyServer bind /0.0.0.0:20880, export /192.168.116.1:20880, dubbo version: , current host: 192.168.116.1
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZookeeperRegistry: [DUBBO] Load registry store file C:\Users\wenchao.ren\.dubbo\dubbo-registry-demo-provider-localhost:2181.cache, data: {org.apache.dubbo.demo.DemoService=empty://192.168.116.1:20880/org.apache.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.2&generic=false&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=18588&side=provider&timestamp=1534778576083}, dubbo version: , current host: 192.168.116.1
[20/08/18 11:43:22:022 CST] main INFO imps.CuratorFrameworkImpl: Starting
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.9-1757313, built on 08/23/2016 06:50 GMT
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZooKeeper: Client environment:host.name=c
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZooKeeper: Client environment:java.version=1.8.0_181
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZooKeeper: Client environment:java.home=C:\Program Files\Java\jdk1.8.0_181\jre
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZooKeeper: Client environment:java.class.path=C:\Program Files\Java\jdk1.8.0_181\jre\lib\charsets.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\deploy.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\access-bridge-64.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\cldrdata.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\dnsns.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\jaccess.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\jfxrt.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\localedata.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\nashorn.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\sunec.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\sunjce_provider.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\sunmscapi.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\sunpkcs11.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\zipfs.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\javaws.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\jce.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\jfr.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\jfxswt.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\jsse.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\management-agent.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\plugin.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\resources.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\rt.jar;E:\workstations\dubbo\incubator-dubbo\dubbo-demo\dubbo-demo-provider\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-demo\dubbo-demo-api\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-config\dubbo-config-spring\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-config\dubbo-config-api\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-monitor\dubbo-monitor-api\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-filter\dubbo-filter-validation\target\classes;C:\Users\wenchao.ren\.m2\repository\javax\validation\validation-api\1.1.0.Final\validation-api-1.1.0.Final.jar;E:\workstations\dubbo\incubator-dubbo\dubbo-filter\dubbo-filter-cache\target\classes;C:\Users\wenchao.ren\.m2\repository\javax\cache\cache-api\1.0.0\cache-api-1.0.0.jar;E:\workstations\dubbo\incubator-dubbo\dubbo-bootstrap\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-common\target\classes;C:\Users\wenchao.ren\.m2\repository\commons-logging\commons-logging\1.2\commons-logging-1.2.jar;C:\Users\wenchao.ren\.m2\repository\log4j\log4j\1.2.16\log4j-1.2.16.jar;C:\Users\wenchao.ren\.m2\repository\org\javassist\javassist\3.20.0-GA\javassist-3.20.0-GA.jar;C:\Users\wenchao.ren\.m2\repository\org\springframework\spring-beans\4.3.16.RELEASE\spring-beans-4.3.16.RELEASE.jar;C:\Users\wenchao.ren\.m2\repository\org\springframework\spring-core\4.3.16.RELEASE\spring-core-4.3.16.RELEASE.jar;C:\Users\wenchao.ren\.m2\repository\org\springframework\spring-web\4.3.16.RELEASE\spring-web-4.3.16.RELEASE.jar;C:\Users\wenchao.ren\.m2\repository\org\springframework\spring-aop\4.3.16.RELEASE\spring-aop-4.3.16.RELEASE.jar;C:\Users\wenchao.ren\.m2\repository\org\springframework\spring-context\4.3.16.RELEASE\spring-context-4.3.16.RELEASE.jar;C:\Users\wenchao.ren\.m2\repository\org\springframework\spring-expression\4.3.16.RELEASE\spring-expression-4.3.16.RELEASE.jar;E:\workstations\dubbo\incubator-dubbo\dubbo-rpc\dubbo-rpc-injvm\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-registry\dubbo-registry-zookeeper\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-registry\dubbo-registry-api\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-cluster\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-remoting\dubbo-remoting-zookeeper\target\classes;C:\Users\wenchao.ren\.m2\repository\org\apache\zookeeper\zookeeper\3.4.9\zookeeper-3.4.9.jar;C:\Users\wenchao.ren\.m2\repository\org\slf4j\slf4j-api\1.7.25\slf4j-api-1.7.25.jar;C:\Users\wenchao.ren\.m2\repository\org\slf4j\slf4j-log4j12\1.6.1\slf4j-log4j12-1.6.1.jar;C:\Users\wenchao.ren\.m2\repository\jline\jline\0.9.94\jline-0.9.94.jar;C:\Users\wenchao.ren\.m2\repository\io\netty\netty\3.10.5.Final\netty-3.10.5.Final.jar;C:\Users\wenchao.ren\.m2\repository\com\101tec\zkclient\0.2\zkclient-0.2.jar;C:\Users\wenchao.ren\.m2\repository\org\apache\curator\curator-framework\2.12.0\curator-framework-2.12.0.jar;C:\Users\wenchao.ren\.m2\repository\org\apache\curator\curator-client\2.12.0\curator-client-2.12.0.jar;C:\Users\wenchao.ren\.m2\repository\com\google\guava\guava\16.0.1\guava-16.0.1.jar;E:\workstations\dubbo\incubator-dubbo\dubbo-registry\dubbo-registry-multicast\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-rpc\dubbo-rpc-dubbo\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-rpc\dubbo-rpc-api\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-remoting\dubbo-remoting-api\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-container\dubbo-container-api\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-remoting\dubbo-remoting-netty4\target\classes;C:\Users\wenchao.ren\.m2\repository\io\netty\netty-all\4.1.25.Final\netty-all-4.1.25.Final.jar;E:\workstations\dubbo\incubator-dubbo\dubbo-serialization\dubbo-serialization-hessian2\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-serialization\dubbo-serialization-api\target\classes;C:\Users\wenchao.ren\.m2\repository\com\alibaba\fastjson\1.2.46\fastjson-1.2.46.jar;C:\Users\wenchao.ren\.m2\repository\com\esotericsoftware\kryo\4.0.1\kryo-4.0.1.jar;C:\Users\wenchao.ren\.m2\repository\com\esotericsoftware\reflectasm\1.11.3\reflectasm-1.11.3.jar;C:\Users\wenchao.ren\.m2\repository\org\ow2\asm\asm\5.0.4\asm-5.0.4.jar;C:\Users\wenchao.ren\.m2\repository\com\esotericsoftware\minlog\1.3.0\minlog-1.3.0.jar;C:\Users\wenchao.ren\.m2\repository\de\javakaffee\kryo-serializers\0.42\kryo-serializers-0.42.jar;C:\Users\wenchao.ren\.m2\repository\de\ruedigermoeller\fst\2.48-jdk-6\fst-2.48-jdk-6.jar;C:\Users\wenchao.ren\.m2\repository\com\fasterxml\jackson\core\jackson-core\2.8.6\jackson-core-2.8.6.jar;C:\Users\wenchao.ren\.m2\repository\com\cedarsoftware\java-util\1.9.0\java-util-1.9.0.jar;C:\Users\wenchao.ren\.m2\repository\com\cedarsoftware\json-io\2.5.1\json-io-2.5.1.jar;C:\Users\wenchao.ren\.m2\repository\com\alibaba\hessian-lite\3.2.3\hessian-lite-3.2.3.jar;C:\Users\wenchao.ren\.m2\repository\org\objenesis\objenesis\2.6\objenesis-2.6.jar;C:\Users\wenchao.ren\AppData\Local\JetBrains\Toolbox\apps\IDEA-U\ch-0\182.3684.40\lib\idea_rt.jar;C:\Users\wenchao.ren\AppData\Local\JetBrains\Toolbox\apps\IDEA-U\ch-0\182.3684.40\lib\rt\debugger-agent.jar
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZooKeeper: Client environment:java.library.path=C:\Program Files\Java\jdk1.8.0_181\bin;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files\Git\cmd;C:\Program Files\Microsoft VS Code\bin;C:\WINDOWS\System32\OpenSSH\;C:\Users\wenchao.ren\AppData\Local\Microsoft\WindowsApps;;.
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=C:\Users\wenchao.ren\AppData\Local\Temp\
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZooKeeper: Client environment:os.name=Windows 10
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZooKeeper: Client environment:os.version=10.0
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZooKeeper: Client environment:user.name=wenchao.ren
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZooKeeper: Client environment:user.home=C:\Users\wenchao.ren
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZooKeeper: Client environment:user.dir=E:\workstations\dubbo\incubator-dubbo
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@3dd69f5a
[20/08/18 11:43:22:022 CST] main-SendThread(127.0.0.1:2181) INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
[20/08/18 11:43:22:022 CST] main-SendThread(127.0.0.1:2181) INFO zookeeper.ClientCnxn: Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZookeeperRegistry: [DUBBO] Register: dubbo://192.168.116.1:20880/org.apache.dubbo.demo.DemoService?anyhost=true&application=demo-provider&dubbo=2.0.2&generic=false&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=6872&side=provider&timestamp=1534779801013, dubbo version: , current host: 192.168.116.1
[20/08/18 11:43:22:022 CST] main-SendThread(127.0.0.1:2181) INFO zookeeper.ClientCnxn: Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x100001d37250002, negotiated timeout = 40000
[20/08/18 11:43:22:022 CST] main-EventThread INFO state.ConnectionStateManager: State change: CONNECTED
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZookeeperRegistry: [DUBBO] Subscribe: provider://192.168.116.1:20880/org.apache.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.2&generic=false&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=6872&side=provider&timestamp=1534779801013, dubbo version: , current host: 192.168.116.1
[20/08/18 11:43:22:022 CST] main INFO zookeeper.ZookeeperRegistry: [DUBBO] Notify urls for subscribe url provider://192.168.116.1:20880/org.apache.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.2&generic=false&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=6872&side=provider&timestamp=1534779801013, urls: [empty://192.168.116.1:20880/org.apache.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.2&generic=false&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=6872&side=provider&timestamp=1534779801013], dubbo version: , current host: 192.168.116.1

同时我们查看zk的话会发现:

1
2
3
4
5
6
7
8
9
10
[zk: localhost:2181(CONNECTED) 1] ls /dubbo
[org.apache.dubbo.demo.DemoService]
[zk: localhost:2181(CONNECTED) 2] ls /dubbo/org.apache.dubbo.demo.DemoService
[configurators, providers]
[zk: localhost:2181(CONNECTED) 3]
[zk: localhost:2181(CONNECTED) 3] ls /dubbo/org.apache.dubbo.demo.DemoService/providers
[dubbo%3A%2F%2F192.168.116.1%3A20880%2Forg.apache.dubbo.demo.DemoService%3Fanyhost%3Dtrue%26application%3Ddemo-provider%26dubbo%3D2.0.2%26generic%3Dfalse%26interface%3Dorg.apache.dubbo.demo.DemoService%26methods%3DsayHello%26pid%3D6872%26side%3Dprovider%26timestamp%3D1534779801013]
[zk: localhost:2181(CONNECTED) 4]
[zk: localhost:2181(CONNECTED) 4] ls /dubbo/org.apache.dubbo.demo.DemoService/configurators
[]

当然此时我们仅仅启动了provider,并没有启动consumer

启动consumer

同样的修改dubbo-demo-consumer模块中的dubbo-demo-consumer.xml中的注册中心,使得consumer和provider连接同一个注册中心,
修改后的内容为:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:dubbo="http://dubbo.apache.org/schema/dubbo"
xmlns="http://www.springframework.org/schema/beans"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.3.xsd
http://dubbo.apache.org/schema/dubbo http://dubbo.apache.org/schema/dubbo/dubbo.xsd">

<!-- consumer's application name, used for tracing dependency relationship (not a matching criterion),
don't set it same as provider -->
<dubbo:application name="demo-consumer"/>

<!-- use multicast registry center to discover service -->
<!--<dubbo:registry address="multicast://224.5.6.7:1234"/>-->
<dubbo:registry protocol="zookeeper" address="localhost:2181"/>

<!-- generate proxy for the remote service, then demoService can be used in the same way as the
local regular interface -->
<dubbo:reference id="demoService" check="false" interface="org.apache.dubbo.demo.DemoService"/>

</beans>

然后运行Consumer类,输出如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
Connected to the target VM, address: '127.0.0.1:8434', transport: 'socket'
[20/08/18 11:49:40:040 CST] main INFO support.ClassPathXmlApplicationContext: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@551aa95a: startup date [Mon Aug 20 23:49:40 CST 2018]; root of context hierarchy
[20/08/18 11:49:40:040 CST] main INFO xml.XmlBeanDefinitionReader: Loading XML bean definitions from class path resource [META-INF/spring/dubbo-demo-consumer.xml]
[20/08/18 11:49:40:040 CST] main INFO logger.LoggerFactory: using logger: org.apache.dubbo.common.logger.log4j.Log4jLoggerAdapter
[20/08/18 11:49:41:041 CST] main WARN extension.SpringExtensionFactory: [DUBBO] No spring extension(bean) named:defaultCompiler, try to find an extension(bean) of type java.lang.String, dubbo version: , current host: 10.254.0.157
[20/08/18 11:49:41:041 CST] main WARN extension.SpringExtensionFactory: [DUBBO] No spring extension(bean) named:defaultCompiler, type:java.lang.String found, stop get bean., dubbo version: , current host: 10.254.0.157
[20/08/18 11:49:42:042 CST] main INFO imps.CuratorFrameworkImpl: Starting
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.9-1757313, built on 08/23/2016 06:50 GMT
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZooKeeper: Client environment:host.name=c
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZooKeeper: Client environment:java.version=1.8.0_181
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZooKeeper: Client environment:java.home=C:\Program Files\Java\jdk1.8.0_181\jre
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZooKeeper: Client environment:java.class.path=C:\Program Files\Java\jdk1.8.0_181\jre\lib\charsets.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\deploy.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\access-bridge-64.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\cldrdata.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\dnsns.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\jaccess.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\jfxrt.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\localedata.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\nashorn.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\sunec.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\sunjce_provider.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\sunmscapi.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\sunpkcs11.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\ext\zipfs.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\javaws.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\jce.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\jfr.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\jfxswt.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\jsse.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\management-agent.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\plugin.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\resources.jar;C:\Program Files\Java\jdk1.8.0_181\jre\lib\rt.jar;E:\workstations\dubbo\incubator-dubbo\dubbo-demo\dubbo-demo-consumer\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-demo\dubbo-demo-api\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-config\dubbo-config-spring\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-config\dubbo-config-api\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-monitor\dubbo-monitor-api\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-filter\dubbo-filter-validation\target\classes;C:\Users\wenchao.ren\.m2\repository\javax\validation\validation-api\1.1.0.Final\validation-api-1.1.0.Final.jar;E:\workstations\dubbo\incubator-dubbo\dubbo-filter\dubbo-filter-cache\target\classes;C:\Users\wenchao.ren\.m2\repository\javax\cache\cache-api\1.0.0\cache-api-1.0.0.jar;E:\workstations\dubbo\incubator-dubbo\dubbo-bootstrap\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-common\target\classes;C:\Users\wenchao.ren\.m2\repository\commons-logging\commons-logging\1.2\commons-logging-1.2.jar;C:\Users\wenchao.ren\.m2\repository\log4j\log4j\1.2.16\log4j-1.2.16.jar;C:\Users\wenchao.ren\.m2\repository\org\javassist\javassist\3.20.0-GA\javassist-3.20.0-GA.jar;C:\Users\wenchao.ren\.m2\repository\org\springframework\spring-beans\4.3.16.RELEASE\spring-beans-4.3.16.RELEASE.jar;C:\Users\wenchao.ren\.m2\repository\org\springframework\spring-core\4.3.16.RELEASE\spring-core-4.3.16.RELEASE.jar;C:\Users\wenchao.ren\.m2\repository\org\springframework\spring-web\4.3.16.RELEASE\spring-web-4.3.16.RELEASE.jar;C:\Users\wenchao.ren\.m2\repository\org\springframework\spring-aop\4.3.16.RELEASE\spring-aop-4.3.16.RELEASE.jar;C:\Users\wenchao.ren\.m2\repository\org\springframework\spring-context\4.3.16.RELEASE\spring-context-4.3.16.RELEASE.jar;C:\Users\wenchao.ren\.m2\repository\org\springframework\spring-expression\4.3.16.RELEASE\spring-expression-4.3.16.RELEASE.jar;E:\workstations\dubbo\incubator-dubbo\dubbo-rpc\dubbo-rpc-injvm\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-registry\dubbo-registry-zookeeper\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-registry\dubbo-registry-api\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-cluster\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-remoting\dubbo-remoting-zookeeper\target\classes;C:\Users\wenchao.ren\.m2\repository\org\apache\zookeeper\zookeeper\3.4.9\zookeeper-3.4.9.jar;C:\Users\wenchao.ren\.m2\repository\org\slf4j\slf4j-api\1.7.25\slf4j-api-1.7.25.jar;C:\Users\wenchao.ren\.m2\repository\org\slf4j\slf4j-log4j12\1.6.1\slf4j-log4j12-1.6.1.jar;C:\Users\wenchao.ren\.m2\repository\jline\jline\0.9.94\jline-0.9.94.jar;C:\Users\wenchao.ren\.m2\repository\io\netty\netty\3.10.5.Final\netty-3.10.5.Final.jar;C:\Users\wenchao.ren\.m2\repository\com\101tec\zkclient\0.2\zkclient-0.2.jar;C:\Users\wenchao.ren\.m2\repository\org\apache\curator\curator-framework\2.12.0\curator-framework-2.12.0.jar;C:\Users\wenchao.ren\.m2\repository\org\apache\curator\curator-client\2.12.0\curator-client-2.12.0.jar;C:\Users\wenchao.ren\.m2\repository\com\google\guava\guava\16.0.1\guava-16.0.1.jar;E:\workstations\dubbo\incubator-dubbo\dubbo-registry\dubbo-registry-multicast\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-rpc\dubbo-rpc-dubbo\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-rpc\dubbo-rpc-api\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-remoting\dubbo-remoting-api\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-container\dubbo-container-api\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-remoting\dubbo-remoting-netty4\target\classes;C:\Users\wenchao.ren\.m2\repository\io\netty\netty-all\4.1.25.Final\netty-all-4.1.25.Final.jar;E:\workstations\dubbo\incubator-dubbo\dubbo-serialization\dubbo-serialization-hessian2\target\classes;E:\workstations\dubbo\incubator-dubbo\dubbo-serialization\dubbo-serialization-api\target\classes;C:\Users\wenchao.ren\.m2\repository\com\alibaba\fastjson\1.2.46\fastjson-1.2.46.jar;C:\Users\wenchao.ren\.m2\repository\com\esotericsoftware\kryo\4.0.1\kryo-4.0.1.jar;C:\Users\wenchao.ren\.m2\repository\com\esotericsoftware\reflectasm\1.11.3\reflectasm-1.11.3.jar;C:\Users\wenchao.ren\.m2\repository\org\ow2\asm\asm\5.0.4\asm-5.0.4.jar;C:\Users\wenchao.ren\.m2\repository\com\esotericsoftware\minlog\1.3.0\minlog-1.3.0.jar;C:\Users\wenchao.ren\.m2\repository\de\javakaffee\kryo-serializers\0.42\kryo-serializers-0.42.jar;C:\Users\wenchao.ren\.m2\repository\de\ruedigermoeller\fst\2.48-jdk-6\fst-2.48-jdk-6.jar;C:\Users\wenchao.ren\.m2\repository\com\fasterxml\jackson\core\jackson-core\2.8.6\jackson-core-2.8.6.jar;C:\Users\wenchao.ren\.m2\repository\com\cedarsoftware\java-util\1.9.0\java-util-1.9.0.jar;C:\Users\wenchao.ren\.m2\repository\com\cedarsoftware\json-io\2.5.1\json-io-2.5.1.jar;C:\Users\wenchao.ren\.m2\repository\com\alibaba\hessian-lite\3.2.3\hessian-lite-3.2.3.jar;C:\Users\wenchao.ren\.m2\repository\org\objenesis\objenesis\2.6\objenesis-2.6.jar;C:\Users\wenchao.ren\AppData\Local\JetBrains\Toolbox\apps\IDEA-U\ch-0\182.3684.40\lib\idea_rt.jar;C:\Users\wenchao.ren\AppData\Local\JetBrains\Toolbox\apps\IDEA-U\ch-0\182.3684.40\lib\rt\debugger-agent.jar
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZooKeeper: Client environment:java.library.path=C:\Program Files\Java\jdk1.8.0_181\bin;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files\Git\cmd;C:\Program Files\Microsoft VS Code\bin;C:\WINDOWS\System32\OpenSSH\;C:\Users\wenchao.ren\AppData\Local\Microsoft\WindowsApps;;.
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=C:\Users\wenchao.ren\AppData\Local\Temp\
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZooKeeper: Client environment:os.name=Windows 10
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZooKeeper: Client environment:os.version=10.0
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZooKeeper: Client environment:user.name=wenchao.ren
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZooKeeper: Client environment:user.home=C:\Users\wenchao.ren
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZooKeeper: Client environment:user.dir=E:\workstations\dubbo\incubator-dubbo
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@59b38691
[20/08/18 11:49:42:042 CST] main-SendThread(0:0:0:0:0:0:0:1:2181) INFO zookeeper.ClientCnxn: Opening socket connection to server 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
[20/08/18 11:49:42:042 CST] main-SendThread(0:0:0:0:0:0:0:1:2181) INFO zookeeper.ClientCnxn: Socket connection established to 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181, initiating session
[20/08/18 11:49:42:042 CST] main-SendThread(0:0:0:0:0:0:0:1:2181) INFO zookeeper.ClientCnxn: Session establishment complete on server 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181, sessionid = 0x100001d37250004, negotiated timeout = 40000
[20/08/18 11:49:42:042 CST] main-EventThread INFO state.ConnectionStateManager: State change: CONNECTED
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZookeeperRegistry: [DUBBO] Register: consumer://10.254.0.157/org.apache.dubbo.demo.DemoService?application=demo-consumer&category=consumers&check=false&dubbo=2.0.2&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=2720&qos.port=33333&side=consumer&timestamp=1534780181816, dubbo version: , current host: 10.254.0.157
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZookeeperRegistry: [DUBBO] Subscribe: consumer://10.254.0.157/org.apache.dubbo.demo.DemoService?application=demo-consumer&category=providers,configurators,routers&check=false&dubbo=2.0.2&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=2720&qos.port=33333&side=consumer&timestamp=1534780181816, dubbo version: , current host: 10.254.0.157
[20/08/18 11:49:42:042 CST] main INFO zookeeper.ZookeeperRegistry: [DUBBO] Notify urls for subscribe url consumer://10.254.0.157/org.apache.dubbo.demo.DemoService?application=demo-consumer&category=providers,configurators,routers&check=false&dubbo=2.0.2&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=2720&qos.port=33333&side=consumer&timestamp=1534780181816, urls: [dubbo://192.168.116.1:20880/org.apache.dubbo.demo.DemoService?anyhost=true&application=demo-provider&dubbo=2.0.2&generic=false&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=6872&side=provider&timestamp=1534779801013, empty://10.254.0.157/org.apache.dubbo.demo.DemoService?application=demo-consumer&category=configurators&check=false&dubbo=2.0.2&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=2720&qos.port=33333&side=consumer&timestamp=1534780181816, empty://10.254.0.157/org.apache.dubbo.demo.DemoService?application=demo-consumer&category=routers&check=false&dubbo=2.0.2&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=2720&qos.port=33333&side=consumer&timestamp=1534780181816], dubbo version: , current host: 10.254.0.157
[20/08/18 11:49:43:043 CST] main INFO transport.AbstractClient: [DUBBO] Successed connect to server /192.168.116.1:20880 from NettyClient 10.254.0.157 using dubbo version , channel is NettyChannel [channel=[id: 0x5a2b2c61, L:/192.168.116.1:8464 - R:/192.168.116.1:20880]], dubbo version: , current host: 10.254.0.157
[20/08/18 11:49:43:043 CST] main INFO transport.AbstractClient: [DUBBO] Start NettyClient c/10.254.0.157 connect to the server /192.168.116.1:20880, dubbo version: , current host: 10.254.0.157
[20/08/18 11:49:43:043 CST] main INFO config.AbstractConfig: [DUBBO] Refer dubbo service org.apache.dubbo.demo.DemoService from url zookeeper://localhost:2181/org.apache.dubbo.registry.RegistryService?anyhost=true&application=demo-consumer&check=false&dubbo=2.0.2&generic=false&interface=org.apache.dubbo.demo.DemoService&methods=sayHello&pid=2720&qos.port=33333&register.ip=10.254.0.157&remote.timestamp=1534779801013&side=consumer&timestamp=1534780181816, dubbo version: , current host: 10.254.0.157
Hello world, response from provider: 192.168.116.1:20880
Hello world, response from provider: 192.168.116.1:20880
Hello world, response from provider: 192.168.116.1:20880

此时我们看zk节点:

1
2
3
4
5
6
7
8
9
10
11
[zk: localhost:2181(CONNECTED) 7] ls /dubbo/org.apache.dubbo.demo.DemoService
[consumers, configurators, routers, providers]
[zk: localhost:2181(CONNECTED) 8] ls /dubbo/org.apache.dubbo.demo.DemoService/consumers
[consumer%3A%2F%2F10.254.0.157%2Forg.apache.dubbo.demo.DemoService%3Fapplication%3Ddemo-consumer%26category%3Dconsumers%26check%3Dfalse%26dubbo%3D2.0.2%26interface%3Dorg.apache.dubbo.demo.DemoService%26methods%3DsayHello%26pid%3D2720%26qos.port%3D33333%26side%3Dconsumer%26timestamp%3D1534780181816]
[zk: localhost:2181(CONNECTED) 9] ls /dubbo/org.apache.dubbo.demo.DemoService/configurators
[]
[zk: localhost:2181(CONNECTED) 10] ls /dubbo/org.apache.dubbo.demo.DemoService/routers
[]
[zk: localhost:2181(CONNECTED) 11] ls /dubbo/org.apache.dubbo.demo.DemoService/providers
[dubbo%3A%2F%2F192.168.116.1%3A20880%2Forg.apache.dubbo.demo.DemoService%3Fanyhost%3Dtrue%26application%3Ddemo-provider%26dubbo%3D2.0.2%26generic%3Dfalse%26interface%3Dorg.apache.dubbo.demo.DemoService%26methods%3DsayHello%26pid%3D6872%26side%3Dprovider%26timestamp%3D1534779801013]
[zk: localhost:2181(CONNECTED) 12]

可以看到多了consumerrouters节点。

同时consumer端不断输出Hello world, response from provider: 192.168.116.1:20880的日志,说明我们的环境以及搭建完成了。

参考文章

git还原到之前版本

命令行操作:

  • git log 查看之前的commit的id,找到想要还原的版本
  • git reset --hard 44bd896bb726be3d3815f1f25d738a9cd402a477 还原到之前的某个版本
  • git push -f origin master 强制push到远程

kafka listeners & advertised.listeners

今天在日常使用spring-kafka消费kafka数据时发现连接不是kafka,出现下面的异常:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
2019-01-21 16:55:58,675 WARN wtraceId[] wtracer[] [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] c.w.f.w.k.LogConsumerConfiguration:45 - [wcollect]batch pull data from kafka error
java.lang.IllegalStateException: No entry found for connection 30
at org.apache.kafka.clients.ClusterConnectionStates.nodeState(ClusterConnectionStates.java:330)
at org.apache.kafka.clients.ClusterConnectionStates.disconnected(ClusterConnectionStates.java:134)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:921)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:287)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.trySend(ConsumerNetworkClient.java:474)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:255)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:236)
at org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1243)
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1188)
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1164)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:728)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.lang.Thread.run(Thread.java:748)

从上面的异常中大致可以看到是初始化NetworkClient连接时出现问题,基本可以证明是连接不是kafka broker。

后来排查发现是因为我在我本地的代码中是通过ip+port的形式访问的,而我们的kafka broker配置了下面的参数:

1
2
3
4
5
6
7
8
9
10
############################# Socket Server Settings #############################

# The address the socket server listens on. It will get the value returned from
# java.net.InetAddress.getCanonicalHostName() if not configured.
# FORMAT:
# listeners = listener_name://host_name:port
# EXAMPLE:
# listeners = PLAINTEXT://your.host.name:9092
listeners=PLAINTEXT://xxx.xx.xxx.xxx.xxx.com:9092
advertised.listeners=PLAINTEXT://xxx.xxx.xxx.xxx:9092

因此导致我连接不上kafka的broker。解决办法也很简单,使用

因此我们需要研究一下这几个参数的作用

listeners

官网的解释如下:

Listener List - Comma-separated list of URIs we will listen on and their protocols. Specify hostname as 0.0.0.0 to bind to all interfaces. Leave hostname empty to bind to default interface. Examples of legal listener lists: PLAINTEXT://myhost:9092,TRACE://:9091 PLAINTEXT://0.0.0.0:9092, TRACE://localhost:9093

string类型,默认值为null, 都好分隔的url列表。例子:PLAINTEXT://myhost:9092,TRACE://:9091 PLAINTEXT://0.0.0.0:9092, TRACE://localhost:9093

host.name

DEPRECATED: only used when listeners is not set. Use listeners instead. hostname of broker. If this is set, it will only bind to this address. If this is not set, it will bind to all interfaces

string类型,默认值为""

advertised.listeners

Listeners to publish to ZooKeeper for clients to use, if different than the listeners above. In IaaS environments, this may need to be different from the interface to which the broker binds. If this is not set, the value for listeners will be used.

string类型,默认值为null

advertised.port

DEPRECATED: only used when advertised.listeners or listeners are not set. Use advertised.listeners instead. The port to publish to ZooKeeper for clients to use. In IaaS environments, this may need to be different from the port to which the broker binds. If this is not set, it will publish the same port that the broker binds to.

int类型,默认值为null 。如果在IaaS的环境下(IaaS 是云服务的最底层,相当于只提供了基础服务器),可能要为broker配置不同的接口绑定(以免端口冲突等问题),如果是这种情况,并没有设置advertised.port, 那么就会使用和broker绑定的端口一样的端口

advertised.host.name

DEPRECATED: only used when advertised.listeners or listeners are not set. Use advertised.listeners instead. Hostname to publish to ZooKeeper for clients to use. In IaaS environments, this may need to be different from the interface to which the broker binds. If this is not set, it will use the value for host.name if configured. Otherwise it will use the value returned from java.net.InetAddress.getCanonicalHostName().

string类型,默认值为null

这个参数已经不推荐使用了,一般使用advertised.listenerslisteners

参考资料

java中的日期pattern

经常搞混java中的日期pattern,比如经常记混Hh的区别,所以专门整理一下,便于我以后查找

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
yyyy:年
MM:月
dd:日
hh:1~12小时制(1-12)
HH:24小时制(0-23)
mm:分
ss:秒
S:毫秒
E:星期几
D:一年中的第几天
F:一月中的第几个星期(会把这个月总共过的天数除以7)
w:一年中的第几个星期
W:一月中的第几星期(会根据实际情况来算)
a:上下午标识
k:和HH差不多,表示一天24小时制(1-24)。
K:和hh差不多,表示一天12小时制(0-11)。
z:表示时区
```java

常用pattern:

```java
yyyy-MM-dd HH:mm:ss.SSS
yyyy-MM-dd'T'HH:mm:ss.SSS'Z'

常用时区:

1
@JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'", timezone = "GMT+8")

日期和字符串互转:

1
2
3
4
5
6
private final static DateTimeFormatter fmt1 = DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss.SSS");
DateTime dateTime = DateTime.parse(date, fmt1)

new DateTime().toString("yyyy-MM-dd HH:mm:ss.SSS")

org.joda.time#Days

优化Elasticsearch的写入

我们这边因为需要对trace系统的数据做一些高级查询,所以会将Span的可能会被用作搜索条件的信息写入elasticsearch中。
由于trace系统的数据量比较大,虽然trace系统本身的设计会有采样率这个东西来降低trace采集的数据量,但是本身还是
比较大的数据量。所以需要对es的写入做一些优化。这篇文章记录一下我们的优化项

分析我们场景的特点:

  • 写请求特别大
  • 读请求很少,实时性要求低
  • trace系统对数据的可靠性要求低,但是要求写入及时(数据的价值会随着时间而降低)

贴一下我们优化以后的template设置:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
{
"order": 0,
"index_patterns": [
"trace.advanced.query-*"
],
"settings": {
"index": {
"refresh_interval": "120s",
"number_of_shards": "10",
"translog": {
"flush_threshold_size": "1024mb",
"sync_interval": "120s",
"durability": "async"
},
"number_of_replicas": "0"
}
},
"mappings": {},
"aliases": {}
}

主要的优化步骤:

  • 关闭副本,设置number_of_replicas为0 在我们的场景下数据丢了是可以忍受的
  • 调大translog.flush_threshold_size,我们设置的为1024mb
  • 调大translog.sync_interval为120s
  • 我们容许数据丢失,设置async模式

因为我们目前的es集群性能是足够的,所以并没有完全按照参考文章将 ELASTICSEARCH 写入速度优化到极限中的所有项目都优化。推荐大家阅读这个参考文章,比我写的好多了。

参考资料

测试hexo-deployer-git自动部署github私有仓库

因为之前github支持了个人免费的私有仓库,所以就打算把自己的博客的仓库设置为private的,但是因为我有点懒,
担心从库从publish修改为private以后而导致hexo-deployer-git需要额外配置,后来测试以后发现我想多了。一切
都不需要做,完美,感谢github。

单据的设计

本文主要是初步记录一下关于单据零散的想法,看看是否可以形成一套可行的方法论。

单据实体

单据这个实体,其实从抽象的层面来说,它其实描述了一次行为:

谁,在什么时间点,用了什么样的成本,对什么目标产生了一次什么样的行为

如果让我们来对上面这句话进行解析,其实可以发现它可以拆分为下面几块:

  • 时间信息
  • 成本
  • 行为描述
  • 行为目标

下面我们来一次分析这个问题。

一般订单的发起者其实都是【用户】,当然这个用户并不一定是C端用户。而用户的信息如何描述呢?一般的公司都会有【用户中心】,或者【供应商管理系统】等等的来描述【用户】的信息。做的好的公司也会有【用户画像】

时间信息

时间信息一般分为2类:

  • 行为动作的产生时间
    • 比如用户的下单时间
  • 行为动作所带来的影响发生的时间
    • 用户购买的商品的出库时间
    • 用户购买的商品的送货时间
    • 用户购买的商品的预计收货时间
      前者是用户主动产生的,后者其实对用户来说是被动的。

成本信息

用户在下订单的时候往往都会消耗一定的成本,这个成本并不一定是金钱。可能是:

  • 金钱
  • 积分
  • 优惠券

##行为描述

行为描述主要是用来解释行为,比如:

  • 购买行为
  • 退货行为

不同的行为会有不同的属性信息,比如购买行为会有购买的数量,购买服务的持续时间等。

行为目标

行为的目标可以是物理实体,比如商品(一瓶可乐),也可以是虚拟实体,比如一次服务。

  • 商品
  • 服务
  • 流量

而具体的目标也会有一大套自己的模型和属性。比如商品的sku就是一套很经典的模型。

单据本身的信息

单据本身也需要一些自己的属性信息:

  • 单号
  • 单据类型
  • 状态机

关于单据号的设计可以很有讲究,一般会有如下的考虑:

  • 单据号的形式,长度
  • 是否有业务含义
  • 业务含义的具体信息
  • 业务含义占单据号的长度
  • 是否连续(可遍历)
  • 单据创建时间
  • 单据生成的serverIP
  • 路由信息
  • 分库分表的依据
  • 其他路由形式
  • 单据数量增长对单据号的影响
  • 单据号长度的划分
  • 每秒最大单据数量预估
  • 关于路由信息也有一个需要注意的点,就是要尽量做到数据分布平衡。

单据流程

单据的流程一般有正向流程和逆向流程。考虑到解耦和业务复杂度不可控导致的变数,一般情况下建议正向和逆向分开设计和实现。然后通过单号等信息进行关联。

  • 正向流程
    • 订单正常生产到配送的过程
  • 逆向流程
    • 逆向流程则指订单发生取消、退货等情况时引发的订单流程过程。

快速的熟悉陌生的系统

工作和学习过程中经常会遇到陌生的系统需要去熟悉,下面是我总结的一些自己的方法论,希望对你有所帮助。

了解系统Overview

首先我们需要对系统有一个Overview的了解,了解的方式可以是自己摸索,找对应系统负责人,团队leader聊,也可以自己找wiki,文档等方法。我简单的总结了一下步骤:

  • 熟悉系统的定位,明确系统要解决的主要问题
  • 如果系统有对应的UI页面,比如对于web系统,可以申请系统的权限,然后登陆系统看看系统主要有哪些菜单(一般情况下菜单其实会对于系统的功能模块),每个菜单下面都有哪些菜单项,然后浏览一下每个菜单项对应的页面有哪些按钮,哪些表格,哪些表单等。这样会对系统有哪些模块、功能有一个大概的了解。
  • 如果你幸运的话,你可以阅读系统使用手册、设计文档、需求文档等,当然大多数情况下是没有的。这个时候你其实更多的需要上一步骤对系统的大概理解,然后结合自己的工作经验,大致猜猜系统怎么设计实现的,其实对于大多数业务系统来说,看看功能模块其实是可以猜出个七七八八的系统设计和使用的情况的。当然了对于类似中间件那些系统(比如MQ,Config,Schedule)等,其实了解起来更加的迅速,因为大家基本在日常工作过程中都使用过这些系统,所以对于系统需要提供的基本功能和如何使用都比较熟悉,可能就是在系统的设计和实现上有所差异。
  • 了解系统的上下游。上下游都是哪些团队的哪些系统?这些团队的负责人是谁?系统和上下游系统是如何交互的?交互方式是什么,交互的时机等。
  • 了解系统的核心数据流向。清楚一个完成的数据链路是怎么样的。
  • 如果可以的话,自己动手在系统的dev环境或者beta环境来点一点,创建一些数据,然后走走核心数据链路。
  • 了解系统的核心监控指标。了解清楚核心监控指标,其实便于你理解系统的核心功能,并且方便日后对系统修改的时候,看看是否对核心监控指标有影响。
  • 了解系统的部署情况。一般一个系统都会有好几个模块(module),清楚每个模块干啥的,这些模块之间的依赖关系(比如admin模块,Server模块,client模块),模块或者系统系统如何部署的,是部署在KVM上?Kubernetes上?多少个实例?多少台Server?Server啥配置?存储用的啥?MySQL?HBase?DB数据量大致多少?存储多少个G/T? 存储增长量多少?系统的QPS大致多少?
  • 按照上面的步骤操作完以后,相信你一定会对系统有一个明确的了解。这个时候其实你应该可以对系统的存储模型(数据模型)猜出个七七八八了,接下来可以看看系统的存储模型了,来验证我们的猜测。

阅读系统存储模型

此处说的存储模型也可以称为是数据模型。可以是MySQL表设计(有哪些表,表之间的关联关系,表的索引是哪些,唯一索引是哪些),Hbase的存储方式(rowKey是如何设计的,列簇如何弄的,有哪些列),也可以是抽象意义上的数据模型。

阅读系统的源码

基本了解系统功能以后,知道存储模型以后,大致怎么实现其实 应该也可以猜出来了。当然了很多功能其实的实现方式会有很多种,通过阅读系统的源码我们就可以知道系统是采用的哪种方式。如果遇到和我们猜测的实现方式不一样的话,其实可以思考一下为啥不一样?两种实现方式的区别在哪,孰优孰劣。

关于阅读源码,我整理了几个注意点

  • 首先让系统在本地/dev/beta跑起来,最好在本地,这样其实对系统的运行环境可以更加熟悉。
  • 先从从核心数据链路的最开始进行源码阅读,第一次阅读只需要关心核心链路,分支链路第一次阅读的时候请忽略。
  • 对核心逻辑流程进行debug。因为设计的比较好的系统,往往接口和实现是分离的,而且会有多个实现,光看接口其实有时候根本不清楚走的哪个实现,这个时候就需要进行debug了。有时候一些方法实现的可能很不清晰,这个时候通过debug,或者写main方法,单元测试等手段来了解。
  • 开业手动画出系统核心流程的关键类的调用关系
  • 阅读系统非核心逻辑代码
  • 尝试讲给系统的开发人员,确认你理解的是正确的。对一些感觉设计不合理的点,咨询当时那么设计的原因

解决自己的疑惑,确认可优化点

一般情况下,我们在阅读完系统代码之后,会有自己的疑惑,也会发现一些设计不合理的点(虽然可能这个设计在当时的情况下是合理的),我们可以提出自己的优化方案,然后找对应系统的产品、开发、负责人确认。

结语

如果你完全按照上面的方式做了,还是没有对某个系统熟悉的话,那么就只能找对应系统的产品、开发、负责人具体咨询了。

Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now

×