Some Knowledge about Apache Kafka(1)

Apache Kafka is an open source, distributed publish-subscribe messaging system,and its main characters include:-Persistent Messaging-High throughput-Distributed: support messages partitioning over Kafka Servers and distributing consumption over a cluster of consumer machines while maintaining per-partition ordering machines Producer(frontend, services, proxies, adapters, other producers) | Kafka ———> Zookeeper | Consumer(realtime, NoSQL, Hadoop, Warehouses)Install KafkaDownload Kafka[root@localhost opt] https://www.apache.org/dyn/closer.cgi/incubator/kafka/kafka-0.7.2-incubating/kafka-0.7.2-incubating-src.tgzExtract the Kafka downloaded package[root@localhost opt]# tar xzf kafka-0.8.0-beta1-src.tgzSetup Kafka Cluster ZooKeeper —————————— | | | | Producer –> Kafka Broker –> ConsumerZookeeperKafka provides the default and simple ZooKeeper configuration file used for lauching a single local ZooKeeper instance. Zookeeper allows distributed processes coordinating with each other through a shared hierarchical name space of data registersStart Zookeeper Server [root@localhost kafka-0.8]# bin/zookeeper-server-start.sh config/zookeeper.propertiesData directory where the zookeeper snapshot is stored ## dataDir=/tmp/zookeeperThe port listening for client request ## clientPort=2181Consumer group id ## groupid=test-consumer-groupzookeeper connection string ## zookeeper.connect=localhost:2181Start the Kafka broker[root@localhost kafka-0.8]# bin/kafka-server-start.sh config/server.propertiesCreate a Kafka Topic[root@localhost kafka-0.8]# bin/kafka-create-topic.sh –zookeeper localhost:2181 –replica 1 –partition 1 –topic kafkatopicStart a producer for sending messages[root@localhost kafka-0.8]# bin/kafka-console-producer.sh –broker-list localhost:9092 –topic kafkatopicStart a consumer for consuming messages[root@localhost kafka-0.8]# bin/kafka-console-consumer.sh –zookeeper localhost:2181 –topic kafkatopic –from-beginningCreate a Kafka topic[root@localhost kafka-0.8]# bin/kafka-create-topic.sh –zookeeper localhost:2181 –replica 2 –partition 2 –topic othertopicStart a producer for sending messages[root@localhost kafka-0.8]# bin/kafka-console-producer.sh –broker-list localhost:9092,localhost:9093 –topic othertopicStart a consumer for consuming messages[root@localhost kafka-0.8]# bin/kafka-console-consumer.sh –zookeeper localhost:2181 –topic othertopic –from-beginning