Pulsar DevOps with Pulsar Shell and CLI Tools
Using Pulsar admin tools
Pulsar Shell
bin/pulsar-shell
Using directory: /Users/tspann/.pulsar-shell
Welcome to Pulsar shell!
Service URL: pulsar://localhost:6650/
Admin URL: http://localhost:8080/
Type help to get started or try the autocompletion (TAB button).
Type exit or quit to end the shell session.For Dev Ops and repeatable processes, this is a good example.
echo "
# Configure the pulsar1 cluster
config use pulsar1
# Create a tenant
admin tenants create chatgpt
# Create a new namespace
admin namespaces create chatgpt/articles
# Build a topic
admin topics create persistent://chatgpt/articles/genitems
# List topics
admin topics list public/default
" > setup-shell.txt
./bin/pulsar-shell -f ./setup-shell.txt --fail-on-errorIf you aren’t sure what you want to do, just hit “tab”.
If you aren’t sure of the full parameters required, just enter it empty.
admin topics create
Main parameters are required ("persistent://tenant/namespace/topic")
Create a non-partitioned topic.
Usage: create [options] persistent://tenant/namespace/topic
Options:
--metadata, -m
key value pair properties(a=a,b=b,c=c)As you can see these are just simple version of pulsar-client, pulsar-admin and config.
Pulsar is a distributed messaging system that allows for the efficient exchange of data between different components of a system. Pulsar Shell is a command-line interface (CLI) tool that can be used to interact with a Pulsar cluster and perform various operations such as creating topics, publishing and consuming messages, and managing the overall cluster.
To start using Pulsar Shell, you will first need to have a Pulsar cluster set up and running. This can be done by following the instructions on the Pulsar website or by using a managed service such as Apache Pulsar on Amazon Web Services (AWS).
You can connect to a Pulsar machine (could be EC2, Docker, Kube Pods or local) and use the commands there.
Otherwise, download to your laptop.
wget https://archive.apache.org/dist/pulsar/pulsar-2.11.0/apache-pulsar-shell-2.11.0-bin.tar.gz
tar xzvf apache-pulsar-shell-2.11.0-bin.tar.gz
cd apache-pulsar-shell-2.11.0-bin.tar.gz
Once you have a Pulsar cluster up and running, you can start using Pulsar Shell by connecting to the cluster using the pulsar-shell command.
Pulsar CLI Tools
Below is how to use the other Pulsar tools, this was incorrectly generated by ChatGPT which did not know that pulsar-shell is not the same as pulsar-admin and pulsar-client command line tools.
Once you have a Pulsar cluster up and running, you can start using Pulsar CLI by connecting to the cluster using the pulsar-admin and pulsar-client commands. Seepulsar-admin command. This command takes the following format:
pulsar-admin --url http://<pulsar-broker-address>:<pulsar-broker-port>Replace <pulsar-broker-address> and <pulsar-broker-port> with the address and port of your Pulsar cluster. For example, if your Pulsar cluster is running on localhost on port 6650, you would use the following command to connect:
pulsar-admin --url http://localhost:6650Once connected to the Pulsar cluster, you can use various commands to interact with it. Some of the most commonly used commands include:
topics create: Creates a new topic on the cluster.topics list: Lists all topics on the cluster.topics delete: Deletes a topic from the cluster.produce: Publishes a message to a topic.consume: Consumes messages from a topic.clusters list: Lists all clusters connected to the current cluster.clusters unload: Unloads a cluster from the current cluster.
In addition to these basic commands, Pulsar CLI also provides a number of advanced features such as the ability to manage subscriptions, configure authentication and authorization, and collect metrics and statistics on the cluster.
Overall, Pulsar CLI is a powerful tool for managing and interacting with a Pulsar cluster. By following the steps outlined in this article, you should be able to start using Pulsar CLI to create topics, publish and consume messages, and manage your cluster effectively.
Another important feature of Pulsar CLI is the ability to manage the schema for topics. Schemas are used to define the structure of messages that are published and consumed on a topic. Pulsar supports different types of schemas such as Avro, JSON, and Protobuf.
To create a new topic with a schema, you can use the topics create command and specify the schema type and the schema definition. For example, the following command creates a topic named my-topic with an Avro schema:
pulsar-admin topics create my-topic --schema-type avro --schema-file my-schema.avscWhere my-schema.avsc is the file containing the Avro schema definition.
Once the topic is created, you can use the produce command to publish messages to the topic and consume command to consume messages from the topic. When publishing messages, you need to provide the message payload and the schema type, and when consuming messages, you will receive the message payload and the schema type.
It’s worth noting that Pulsar CLI also allows you to manage permissions, configure authentication and authorization, and collect metrics and statistics on the cluster.
For example, you can use the permissions grant command to grant a user or a role a set of permissions on a specific topic. And you can use the metrics command to collect various metrics about the cluster such as the number of messages published, the number of messages consumed, the number of active subscriptions, and so on.
In conclusion, Pulsar CLI is an essential tool for managing and interacting with a Pulsar cluster. It provides a wide range of functionality that allows you to create, publish, and consume messages, manage schemas, configure authentication and authorization, and collect metrics and statistics. By learning how to use Pulsar CLI, you can become proficient in managing and monitoring your Pulsar cluster and make sure that it runs smoothly.
Author Part 2: ChatGPT
Prompt: Write me a full length article on how to start using Pulsar shell.
Prompt 2: continue
