Packages
brod
3.3.0
4.5.7
4.5.6
4.5.5
4.5.4
4.5.3
4.5.2
4.5.1
4.5.0
4.4.7
4.4.6
4.4.5
4.4.4
4.4.3
4.4.2
4.4.1
4.4.0
4.3.3
4.3.2
4.3.1
4.3.0
4.2.0
4.1.1
4.1.0
4.0.0
3.19.1
3.19.0
3.18.0
3.17.1
3.17.0
3.16.5
3.16.4
3.16.3
3.16.2
3.16.1
3.16.0
3.15.6
3.15.5
3.15.4
3.15.3
3.15.1
3.15.0
3.14.0
3.13.0
3.12.0
3.11.0
3.10.0
3.9.5
3.9.3
3.9.2
3.9.1
3.9.0
3.8.1
3.8.0
3.7.11
3.7.10
3.7.9
3.7.8
3.7.7
3.7.6
3.7.5
3.7.4
3.7.3
3.7.2
3.7.1
3.7.0
3.6.2
3.6.1
3.6.0
3.5.2
3.5.1
3.5.0
3.4.0
3.3.5
3.3.4
3.3.3
3.3.2
3.3.1
3.3.0
3.2.0
3.0.0
2.5.0
2.4.1
2.4.0
2.3.7
2.3.6
2.3.5
2.3.4
2.3.3
2.3.1
2.2.16
2.2.15
2.2.14
2.2.12
2.2.11
2.2.10
2.2.9
2.2.8
2.2.7
2.2.6
2.2.5
2.2.4
2.2.3
2.2.2
2.2.1
2.2.0
2.1.12
2.1.11
2.1.10
2.1.8
2.1.7
2.1.4
2.1.2
2.0.0
Apache Kafka Erlang client library
Current section
113 Versions
Jump to
Current section
113 Versions
Compare versions
21
files changed
+834
additions
-234
deletions
| @@ -1,6 +1,6 @@ | |
| 1 1 | PROJECT = brod |
| 2 2 | PROJECT_DESCRIPTION = Kafka client library in Erlang |
| 3 | - PROJECT_VERSION = 3.2.0 |
| 3 | + PROJECT_VERSION = 3.3.0 |
| 4 4 | |
| 5 5 | DEPS = supervisor3 kafka_protocol |
| 6 6 | TEST_DEPS = docopt jsone meck proper |
| @@ -14,6 +14,7 @@ dep_kafka_protocol_commit = 1.1.0 | |
| 14 14 | dep_docopt = git https://github.com/zmstone/docopt-erl.git 0.1.3 |
| 15 15 | |
| 16 16 | ESCRIPT_FILE = scripts/brod |
| 17 | + ESCRIPT_EMU_ARGS = -sname brod_cli |
| 17 18 | |
| 18 19 | COVER = true |
| @@ -424,13 +424,14 @@ Start an Erlang shell with brod started | |
| 424 424 | ./scripts/brod meta -b localhost |
| 425 425 | ``` |
| 426 426 | |
| 427 | - ### Produce a message |
| 427 | + ### Produce a Message |
| 428 | + |
| 428 429 | ``` |
| 429 430 | ./scripts/brod send -b localhost -t test-topic -p 0 -k "key" -v "value" |
| 430 431 | |
| 431 432 | ``` |
| 432 433 | |
| 433 | - ### Fetch a message |
| 434 | + ### Fetch a Message |
| 434 435 | |
| 435 436 | ``` |
| 436 437 | ./scripts/brod fetch -b localhost -t test-topic -p 0 --fmt 'io:format("offset=~p, ts=~p, key=~s, value=~s\n", [Offset, Ts, Key, Value])' |
| @@ -445,3 +446,54 @@ Bound variables to be used in `--fmt` expression: | |
| 445 446 | - `TsType`: Timestamp type either `create` or `append` |
| 446 447 | - `Ts`: Timestamp, `-1` as no value |
| 447 448 | |
| 449 | + ### Stream Messages to Kafka |
| 450 | + |
| 451 | + Send `README.md` to kafka one line per kafka message |
| 452 | + ``` |
| 453 | + ./scripts/brod pipe -b localhost:9092 -t test-topic -p 0 -s @./README.md |
| 454 | + ``` |
| 455 | + |
| 456 | + ### Resolve Offset |
| 457 | + |
| 458 | + ``` |
| 459 | + ./scripts/brod offset -b localhost:9092 -t test-topic -p 0 |
| 460 | + ``` |
| 461 | + |
| 462 | + ### List or Describe Groups |
| 463 | + |
| 464 | + ``` |
| 465 | + # List all groups |
| 466 | + ./scripts/brod groups -b localhost:9092 |
| 467 | + |
| 468 | + # Describe groups |
| 469 | + ./scripts/brod groups -b localhost:9092 --ids group-1,group-2 |
| 470 | + ``` |
| 471 | + |
| 472 | + ### Display Committed Offsets |
| 473 | + |
| 474 | + ``` |
| 475 | + # all topics |
| 476 | + ./scripts/brod commits -b localhost:9092 --id the-group-id --describe |
| 477 | + |
| 478 | + # a specific topic |
| 479 | + ./scripts/brod commits -b localhost:9092 --id the-group-id --describe --topic topic-name |
| 480 | + ``` |
| 481 | + |
| 482 | + ### Commit Offsets |
| 483 | + |
| 484 | + NOTE: This feature is designed for force overwriting commits, not for regular use of offset commit. |
| 485 | + |
| 486 | + ``` |
| 487 | + # Commit 'latest' offsets of all partitions with 2 days retention |
| 488 | + ./scripts/brod commits -b localhost:9092 --id the-group-id --topic topic-name --offsets latest --retention 2d |
| 489 | + |
| 490 | + # Commit offset=100 for partition 0 and 200 for partition 1 |
| 491 | + ./scripts/brod commits -b localhost:9092 --id the-group-id --topic topic-name --offsets "0:100,1:200" |
| 492 | + |
| 493 | + # Use --retention 0 to delete commits (may linger in kafka before cleaner does its job) |
| 494 | + ./scripts/brod commits -b localhost:9092 --id the-group-id --topic topic-name --offsets latest --retention 0 |
| 495 | + |
| 496 | + # Try join an active consumer group using 'range' protocol and steal one partition assignment then commit offset=10000 |
| 497 | + ./scripts/brod commits -b localhost:9092 -i the-group-id -t topic-name -o "0:10000" --protocol range |
| 498 | + ``` |
| 499 | + |
| @@ -1,7 +1,7 @@ | |
| 1 1 | FROM java:openjdk-8-jre |
| 2 2 | |
| 3 3 | ENV SCALA_VERSION 2.11 |
| 4 | - ENV KAFKA_VERSION 0.10.1.1 |
| 4 | + ENV KAFKA_VERSION 0.10.2.1 |
| 5 5 | |
| 6 6 | RUN apt-get update && \ |
| 7 7 | apt-get install -y zookeeper wget supervisor dnsutils && \ |
| @@ -1,5 +1,5 @@ | |
| 1 1 | {<<"name">>,<<"brod">>}. |
| 2 | - {<<"version">>,<<"3.2.0">>}. |
| 2 | + {<<"version">>,<<"3.3.0">>}. |
| 3 3 | {<<"requirements">>, |
| 4 4 | #{<<"kafka_protocol">> => #{<<"app">> => <<"kafka_protocol">>, |
| 5 5 | <<"optional">> => false, |
| @@ -23,9 +23,9 @@ | |
| 23 23 | <<"rebar.config.script">>,<<"scripts/.gitignore">>,<<"scripts/brod">>, |
| 24 24 | <<"scripts/cover-print-not-covered-lines.escript">>, |
| 25 25 | <<"scripts/cover-summary.escript">>,<<"scripts/generate-certificates.sh">>, |
| 26 | - <<"scripts/line-width-check.sh">>,<<"scripts/setup-test-env.sh">>, |
| 27 | - <<"scripts/vsn-check.sh">>,<<"src/brod.erl">>, |
| 28 | - <<"src/brod_auth_backend.erl">>,<<"src/brod_cli.erl">>, |
| 26 | + <<"scripts/setup-test-env.sh">>,<<"scripts/vsn-check.sh">>, |
| 27 | + <<"src/brod.erl">>,<<"src/brod_auth_backend.erl">>, |
| 28 | + <<"src/brod_cg_commits.erl">>,<<"src/brod_cli.erl">>, |
| 29 29 | <<"src/brod_cli_pipe.erl">>,<<"src/brod_client.erl">>, |
| 30 30 | <<"src/brod_consumer.erl">>,<<"src/brod_consumers_sup.erl">>, |
| 31 31 | <<"src/brod_group_coordinator.erl">>,<<"src/brod_group_member.erl">>, |
unknownscripts/brod
File is too large to be displayed (100 KB limit).
Loading more files…