Scribe nodes are typically configured into a fan-in tree topology.

If the message batch is throttled, the server returns “try again” to client and the client should retry sending at a later time. When a client sends a batch of messages to a scribe server, the scribe server forwards each message in the batch to the corresponding upstream and returns “ok” to the client to indicate success. Log(list of (category, message)-tuples) -> status The upstreams may be either other scribe nodes, hdfs (or local file), or a sink which drops all messages. The server’s configurations, including its upstream locations, are loaded on startup from an xml-like config file.

Feel free to skip this section if you are already familiar with scribe.Ī scribe server can be thought of as an input-queued network switch which performs routing based on the message’s category.

This section describes the scribe pipeline with respect to how it is setup at Dropbox (we suspect most companies deploy/use scribe in similar fashion).

How much memory is dropbox free