Stop a Node

On this page Carat arrow pointing down

This page shows you how to use the cockroach quit command to stop a node.

Overview

How It Works

When you stop a node, CockroachDB lets the node finish in-flight requests and transfers all range leases off the node before shutting it down. If the node then stays offline for more than 5 minutes, the cluster considers the node dead and starts to transfer its range replicas to other nodes as well.

After that, if the node comes back online, its range replicas will determine whether or not they are still valid members of replica groups. If a range replica is still valid and any data in its range has changed, it will receive updates from another replica in the group. If a range replica is no longer valid, it will be removed from the node.

Basic terms:

  • Range: CockroachDB stores all user data and almost all system data in a giant sorted map of key value pairs. This keyspace is divided into "ranges", contiguous chunks of the keyspace, so that every key can always be found in a single range.
  • Range Replica: CockroachDB replicates each range (3 times by default) and stores each replica on a different node.
  • Range Lease: For each range, one of the replicas holds the "range lease". This replica, referred to as the "leaseholder", is the one that receives and coordinates all read and write requests for the range.

Considerations

Before temporarily stopping a node to upgrade its version of CockroachDB, if you expect the node to be offline for longer than 5 minutes, you should first set the server.time_until_store_dead cluster setting to higher than the 5m0s default. For example, if you think the node might be offline for up to 8 minutes, you might change this setting as follows:

> SET CLUSTER SETTING server.time_until_store_dead = 10m0s;

Synopsis

# Stop a node:
$ cockroach quit <flags>

# View help:
$ cockroach quit --help

Flags

The quit command supports the following general-use and logging flags.

General

Flag Description
--certs-dir The path to the certificate directory. If the cluster is secure, this directory must contain a valid CA certificate and a client certificate and key for the root user. Client certificates for other users are not supported.

Env Variable: COCKROACH_CERTS_DIR
Default: ${HOME}/.cockroach-certs/
--host The server host to connect to. This can be the address of any node in the cluster.

Env Variable: COCKROACH_HOST
Default:localhost
--insecure Run in insecure mode. If this flag is not set, the --certs-dir flag must point to valid certificates.

Env Variable: COCKROACH_INSECURE
Default: false
--port The server port to connect to.

Env Variable: COCKROACH_PORT
Default: 26257

Logging

By default, the quit command logs errors to stderr.

If you need to troubleshoot this command's behavior, you can change its logging behavior.

Examples

Stop a Node from the Machine Where It's Running

  1. SSH to the machine where the node is running.

  2. If the node is running in the background and you are using a process manager for automatic restarts, use the process manager to stop the cockroach process without restarting it.

    If the node is running in the background and you are not using a process manager, send a kill signal to the cockroach process, for example:

    $ pkill cockroach
    

    If the node is running in the foreground, press CTRL-C.

  3. Verify that the cockroach process has stopped:

    $ ps aux | grep cockroach
    

    Alternately, you can check the node's logs for the message server drained and shutdown completed.

Stop a Node from Another Machine

  1. Install the cockroach binary on a machine separate from the node.

  2. Create a certs directory and copy the CA certificate and the client certificate and key for the root user into the directory.

  3. Run the cockroach quit command without the --decommission flag:

    $ cockroach quit --certs-dir=certs --host=<address of node to stop>
    
  1. Install the cockroach binary on a machine separate from the node.

  2. Run the cockroach quit command without the --decommission flag:

    $ cockroach quit --insecure --host=<address of node to stop>
    

See Also


Yes No
On this page

Yes No