Grenade assumes it is running on the system that will be hosting the upgrade processes

grenade.sh [-b] [-t] [-s stop-label] [-q]

-b Run only the base part -t Run only the target part (assumes a base run is in place) -q Quiet mode -s stop-label is the name of the step after which the script will stop. This is useful for debugging upgrades.

GRENADE_DIR is set once by the top level grenade.sh and exported so that all subsequent scripts can find their way back to the grenade root directory. No other scripts should set this variable

Source the bootstrapping facilities

Grenade attempts to reuse as much content from devstack on the target side as possible, but we need enough of our own code to get there.

grenaderc is a set of X=Y declarations that don't need any of the devstack functions to work

inc/bootstrap is the most minimal amount of functions that grenade needs to get going. This includes things like echo functions, trueorfalse, and the functions related to git cloning, so that we can get our devstack trees.

Create all the base directory structures needed for the rest of the environment to run.

This will give you an STACK_ROOT tree that you expect, and that your normal user owns for follow on activities.

TODO(sdague): should this extract into inc/bootstrap?

Set up logging Set LOGFILE to turn on logging Append '.xxxxxxxx' to the given name to maintain history where 'xxxxxxxx' is a representation of the date the file was created

First clean up old log files. Use the user-specified LOGFILE as the template to search for, appending '.*' to match the date we added on earlier runs.

Redirect output according to config Copy stdout to fd 3

Redirect stdout/stderr to tee to write the log file

Set up a second fd for output

Set fd 1 and 2 to primary logfile

Set fd 6 to summary logfile and stdout

Specified logfile name always links to the most recent log

Set up output redirection without log files Copy stdout to fd 3

Throw away stdout and stderr

Always send summary fd to original stdout

Set up logging of screen windows Set SCREEN_LOGDIR to turn on logging of screen windows to the directory specified in SCREEN_LOGDIR, we will log to the file screen-$SERVICE_NAME-$TIMESTAMP.log in that dir and have a link screen-$SERVICE_NAME.log to the latest log file. Logs are kept for as long specified in LOGDAYS.

We make sure the directory is created.

We cleanup the old logs

This script exits on an error so that errors don't compound and you see only the first error that occurred.

Print the commands being run so that we can see the command that triggers an error. It is also useful for following allowing as the install occurs.

Get both devstack trees, so that BASE_DEVSTACK_DIR, and TARGET_DEVSTACK_DIR are now fully populated.

Source the rest of the Grenade functions. For convenience $GRENADE_DIR/functions implicitly sources $TARGET_DEVSTACK_DIR/functions. So this line can't happen until we have the devstacks pulled down.

We now have the 'short_source' function available, so setup our PS4 variable

Many calls inside of devstack functions reference $TOP_DIR, which is the root of devstack. We export $TOP_DIR to all child processes here to be the TARGET_DEVSTACK_DIR.

If you want a script to use functions off of BASE_DEVSTACK_DIR (like the shutdown phase) you must explicitly reset TOP_DIR in those scripts.

Oh the complexity of bootstrapping. We need to populate enabled services from what devstack-gate might have set it as.

Collect the ENABLED_SERVICES from the base directory, this is what we are starting with.

Fetch all the grenade plugins which were registered in pluginrc via the enable_grenade_plugin stanza. This must be done before settings are loaded, but has to be this late to give access to all the devstack functions.

Load the settings files for all the in tree projects/. This registers all the projects in order that we're going to be upgrading when the time is right.

And ensure that we setup the target localrc.auto, because stack.sh isn't run there. This has to be run after load_settings because plugins might change the service list during this phase.

Nova should use singleconductor as Grenade doesn't setup multi-cell rabbit for now

Run the base install of the environment

Initialize grenade_db local storage, used for resource tracking

By the time we get here the sub nodes are already setup with localrc files as those are transferred in devstack-gate even before grenade.sh is called We hack the ./post-stack.sh to inject what we need. if we don't set CELLSV2_SETUP, the default devstack assumes "superconductor" and fails.

NOTE(sileht): If glance is not enabled the directory cannot exists.

Validate the install

NOTE(gmann): This run is on old node which is stable/rocky in stein case. Use branch constraint because Tempest is pinned on stable/rocky. We need to export it via env var UPPER_CONSTRAINTS_FILE so that initial creation of tempest tox use stable branch constraint instead of master constraint which is hard coded in tempest/tox.ini

once we are done, copy our created artifacts to the target

Early Create resources, used largely for network setup

Create resources

Verify the resources were created

Save some stuff before we shut that whole thing down

Shut down running code

Verify the resources still exist after the shutdown

Clone all devstack plugins on the new side, because we're not running a full stack.sh

Get target devstack tree ready for services to be run from it, including trying to reuse any existing files we pulled during the base run.

upgrade all the projects in order

Upgrade Tempest

Verify the resources still exist after the upgrade

Validate the upgrade This is used for testing grenade locally, and should not be used in the gate. Instead, grenade.sh runs smoke tests on the old cloud above, but smoke tests are run on the upgraded cloud by the gate script after grenade.sh has finished. If this option is enabled in the gate, tempest will be run twice against the new cloud.

Cleanup the resources

Indicate how long this took to run (bash maintained variable SECONDS)