Role - tripleo_cephadm

Role Documentation

Welcome to the “tripleo_cephadm” role documentation.

Role Defaults

This section highlights all of the defaults and variables set within the “tripleo_cephadm” role.

# defaults file for tripleo_cephadm
tripleo_cephadm_spec_on_bootstrap: false  # not recommended due to https://tracker.ceph.com/issues/49277
tripleo_cephadm_ssh_user: ceph-admin
tripleo_cephadm_bin: /usr/sbin/cephadm
tripleo_cephadm_cluster: ceph
tripleo_cephadm_config_home: /etc/ceph
tripleo_cephadm_config_home_container: /var/lib/ceph/{{ tripleo_cephadm_fsid }}/config/
tripleo_cephadm_verbose: false
tripleo_cephadm_container_ns: quay.io/ceph
tripleo_cephadm_container_image: ceph
tripleo_cephadm_container_tag: v16
tripleo_cephadm_container_cli: podman
tripleo_cephadm_container_options: --net=host --ipc=host
tripleo_cephadm_registry_password: ''
tripleo_cephadm_registry_username: ''
tripleo_cephadm_registry_url: ''
tripleo_cephadm_keyring_prefix: '{{ tripleo_cephadm_config_home }}/{{ tripleo_cephadm_cluster
  }}.client'
tripleo_cephadm_admin_keyring: '{{ tripleo_cephadm_keyring_prefix }}.admin.keyring'
tripleo_cephadm_conf: '{{ tripleo_cephadm_config_home }}/{{ tripleo_cephadm_cluster
  }}.conf'
tripleo_cephadm_assimilate_conf: /home/{{ tripleo_cephadm_ssh_user }}/assimilate_{{
  tripleo_cephadm_cluster }}.conf
tripleo_cephadm_assimilate_conf_container: /home/assimilate_{{ tripleo_cephadm_cluster
  }}.conf
# path on ansible host (i.e. undercloud) of the ceph spec
tripleo_cephadm_spec_ansible_host: '{{ playbook_dir }}/ceph_spec.yaml'
# path on bootstrap node of ceph spec (scp'd from above var)
tripleo_cephadm_spec: /home/{{ tripleo_cephadm_ssh_user }}/specs/ceph_spec.yaml
# path in container on bootstrap node of spec (podman -v'd from above var)
tripleo_cephadm_container_spec: /home/ceph_spec.yaml
# path of other ceph specs podman -v mounted into running container
tripleo_cephadm_spec_home: /home/{{ tripleo_cephadm_ssh_user }}/specs
tripleo_cephadm_bootstrap_files:
- /home/{{ tripleo_cephadm_ssh_user }}/.ssh/id_rsa
- /home/{{ tripleo_cephadm_ssh_user }}/.ssh/id_rsa.pub
tripleo_cephadm_uid: '167'
tripleo_cephadm_mode: '0755'
tripleo_cephadm_keyring_permissions: '0644'
tripleo_ceph_client_config_home: /etc/ceph
tripleo_cephadm_dashboard_enabled: false
tripleo_cephadm_observability_data_only: false
tripleo_cephadm_wait_for_mons: true
tripleo_cephadm_wait_for_mons_retries: 10
tripleo_cephadm_wait_for_mons_delay: 20
tripleo_cephadm_wait_for_mons_ignore_errors: false
tripleo_cephadm_wait_for_osds: true
tripleo_cephadm_wait_for_osds_retries: 40
tripleo_cephadm_wait_for_osds_delay: 30
tripleo_cephadm_wait_for_osds_ignore_errors: false
tripleo_cephadm_num_osd_expected: 1
tripleo_cephadm_predeployed: true
tripleo_cephadm_conf_overrides: {}
tripleo_cephadm_fsid_list: []
tripleo_cephadm_fqdn: false
tripleo_cephadm_crush_rules: []
tripleo_cephadm_internal_tls_enabled: false
tripleo_cephadm_nfs_rados_export_index: ganesha-export-index
tripleo_cephadm_ceph_nfs_rados_backend: true
tripleo_cephadm_certs: /etc/pki/tls
tripleo_cephadm_idmap_conf: /etc/ganesha/idmap.conf
tripleo_cephadm_idmap_overrides: {}
tripleo_cephadm_rbd_mirror_pool: ''
tripleo_cephadm_rbd_mirror_remote_cluster: not-ceph
tripleo_cephadm_rbd_mirror_remote_user: ''
tripleo_cephadm_debug: false
tripleo_cephadm_min_compat_client: ''
tripleo_cephadm_deployed_ceph: false
tripleo_cephadm_deployed_ceph_tht_path: /home/stack/deployed_ceph.yaml
tripleo_cephadm_backend: ''
tripleo_cephadm_action: disable
tripleo_cephadm_rbd_trash_interval: 15
tripleo_cephadm_enable_trash_scheduler: false
tripleo_cephadm_apply_ceph_conf_overrides_on_update: false
tripleo_cephadm_standalone: false
tripleo_cephadm_single_host_defaults: false
tripleo_cephadm_vip_path: /home/stack/ceph_vips.yaml
tripleo_cephadm_default_container: false
tripleo_cephadm_nfs_ingress_frontend_port: 2049
tripleo_cephadm_nfs_ingress_monitor_port: 8999
tripleo_cephadm_ceph_vips: {}
tripleo_cephadm_extra_args: ''

Molecule Scenarios

Molecule is being used to test the “tripleo_cephadm” role. The following section highlights the drivers in service and provides an example playbook showing how the role is leveraged.

Scenario: default

Driver: podman

Molecule Inventory

hosts:
  all:
    hosts:
      centos:
        ansible_python_interpreter: /usr/bin/python3

Example default playbook

- name: Converge
  hosts: all
  vars:
    tripleo_cephadm_wait_for_mons: false
    tripleo_ceph_client_vars: ceph_client.yaml
    tripleo_cephadm_fsid_list:
    - cc7adc82-32eb-531e-b173-54047020ea3c
  tasks:
  - name: Satisfy Ceph prerequisites
    import_role:
      name: tripleo_cephadm
      tasks_from: pre

  - name: Bootstrap Ceph
    import_role:
      name: tripleo_cephadm
      tasks_from: bootstrap

  - name: Mock ceph_mon_dump command
    shell: cat mock/mock_ceph_mon_dump.json
    register: ceph_mon_mock_dump
    delegate_to: localhost

  - name: Mock ceph_keys_module_output
    include_vars: mock_ceph_keys.yml

  - name: Export configuration for tripleo_ceph_client
    import_role:
      name: tripleo_cephadm
      tasks_from: export
    vars:
      ceph_mon_dump: '{{ ceph_mon_mock_dump }}'
      tripleo_cephadm_client_keys: '{{ mock_ceph_keys }}'

  - name: Run verify tasks
    include_tasks: tasks/verify.yml

About

An Ansible role for TripleO integration with Ceph clusters deployed with cephadm and managed with Ceph orchestrator.

This role is provided as part of the implementation of the tripleo_ceph_spec. It is an Ansible wrapper to call the Ceph tools cephadm and orchestrator and it contains the Ansible module ceph_key from ceph-ansible.

Assumptions

• This role assumes it has an inventory with a single host, known as the bootstrap_host. An inventory genereated by tripleo-ansible-inventory will have a mons group so the first node in this group is a good candidate for this host.

• The cephadm binary must be installed on the bootstrap_host.

• Though there only needs to be one Ceph node in the inventory cephadm will configure the other servers with SSH. Thus, the following playbook should be run before one which uses this role to configure the ceph-admin user on the overcloud with the SSH keys that cephadm requires.

  ansible-playbook -i $INV \
    tripleo-ansible/tripleo_ansible/playbooks/cli-enable-ssh-admin.yaml \
    -e @ceph-admin.yml
  

  Where ceph-admin.yml contains something like the following:

  ---
  tripleo_admin_user: ceph-admin
  ssh_servers: "{{ groups['mons'] }}"
  distribute_private_key: true
  

  The ssh_servers variable should be expanded to contain another other nodes hosting Ceph, e.g. osds.

• A cephadm-spec file should be provided which references the Ceph services to be run on the other ssh_hosts. The path to this file can be set with the ceph_spec variable.

Usage

Here is an example of a playbook which bootstraps the first Ceph monitor and then applies a spec file to add other hosts. It then creates RBD pools for Nova, Cinder, and Glance and a cephx keyring called openstack to access those pools. It then creates a file which can be passed as input to the role tripleo_ceph_client so that an overcloud can be configured to use the deployed Ceph cluster.

- name: Deploy Ceph with cephadm
  hosts: mons[0]
  vars:
    bootstrap_host: "{{ groups['mons'][0] }}"
    tripleo_cephadm_spec_on_bootstrap: false
    pools:
      - vms
      - volumes
      - images
  tasks:
    - name: Satisfy Ceph prerequisites
      import_role:
        role: tripleo_cephadm
        tasks_from: pre

    - name: Bootstrap Ceph
      import_role:
        role: tripleo_cephadm
        tasks_from: bootstrap

    - name: Apply Ceph spec
      import_role:
        role: tripleo_cephadm
        tasks_from: apply_spec
      when: not tripleo_cephadm_spec_on_bootstrap

    - name: Create Pools
      import_role:
        role: tripleo_cephadm
        tasks_from: pools

    - name: Create Keys
      import_role:
        role: tripleo_cephadm
        tasks_from: keys

    - name: Export configuration for tripleo_ceph_client
      import_role:
        role: tripleo_cephadm
        tasks_from: export
      vars:
        cephx_keys:
          - client.openstack