Development Environment¶

Prerequisites¶

Install prerequisites:

# Ubuntu/Debian:
sudo apt-get update
sudo apt-get install python-dev libssl-dev python-pip libxml2-dev \
                     libxslt-dev git git-review libffi-dev gettext \
                     python-tox

# Fedora/RHEL:
sudo yum install python-devel openssl-devel python-pip libxml2-devel \
                 libxslt-devel git git-review libffi-devel gettext

# openSUSE/SLE 12:
sudo zypper install git git-review libffi-devel \
                    libopenssl-devel libxml2-devel libxslt-devel \
                    python-devel python-flake8 \
                    python-nose python-pip python-setuptools-git \
                    python-testrepository python-tox python-virtualenv \
                    gettext-runtime

sudo easy_install nose
sudo pip install virtualenv setuptools-git flake8 tox testrepository

If using RHEL and yum reports “No package python-pip available” and “No package git-review available”, use the EPEL software repository. Instructions can be found at http://fedoraproject.org/wiki/EPEL/FAQ#howtouse.

You may need to explicitly upgrade virtualenv if you’ve installed the one from your OS distribution and it is too old (tox will complain). You can upgrade it individually, if you need to:

sudo pip install -U virtualenv

This guide assumes that you also have the following services minimally available:

• Elasticsearch (See Installing and Configuring Elasticsearch for install information).
• Keystone

Additional services required to be installed will depend on the plugins activated in searchlight.

Installing Searchlight¶

1. Clone the Searchlight repo from git.openstack.org

$ mkdir openstack
$ cd openstack
$ git clone https://git.openstack.org/openstack/searchlight.git
$ cd searchlight

2. Setup a virtualenv

$ tox -evenv --notest

Activate the virtual environment whenever you want to work in it. All further commands in this section should be run with the venv active:

$ . .tox/venv/bin/activate

4. Install Searchlight and its dependencies

(venv) $ python setup.py develop

5. Generate sample config.

(venv) $ oslo-config-generator --config-file
etc/oslo-config-generator/searchlight.conf

6. Create Searchlight’s config files by copying the sample config files

$ cd etc/
$ ls *.sample | while read f; do cp $f $(echo $f | sed "s/.sample$//g"); done

7. Make the directory for Searchlight’s log files

$ mkdir -p ../log

8. Make the directory for Searchlight’s state files

$ mkdir -p ../state

Configuring Searchlight¶

Searchlight has several configuration files. The following are the basic configuration suggestions.

[DEFAULT]
debug = true
log_file = log/searchlight.log

Initialize the Elasticsearch Index¶

Perform initial sync of the resources to Elasticsearch. All plugins for searchlight must have their services installed and available at an endpoint registered in keystone (e.g. glance).

$ cd <install dir. eg: openstack/searchlight>

# Make sure your virtualenv is sourced
$ . .tox/venv/bin/activate

# Run the index operation.
(venv) $ searchlight-manage index sync

# Alternatively, you can directly invoke the command using the following.
(venv) $ python searchlight/cmd/manage.py --config-file etc/searchlight.conf index sync

This command may be re-run at any time to perform a full re-index.

# Assume elasticsearch is running on localhost, and the default
# 'searchlight' index is in use
curl -X DELETE localhost:9200/searchlight

(venv) $ searchlight-manage --config-file etc/searchlight.conf index sync --no-delete

# Change THIS
elasticsearch>=1.3.0
# To THIS
elasticsearch>=1.3.0,<2.0.0

(venv) $ pip install -r requirements.txt
(venv) $ searchlight-manage --config-file etc/searchlight.conf index sync

Start Index Update Monitoring¶

The index is updated continually based on updates to the source resource data. Start this service to start update monitoring. Note, depending on the resource type, this will typically require that you have configured notifications properly for the service which owns the resource (e.g. Glance images).

$ cd <install dir. eg: openstack/searchlight>

# Make sure your virtualenv is sourced
$ . .tox/venv/bin/activate

# Start the index update monitoring.
(venv) $ searchlight-listener --config-file etc/searchlight.conf

# Alternatively, you can directly invoke the command using the following.
(venv) $ python searchlight/cmd/listener.py --config-file
etc/searchlight.conf

Initialize & Start the API Service¶

Open up a new ssh window and log in to your server (or however you’re communicating with your server).

$ cd <install dir. eg: openstack/searchlight>

# Make sure your virtualenv is sourced
$ . .tox/venv/bin/activate

# Start the API Service.
(venv) $ searchlight-api --config-file etc/searchlight.conf

# Alternatively, you can directly invoke the command using the following.
(venv) $ python searchlight/cmd/api.py --config-file etc/searchlight.conf

You should now see the log from the API service.

Exercising the API¶

Using a web browser, curl statement, or a REST client, calls can be made to the Searchlight API using the following format where “api_version” is v1 and “command” is any of the commands listed under the Searchlight API

http://IP.Address:9393/api_version/command

Example: List plugins:

$ curl http://localhost:9393/v1/search/plugins

Example: List all data:

# Prerequisite Setup:
$ token=<insert token>
$ touch search.json
# Paste content
{
  "query": {
    "match_all": {}
  },
  "limit": 1000
}

# Execute query
$ curl -X POST http://localhost:9393/v1/search -H "X-Auth-Token:$token" \
-d @search.json -H "Content-Type: application/json" | python -mjson.tool

You can find the IP Address of your server by running

curl -s checkip.dyndns.org | sed -e 's/.*Current IP Address: //' -e 's/<.*$//'

Troubleshooting¶

Elasticsearch:

You can directly connect to the Elasticsearch server and examine its contents. We recommend using the Sense extension for google chrome.

Notifications:

Use rabbitmqctl to examine unconsumed notifications:

sudo rabbitmqctl list_queues | grep info

There are also a number of management tools available to help with troubleshooting.

Please see: https://www.rabbitmq.com/management.html

If you have searchlight setup to share a notification topic, but do not have a notification pool configured, then notifications may be consumed by other listeners and searchlight will not receive the notifications.