Role Based Access Control - Testing

The Role Based Access control testing is a minor departure from the Ironic standard pattern of entirely python based unit testing. In part this was done for purposes of speed and to keep the declaration of the test context.

This also lended itself to be very useful due to the nature of A/B testing which is required to properly migrate the Ironic project from a project scoped universe where an admin project is utilized as the authenticating factor coupled with two custom roles, baremetal_admin, and baremetal_observer.

As a contributor looking back after getting a over a thousand additional tests in place using this method, it definitely helped the speed at which these were created, and then ported to support additional.

How these tests work

These tests execute API calls through the API layer, using the appropriate verb and header, which settings to prevent the keystonemiddleware from intercepting and replacing the headers we’re passing. Ultimately this is a feature, and it helps quite a bit.

The second aspect of how this works is we’re mocking the conductor RPC get_topic_for and get_random_topic_for methods. These calls raise Temporary Unavailable, since trying to execute the entire interaction into the conductor is moderately pointless because all policy enforement is located with-in the API layer.

At the same time wiring everything up to go from API to conductor code and back would have been a heavier lift. As such, the tests largely look for one of the following error codes.

• 200 - Got the item from the API - This is an database driven interaction.

• 201 - Created - This is database driven interaction. These are rare.

• 204 - Accepted - This is a database driven interaction. These are rare.

• 403 - Forbidden - This tells us the policy worked as expected where

  access was denied.

• 404 - NotFound - This is typically when objects were not found. Before

  Ironic becomes scope aware, these are generally only in the drivers API endpoint’s behavior. In System scope aware Project scoped configuration, i.e. later RBAC tests, this will become the dominant response for project scoped users as responding with a 403 if they could be an owner or lessee would provide insight into the existence of a node.

• 503 - Service Unavailable - In the context of our tests, we expect this

  when a request has been successfully authenticated and would have been sent along to the conductor.

How to make changes or review these tests?

The tests cycle through the various endpoints, and repeating patterns are clearly visible. Typically this means a given endpoint is cycled through with the same basic test using slightly different parameters such as different authentication parameters. When it comes to system scope aware tests supporting node owners and lessee, these tests will cycle a little more with slightly different attributes as the operation is not general against a shared common node, but different nodes.

Some tests will test body contents, or attributes. some will validate the number of records returned. This is important later with owner and lessee having slightly different views of the universe.

Some general rules apply

• Admins can do things, at least as far as their scope or rights apply. Remember: owner and lessee admins are closer to System scoped Admin Members.

• Members can do some things, but not everything

• Readers can always read, but as we get into sensitive data later on such as fields containing infrastructure internal addresses, these values will become hidden and additional tests will examine this.

• Third party, or external/other Admins will find nothing but sadness in empty lists, 403, 404, or even 500 errors.

What is/will be tested?

The idea is to in essence test as much as possible, however as these tests Role Based Access Control related capabilities will come in a series of phases, styles vary a little.

The first phase is "legacy". In essence these are partially programmatically generated and then human reviewed and values populated with expected values.

The second phase is remarkably similar to legacy. It is the safety net where we execute the legacy tests with the updated oslo.policy configuration to help enforce scopes. These tests will intentionally begin to fail in phase three.

The third phase is the implementation of System scope awareness for the API. In this process, as various portions of the API are made system scope aware. The legacy tests are marked as deprecated which signals to the second phase test sequences that they are expected to fail. New system scoped tests are also implemented which are matched up by name to the legacy tests. The major difference being some header values, and a user with a member role in the system scope now has some rights.

The forth phase, is implementation of owner and lessee aware project scoping. The testing approach is similar, however it is much more of a “shotgun” approach. We test what we know should work, and what know should not work, but we do not have redundant testing for each role as admin users are also members, and since the policy rules are designed around thresholds of access, it just made no sense to run the same test for admin and members, where member was the threshold. These thresholds will vary with the proposed default policy. The forth scope also tests a third party external admin as a negative test to ensure that we are also denying access to resources appropriately.