Commit ce4c9d79 authored by Fulvio Galeazzi's avatar Fulvio Galeazzi
Browse files

2019-02-07: FG; Reverted back to functional version, very close to charm version 31.

parent b1a48e4d
Overview
========
This charm provides Keystone, the Openstack identity service. Its target
This charm provides Keystone, the Openstack identity service. It's target
platform is (ideally) Ubuntu LTS + Openstack.
This is a modified version, which adds support for Identity
......@@ -14,7 +14,7 @@ The following interfaces are provided:
- nrpe-external-master: Used to generate Nagios checks.
- identity-service: Openstack API endpoints request an entry in the
- identity-service: Openstack API endpoints request an entry in the
Keystone service catalog + endpoint template catalog. When a relation
is established, Keystone receives: service name, region, public_url,
admin_url and internal_url. It first checks that the requested service
......@@ -97,28 +97,33 @@ If 'dns-ha' is set and none of the os-{admin,internal,public}-hostname(s) are se
SSL/HTTPS
---------
Support for SSL and https endpoint is provided via various charm configuration
options.
Support for SSL and https endpoint is provided via a set of configuration
options on the charm. There are two types supported;
To enable SSL and https endpoint with a charm-generated CA, set the following
configuration options:
use-https - if enabled this option tells Keystone to configure the identity
endpoint as https. Under this model the keystone charm will either use the CA
as provided by the user (see ssl_* options below) or will generate its own and
sync across peers. The cert will be distributed to all service endpoints which
will be configured to use https.
- use-https - if enabled this option tells Keystone to configure the identity
endpoint as https, and the keystone charm will generate its own CA and sync
across peers. The cert will be distributed to all service endpoints which
will be configured to use https.
https-service-endpoints - if enabled this option tells Keystone to configure
ALL endpoints as https. Under this model the keystone charm will either use the
CA as provided by the user (see ssl_* options below) or will generate its own
and sync across peers. The cert will be distributed to all service endpoints
which will be configured to use https as well as configuring themselves to be
used as https.
- https-service-endpoints - if enabled this option tells Keystone to configure
ALL endpoints as https. Under this model the keystone charm will generate its
own CA and sync across peers. The cert will be distributed to all service
endpoints which will be configured to use https as well as configuring
themselves to be used as https.
When configuring the charms to use SSL there are three charm config options as
ssl_ca, ssl_cert and ssl_key.
To enable SSL and https endpoint with your own CA, SSL cert, and key set the
following configuration options: ssl_ca, ssl_cert, and ssl_key. The user can
provide SSL cert and key using ssl_cert and ssl_key only when the cert is
signed by a trusted CA. These options should not be used with use-https and
https-service-endpoints.
- The user can provide their own CA, SSL cert and key using the options ssl_ca,
ssl_cert, ssl_key.
- The user can provide SSL cert and key using ssl_cert and ssl_key when the cert
is signed by a trusted CA.
- If not provided, the keystone charm will automatically generate a CA and certs
to distribute to endpoints.
When the charm configures itself as a CA (generally only recommended for test
purposes) it will elect an "ssl-cert-master" whose duty is to generate the CA
......@@ -164,7 +169,7 @@ To use this feature, use the --bind option when deploying the charm:
juju deploy keystone --bind "public=public-space internal=internal-space admin=admin-space shared-db=internal-space"
Alternatively these can also be provided as part of a juju native bundle configuration:
alternatively these can also be provided as part of a juju native bundle configuration:
keystone:
charm: cs:xenial/keystone
......@@ -177,7 +182,7 @@ Alternatively these can also be provided as part of a juju native bundle configu
NOTE: Spaces must be configured in the underlying provider prior to attempting to use them.
NOTE: Existing deployments using os\-\*-network configuration options will continue to function; these options are preferred over any network space binding provided if set.
NOTE: Existing deployments using os-*-network configuration options will continue to function; these options are preferred over any network space binding provided if set.
Federated Authentication
------------------------
......
git-reinstall:
description: Reinstall keystone from the openstack-origin-git repositories.
pause:
description: |
Pause keystone services.
......
../charmhelpers
\ No newline at end of file
#!/usr/bin/python
#
# Copyright 2016 Canonical Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import traceback
from charmhelpers.contrib.openstack.utils import (
git_install_requested,
)
from charmhelpers.core.hookenv import (
action_set,
action_fail,
config,
)
from hooks.keystone_utils import (
git_install,
)
from hooks.keystone_hooks import (
config_changed,
)
def git_reinstall():
"""Reinstall from source and restart services.
If the openstack-origin-git config option was used to install openstack
from source git repositories, then this action can be used to reinstall
from updated git repositories, followed by a restart of services."""
if not git_install_requested():
action_fail('openstack-origin-git is not configured')
return
try:
git_install(config('openstack-origin-git'))
config_changed()
except:
action_set({'traceback': traceback.format_exc()})
action_fail('git-reinstall resulted in an unexpected error')
if __name__ == '__main__':
git_reinstall()
#!/usr/bin/python
#
# Copyright 2016 Canonical Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import traceback
from charmhelpers.contrib.openstack.utils import (
git_install_requested,
)
from charmhelpers.core.hookenv import (
action_set,
action_fail,
config,
)
from hooks.keystone_utils import (
git_install,
)
from hooks.keystone_hooks import (
config_changed,
)
def git_reinstall():
"""Reinstall from source and restart services.
If the openstack-origin-git config option was used to install openstack
from source git repositories, then this action can be used to reinstall
from updated git repositories, followed by a restart of services."""
if not git_install_requested():
action_fail('openstack-origin-git is not configured')
return
try:
git_install(config('openstack-origin-git'))
config_changed()
except:
action_set({'traceback': traceback.format_exc()})
action_fail('git-reinstall resulted in an unexpected error')
if __name__ == '__main__':
git_reinstall()
../hooks
\ No newline at end of file
../charmhelpers
\ No newline at end of file
......@@ -65,8 +65,7 @@ def get_ca_cert():
if ca_cert is None:
log("Inspecting identity-service relations for CA SSL certificate.",
level=INFO)
for r_id in (relation_ids('identity-service') +
relation_ids('identity-credentials')):
for r_id in relation_ids('identity-service'):
for unit in relation_list(r_id):
if ca_cert is None:
ca_cert = relation_get('ca_cert',
......@@ -77,7 +76,7 @@ def get_ca_cert():
def retrieve_ca_cert(cert_file):
cert = None
if os.path.isfile(cert_file):
with open(cert_file, 'rb') as crt:
with open(cert_file, 'r') as crt:
cert = crt.read()
return cert
......
......@@ -223,11 +223,6 @@ def https():
return True
if config_get('ssl_cert') and config_get('ssl_key'):
return True
for r_id in relation_ids('certificates'):
for unit in relation_list(r_id):
ca = relation_get('ca', rid=r_id, unit=unit)
if ca:
return True
for r_id in relation_ids('identity-service'):
for unit in relation_list(r_id):
# TODO - needs fixing for new helper as ssl_cert/key suffixes with CN
......@@ -376,7 +371,6 @@ def distributed_wait(modulo=None, wait=None, operation_name='operation'):
''' Distribute operations by waiting based on modulo_distribution
If modulo and or wait are not set, check config_get for those values.
If config values are not set, default to modulo=3 and wait=30.
:param modulo: int The modulo number creates the group distribution
:param wait: int The constant time wait value
......@@ -388,17 +382,10 @@ def distributed_wait(modulo=None, wait=None, operation_name='operation'):
:side effect: Calls time.sleep()
'''
if modulo is None:
modulo = config_get('modulo-nodes') or 3
modulo = config_get('modulo-nodes')
if wait is None:
wait = config_get('known-wait') or 30
if juju_is_leader():
# The leader should never wait
calculated_wait = 0
else:
# non_zero_wait=True guarantees the non-leader who gets modulo 0
# will still wait
calculated_wait = modulo_distribution(modulo=modulo, wait=wait,
non_zero_wait=True)
wait = config_get('known-wait')
calculated_wait = modulo_distribution(modulo=modulo, wait=wait)
msg = "Waiting {} seconds for {} ...".format(calculated_wait,
operation_name)
log(msg, DEBUG)
......
......@@ -27,7 +27,6 @@ from charmhelpers.core.hookenv import (
network_get_primary_address,
unit_get,
WARNING,
NoNetworkBinding,
)
from charmhelpers.core.host import (
......@@ -110,12 +109,7 @@ def get_address_in_network(network, fallback=None, fatal=False):
_validate_cidr(network)
network = netaddr.IPNetwork(network)
for iface in netifaces.interfaces():
try:
addresses = netifaces.ifaddresses(iface)
except ValueError:
# If an instance was deleted between
# netifaces.interfaces() run and now, its interfaces are gone
continue
addresses = netifaces.ifaddresses(iface)
if network.version == 4 and netifaces.AF_INET in addresses:
for addr in addresses[netifaces.AF_INET]:
cidr = netaddr.IPNetwork("%s/%s" % (addr['addr'],
......@@ -584,9 +578,6 @@ def get_relation_ip(interface, cidr_network=None):
except NotImplementedError:
# If network-get is not available
address = get_host_ip(unit_get('private-address'))
except NoNetworkBinding:
log("No network binding for {}".format(interface), WARNING)
address = get_host_ip(unit_get('private-address'))
if config('prefer-ipv6'):
# Currently IPv6 has priority, eventually we want IPv6 to just be
......
......@@ -21,9 +21,6 @@ from collections import OrderedDict
from charmhelpers.contrib.amulet.deployment import (
AmuletDeployment
)
from charmhelpers.contrib.openstack.amulet.utils import (
OPENSTACK_RELEASES_PAIRS
)
DEBUG = logging.DEBUG
ERROR = logging.ERROR
......@@ -274,8 +271,11 @@ class OpenStackAmuletDeployment(AmuletDeployment):
release.
"""
# Must be ordered by OpenStack release (not by Ubuntu release):
for i, os_pair in enumerate(OPENSTACK_RELEASES_PAIRS):
setattr(self, os_pair, i)
(self.trusty_icehouse, self.trusty_kilo, self.trusty_liberty,
self.trusty_mitaka, self.xenial_mitaka, self.xenial_newton,
self.yakkety_newton, self.xenial_ocata, self.zesty_ocata,
self.xenial_pike, self.artful_pike, self.xenial_queens,
self.bionic_queens,) = range(13)
releases = {
('trusty', None): self.trusty_icehouse,
......@@ -291,8 +291,6 @@ class OpenStackAmuletDeployment(AmuletDeployment):
('zesty', None): self.zesty_ocata,
('artful', None): self.artful_pike,
('bionic', None): self.bionic_queens,
('bionic', 'cloud:bionic-rocky'): self.bionic_rocky,
('cosmic', None): self.cosmic_rocky,
}
return releases[(self.series, self.openstack)]
......@@ -308,7 +306,6 @@ class OpenStackAmuletDeployment(AmuletDeployment):
('zesty', 'ocata'),
('artful', 'pike'),
('bionic', 'queens'),
('cosmic', 'rocky'),
])
if self.openstack:
os_origin = self.openstack.split(':')[1]
......
......@@ -40,7 +40,6 @@ import novaclient
import pika
import swiftclient
from charmhelpers.core.decorators import retry_on_exception
from charmhelpers.contrib.amulet.utils import (
AmuletUtils
)
......@@ -51,13 +50,6 @@ ERROR = logging.ERROR
NOVA_CLIENT_VERSION = "2"
OPENSTACK_RELEASES_PAIRS = [
'trusty_icehouse', 'trusty_kilo', 'trusty_liberty',
'trusty_mitaka', 'xenial_mitaka', 'xenial_newton',
'yakkety_newton', 'xenial_ocata', 'zesty_ocata',
'xenial_pike', 'artful_pike', 'xenial_queens',
'bionic_queens', 'bionic_rocky', 'cosmic_rocky']
class OpenStackAmuletUtils(AmuletUtils):
"""OpenStack amulet utilities.
......@@ -71,34 +63,7 @@ class OpenStackAmuletUtils(AmuletUtils):
super(OpenStackAmuletUtils, self).__init__(log_level)
def validate_endpoint_data(self, endpoints, admin_port, internal_port,
public_port, expected, openstack_release=None):
"""Validate endpoint data. Pick the correct validator based on
OpenStack release. Expected data should be in the v2 format:
{
'id': id,
'region': region,
'adminurl': adminurl,
'internalurl': internalurl,
'publicurl': publicurl,
'service_id': service_id}
"""
validation_function = self.validate_v2_endpoint_data
xenial_queens = OPENSTACK_RELEASES_PAIRS.index('xenial_queens')
if openstack_release and openstack_release >= xenial_queens:
validation_function = self.validate_v3_endpoint_data
expected = {
'id': expected['id'],
'region': expected['region'],
'region_id': 'RegionOne',
'url': self.valid_url,
'interface': self.not_null,
'service_id': expected['service_id']}
return validation_function(endpoints, admin_port, internal_port,
public_port, expected)
def validate_v2_endpoint_data(self, endpoints, admin_port, internal_port,
public_port, expected):
public_port, expected):
"""Validate endpoint data.
Validate actual endpoint data vs expected endpoint data. The ports
......@@ -127,7 +92,7 @@ class OpenStackAmuletUtils(AmuletUtils):
return 'endpoint not found'
def validate_v3_endpoint_data(self, endpoints, admin_port, internal_port,
public_port, expected, expected_num_eps=3):
public_port, expected):
"""Validate keystone v3 endpoint data.
Validate the v3 endpoint data which has changed from v2. The
......@@ -173,89 +138,10 @@ class OpenStackAmuletUtils(AmuletUtils):
if ret:
return 'unexpected endpoint data - {}'.format(ret)
if len(found) != expected_num_eps:
if len(found) != 3:
return 'Unexpected number of endpoints found'
def convert_svc_catalog_endpoint_data_to_v3(self, ep_data):
"""Convert v2 endpoint data into v3.
{
'service_name1': [
{
'adminURL': adminURL,
'id': id,
'region': region.
'publicURL': publicURL,
'internalURL': internalURL
}],
'service_name2': [
{
'adminURL': adminURL,
'id': id,
'region': region.
'publicURL': publicURL,
'internalURL': internalURL
}],
}
"""
self.log.warn("Endpoint ID and Region ID validation is limited to not "
"null checks after v2 to v3 conversion")
for svc in ep_data.keys():
assert len(ep_data[svc]) == 1, "Unknown data format"
svc_ep_data = ep_data[svc][0]
ep_data[svc] = [
{
'url': svc_ep_data['adminURL'],
'interface': 'admin',
'region': svc_ep_data['region'],
'region_id': self.not_null,
'id': self.not_null},
{
'url': svc_ep_data['publicURL'],
'interface': 'public',
'region': svc_ep_data['region'],
'region_id': self.not_null,
'id': self.not_null},
{
'url': svc_ep_data['internalURL'],
'interface': 'internal',
'region': svc_ep_data['region'],
'region_id': self.not_null,
'id': self.not_null}]
return ep_data
def validate_svc_catalog_endpoint_data(self, expected, actual,
openstack_release=None):
"""Validate service catalog endpoint data. Pick the correct validator
for the OpenStack version. Expected data should be in the v2 format:
{
'service_name1': [
{
'adminURL': adminURL,
'id': id,
'region': region.
'publicURL': publicURL,
'internalURL': internalURL
}],
'service_name2': [
{
'adminURL': adminURL,
'id': id,
'region': region.
'publicURL': publicURL,
'internalURL': internalURL
}],
}
"""
validation_function = self.validate_v2_svc_catalog_endpoint_data
xenial_queens = OPENSTACK_RELEASES_PAIRS.index('xenial_queens')
if openstack_release and openstack_release >= xenial_queens:
validation_function = self.validate_v3_svc_catalog_endpoint_data
expected = self.convert_svc_catalog_endpoint_data_to_v3(expected)
return validation_function(expected, actual)
def validate_v2_svc_catalog_endpoint_data(self, expected, actual):
def validate_svc_catalog_endpoint_data(self, expected, actual):
"""Validate service catalog endpoint data.
Validate a list of actual service catalog endpoints vs a list of
......@@ -424,7 +310,6 @@ class OpenStackAmuletUtils(AmuletUtils):
self.log.debug('Checking if tenant exists ({})...'.format(tenant))
return tenant in [t.name for t in keystone.tenants.list()]
@retry_on_exception(num_retries=5, base_delay=1)
def keystone_wait_for_propagation(self, sentry_relation_pairs,
api_version):
"""Iterate over list of sentry and relation tuples and verify that
......@@ -443,7 +328,7 @@ class OpenStackAmuletUtils(AmuletUtils):
if rel.get('api_version') != str(api_version):
raise Exception("api_version not propagated through relation"
" data yet ('{}' != '{}')."
"".format(rel.get('api_version'), api_version))
"".format(rel['api_version'], api_version))
def keystone_configure_api_version(self, sentry_relation_pairs, deployment,
api_version):
......@@ -465,13 +350,16 @@ class OpenStackAmuletUtils(AmuletUtils):
deployment._auto_wait_for_status()
self.keystone_wait_for_propagation(sentry_relation_pairs, api_version)
def authenticate_cinder_admin(self, keystone, api_version=2):
def authenticate_cinder_admin(self, keystone_sentry, username,
password, tenant, api_version=2):
"""Authenticates admin user with cinder."""
self.log.debug('Authenticating cinder admin...')
# NOTE(beisner): cinder python client doesn't accept tokens.
keystone_ip = keystone_sentry.info['public-address']
ept = "http://{}:5000/v2.0".format(keystone_ip.strip().decode('utf-8'))
_clients = {
1: cinder_client.Client,
2: cinder_clientv2.Client}
return _clients[api_version](session=keystone.session)
return _clients[api_version](username, password, tenant, ept)
def authenticate_keystone(self, keystone_ip, username, password,
api_version=False, admin_port=False,
......@@ -479,36 +367,13 @@ class OpenStackAmuletUtils(AmuletUtils):
project_domain_name=None, project_name=None):
"""Authenticate with Keystone"""
self.log.debug('Authenticating with keystone...')
if not api_version:
api_version = 2
sess, auth = self.get_keystone_session(
keystone_ip=keystone_ip,
username=username,
password=password,
api_version=api_version,
admin_port=admin_port,
user_domain_name=user_domain_name,
domain_name=domain_name,
project_domain_name=project_domain_name,
project_name=project_name
)
if api_version == 2:
client = keystone_client.Client(session=sess)
else:
client = keystone_client_v3.Client(session=sess)
# This populates the client.service_catalog
client.auth_ref = auth.get_access(sess)
return client
def get_keystone_session(self, keystone_ip, username, password,
api_version=False, admin_port=False,
user_domain_name=None, domain_name=None,
project_domain_name=None, project_name=None):
"""Return a keystone session object"""
ep = self.get_keystone_endpoint(keystone_ip,
api_version=api_version,
admin_port=admin_port)
if api_version == 2:
port = 5000
if admin_port:
port = 35357
base_ep = "http://{}:{}".format(keystone_ip.strip().decode('utf-8'),
port)
if not api_version or api_version == 2:
ep = base_ep + "/v2.0"
auth = v2.Password(
username=username,
password=password,
......@@ -516,7 +381,12 @@ class OpenStackAmuletUtils(AmuletUtils):
auth_url=ep
)
sess = keystone_session.Session(auth=auth)
client = keystone_client.Client(session=sess)
# This populates the client.service_catalog
client.auth_ref = auth.get_access(sess)
return client
else:
ep = base_ep + "/v3"
auth = v3.Password(
user_domain_name=user_domain_name,
username=username,
......@@ -527,57 +397,10 @@ class OpenStackAmuletUtils(AmuletUtils):
auth_url=ep
)
sess = keystone_session.Session(auth=auth)
return (sess, auth)
def get_keystone_endpoint(self, keystone_ip, api_version=None,
admin_port=False):
"""Return keystone endpoint"""
port = 5000
if admin_port:
port = 35357
base_ep = "http://{}:{}".format(keystone_ip.strip().decode('utf-8'),