Kosmos Mid Cycle

https://upload.wikimedia.org/wikipedia/commons/5/58/Seattle_Center_as_night_falls.jpg

[1]

Kosmos Mid Cycle

We sat down in Seattle last week to plan out how we were going to kick start progess on Kosmos

(For those of you who don't know what Kosmos is - here is the overview on the wiki )

We decided on a few things, I managed to clarify my (admitedly weird and convaluted) thought process, and we started on the inital sets of patches.

It was a small group - some people were sick from the LBaaS / VPN / FW mid cycle the week before, but enough to start.

We covered the DB, the layout, and how we will interact with it. I showed our basic use of the olso_versionedobjects lib, and the basic pecan framework that was in place.

We have the basic design up on our specs repo here ( it is not publishing anywhere yet - I will fix that soon )

If anyone is interested in the project, give me a ping - be it for developing drivers, code, requirements or general interest. (But, really - if you want to write code, just jump in, or ping me for what the hot topics are)

In 2 weeks time I will be off the Designate Mid Cycle - a little closer to home this time - in Galway, Ireland.

[1] By Jeffery Hayes (Jeffery Hayes) [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons

Big Rename - Complete!!

http://replygif.net/i/789.gif

It is done !! :)

So, yesterday Tim and Kiall +2'd the change.

It was quite a bit bigger than expected (+4444, -4230) so thanks to all the reviewers who put time and effort into checking everything worked, and I hadn't made any glaring mistakes :).

For anyone interested (or mad) the change is here: Review

The Big Rename - why?

Update

Due to unforeseen circumstances I will not be able to continue work on this until Monday, so we will have to extend the freeze until Tuesday the 17th.

The following message will be put on the patches affected by the freeze.

big-rename-2-update-message.txt (Source)

Due to unforeseen circumstances I will not be able to continue work on this until
Monday, so we will have to extend the freeze until Tuesday the 17th.

On Tuesday 2015-11-17 this -2 will be removed, and code review as normal will continue.

I apologise for the delay, and will get this completed ASAP.

Further updates will be posted to http://graham.hayes.ie/posts/the-big-rename/
https://upload.wikimedia.org/wikipedia/commons/1/1a/Code.jpg

[1]

The Big Rename

So from tomorrow (2015-11-10) until Friday (2015-11-13) Designate will be frozen for new code.

I send an email to the openstack-dev list before the summit [2] so, it shouldn't be a surprise (I hope)

So, why are we doing this?

When Designate started it had a v1 API, which used the term domains for DNS Domains. This was pre Keystone's Domain support, but unfortunately they also decided to use it. AS Designate was not even incubated at the time, there was not much we could do, but continue on.

When we started the v2 API, we decided to avoid confusion and expose "Domains" as "Zones".

This worked for a while, but as we worked on the V2 API, we had to write code to translate the internal code (which referenced Domains) to Zones. this caused code like:

big-rename-render.py (Source)

def _render_object(cls, object, *args, **kwargs):
    # The dict we will return to be rendered to JSON / output format
    r_obj = {}
    # Loop over all fields that are supposed to be output
    for key, value in cls.MODIFICATIONS['fields'].items():
        # Get properties for this field
        field_props = cls.MODIFICATIONS['fields'][key]
        # Check if it has to be renamed
        if field_props.get('rename', False):
            obj = getattr(object, field_props.get('rename'))
            # if rename is specified we need to change the key
            obj_key = field_props.get('rename')
        else:
            # if not, move on
            obj = getattr(object, key, None)
            obj_key = key

and

big-rename-error-render.py (Source)

@classmethod
def _rename_path_segment(cls, obj_adapter, object, path_segment):

    # Check if the object is a list - lists will just have an index as a
    # value, ands this can't be renamed
    if issubclass(obj_adapter.ADAPTER_OBJECT, objects.ListObjectMixin):
        obj_adapter = cls.get_object_adapter(
            cls.ADAPTER_FORMAT,
            obj_adapter.ADAPTER_OBJECT.LIST_ITEM_TYPE.obj_name())
        # Return the segment as is, and the next adapter (which is the
        # LIST_ITEM_TYPE)
        return path_segment, obj_adapter

    for key, value in obj_adapter.MODIFICATIONS.get(
            'fields', {}).items():

        # Check if this field as actually a nested object
        if object.FIELDS.get(path_segment, {}).get('relation', False):

            obj_cls = object.FIELDS.get(path_segment).get('relation_cls')
            obj_adapter = cls.get_object_adapter(
                cls.ADAPTER_FORMAT,
                obj_cls)

            object = objects.DesignateObject.obj_cls_from_name(obj_cls)()
            # Recurse down into this object
            path_segment, obj_adapter = cls._rename_path_segment(
                obj_adapter, object, path_segment)

            # No need to continue the loop
            break

        if not isinstance(
            value.get(
                'rename', NotSpecifiedSential()), NotSpecifiedSential)\
                and path_segment == value.get('rename'):
            # Just do the rename
            path_segment = key

    return path_segment, obj_adapter

which proved to be quite fragile. (Yes that last bit of code is trying to walk the error path of a JSONSchema Error and rename the path to the new terminology)

We are also logging messages about domains - which could prove to be quite confusing when we remove the v1 API and users are only interacting with zones.

So tomorrow morning-ish (UTC) we will start to rename every occurrence of "domain" in the code base to "zone" (without breaking Keystone domain support).

Anything that does not have the "the-big-rename" topic on Gerrit will be getting a procedural -2 from me or the designate-core team.

big-rename-2-message.txt (Source)

Currently Designate is undergoing a code freeze to allow us re-factor the code base,
as announced here: - http://lists.openstack.org/pipermail/openstack-dev/2015-October/077442.html

All code that is not in the "the-big-rename" topic will be getting this procedural -2.

More information can be found here: - http://graham.hayes.ie/posts/the-big-rename/

On Friday 2015-11-13 this -2 will be removed, and code review as normal will continue.

This re-factor will break the majority of patches that are outstanding, and you may need to manually
rebase your patch when we remove the -2.
[1] By Crusher95 (Own work) [CC BY-SA 4.0 (http://creativecommons.org/licenses/by-sa/4.0)], via Wikimedia Commons

OpenStack Summit - Designate Report

https://upload.wikimedia.org/wikipedia/commons/7/70/Ginza_at_Night,_Tokyo.jpg

[1]

Tokyo

Oh, what a place. I thought I was ready for the experience, but this city has to be visited to understand. From the 2 or 3 different metro systems that snake across the city, to the 9 story electronics stores, it is something else.

Living in a city that has 3 metro lines that don't actually connect it was a nice change to be able to get reliable public transport :).

Design Summit

This was a much more relaxed summit for Designate. We had done a huge amount of work in Vancouver, and we were nailing down details and doing cross project work.

We got a few major features discussed, and laid out our priorities for the next cycle.

We decided on the following:

  1. Nova / Neutron Integration
  2. Pool Scheduler
  3. Pool Configuration migration to database
  4. IXFR (Incremental Zone Transfer)
  5. ALIAS Record type (Allows for CNAME like records at the root of a DNS Zone)
  6. DNSSEC (this may drag on for a cycle or two)

Nova & Neutron Integration

This is progressing pretty well, and Miguel Lavalle has patches up for this. He, Kiall Mac Innes and Carl Baldwin demoed this in a session on the Thursday. If you are interested in the idea, it is definitely worth a watch here

Pool Scheduler

A vital piece of the pools re architecture that needs to be finished out. There is no great debate on what we need, and I have taken on the task of finishing this out.

Pool Configuration migration to database

Are current configuration file format is quite complex, and moving it to an API allows us to iterate on it much quicker, while reducing the complexity of the config file. I recently had to write an ansible play to write out this file, and it was not fun.

Kiall had a patch up, so we should be able to continue based on that.

IXFR

There was quite a lot of discussion on how this will be implemented, both in the work session, and the 1/2 day session on the Friday. Tim Simmons has stepped up, to continue the work on this.

ALIAS

This is quite a sort after feature - but is quite complex to implement. The DNS RFCs explicitly ban this behavior, so we have to work the solution around them. Eric Larson has been doing quite a lot of work on this in Liberty, and is going to continue in Mitaka.

DNSSEC

This is a feature that we have been looking at for a while, but we started to plan out our roadmap for it recently.

We (I) am allergic to storing private encryption keys in Designate, so we had a good conversation with Barbican about implementing a signing endpoint that we would post a hash to. This work is on me to now drive for Mitaka, so we can consume it in N.

There is some raw notes in the etherpad and I expect we will soon be seeing specs building out of them.

[1] By Francisco Diez (Flickr: Ginza at Night, Tokyo) [CC BY 2.0 (http://creativecommons.org/licenses/by/2.0)], via Wikimedia Commons

Starting Blogging (Again)

Having taken a year or two hiatus on blogging, I have decided to give it another shot.

A lot has happened since then, and recently I have started to do more and more work in upstream OpenStack. Starting as a PTL I thought it might be useful to have somewhere for me to rant, rave, and generally explain my (admittedly from time to time bizarre) thought process.

I reckon there will be flurries of activity here, followed by silence, but we shall see what happens in time :)